in Ideas · 100 words

When I run rawdog, typically one of two things happens: it decides that it needs to do nothing at all, or it decides that it needs to fetch lots of feeds so it takes ages. A smarter scheduling algorithm could try to spread out feed fetches so that on average the delay between fetches is about what was specified in the config file, but the number of feeds fetched in any one session would be minimised. A really smart scheduler could try to balance the size of data to be fetched (using the size from last time as an estimate).