Feed timeout not configurable

[kurt@…]

The timeout used when fetching a feed seems to be hard-coded to some very short value; I have some feeds which Gregarius is only able to fetch 10% of the time or so because the server takes a while to respond. When I fetch the feed with curl, it may take 5-7 seconds or more, but it does come through eventually. I'd rather that Gregarius take longer to update and give each feed enough time to respond.

Obviously this would need to be user-configurable; people with hundreds of feeds may not want to wait very long for a feed to respond.

[sdcosta]

The reason for this low time out is because by default Apache kills your PHP script after 5 minutes. People with hundreds of feeds will not be able to update all their feeds if we increase the timeout.

That being said, I think the ajax-updater could be modified to increase the timeout, however it is quite heavily server intensive already.

I think ticket #189 (updating from the command line) would be a better option if you want to increase the timeout. Please feel free to reopen this ticket or post on #189 if you have further concerns.

[kurt@…]

Well, I understand that increasing the timeout won't work for everyone. I suppose that's why it made sense to me to expose MAGPIE_FETCH_TIME_OUT in the admin interface (or at least I think that's what it would take to increase the timeout). Best of all would be to make it per-feed (so that known slow feeds could get special treatment), but I suspect that that's just too much work. I see that there is also a ten second timeout in update.php, but I don't suppose that that matters if Snoopy times out after five seconds.

Or am I missing something larger here?

I have not done much hacking on the Gregarius code yet, but I do have experience with PHP, so I will see what I can do in the way of putting together a patch for this.

[sdcosta]

I guess we could having something in the new feed properties database that we are thinking about....

[gregarius@…]

what about using set_time_limit to up the script's max runtime?

http://de3.php.net/manual/en/function.set-time-limit.php

Maybe make it a config option with a note that it gets overrode by the ini value?

[gregarius@…]

what about using set_time_limit to up the script's max runtime?

http://de3.php.net/manual/en/function.set-time-limit.php

Maybe make it a config option with a note that it gets overrode by the ini value?

[sdcosta]

I dont think that this timeout can over-ride the webserver's time out. Anyway just realised that this is a dupe of #243

There is a discussion about it here http://sinless.org/pipermail/gregarius-dev/2005-October/thread.html#628 http://sinless.org/pipermail/gregarius-dev/2005-October/000631.html

I think this feature will be included in with the new feed properties table.

[kurt@…]

Sorry, but this is not a duplicate of bug 243, that bug deals with the update interval (how long to go between each update of the feed). This issue deals with the update timeout (how long to wait for the server to respond when updating). They're two different parameters.

The patches attached to bug 243 could probably be used as a basis for resolving this issue as well, though.

[gregarius@…]

I'm realtively certain it can, as I wrote a irc bot in php using that command, but then, my web host might have had things setup very very oddly.

[sdcosta]

You are right it is not a duplicate, but can be fixed at the same time. My fault. Here we go again. Sorry.

[mbonetti]

Test, please ignore