Hi,
It sounds like you've got the basic idea. You'll start a task that
will fetch the twitter stream you're interested in, that task will
insert another task to fetch again, and the process continues. You
need to name your tasks to prevent a task-bomb (fork bomb). I think
Nick talks about that in his blog:
http://blog.notdot.net/2010/03/Task-Queue-task-chaining-done-right
Although if it is something you just need to do over and over every
five minutes I would look into using cron.
Robert
On Tue, Nov 30, 2010 at 10:27, [email protected] <[email protected]> wrote:
> Can somebody please exlpain this more closely. I have more or less the
> same issue. I need to get a constant uptodate twitter stream to my
> webapp. Dont understand at all how I can substitude the daemon like
> behaviour. I understand that the first task hast do to the parsing job
> but what about the second? It just calls the first again and again
> like a loop?
>
> On Nov 1, 5:13 am, Robert Kluin <[email protected]> wrote:
>> http://code.google.com/appengine/docs/python/taskqueue/overview.html
>>
>> First, pop a task to start your crawl. From that task fire another
>> task to continue it. Keep doing that until you get what you need or
>> run out of resources. You'll probably want to use either named or
>> transactional tasks to prevent a fork bomb.
>>
>> Use cron, or a user request to initiate your 'spider.' Or to ensure
>> the process keeps running.
>>
>> Robert
>>
>> On Sun, Oct 31, 2010 at 22:34, sombra XX <[email protected]> wrote:
>> > task-queue? how?
>>
>> > 2010/10/31 Robert Kluin <[email protected]>
>>
>> >> Then use the other part of djidjadji's suggestion: task-queue.
>>
>> >> Robert
>>
>> >> On Sun, Oct 31, 2010 at 12:17, sombra XX <[email protected]> wrote:
>> >> > djidjadji: Yes, I thought about the cron, but the intervals are every 1
>> >> > minute, and I need to be every 5 seconds
>>
>> >> > 2010/10/31 djidjadji <[email protected]>
>>
>> >> >> Every request should finish within 30 seconds.
>> >> >> You can setup a cron job and/or task queue chain to simulate a
>> >> >> continuous processing.
>> >> >> You have to keep state information in the datastore or part of the
>> >> >> task arguments.
>>
>> >> >> 2010/10/31 SombraX <[email protected]>:
>> >> >> > Hello everyone. I try to build a program that works non-stop, is an
>> >> >> > infinite while that collects information from different Web sites and
>> >> >> > record the information in the database. The problem is that at
>> >> >> > startup
>> >> >> > is interpreted as a request and get the timeout error.
>> >> >> > Is it possible to build a servicedaemonwith python and boot from
>> >> >> > the
>> >> >> > console app enginne?
>>
>> >> >> > (my english is terrible , sorry....cago en die).
>>
>> >> >> > --
>> >> >> > You received this message because you are subscribed to the Google
>> >> >> > Groups "Google App Engine" group.
>> >> >> > To post to this group, send email to
>> >> >> > [email protected].
>> >> >> > To unsubscribe from this group, send email to
>> >> >> > [email protected].
>> >> >> > For more options, visit this group at
>> >> >> >http://groups.google.com/group/google-appengine?hl=en.
>>
>> >> >> --
>> >> >> You received this message because you are subscribed to the Google
>> >> >> Groups
>> >> >> "Google App Engine" group.
>> >> >> To post to this group, send email to [email protected].
>> >> >> To unsubscribe from this group, send email to
>> >> >> [email protected].
>> >> >> For more options, visit this group at
>> >> >>http://groups.google.com/group/google-appengine?hl=en.
>>
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "Google App Engine" group.
>> >> > To post to this group, send email to [email protected].
>> >> > To unsubscribe from this group, send email to
>> >> > [email protected].
>> >> > For more options, visit this group at
>> >> >http://groups.google.com/group/google-appengine?hl=en.
>>
>> >> --
>> >> You received this message because you are subscribed to the Google Groups
>> >> "Google App Engine" group.
>> >> To post to this group, send email to [email protected].
>> >> To unsubscribe from this group, send email to
>> >> [email protected].
>> >> For more options, visit this group at
>> >>http://groups.google.com/group/google-appengine?hl=en.
>>
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "Google App Engine" group.
>> > To post to this group, send email to [email protected].
>> > To unsubscribe from this group, send email to
>> > [email protected].
>> > For more options, visit this group at
>> >http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
