I've found the exact example I needed. Unfortunately, it is commented and described in Russian. I did not need nothing like minion
On Mon, Nov 14, 2016 at 6:46 PM, Scott Wiersdorf <[email protected]> wrote: > I agree with Alexander Karelas: your question is a textbook job queue > application. > > It is possible to solve this using only non-blocking user agents, but in > either case you'll have to write some kind of controller that keeps track > of "in-flight" requests, a queue of pending requests, and something to > manage the queue. It's not a trivial application to do it well—you're > essentially implementing Minion. This is likely not the right forum to ask > for someone to write this for you. > > Scott > > On Monday, November 14, 2016 at 5:53:50 AM UTC-7, Alex Povolotsky wrote: >> >> I do not need an extra job manager with forks, I need to employ Mojo's >> event-based nonblocking I/O >> >> On Mon, Nov 14, 2016 at 3:46 PM, Alexander Karelas <[email protected]> >> wrote: >> >>> I'm not an expert, but what you're asking for looks like a job for >>> Minion: https://metacpan.org/pod/Minion >>> >>> On 14/11/16 14:41, Alex Povolotsky wrote: >>> >>> Looks like I was unclear. I do not need only extract all links, I must >>> continue fetching and parsing them, not more than N at a time and if >>> possible not less. I need some king of mirroring crawler. >>> >>> On Sat, Nov 12, 2016 at 6:25 PM, Scott Wiersdorf <[email protected]> >>> wrote: >>> >>>> Here is a complete working example you can run: >>>> >>>> #!/usr/bin/env perl >>>> use Mojolicious::Lite; >>>> >>>> get '/random-urls' => sub { >>>> my $c = shift; >>>> $c->render_later; >>>> >>>> $c->delay( >>>> sub { ## first step >>>> my $delay = shift; >>>> >>>> $c->ua->get('https://www.random.org/bytes/', >>>> $delay->begin); >>>> $c->ua->get('https://www.random.org/integer-sets/', >>>> $delay->begin); >>>> $c->ua->get('https://www.random.org/strings/', >>>> $delay->begin); >>>> $c->ua->get('https://www.random.org/audio-noise/', >>>> $delay->begin); >>>> }, >>>> >>>> sub { ## second step >>>> my $delay = shift; >>>> >>>> for my $dom (map { $_->res->dom } @_) { >>>> say STDERR $dom->at('title')->text; >>>> } >>>> >>>> $c->render(text => "Got all the links"); >>>> } >>>> ); >>>> }; >>>> >>>> app->start; >>>> >>>> >>>> When the results finally all come back from the first step, the second >>>> step will print out the page titles to STDERR, and return to the client >>>> "Got all the links". >>>> >>>> Scott >>>> >>>> On Saturday, November 12, 2016 at 2:47:57 AM UTC-7, Alex Povolotsky >>>> wrote: >>>>> >>>>> Hello >>>>> >>>>> Examples of Mojo::UserAgent are limited to fetching a set of files. >>>>> >>>>> But I need a bit more complex thing: I need to parse a site, reading >>>>> pages, parsing it and reading links, using non-blocking UA with, say, 4 >>>>> downloads at a time, no more and if possible no less. >>>>> >>>>> Can someone give me a good example? >>>>> >>>>> Alex >>>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Mojolicious" group. >>>> To unsubscribe from this topic, visit https://groups.google.com/d/to >>>> pic/mojolicious/wdp_pgd4e0k/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/mojolicious. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Mojolicious" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/mojolicious. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Mojolicious" group. >>> To unsubscribe from this topic, visit https://groups.google.com/d/to >>> pic/mojolicious/wdp_pgd4e0k/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/mojolicious. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "Mojolicious" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/mojolicious/wdp_pgd4e0k/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/mojolicious. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
