Here is how you can fetch simultaneously many urls, check if every
response is 200 OK and print title tags (or catch an exception if
something goes wrong) .
If you're not familiar with promises, here is a perfect documentation
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Promise
# cpanm Mojo::Pua
use Evo 'Mojo::Pua want_code; Mojo::Promise all';
my $ua = Mojo::Pua->new;
my @urls = qw(http://alexbyk.com https://metacpan.org https://www.perl.org);
all(map { $ua->get($_)->then(want_code 200)->then(sub {
shift->dom->at('title') }) } @urls)
->spread(sub(@titles) { say $_ for @titles })
->catch(sub($e) { warn $e })->finally(sub { Mojo::IOLoop->stop });
Mojo::IOLoop->start;
Throttling requests is a little bit more complex
On 11/12/2016 11:47 AM, Alex Povolotsky wrote:
Hello
Examples of Mojo::UserAgent are limited to fetching a set of files.
But I need a bit more complex thing: I need to parse a site, reading
pages, parsing it and reading links, using non-blocking UA with, say,
4 downloads at a time, no more and if possible no less.
Can someone give me a good example?
Alex
--
You received this message because you are subscribed to the Google
Groups "Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
