I have a txt file with contains one url per line and many lines (200 000).
myTxtUrlfile
url1
url2
url3
url4
url5
...
I load my file with fs.readFileSync and stock a line var line
fs.readFileSync('./myUrlFile.txt').toString().split('\n').forEach(function
(line) {
});
So, i want to request each urkl, but not in the same time, because sending
200 000 http requests trigger error (web server : too many sockets
connections)
I use requestify.get(url).then(function(response), this object is asych,
and i would like to send just 50 http request in the same time :
1 : 1 block of 1 to 50 urls
2 : 1 block of 51 to 101 urls
3 : 1 block of 102 to 152 urls
....
End block : last line of my url file.
Any idea, to do this.
Thanks.
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
