Parallelizing is not very difficult. Galaxy gives you two ways to do it.
The first one is to call galaxy.spin. This gives you another generator
function on which you can yield later to get the result. This is very much
like a future.
var future = galaxy.spin(generatorFunction(args));
// do some awesome stuff while future can run every time you yield
// you can also pass future around, etc.
// somewhere down the line
var result = yield future(); // will throw here if generatorFunction failed.
The second one is to pass an extra argument to forEachStar:
// run 8 loop iterations in parallel (pass -1 if you don't want to limit
parallelism)
yield contents.split('\n').forEachStar(8, function* (id) {
var info = yield
request.get('http://www.example.com?id='+id<http://www.example.com/?id=%27+id>,
resume);
});
Bruno
On Monday, July 15, 2013 8:30:10 PM UTC+2, Raynos wrote:
>
> The fact that you can't trivially do deep yields in callbacks is a good
> thing.
>
> For example
>
> ```
> contents.split('\n').forEach(function(id) {
> var info = yield
> request.get('http://www.example.com?id='+id<http://www.example.com/?id='+id>,
> resume);
> });
> ```
>
> Would have done N get request in series when you probably really want to
> do something like
>
> ```
> var infos = yield parallel(contents.split('\n').map(function (id) {
> return request.get.bind(null, 'http://www.examples/com?id='+id)
> }))
> ```
>
> Which is yielding a single thing, where that single thing is do N get
> requests in parallel.
>
> If we make yielding too easy you will write sequential programs when you
> really want to write parallel programs.
>
>
> On Mon, Jul 15, 2013 at 12:11 AM, cpprototypes
> <[email protected]<javascript:>
> > wrote:
>
>> Thanks for the replies, I have a better understanding now of how these
>> libraries are using generators. I hope that one of these types of
>> libraries becomes as popular as the async library and "standard" within the
>> node community. This way of writing async code is much more elegant than
>> the current way of using only callbacks and can help expand node.js usage
>> into other areas. For instance, the script example I wrote is part of a
>> larger command line script I was writing to test some services. Such
>> scripting tends to follow a synchronous flow. I got really frustrated
>> while writing it in an async way (since the natural flow is synchronous, it
>> maps very poorly to the current callback-only style, even libraries like
>> async don't help much). I eventually gave up and quickly wrote the script
>> in python because I had to get the task done for the day. But I want
>> node.js to eventually take python's place in my tool set. So I went back
>> and tried it again with this new generators feature and it was going well
>> until I ran into this issue. Some may say that such use is beyond the
>> scope of node.js, but I think JS and node.js have potential to be more than
>> just network async programming.
>>
>>
>>
>> On Friday, July 12, 2013 2:47:28 PM UTC-7, cpprototypes wrote:
>>>
>>> I'm using node 0.11.3 with --harmony-generators to try the new feature.
>>> I found two libraries that should help use existing node callback-based
>>> code with generators (suspend and galaxy). The code I'm trying to run was
>>> similar to the following:
>>>
>>> (using suspend)
>>>
>>> var fs = require('fs');
>>> var request = require('request');
>>>
>>> suspend(function* (resume) {
>>> var contents = yield fs.readFile('idList.json', 'utf8', resume);
>>> contents.split('\n').forEach(**function(id) {
>>> var info = yield
>>> request.get('http://www.**example.com?id='+id<http://www.example.com?id='+id>,
>>>
>>> resume);
>>> });
>>> })();
>>>
>>> (using galaxy)
>>>
>>> var galaxy = require('galaxy');
>>> var fs = galaxy.star(require('fs'));
>>> var request = galaxy.star(require('request')**);
>>>
>>> function* main() {
>>> var contents = yield fs.readFile('idList.json', 'utf8');
>>> contents.split('\n').forEach(**function(id) {
>>> var info = yield
>>> request.get('http://www.**example.com?id='+id<http://www.example.com?id='+id>
>>> );
>>> });
>>> };
>>>
>>> galaxy.unstar(main)(function(**err, result) {
>>> console.log('done');
>>> });
>>>
>>> Using either library, when node tries to execute the get request ("var
>>> info = yield request.get...") it exits with the following error:
>>>
>>> SyntaxError: Unexpected identifier
>>>
>>> And the error highlights the "request" part in "var info = yield
>>> request.get..." I'm guessing that the creation of the new function scope
>>> in the forEach is somehow causing an issue. But I'm not sure why it's not
>>> working.
>>>
>>>
>>> --
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to [email protected]<javascript:>
>> To unsubscribe from this group, send email to
>> [email protected] <javascript:>
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
