It may be the overhead of communicating the data is dominating the
time spent working.
Would it work to the main place open the file, count the number of
lines, and then just tell the worker places which chunks of the file
are theirs? Or maybe just do the counting at the byte level and then
have
On Thursday, January 21, 2016 at 7:48:44 AM UTC-5, Robby Findler wrote:
> It may be the overhead of communicating the data is dominating the
> time spent working.
>
> Would it work to the main place open the file, count the number of
> lines, and then just tell the worker places which chunks of
On Thursday, January 21, 2016 at 7:48:44 AM UTC-5, Robby Findler wrote:
> It may be the overhead of communicating the data is dominating the
> time spent working.
>
> Would it work to the main place open the file, count the number of
> lines, and then just tell the worker places which chunks of
If I understand correctly, you're ultimately looking for a general way
that you can write this kind of record processing code simply in the
future. And that, right now, you're investing some one-time
experimental effort, to assess feasibility and to find an
approach/guidelines that you can
On Thursday, January 21, 2016 at 6:54:54 PM UTC-5, Neil Van Dyke wrote:
> If I understand correctly, you're ultimately looking for a general way
> that you can write this kind of record processing code simply in the
> future. And that, right now, you're investing some one-time
> experimental
5 matches
Mail list logo
