Re: [racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Robby Findler
It may be the overhead of communicating the data is dominating the time spent working. Would it work to the main place open the file, count the number of lines, and then just tell the worker places which chunks of the file are theirs? Or maybe just do the counting at the byte level and then have

Re: [racket-users] Abridged summary of racket-users@googlegroups.com - 16 updates in 7 topics

2016-01-21 Thread Neil Van Dyke
sagar tripathy wrote on 01/21/2016 02:32 AM: is it possible to create a 'os' using 'racket' If you mean an operating system, it depends what kind. For example, this work was done using an early version of Racket (called MrEd in the paper): http://www.ccs.neu.edu/racket/pubs/icfp99-ffkf.pdf

Re: [racket-users] Places performance & channel capacity

2016-01-21 Thread Sam Tobin-Hochstadt
Hi Brian, A few suggestions: 1. You really want to use synchronization to determine when to end, not sleeping. Have each place write a message back to its parent when it is done, and have the parent wait for that message before finishing. 2. Async channels are implemented with threads, and

Re: [racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 7:48:44 AM UTC-5, Robby Findler wrote: > It may be the overhead of communicating the data is dominating the > time spent working. > > Would it work to the main place open the file, count the number of > lines, and then just tell the worker places which chunks of

Re: [racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 7:48:44 AM UTC-5, Robby Findler wrote: > It may be the overhead of communicating the data is dominating the > time spent working. > > Would it work to the main place open the file, count the number of > lines, and then just tell the worker places which chunks of

Re: [racket-users] Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 10:58:51 AM UTC-5, Sam Tobin-Hochstadt wrote: > Hi Brian, > > A few suggestions: > > 1. You really want to use synchronization to determine when to end, > not sleeping. Have each place write a message back to its parent when > it is done, and have the parent wait

Re: [racket-users] Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 12:50:26 PM UTC-5, antoine wrote: > Bonjour, > > Maybe for avoiding transforming mutable data to immutable data you could use > make-shared-bytes and transfer the data by this mean? Don't know the > underlying > implementation of make-shared-bytes it didn't seem

Re: [racket-users] Places performance & channel capacity

2016-01-21 Thread antoine
Bonjour, Maybe for avoiding transforming mutable data to immutable data you could use make-shared-bytes and transfer the data by this mean? Don't know the underlying implementation of make-shared-bytes it didn't seem to use POSIX shared memory objects. Use one make-shared-bytes per place and

Re: [racket-users] Places performance & channel capacity

2016-01-21 Thread antoine
To make it clear here is what i have in mind: #lang racket ;; dummy function, just remplace by A (define (do-stuff shared-data data-length) (values (make-bytes data-length 65) data-length)) (define p (place ch (define shared-data (place-channel-get ch)) (let loop ()

Re: [racket-users] Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 2:26:50 PM UTC-5, antoine wrote: > To make it clear here is what i have in mind: > > #lang racket > > ;; dummy function, just remplace by A > (define (do-stuff shared-data data-length) > (values >(make-bytes data-length 65) >data-length)) > > (define p

[racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 2:53:48 PM UTC-5, Brian Adkins wrote: > > Ok, so the huge place channel for each worker isn't the main issue. I changed > the code so the output process sends a message to the main process every N > messages, and the main process waits on a message from the

Re: [racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Neil Van Dyke
If I understand correctly, you're ultimately looking for a general way that you can write this kind of record processing code simply in the future. And that, right now, you're investing some one-time experimental effort, to assess feasibility and to find an approach/guidelines that you can

Re: [racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 6:54:54 PM UTC-5, Neil Van Dyke wrote: > If I understand correctly, you're ultimately looking for a general way > that you can write this kind of record processing code simply in the > future. And that, right now, you're investing some one-time > experimental

[racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Wednesday, January 20, 2016 at 11:47:57 PM UTC-5, Brian Adkins wrote: > On Wednesday, January 20, 2016 at 11:28:59 PM UTC-5, Brian Adkins wrote: > > My initial experiment with places is a bit disappointing: > > > > Sequential version: cpu time: 2084 real time: 2091 gc time: 91 > > > > Places

[racket-users] Re: Places performance & channel capacity

2016-01-21 Thread Brian Adkins
On Thursday, January 21, 2016 at 2:12:51 PM UTC-5, Brian Adkins wrote: > On Wednesday, January 20, 2016 at 11:47:57 PM UTC-5, Brian Adkins wrote: > > On Wednesday, January 20, 2016 at 11:28:59 PM UTC-5, Brian Adkins wrote: > > > My initial experiment with places is a bit disappointing: > > > > >