Opps, forgot to send this to mailing list as well... On Tue, Mar 30, 2010 at 8:32 PM, Jordan Neumeyer < [email protected]> wrote:
> Hi Glynn, > > On Tue, Mar 30, 2010 at 8:29 AM, Glynn Clements > <[email protected]>wrote: > >> >> Jordan Neumeyer wrote: >> >> > Just kind of my thought process about how I would try to go about >> > parallelizing a module. >> >> The main issue with parallelising raster input is that the library >> keeps a copy of the current row's data, so that consecutive reads of >> the same row (as happen when upsampling) don't re-read the data. >> >> For concurrent access to a single map, you would need to either keep >> one row per thread, or abandon caching. Also, you would need to use >> pread() rather than fseek()+read(). >> > > It sounds like you're talking about parallelism in I/O from a file or > database. Neither of which is my intent or goal for this project. I will > parallelize things after they have already been read into memory, and tasks > are processor intensive. I wouldn't want parallelize any I/O, but if I were > to optimize I/O. I would make all operations I/O asynchronous, which is can > mimic parallelism in a sense. Queuing up the chunks of data and then > processing them as resources become available. > > >> It's more straightfoward to read multiple maps concurrently. In 7.0, >> this case should be thread-safe. >> >> Alternatively, you could have one thread for reading, one for writing, >> and multiple worker threads for the actual processing. However, unless >> the processing is complex, I/O will be the bottleneck. >> > > I/O is generally a bottleneck anyway. Something always tends to be waiting > on another. > > >> -- >> >> Glynn Clements <[email protected]> >> >> > >
_______________________________________________ grass-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-dev
