Drew Van Zandt wrote: > Have you considered using a fast compression/decompression algorithm > before you transmit, one that isn't too computationally intensive for > either compression or decompression? You won't get high compression > (factor of 10) as you might with slower ones, but if you get even a > factor of 2, you've just doubled the effective network speed. > I hadn't thought of this. Typically a lot of the data is random, so compression doesn't help too much. Or at least, that is what I thought. If I'm wrong, I hope someone will cheerfully correct me :)
I did do an experiment that had curious results. Instead of sending double precision binary data, I sent single precision or 'float'. I was expecting to halve my transmission time, since it is half the size. Instead, there was only a 10-15% speed increase, not a 100%. This result is telling me something, although at this time, I'm too brain dead to really ascertain what it really means. > It appears to me that this would only gain you 50% or so, as the > obvious fast compression algorithms are only about twice as fast > (http://www.quicklz.com/bench.html) as Gigabit Ethernet's theoretical > speed, but it's worth consideration. The decompression is faster than > the compression, which would hopefully result in a net win on > processing time on remote server. > > --DTVZ > Thanks for your suggestions. Compression would certainly help out. Bruce > On Sun, Oct 11, 2009 at 10:29 AM, Joshua Judson Rosen > <roz...@geekspace.com <mailto:roz...@geekspace.com>> wrote: > > Bruce Labitt <bruce.lab...@myfairpoint.net > <mailto:bruce.lab...@myfairpoint.net>> writes: > > > > What I'm trying to do: Optimizer for a radar power spectral > density problem > > > > Problem: FFTs required in optimization loop take too long on > current > > workstation for the optimizer to even be viable. > > > > Attempted solution: FFT engine on remote server to reduce overall > > execution time > > > > Builds client - server app implementing above solution. Server uses > > OpenMP and FFTW to exploit all cores. > [...] > > Implements better binary packing unpacking in code. Stuff works > > > > Nit in solution: TCP transport time >> FFT execution time, > rendering > > attempted solution non-viable > > > > Researches TCP optimization: Reads countless papers on tcp > optimization > > techniques... Fails to find a robust solutions or methodology for > > problem. Tries most techniques written in papers, only > realizing a 10% > > gain. Not good enough. Still needs to be faster > > > > Driven to more exotic techniques to reduce transport time. Explores > > parallel sockets, other techniques > [...] > > Hey, that is my bigger picture... Any and all suggestions are > > appreciated. Undoubtedly, a few dumb questions will follow. I > appear > > to be good at it. :P Maybe this context will help list subscribers > > frame their answers if they have any, or ask insightful questions. > > Where exactly does NFS fit into this? > > -- > Don't be afraid to ask (Lf.((Lx.xx) (Lr.f(rr)))). > _______________________________________________ > gnhlug-discuss mailing list > gnhlug-discuss@mail.gnhlug.org <mailto:gnhlug-discuss@mail.gnhlug.org> > http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > gnhlug-discuss mailing list > gnhlug-discuss@mail.gnhlug.org > http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ > _______________________________________________ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
