Bill: Thanks for your reply. Bill Barry wrote: > On Wed, Jun 1, 2011 at 9:13 PM, Wayne E. Van Loon Sr. <[email protected]> > wrote: > >> I have a question about user applications using broadcast on a LAN. >> >> I usually have several processes that connect via TCP/IP to a central >> server that solves some optimization problems. In more demanding project >> that I am working on now, I hope to distribute the optimization problem >> between 24 to 36 helper processes. >> >> At the start of each optimization job, I will have a few data files that >> need to be distributed to each of the helper optimization processes. >> Rather than have the central optimizer / server send these files 24 >> times, some way to broadcast these files to all machines / processes at >> the same time might be nice. >> >> Is that the kind of thing that broadcasting could accomplish? >> >> And if so, and in a big picture way, how would one do it? >> >> An example of my network topology would be: >> TCP/IP server @ 192.168.1.40 >> 4 helper processes @ 192.168.1.40 >> 4 helper processes @ 192.168.1.41 >> 4 helper processes @ 192.168.1.42 >> 4 helper processes @ 192.168.1.43 >> 4 helper processes @ 192.168.1.44 >> 4 helper processes @ 192.168.1.45 >> >> Thanks for looking. >> Wayne >> > > > This project looks like it might help http://en.wikipedia.org/wiki/GridFTP > I have never used it, but apt-cache search shows there are Debian > packages for it. > I see that GridFTP is on top of FTP which is a client / server paradigm that it uses TCP/IP. Is GridFTP intended for many sources of data to one receiver? Or many receivers from one source of data?
> The real question to ask I think is where is your bottleneck in the > computation. Unquestionably, the optimization. > What should you optimize first. If you are going to > distribute the computing, are you transferring large amounts of data > to the nodes or just small datasets. Do the nodes take a long time to > do the computation compared to the data transfer? Yes. > If so, maybe you can > make more progress optimizing the computation than the data transfer. > For sure. That is how I came to distributing the optimization. I have been exploring some sort of broadcast or multicast. If it wasn't difficult to implement and keep synchronized, I would consider it. Your question is insightful, the optimization is where the time is. Again, thanks very much for your reply. Wayne _______________________________________________ PLUG mailing list [email protected] http://lists.pdxlinux.org/mailman/listinfo/plug
