On Wed, 27 Nov 2013 13:29:16 -0300 Antonio Terceiro <antonio.terce...@linaro.org> wrote:
> On Wed, Nov 27, 2013 at 04:13:14PM +0000, Neil Williams wrote: > > Data cannot go over the existing API connection, it has to be > > configured separately over something like TCP/IP and root on node01 > > does not necessarily have access to anything on node02 without > > node02 being explicitly configured in advance to either serve files > > anonymously or allow login. > > good point. my idea was that such helper would be using lava-network > under the hood, but I neglected the sending side. We probably also > need a matching call on the sending side (`lava-serve PATH`)? > > so the data analysis node does > > lava-sync processing-done > lava-collect RESULTS-PATH DEST > # calculate ... > > and the others > > # run ... > lava-sync processing-done > lava-serve RESULTS-PATH hmmm.... an extra sync would be required as the listener on each node must open the port before the transmission can start and it is the listener which is doing the initial tar operation, so that will take time. > (sure we could find better names than lava-collect and lava-serve) lava-nc-listen - offers files on a known port, opens the port lava-nc-connect - makes connections to each node using the port. > lava-sync processing-done lava-sync connection-ready lava-nc-connect RESULTS-PATH DEST DEST DEST > # calculate ... > # run ... > lava-sync processing-done lava-nc-listen RESULTS-PATH lava-sync connection-ready Each call to lava-nc-listen would have to put the call to nc into the background on each node and blindly trust nc to open the port on each node and close when it is done. The only indication that anything happened would then be when the connecting end completes the cycle of connecting to each node. If tests want error checking, checksums, progress indication, resume support, incremental downloads or any other "typical" support, netcat isn't going to be suitable. I'm not sure we get much from the helpers that wouldn't be a lot easier to debug by using calls to tar and nc directly. It seems very flaky to me. SSH or HTTPServer&wget are a lot more work to setup in a test but much more usable. Therefore, I'd be much happier with test writers doing their own connection setup using more reliable tools, rather than possibly giving a false sense of security just because there is a helper available. After all, HTTPServer&wget is exactly how LAVA sends data between the dispatcher and the board during deployment. It means installing python (or some other http daemon) on each node and wget on the receiver. > > I'm not sure this is appropriate as a helper across all LAVA > > deployments and if restricted to the same deployments as > > lava-network, it would require particular services to be installed > > and running on all nodes which lava-network does not enforce. > > with the matching call on the sending side, `tar | nc -` ought to be > good enough on the sending side and `nc | tar` should do it for the > receiving side. It's still worth investigating whether the whole analysis can be done without needing to transmit data between nodes during the test as this data will be lost at the end of the test and any chance of further analysis of the data is thrown away. I'm not sure it's helpful to provide tools which encourage test writers to throw data away at the end of a test. -- Neil Williams ============= http://www.linux.codehelp.co.uk/
signature.asc
Description: PGP signature
_______________________________________________ linaro-validation mailing list linaro-validation@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-validation
