Hello, Can I proceed with commit these up coming days ? Thanks Regards Philippe
On Monday, October 28, 2013, Philippe Mouawad wrote: > > > On Monday, October 28, 2013, sebb wrote: > >> On 28 October 2013 01:26, sebb <[email protected]> wrote: >> > On 26 October 2013 20:23, Philippe Mouawad <[email protected]> >> wrote: >> >> Hello, >> >> These days Cloud based testing is becoming popular and having to >> distribute >> >> test data accross many servers through CSV can become painful if not >> >> impossible. >> >> >> >> Even without Cloud, when using distributed testing you always have to >> >> replicate your data on all servers, which is a painful manual step. >> >> >> >> Shouldn't we introduce a new DataSet more suitable for these use cases >> ? >> >> >> >> We could do it in many different ways: >> >> - Integrate an automatic CSV replicator, this would remain simple and >> would >> >> not introduce any tier, but with cloud based it would not horizontally >> >> scale easily >> > > > > > > >> > Not sure I follow the scaling argument. The file would only have to be >> > copied once before the test proper starts. >> > From then on, the data is accessed locally. >> > > > > The scale word was not good, In my thinking the issue is more about > deploying/copying/splitting the file among nodes or cloud members. A > centralized access makes it far easier. > > >> > However, with a database, each node will need at least one connection >> > to the database, and every time more data is needed there will be >> > network traffic. >> > Or the database has to be running on the server node. >> > > > > Agree, I was not saying anything different. But as I said this can be > useful for middle or low volume > > >> >> - Use a JDBC DataSet, but we would need to ensure it performs fine, and >> >> jdbc protocol is not well suited for cloud based deployment (But it >> could >> >> also be an interesting feature for Continuous Integration) >> >> - Use a NOSQL repository (Redis seems to me the best choice) , see >> this >> >> high level summary which I find interesting >> >> >> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml >> >> >> >> I have implemented a new Redis (based on Java library for Redis) >> DataSet >> >> which I plan to commit if no objection. >> >> >> >> It will introduce 2 new dependencies with Apache License: >> >> - Jedis (http://code.google.com/p/jedis/) >> >> - commons-pool >> > >> > There is also a dependency on Redis, but I guess that would not be >> bundled. >> >> > no need to anything else than jedis + commons-pool > > >> I've just noticed that Redis is provided as source which needs to be >> built before use. >> If it's difficult to copy CSV files to cloud nodes, it seems to me >> it's going to be much harder to install Redis. >> In which case additional network traffic will be needed to access the >> database. >> >> Also there is no official Windows release. >> >> >> Thoughts ? >> > >> > Is MongoDB not suitable? >> > We already include a jar for it. >> > > > > Mongodb is more document oriented and has another type of use cases. One > interesting feature of redis is lists where you can pop a line it will not > be available to next calls, it is suitable for tests that need uniqueness > accross all nodes. > >> >> -- >> >> Regards. >> >> Philippe M. >> >> @philmdot <https://twitter.com/philmdot> >> > > > -- > Cordialement. > Philippe Mouawad. > > > > -- Cordialement. Philippe Mouawad.
