On 28 October 2013 01:26, sebb <seb...@gmail.com> wrote: > On 26 October 2013 20:23, Philippe Mouawad <philippe.moua...@gmail.com> wrote: >> Hello, >> These days Cloud based testing is becoming popular and having to distribute >> test data accross many servers through CSV can become painful if not >> impossible. >> >> Even without Cloud, when using distributed testing you always have to >> replicate your data on all servers, which is a painful manual step. >> >> Shouldn't we introduce a new DataSet more suitable for these use cases ? >> >> We could do it in many different ways: >> - Integrate an automatic CSV replicator, this would remain simple and would >> not introduce any tier, but with cloud based it would not horizontally >> scale easily > > Not sure I follow the scaling argument. The file would only have to be > copied once before the test proper starts. > From then on, the data is accessed locally. > > However, with a database, each node will need at least one connection > to the database, and every time more data is needed there will be > network traffic. > Or the database has to be running on the server node. > >> - Use a JDBC DataSet, but we would need to ensure it performs fine, and >> jdbc protocol is not well suited for cloud based deployment (But it could >> also be an interesting feature for Continuous Integration) >> - Use a NOSQL repository (Redis seems to me the best choice) , see this >> high level summary which I find interesting >> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml >> >> I have implemented a new Redis (based on Java library for Redis) DataSet >> which I plan to commit if no objection. >> >> It will introduce 2 new dependencies with Apache License: >> - Jedis (http://code.google.com/p/jedis/) >> - commons-pool > > There is also a dependency on Redis, but I guess that would not be bundled.
I've just noticed that Redis is provided as source which needs to be built before use. If it's difficult to copy CSV files to cloud nodes, it seems to me it's going to be much harder to install Redis. In which case additional network traffic will be needed to access the database. Also there is no official Windows release. >> Thoughts ? > > Is MongoDB not suitable? > We already include a jar for it. > >> -- >> Regards. >> Philippe M. >> @philmdot <https://twitter.com/philmdot>
