Hi Kevin, I just tried jedis for the initial implementation, no actual reason reason to be honest. I will try other java-clients you mentioned, and figure out what will fit the best for the project scenario.
Best Regards, On Fri, Jul 14, 2017 at 11:17 PM, Kevin Ratnasekera <[email protected]> wrote: > Hi Wishmitha, > > There are many java clients available for redis, there are several very > good clients, see comparison [1]. Any reason using jedis over others? > > [1] https://redis.io/clients#java > > Regards > Kevin > > > > On Fri, Jul 14, 2017 at 11:02 PM, Wishmitha Mendis <[email protected]> > wrote: > >> Hi Indika, >> >> I have started a basic implementation to test the concurrency handling, >> message order and transaction support of Redis. Basic message producers and >> consumers were implemented and messages are stored in a basic Redis queue. >> GitHub : https://github.com/Wishmitha/RedisTesting >> >> Best Regards, >> >> On Thu, Jul 13, 2017 at 6:51 PM, Indika Sampath <[email protected]> wrote: >> >>> Hi Wishmitha, >>> >>> No SQL database is not an option as we have prior experience with >>> Cassandra. Basically, the important part is to find out performance and >>> fault tolerance capability in the database. Message broker >>> read/write/delete records all the time and database should be able to >>> handle high concurrency. Some metadata and message content write as byte >>> data. Also, message order matters when delivery. Hence database should >>> capable to sort records rather than bringing that logic into broker code. >>> As you mentioned transaction is another factor we are looking in the >>> database. Shall we start POC with Redis and see how it goes? >>> >>> Cheers! >>> >>> On Thu, Jul 13, 2017 at 6:07 PM, Wishmitha Mendis <[email protected]> >>> wrote: >>> >>>> Hi all, >>>> >>>> I am working on a project to replace the current RDBMS based database >>>> of the message broker store with a file based database system. I have been >>>> researching on potential file based databases which can be used in the >>>> project scenario. I have evaluated the pros and cons of the each of the >>>> potential databases mainly based on read and write performances, >>>> transaction support, data structure (queue) implementation support and >>>> overhead of replacement. >>>> >>>> A summary of database evaluation is represented in the following table. >>>> >>>> >>>> Database >>>> >>>> Pros >>>> >>>> Cons >>>> >>>> LevelDB >>>> >>>> - >>>> >>>> A simple key value storage >>>> - >>>> >>>> Used in ActiveMQ (replaced with KahaDB) >>>> >>>> >>>> 1. >>>> >>>> Fast due to sorted keys >>>> 2. >>>> >>>> Simple transaction support with batch( ) operation >>>> 3. >>>> >>>> Sublevel feature (provides more organization and transactions >>>> facilitation) >>>> 4. >>>> >>>> LiveStream feature (able to query things which are yet being >>>> receiving) >>>> 5. >>>> >>>> Support triggers, groupby (mapreduce) and join (fanout) etc. >>>> >>>> >>>> 1. >>>> >>>> Performance decrease in concurrent / multi threaded scenarios. >>>> (improved in RocksDB.) >>>> 2. >>>> >>>> No object relational mapping. >>>> 3. >>>> >>>> Poor performance when memory is not enough. >>>> >>>> MongoDB >>>> >>>> >>>> - >>>> >>>> Document oriented >>>> - >>>> >>>> Can define JSON schemas >>>> >>>> >>>> >>>> 1. >>>> >>>> Direct object mapping (good replacement for RDBMS) >>>> 2. >>>> >>>> Can implement queue like data structures (store messages in >>>> separate documents and delete them once delivered / already implemented >>>> data structures : https://www.npmjs.com/package/mongodb-queue) >>>> 3. >>>> >>>> Capped collections (automatically remove old docs to make space for >>>> new docs.) >>>> >>>> >>>> 1. >>>> >>>> No transaction support for multiple documents. >>>> 2. >>>> >>>> Low performance compared to other NoSQL databases. >>>> >>>> Caassandra >>>> >>>> >>>> - >>>> >>>> Column oriented (a key is denoted by row and an attribute is >>>> denoted by a column which can dynamically change.) >>>> >>>> >>>> - >>>> >>>> Node - Cluster configuration (all nodes are identical) >>>> >>>> >>>> 1. >>>> >>>> High availability (No single point of failure. In case of node >>>> failure other identical nodes are available.) >>>> 2. >>>> >>>> No data loss due to replication (identical nodes) >>>> 3. >>>> >>>> Easy to model (column store is very similar to RDBMS tables.) >>>> 4. >>>> >>>> Similar query language to SQL (CQL) >>>> 5. >>>> >>>> High performance (No master node concept. Any node can provide >>>> requested query. No performance bottleneck.) >>>> 6. >>>> >>>> Transaction implementation >>>> >>>> >>>> 1. >>>> >>>> Does not support queue structure (message queue implementation is >>>> an anti-pattern : http://www.datastax.com/dev/bl >>>> og/cassandra-anti-patterns-queues-and-queue-like-datasets >>>> >>>> <http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets> >>>> ) >>>> >>>> Redis >>>> >>>> >>>> - >>>> >>>> A simple key value storage >>>> - >>>> >>>> In memory database >>>> - >>>> >>>> Support data structures like lists, caped lists, hashes, sets etc. ( >>>> https://redis.io/topics/data-types-intro >>>> <https://redis.io/topics/data-types-intro>) >>>> >>>> >>>> 1. >>>> >>>> Fast read,write performances >>>> 2. >>>> >>>> Provide persistence by writing to hard disk in configurable >>>> intervals. (snapshots : https://redis.io/topics/persistence) >>>> 3. >>>> >>>> Can implement message queue structures. ( >>>> http://fiznool.com/blog/2016/02/24/building-a-simple-messag >>>> e-queue-with-redis/ >>>> >>>> <http://fiznool.com/blog/2016/02/24/building-a-simple-message-queue-with-redis/> >>>> ) >>>> 4. >>>> >>>> Use to implement message stores. (https://redis.io/topics/pubsub) >>>> 5. >>>> >>>> In built basic transaction support (MULTI/EXEC commands : >>>> https://redis.io/topics/transactions >>>> <https://redis.io/topics/transactions>) >>>> >>>> >>>> 1. >>>> >>>> Loss of data (in case of power outage etc.) >>>> 2. >>>> >>>> Depends on CPU performances when data amount is too high ( >>>> https://redis.io/topics/persistence >>>> <https://redis.io/topics/persistence>) >>>> >>>> RocksDB >>>> >>>> >>>> - >>>> >>>> A simple key value storage (upgrade of LevelDB) >>>> - >>>> >>>> In memory database >>>> >>>> >>>> 1. >>>> >>>> Fast read,write performances (faster than LevelDB) >>>> 2. >>>> >>>> Can implement queue structures (https://github.com/facebook/r >>>> ocksdb/wiki/Implement-Queue-Service-Using-RocksDB >>>> >>>> <https://github.com/facebook/rocksdb/wiki/Implement-Queue-Service-Using-RocksDB> >>>> ) >>>> 3. >>>> >>>> Support concurrency >>>> 4. >>>> >>>> Highly configurable (Pluggable Architecture) >>>> 5. >>>> >>>> Support persistence (memtables and transactional logs manage in >>>> memory. Memtable is flushed to a sstfile to provide persistence.) >>>> >>>> >>>> 1. >>>> >>>> Loss of data (in case of power outage etc.) >>>> 2. >>>> >>>> No in built transaction support (have to use TansactionDB : >>>> https://github.com/facebook/rocksdb/wiki/Transactions >>>> <https://github.com/facebook/rocksdb/wiki/Transactions> ) >>>> >>>> >>>> According to this evaluation I suggest Redis as the most suitable >>>> database for the message broker store. Even though it has an element of >>>> risk in data loss, the persistence storing is configurable unlike in other >>>> key-value stores. And it provides fast read and write performances compared >>>> to other databases with basic transaction support. >>>> >>>> And this is basically my evaluation and the opinion. It would be great >>>> to have a feedback on this, specially regarding the additional criteria to >>>> consider and other suggested databases. >>>> >>>> Best Regards >>>> >>>> -- >>>> >>>> *Wishmitha Mendis* >>>> >>>> *Intern - Software Engineering* >>>> *WSO2* >>>> >>>> *Mobile : +94 777577706 <+94%2077%20757%207706>* >>>> >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Indika Sampath >>> Associate Technical Lead >>> WSO2 Inc. >>> http://wso2.com >>> >>> Phone: +94 716 424 744 <+94%2071%20642%204744> >>> Blog: http://indikasampath.blogspot.com/ >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> >> *Wishmitha Mendis* >> >> *Intern - Software Engineering* >> *WSO2* >> >> *Mobile : +94 777577706 <+94%2077%20757%207706>* >> >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Kevin Ratnaskera* > Software Engineer > WSO2 Inc. - http://wso2.com > lean . enterprise . middleware > Mobile - +94774696950 <+94%2077%20469%206950> > Linkedin - https://www.linkedin.com/in/djkevincr > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Wishmitha Mendis* *Intern - Software Engineering* *WSO2* *Mobile : +94 777577706*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
