Hi Wishmitha, There are many java clients available for redis, there are several very good clients, see comparison [1]. Any reason using jedis over others?
[1] https://redis.io/clients#java Regards Kevin On Fri, Jul 14, 2017 at 11:02 PM, Wishmitha Mendis <[email protected]> wrote: > Hi Indika, > > I have started a basic implementation to test the concurrency handling, > message order and transaction support of Redis. Basic message producers and > consumers were implemented and messages are stored in a basic Redis queue. > GitHub : https://github.com/Wishmitha/RedisTesting > > Best Regards, > > On Thu, Jul 13, 2017 at 6:51 PM, Indika Sampath <[email protected]> wrote: > >> Hi Wishmitha, >> >> No SQL database is not an option as we have prior experience with >> Cassandra. Basically, the important part is to find out performance and >> fault tolerance capability in the database. Message broker >> read/write/delete records all the time and database should be able to >> handle high concurrency. Some metadata and message content write as byte >> data. Also, message order matters when delivery. Hence database should >> capable to sort records rather than bringing that logic into broker code. >> As you mentioned transaction is another factor we are looking in the >> database. Shall we start POC with Redis and see how it goes? >> >> Cheers! >> >> On Thu, Jul 13, 2017 at 6:07 PM, Wishmitha Mendis <[email protected]> >> wrote: >> >>> Hi all, >>> >>> I am working on a project to replace the current RDBMS based database of >>> the message broker store with a file based database system. I have been >>> researching on potential file based databases which can be used in the >>> project scenario. I have evaluated the pros and cons of the each of the >>> potential databases mainly based on read and write performances, >>> transaction support, data structure (queue) implementation support and >>> overhead of replacement. >>> >>> A summary of database evaluation is represented in the following table. >>> >>> >>> Database >>> >>> Pros >>> >>> Cons >>> >>> LevelDB >>> >>> - >>> >>> A simple key value storage >>> - >>> >>> Used in ActiveMQ (replaced with KahaDB) >>> >>> >>> 1. >>> >>> Fast due to sorted keys >>> 2. >>> >>> Simple transaction support with batch( ) operation >>> 3. >>> >>> Sublevel feature (provides more organization and transactions >>> facilitation) >>> 4. >>> >>> LiveStream feature (able to query things which are yet being >>> receiving) >>> 5. >>> >>> Support triggers, groupby (mapreduce) and join (fanout) etc. >>> >>> >>> 1. >>> >>> Performance decrease in concurrent / multi threaded scenarios. >>> (improved in RocksDB.) >>> 2. >>> >>> No object relational mapping. >>> 3. >>> >>> Poor performance when memory is not enough. >>> >>> MongoDB >>> >>> >>> - >>> >>> Document oriented >>> - >>> >>> Can define JSON schemas >>> >>> >>> >>> 1. >>> >>> Direct object mapping (good replacement for RDBMS) >>> 2. >>> >>> Can implement queue like data structures (store messages in separate >>> documents and delete them once delivered / already implemented data >>> structures : https://www.npmjs.com/package/mongodb-queue) >>> 3. >>> >>> Capped collections (automatically remove old docs to make space for >>> new docs.) >>> >>> >>> 1. >>> >>> No transaction support for multiple documents. >>> 2. >>> >>> Low performance compared to other NoSQL databases. >>> >>> Caassandra >>> >>> >>> - >>> >>> Column oriented (a key is denoted by row and an attribute is denoted >>> by a column which can dynamically change.) >>> >>> >>> - >>> >>> Node - Cluster configuration (all nodes are identical) >>> >>> >>> 1. >>> >>> High availability (No single point of failure. In case of node >>> failure other identical nodes are available.) >>> 2. >>> >>> No data loss due to replication (identical nodes) >>> 3. >>> >>> Easy to model (column store is very similar to RDBMS tables.) >>> 4. >>> >>> Similar query language to SQL (CQL) >>> 5. >>> >>> High performance (No master node concept. Any node can provide >>> requested query. No performance bottleneck.) >>> 6. >>> >>> Transaction implementation >>> >>> >>> 1. >>> >>> Does not support queue structure (message queue implementation is an >>> anti-pattern : http://www.datastax.com/dev/bl >>> og/cassandra-anti-patterns-queues-and-queue-like-datasets >>> >>> <http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets> >>> ) >>> >>> Redis >>> >>> >>> - >>> >>> A simple key value storage >>> - >>> >>> In memory database >>> - >>> >>> Support data structures like lists, caped lists, hashes, sets etc. ( >>> https://redis.io/topics/data-types-intro >>> <https://redis.io/topics/data-types-intro>) >>> >>> >>> 1. >>> >>> Fast read,write performances >>> 2. >>> >>> Provide persistence by writing to hard disk in configurable >>> intervals. (snapshots : https://redis.io/topics/persistence) >>> 3. >>> >>> Can implement message queue structures. ( >>> http://fiznool.com/blog/2016/02/24/building-a-simple-messag >>> e-queue-with-redis/ >>> >>> <http://fiznool.com/blog/2016/02/24/building-a-simple-message-queue-with-redis/> >>> ) >>> 4. >>> >>> Use to implement message stores. (https://redis.io/topics/pubsub) >>> 5. >>> >>> In built basic transaction support (MULTI/EXEC commands : >>> https://redis.io/topics/transactions >>> <https://redis.io/topics/transactions>) >>> >>> >>> 1. >>> >>> Loss of data (in case of power outage etc.) >>> 2. >>> >>> Depends on CPU performances when data amount is too high ( >>> https://redis.io/topics/persistence >>> <https://redis.io/topics/persistence>) >>> >>> RocksDB >>> >>> >>> - >>> >>> A simple key value storage (upgrade of LevelDB) >>> - >>> >>> In memory database >>> >>> >>> 1. >>> >>> Fast read,write performances (faster than LevelDB) >>> 2. >>> >>> Can implement queue structures (https://github.com/facebook/r >>> ocksdb/wiki/Implement-Queue-Service-Using-RocksDB >>> >>> <https://github.com/facebook/rocksdb/wiki/Implement-Queue-Service-Using-RocksDB> >>> ) >>> 3. >>> >>> Support concurrency >>> 4. >>> >>> Highly configurable (Pluggable Architecture) >>> 5. >>> >>> Support persistence (memtables and transactional logs manage in >>> memory. Memtable is flushed to a sstfile to provide persistence.) >>> >>> >>> 1. >>> >>> Loss of data (in case of power outage etc.) >>> 2. >>> >>> No in built transaction support (have to use TansactionDB : >>> https://github.com/facebook/rocksdb/wiki/Transactions >>> <https://github.com/facebook/rocksdb/wiki/Transactions> ) >>> >>> >>> According to this evaluation I suggest Redis as the most suitable >>> database for the message broker store. Even though it has an element of >>> risk in data loss, the persistence storing is configurable unlike in other >>> key-value stores. And it provides fast read and write performances compared >>> to other databases with basic transaction support. >>> >>> And this is basically my evaluation and the opinion. It would be great >>> to have a feedback on this, specially regarding the additional criteria to >>> consider and other suggested databases. >>> >>> Best Regards >>> >>> -- >>> >>> *Wishmitha Mendis* >>> >>> *Intern - Software Engineering* >>> *WSO2* >>> >>> *Mobile : +94 777577706 <+94%2077%20757%207706>* >>> >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Indika Sampath >> Associate Technical Lead >> WSO2 Inc. >> http://wso2.com >> >> Phone: +94 716 424 744 <+94%2071%20642%204744> >> Blog: http://indikasampath.blogspot.com/ >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > > *Wishmitha Mendis* > > *Intern - Software Engineering* > *WSO2* > > *Mobile : +94 777577706 <+94%2077%20757%207706>* > > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Kevin Ratnaskera* Software Engineer WSO2 Inc. - http://wso2.com lean . enterprise . middleware Mobile - +94774696950 Linkedin - https://www.linkedin.com/in/djkevincr
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
