Get ManifoldCF to run on top of a key/value store like Voldemort, for potential massive scalability improvements and speed gains --------------------------------------------------------------------------------------------------------------------------------
Key: CONNECTORS-286 URL: https://issues.apache.org/jira/browse/CONNECTORS-286 Project: ManifoldCF Issue Type: New Feature Components: Framework core Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF next ManifoldCF's reliance on a relational database limits its throughput and scalability. I am now convinced it is possible to build all the structures we need within a distributed key-value store like Voldemort, which has the nice side effect of permitting massive scaling. I envision there will be several layers to this project, some of which may have broader utility in the open-source community at large: (1) An atomic serialization layer, which adds serialization capabilities to an non-transactional substrate; (2) A transaction layer, which uses atomic serialization to build a notion of light transactions; (3) A table and index layer, which defines SQL-like concepts of tables and btree indexes on top of the transaction layer, via a Java API; (4) A generic "database abstraction" layer, which is capable of representing both standard SQL databases as well as this NoSQL variant, so that ManifoldCF can support both models. This is obviously a major development task, and as such is not envisioned to be completed by the next standard release. Work will indeed need to be done in a branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira