Get ManifoldCF to run on top of a key/value store like Voldemort, for potential
massive scalability improvements and speed gains
--------------------------------------------------------------------------------------------------------------------------------
Key: CONNECTORS-286
URL: https://issues.apache.org/jira/browse/CONNECTORS-286
Project: ManifoldCF
Issue Type: New Feature
Components: Framework core
Reporter: Karl Wright
Assignee: Karl Wright
Fix For: ManifoldCF next
ManifoldCF's reliance on a relational database limits its throughput and
scalability. I am now convinced it is possible to build all the structures we
need within a distributed key-value store like Voldemort, which has the nice
side effect of permitting massive scaling. I envision there will be several
layers to this project, some of which may have broader utility in the
open-source community at large:
(1) An atomic serialization layer, which adds serialization capabilities to an
non-transactional substrate;
(2) A transaction layer, which uses atomic serialization to build a notion of
light transactions;
(3) A table and index layer, which defines SQL-like concepts of tables and
btree indexes on top of the transaction layer, via a Java API;
(4) A generic "database abstraction" layer, which is capable of representing
both standard SQL databases as well as this NoSQL variant, so that ManifoldCF
can support both models.
This is obviously a major development task, and as such is not envisioned to be
completed by the next standard release. Work will indeed need to be done in a
branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
