[
https://issues.apache.org/jira/browse/CONNECTORS-286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171178#comment-13171178
]
Karl Wright commented on CONNECTORS-286:
----------------------------------------
Work on a database java API, with a key/value back-end implementation, has been
going on for some six weeks now and is nearly complete. It may be a valuable
first step to convert ManifoldCF to this layer, and create standard SQL
database implementations of it as a replacement for the current ManifoldCF
implementation layer, to see how that works. The functionality needed by the
database in order to participate is limited to the following:
- transaction support
- ability to read tables sequentially
- ability to read through a table index in order, with specific conditions on
that index read
Everything else is synthesized by the Warthog API layer. This should make the
job of porting to different databases much much easier - if it works.
Code as it currently stands is in branches/CONNECTORS-286/warthog.
> Get ManifoldCF to run on top of a key/value store like Voldemort, for
> potential massive scalability improvements and speed gains
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: CONNECTORS-286
> URL: https://issues.apache.org/jira/browse/CONNECTORS-286
> Project: ManifoldCF
> Issue Type: New Feature
> Components: Framework core
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
>
> ManifoldCF's reliance on a relational database limits its throughput and
> scalability. I am now convinced it is possible to build all the structures
> we need within a distributed key-value store like Voldemort, which has the
> nice side effect of permitting massive scaling. I envision there will be
> several layers to this project, some of which may have broader utility in the
> open-source community at large:
> (1) An atomic serialization layer, which adds serialization capabilities to
> an non-transactional substrate;
> (2) A transaction layer, which uses atomic serialization to build a notion of
> light transactions;
> (3) A table and index layer, which defines SQL-like concepts of tables and
> btree indexes on top of the transaction layer, via a Java API;
> (4) A generic "database abstraction" layer, which is capable of representing
> both standard SQL databases as well as this NoSQL variant, so that ManifoldCF
> can support both models.
> This is obviously a major development task, and as such is not envisioned to
> be completed by the next standard release. Work will indeed need to be done
> in a branch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
