[
https://issues.apache.org/jira/browse/NUTCH-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12856360#action_12856360
]
Enis Soztutar commented on NUTCH-808:
-------------------------------------
bq. What do you mean by current implementation? NutchBase?
Indeed. In package o.a.n.storage deals with ORM (though not all classes)
bq. I know that Cascading have various Tape/Sink implementations including
JDBC, HBase but also SimpleDB. Maybe it would be worth having a look at how
they do it?
The way cascading does this is to convert Tuples (cascading data structure) to
HBase/JDBC records. The schema for HBase/JDBC is given as a metadata. Since
they deal with only tuple -> table row, it is not that difficult. But again,
cascading does not allow for mapping lists to columns, etc.
bq. My gut feeling would be to write a custom framework instead of relying on
DataNucleus and use AVRO if possible. I really think that HBase support is
urgently needed but am less convinced that we need MySQL in the very short
term.
Yeah, the more I think about it, the more I come to terms with custom
implementation. However, I think we might benefit a lot from the ideas from JDO
in the long term. Also, JDBC implementation may not be relevant for large scale
deployments, but it will be a very nice side effect of the ORM layer, which
will allow easy deployment, which in turn will hopefully bring more users.
> Evaluate ORM Frameworks which support non-relational column-oriented
> datastores and RDBMs
> ------------------------------------------------------------------------------------------
>
> Key: NUTCH-808
> URL: https://issues.apache.org/jira/browse/NUTCH-808
> Project: Nutch
> Issue Type: Task
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 2.0
>
>
> We have an ORM layer in the NutchBase branch, which uses Avro Specific
> Compiler to compile class definitions given in JSON. Before moving on with
> this, we might benefit from evaluating other frameworks, whether they suit
> our needs.
> We want at least the following capabilities:
> - Using POJOs
> - Able to persist objects to at least HBase, Cassandra, and RDBMs
> - Able to efficiently serialize objects as task outputs from Hadoop jobs
> - Allow native queries, along with standard queries
> Any comments, suggestions for other frameworks are welcome.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
