[
https://issues.apache.org/jira/browse/HBASE-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916599#action_12916599
]
Abhimanyu commented on HBASE-1015:
----------------------------------
I just started using Thrift and HBase. Now this seems like a typical question
but can't seem to find an answer. Our "number-crunching" code is all C++. I am
planning on using thrift to load data from HBase
into the client. The problem is that we're talking about a LOT of data. In a
typical scenario I spawn 100 processes and each loads up about 10GB of data ~
1TB. So I'm not sure if thrift will be fast enough. For management tasks we're
going to write everything in Java so that is not a problem. The question then
is is it better to write custom wrappers in JNI and bypass thrift completely?
Purely for performance considerations.
Also, what seems like the timeline for the C client since like libhdfs my guess
is HBase will provide c++ wrappers using JNI from what the discussion here
looks like.
If thrift is the way to go then we are looking at creating a tool that takes an
ODBC data source and loads all the data from one table to an HBase table. Again
this will be in C++. Only if we find that the overhead of thrift is too much
will we shift to java but that would mean double work writing clients for java
and c++. Anyway, we could provide this code for the community.
> pure C and C++ client libraries
> -------------------------------
>
> Key: HBASE-1015
> URL: https://issues.apache.org/jira/browse/HBASE-1015
> Project: HBase
> Issue Type: New Feature
> Components: client
> Affects Versions: 0.20.6
> Reporter: Andrew Purtell
> Priority: Minor
> Fix For: 0.92.0
>
>
> If via HBASE-794 first class support for talking via Thrift directly to
> HMaster and HRS is available, then pure C and C++ client libraries are
> possible.
> The C client library would wrap a Thrift core.
> The C++ client library can provide a class hierarchy quite close to
> o.a.h.h.client and, ideally, identical semantics. It should be just a
> wrapper around the C API, for economy.
> Internally to my employer there is a lot of resistance to HBase because many
> dev teams have a strong C/C++ bias. The real issue however is really client
> side integration, not a fundamental objection. (What runs server side and how
> it is managed is a secondary consideration.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
