Hi Aditya/James,
As per my understanding Phoenix (having great features while
storing[e.g. secondary index] + Schema) can be directly used for data
insertion (Drill does not come in to picture in this flow).
And for query retrieval; first level we can use Phoenix end user
API in local Drillbit to query data.
DrillUsers -> DrillBit+EmbaddedPhoenixJDBCClient (local mode,
single instance) -> {HBase cluster..n}
In the mean time James can draft a design where DrillBit can use
more lower level API to benefit from Drill's Distributed operators,Joins
and ValueVector memory model, and using Phoenix's Hbase side
coprocessors and custom filters (as operator pushdown)
DrillUsers -> {DrillBit+EmbaddedPhoenixClient (Distributed mode)
..n} -> -> {HBase cluster..n}
Also we should be able to use HBase Table Schema generated by Phoenix.
Your thoughts?
-Dhruv
On 10/29/2013 03:52 AM, James Taylor wrote:
Sure, I'd be happy to.
Thanks,
James
On Mon, Oct 28, 2013 at 3:16 PM, Aditya <[email protected]> wrote:
James,
Jacques and I discussed this recently and we concur that it would be a
beneficial to build such an integration between the two projects.
Would you have some time to have a meeting/hangout with Drill team to talk
this further?
Regards,
Aditya...
Regards,
Aditya...
On Mon, Oct 28, 2013 at 1:31 PM, James Taylor <[email protected]
wrote:
Hi,
The idea for the Phoenix integration was to come up with the right API to
be able to plug Phoenix into Drill as the HBase query engine/store.
Jacque
and I got busy on our respective releases, but now that those are both
out,
I think we should get back to it. If we did it right, then both products
would be able to continue to evolve independently, but Drill would have
top
notch query performance for HBase data and Phoenix would have a new
customer.
Thoughts/ideas?
Thanks,
James
On Mon, Oct 28, 2013 at 5:34 AM, Dhruv <[email protected]>
wrote:
Hi,
I am interested in contributing towards providing HBase Query support
in
DRILL.
A) Can anyone summarize existing plan+Priority for it? (Or we can
discuss
in this thread and I will create WIKI page)
B) I see already following JIRA issues, Anybody working on this
locally?
B-1) https://issues.apache.org/**jira/browse/DRILL-15<
https://issues.apache.org/jira/browse/DRILL-15> last updated 13-Aug-13.
B-2) https://issues.apache.org/**jira/browse/DRILL-88<
https://issues.apache.org/jira/browse/DRILL-88>Unassigned
B-3) https://issues.apache.org/**jira/browse/DRILL-63<
https://issues.apache.org/jira/browse/DRILL-63>James Taylor seems
interested to work
C) As per my understanding implimenting
org.apache.drill.exec.store.**hbase
package with following for local mode drill bit should suffice for POC?
HBASERecordReader.java
HBASEGroupScan
HBASEScanBatchCreator
HbaseSchemaProvider
HBASEStorageEngine
HBASEStorageEngineConfig
HBASESubScan
D) What part of https://github.com/**forcedotcom/phoenix<
https://github.com/forcedotcom/phoenix>can we reuse? Any phonix
contributors around?
-Dhruv
