-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17626/
-----------------------------------------------------------
Review request for drill and Jacques Nadeau.
Repository: drill-git
Description
-------
1. Configuring Hive Metastore as storage engine in Drill
add the metastore config to conf/storage-engines.json.
Ex.
"hive-derby" :
{
"type":"hive",
"config" :
{
"hive.metastore.uris" : "localhost:9083",
"hive.metastore.sasl.enabled" : "false"
}
}
To embed the metastore rather than connecting to a remote server, set the
"hive.metastore.uris" to empty and give metastore backend related properties in
config section.
2. Implement schema interfaces to list tables in Hive metastore and covert hive
types into Drill/Sql types.
Currently hive tables are mapped in two level schema. First level schema is
used for the hive metastore (you can have more than one hive metastore
configured at the same time). Second level of schema is the database within the
metastore. See an example of the "show tables" below. In order to query a table
"kv_seq" in database "testdb1" in hive metastore configured as schema
"hive-derby", refer to the table as "dbname"."tablename". Ex. SELECT * FROM
"testdb1"."kv_seq";
$ ./bin/sqlline -u "jdbc:drill:schema=hive-derby" -n admin -p admin
scan complete in 4062ms
Connecting to jdbc:drill:schema=hive-derby
Connected to: Drill (version 1.0)
Driver: Apache Drill JDBC Driver (version 1.0)
Autocommit status: true
Transaction isolation: TRANSACTION_REPEATABLE_READ
sqlline version ??? by Marc Prud'hommeaux
0: jdbc:drill:schema=hive-derby> !tables
+------------+--------------+--------------------+---------------+----------+-----------+-------------+---------+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | TABLE_TYPE | REMARKS |
TYPE_CAT | TYPE_SCHEM | TYPE_NA |
+------------+--------------+--------------------+---------------+----------+-----------+-------------+---------+
| null | hive-derby | testdb1.kv_seq | TABLE | null |
null | null | null |
| null | hive-derby | default.all_types | TABLE | null |
null | null | null |
| null | hive-derby | default.kv_text | TABLE | null |
null | null | null |
| null | metadata | COLUMNS | SYSTEM_TABLE | null |
null | null | null |
| null | metadata | TABLES | SYSTEM_TABLE | null |
null | null | null |
+------------+--------------+--------------------+---------------+----------+-----------+-------------+---------+
0: jdbc:drill:schema=hive-derby>
Diffs
-----
contrib/pom.xml dc07979
contrib/storage-hive/pom.xml PRE-CREATION
distribution/src/assemble/bin.xml fdd6c70
exec/java-exec/pom.xml 2ffdae9
exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveReadEntry.java
PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveSchemaProvider.java
PRE-CREATION
exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveStorageEngineConfig.java
PRE-CREATION
sqlparser/pom.xml 34622f7
sqlparser/src/main/java/org/apache/drill/jdbc/DrillHandler.java 51600ec
sqlparser/src/main/java/org/apache/drill/sql/client/full/HiveDatabaseSchema.java
PRE-CREATION
sqlparser/src/main/java/org/apache/drill/sql/client/full/HiveSchema.java
PRE-CREATION
sqlparser/src/test/java/org/apache/drill/jdbc/test/FullEngineTest.java
f271bad
sqlparser/src/test/resources/storage-engines.json PRE-CREATION
Diff: https://reviews.apache.org/r/17626/diff/
Testing
-------
Included an e2e test which lists the tables in Sample hive metastore. Sample
derby metastore (sample-data/drill_hive_db) and warehouse
(sample-data/drill_hive_wh) are not included in this patch. They can be found
in this commit:
https://github.com/vkorukanti/incubator-drill/commit/55265368b642e18700ad067a69a3a64f07280067
Thanks,
Venki Korukanti
