paulzanmei commented on issue #1684:
URL: https://github.com/apache/iceberg/issues/1684#issuecomment-719195944
@RussellSpitzer @openinx
My expectation is to use hive to create an iceberg table, use flink to write
data, and then use hive and Presto to query and analyze the data
Operation steps:
hive:
1. add jar /path/to/iceberg-hive-runtime.jar
2. CREATE EXTERNAL TABLE customers
STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
TBLPROPERTIES
('iceberg.mr.table.schema'='{"type":"struct","fields":[{"id":1,"name":"customer_id","required":true,"type":"long"},{"id":2,"name":"first_name","required":true,"type":"string"}]}');
flink:
3.
tableEnv.executeSql("CREATE CATALOG iceberg_catalog WITH (\n" +
" 'type'='iceberg',\n" +
" 'catalog-type'='hive',\n" +
" 'uri'='thrift://dcwork03:9083',\n" +
" 'default-database'='test',\n" +
" 'clients'='5',\n" +
//" 'warehouse'='/user/hive/warehouse',\n" +
"
'hive-conf-dir'='/Users/zhongbaoluo/Applications/app/apache-hive-3.1.2/conf/',\n"
+
" 'property-version'='1'\n" +
")");
4.
tableEnv.executeSql("INSERT INTO iceberg_catalog.test.customers SELECT
`customer_id`,first_name from datagen").print();
5.
tableEnv.executeSql("SELECT `customer_id`,first_name from
iceberg_catalog.test.customers").print();
hive:
6.
select * from customers;
problem:
1. The data actually inserted into the iceberg/data file by flink already
exists, and flink run cannot be stopped. The next select sql query also has no
data.
2. Hive query statement is also empty table
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
