zabetak commented on code in PR #3418:
URL: https://github.com/apache/hive/pull/3418#discussion_r915124542
##########
hbase-handler/src/test/results/positive/hbase_avro_nested_timestamp.q.out:
##########
@@ -0,0 +1,45 @@
+PREHOOK: query: CREATE EXTERNAL TABLE tbl(
+`key` string COMMENT '',
+`data_frv4` struct<`id`:string, `dischargedate`:struct<`value`:timestamp>>)
+ROW FORMAT SERDE
+ 'org.apache.hadoop.hive.hbase.HBaseSerDe'
+STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
+WITH SERDEPROPERTIES (
+'serialization.format'='1',
+'hbase.columns.mapping' = ':key,data:frV4',
+'data.frV4.serialization.type'='avro',
+#### A masked pattern was here ####
+)
+TBLPROPERTIES (
+'hbase.table.name' = 'HiveAvroTable',
+'hbase.struct.autogenerate'='true')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@tbl
+POSTHOOK: query: CREATE EXTERNAL TABLE tbl(
+`key` string COMMENT '',
+`data_frv4` struct<`id`:string, `dischargedate`:struct<`value`:timestamp>>)
+ROW FORMAT SERDE
+ 'org.apache.hadoop.hive.hbase.HBaseSerDe'
+STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
+WITH SERDEPROPERTIES (
+'serialization.format'='1',
+'hbase.columns.mapping' = ':key,data:frV4',
+'data.frV4.serialization.type'='avro',
+#### A masked pattern was here ####
+)
+TBLPROPERTIES (
+'hbase.table.name' = 'HiveAvroTable',
+'hbase.struct.autogenerate'='true')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@tbl
+PREHOOK: query: select data_frV4.dischargedate.value from tbl
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl
+#### A masked pattern was here ####
+POSTHOOK: query: select data_frV4.dischargedate.value from tbl
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl
+#### A masked pattern was here ####
+1970-01-19 20:16:19.2
Review Comment:
The result does not appear to be correct. I was expecting to see something
like `2022-07-05 00:00:00`.
Are we doing something wrong when inserting the data? Are we doing something
wrong while fetching the data?
##########
itests/util/src/main/java/org/apache/hadoop/hive/hbase/HBaseTestSetup.java:
##########
@@ -158,6 +170,41 @@ private void createHBaseTable() throws IOException {
}
}
+ private byte[] createAvroRecordWithNestedTimestamp() throws IOException {
+ String dataDir = System.getProperty("test.data.dir");
+ Schema schema = new Schema.Parser().parse(new
File(dataDir+"/nested_ts.avsc"));
Review Comment:
This path concatenation may not be portable to different OS.
##########
itests/util/src/main/java/org/apache/hadoop/hive/hbase/HBaseTestSetup.java:
##########
@@ -158,6 +170,41 @@ private void createHBaseTable() throws IOException {
}
}
+ private byte[] createAvroRecordWithNestedTimestamp() throws IOException {
+ String dataDir = System.getProperty("test.data.dir");
+ Schema schema = new Schema.Parser().parse(new
File(dataDir+"/nested_ts.avsc"));
Review Comment:
Make method static.
##########
itests/util/src/main/java/org/apache/hadoop/hive/hbase/HBaseTestSetup.java:
##########
@@ -158,6 +170,41 @@ private void createHBaseTable() throws IOException {
}
}
+ private byte[] createAvroRecordWithNestedTimestamp() throws IOException {
+ String dataDir = System.getProperty("test.data.dir");
+ Schema schema = new Schema.Parser().parse(new
File(dataDir+"/nested_ts.avsc"));
+ GenericData.Record rootRecord = new GenericData.Record(schema);
+ rootRecord.put("id", "X338092");
+ GenericData.Record dateRecord = new
GenericData.Record(schema.getField("dischargedate").schema());
+ dateRecord.put("value",
LocalDate.of(2022,7,5).atStartOfDay().toEpochSecond(ZoneOffset.UTC));
+ rootRecord.put("dischargedate", dateRecord);
+
+ try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
+ try (DataFileWriter<GenericRecord> dataFileWriter
+ = new DataFileWriter<GenericRecord>(new
GenericDatumWriter<>(schema))) {
+ dataFileWriter.create(schema, out);
+ dataFileWriter.append(rootRecord);
+ }
+ return out.toByteArray();
+ }
+ }
+
+ private void createAvroTable() throws IOException {
+ final String HBASE_TABLE_NAME = "HiveAvroTable";
+ HTableDescriptor htableDesc = new
HTableDescriptor(TableName.valueOf(HBASE_TABLE_NAME));
+ htableDesc.addFamily(new HColumnDescriptor("data".getBytes()));
+
+ try (Admin hbaseAdmin = hbaseConn.getAdmin()) {
+ hbaseAdmin.createTable(htableDesc);
+ try (Table table =
hbaseConn.getTable(TableName.valueOf(HBASE_TABLE_NAME))) {
Review Comment:
Extract `TableName.valueOf("HiveAvroTable")` to local variable and replace
occurrences.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
