bowenli86 commented on a change in pull request #9067: [FLINK-13069][hive]
HiveTableSink should implement OverwritableTableSink
URL: https://github.com/apache/flink/pull/9067#discussion_r302266277
##########
File path:
flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/batch/connectors/hive/HiveTableSinkTest.java
##########
@@ -235,10 +220,44 @@ public void testInsertIntoStaticPartition() throws
Exception {
// make sure new partition is created
assertEquals(toWrite.size(),
hiveCatalog.listPartitions(tablePath).size());
- CatalogPartition catalogPartition =
hiveCatalog.getPartition(tablePath, new CatalogPartitionSpec(partSpec));
- String partitionLocation =
catalogPartition.getProperties().get(HiveCatalogConfig.PARTITION_LOCATION);
- verifyWrittenData(new Path(partitionLocation, "0"), toWrite, 1);
+ verifyWrittenData(toWrite, hiveShell.executeQuery("select *
from " + tblName));
+
+ hiveCatalog.dropTable(tablePath, false);
+ }
+
+ @Test
+ public void testInsertOverwrite() throws Exception {
+ String dbName = "default";
+ String tblName = "dest";
+ RowTypeInfo rowTypeInfo = createDestTable(dbName, tblName, 0);
+ ObjectPath tablePath = new ObjectPath(dbName, tblName);
+
+ ExecutionEnvironment execEnv =
ExecutionEnvironment.createLocalEnvironment(1);
+ BatchTableEnvironment tableEnv =
BatchTableEnvironment.create(execEnv);
+
+ // write some data and verify
+ List<Row> toWrite = generateRecords(5);
+ tableEnv.registerDataSet("src", execEnv.fromCollection(toWrite,
rowTypeInfo));
+
+ CatalogTable table = (CatalogTable)
hiveCatalog.getTable(tablePath);
+ tableEnv.registerTableSink("destSink", new HiveTableSink(new
JobConf(hiveConf), tablePath, table));
+ tableEnv.sql("select * from src").insertInto("destSink");
+ execEnv.execute();
+
+ verifyWrittenData(toWrite, hiveShell.executeQuery("select *
from " + tblName));
+
+ // write some data to overwrite existing data and verify
+ toWrite = generateRecords(3);
Review comment:
this test doesn't seem to serve its purpose. IIUIC, `generateRecords(3)`
will generate the exact same three records as the first 3 rows from
`generateRecords(5)` at line 240, and thus even if the `setOverwrite()` doesn't
funciton, the table rows will remain the same, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
