[
https://issues.apache.org/jira/browse/TAJO-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587540#comment-14587540
]
ASF GitHub Bot commented on TAJO-1644:
--------------------------------------
Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/601#discussion_r32494015
--- Diff:
tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
---
@@ -548,54 +526,61 @@ public final void
testInsertIntoColumnPartitionedTableByThreeColumns() throws Ex
+ " where l_orderkey = 1 and l_partkey = 1 and l_linenumber = 1");
res.close();
- desc = catalog.getTableDesc(DEFAULT_DATABASE_NAME, tableName);
- assertTrue(fs.isDirectory(path));
- assertTrue(fs.isDirectory(new Path(path.toUri() + "/col1=1")));
- assertTrue(fs.isDirectory(new Path(path.toUri() + "/col1=1/col2=1")));
- assertTrue(fs.isDirectory(new Path(path.toUri() +
"/col1=1/col2=1/col3=17.0")));
- assertTrue(fs.isDirectory(new Path(path.toUri() +
"/col1=1/col2=1/col3=30.0")));
- assertTrue(fs.isDirectory(new Path(path.toUri() + "/col1=2")));
- assertTrue(fs.isDirectory(new Path(path.toUri() + "/col1=2/col2=2")));
- assertTrue(fs.isDirectory(new Path(path.toUri() +
"/col1=2/col2=2/col3=38.0")));
- assertTrue(fs.isDirectory(new Path(path.toUri() + "/col1=3")));
- assertTrue(fs.isDirectory(new Path(path.toUri() + "/col1=3/col2=2")));
- assertTrue(fs.isDirectory(new Path(path.toUri() + "/col1=3/col2=3")));
- assertTrue(fs.isDirectory(new Path(path.toUri() +
"/col1=3/col2=2/col3=45.0")));
- assertTrue(fs.isDirectory(new Path(path.toUri() +
"/col1=3/col2=3/col3=49.0")));
-
+ verifyDirectoriesForThreeColumns(fs, path, 3);
if (!testingCluster.isHiveCatalogStoreRunning()) {
// TODO: If there is existing another partition directory, we must
add its rows number to result row numbers.
+ // desc = catalog.getTableDesc(DEFAULT_DATABASE_NAME, tableName);
// assertEquals(6, desc.getStats().getNumRows().intValue());
}
- res = executeString("select * from " + tableName + " where col2 = 1");
- resultSetData = resultSetToString(res);
- res.close();
- expected = "col4,col1,col2,col3\n" +
- "-------------------------------\n" +
- "N,1,1,17.0\n" +
- "N,1,1,17.0\n" +
- "N,1,1,30.0\n" +
- "N,1,1,36.0\n" +
- "N,1,1,36.0\n";
-
- assertEquals(expected, resultSetData);
+ verifyMaintainExistingData(res, tableName);
// insert overwrite empty result to partitioned table
res = executeString("insert overwrite into " + tableName
- + " select l_returnflag, l_orderkey, l_partkey, l_quantity from
lineitem where l_orderkey" +
- " > 100");
+ + " select l_returnflag, l_orderkey, l_partkey, l_quantity from
lineitem where l_orderkey > 100");
res.close();
- desc = catalog.getTableDesc(DEFAULT_DATABASE_NAME, tableName);
+ verifyDirectoriesForThreeColumns(fs, path, 4);
+ verifyMaintainExistingData(res, tableName);
- ContentSummary summary = fs.getContentSummary(new
Path(desc.getPath()));
+ executeString("DROP TABLE " + tableName + " PURGE").close();
+ }
- assertEquals(summary.getDirectoryCount(), 1L);
- assertEquals(summary.getFileCount(), 0L);
- assertEquals(summary.getLength(), 0L);
+ private final void verifyMaintainExistingData(ResultSet res, String
tableName) throws Exception {
--- End diff --
It would be better if you rename verifyMaintainExistingData to
verifyKeptExistingData.
> When inserting empty data into a partitioned table, existing data would be
> removed.
> -----------------------------------------------------------------------------------
>
> Key: TAJO-1644
> URL: https://issues.apache.org/jira/browse/TAJO-1644
> Project: Tajo
> Issue Type: Bug
> Components: query master
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
> Fix For: 0.11.0, 0.10.2
>
> Attachments: TAJO-1644.patch
>
>
> When inserting empty data into a partitioned table, existing data would be
> removed. Tajo provides column value partition which is hive-style partition.
> In hive, when inserting empty data into a partition, there are two cases. If
> you use dynamic partitions, existing data never would be removed. But if you
> don't use dynamic partitions, existing data would be removed. When inserting
> a data to partition, tajo user don't specify each column and each column
> value. So, it is similar to dynamic partition of hive. It seems to update
> deletion logic of partitioned table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
