[
https://issues.apache.org/jira/browse/HIVE-23887?focusedWorklogId=468235&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-468235
]
ASF GitHub Bot logged work on HIVE-23887:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Aug/20 12:50
Start Date: 08/Aug/20 12:50
Worklog Time Spent: 10m
Work Description: sankarh commented on a change in pull request #1370:
URL: https://github.com/apache/hive/pull/1370#discussion_r467461932
##########
File path: ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java
##########
@@ -566,4 +567,28 @@ public void testMMExportAborted() throws Exception {
TestTxnCommands2.stringifyValues(data), rs);
}
-}
+
+ @Test public void testExportPartitionedOrcWithOutColumnStats() throws
Exception {
+
+ runStatementOnDriver("drop table if exists T");
+ runStatementOnDriver("drop table if exists Tstage");
+ runStatementOnDriver("create table T (a int, b int) partitioned by (p int)
stored"
+ + " as orc tblproperties('transactional'='true')");
+ //Tstage is the target table
+ runStatementOnDriver("create table Tstage (a int, b int) partitioned by (p
int) stored"
+ + " as orc tblproperties('transactional'='true')");
+ //this creates an ORC data file with correct schema under table root
+ runStatementOnDriver("insert into Tstage
values(1,2,10),(3,4,11),(5,6,12)");
+ final int[][] rows = { { 3 } };
+ //now we have an archive with 3 partitions
+ runStatementOnDriver("export table Tstage to '" + getWarehouseDir() +
"/1'");
Review comment:
Check the COLUMN_STATS_ACCURATE property is true on source table and
partition before and after export operation to confirm we don't overwrite
anything in their metadata.
##########
File path: ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java
##########
@@ -566,4 +567,28 @@ public void testMMExportAborted() throws Exception {
TestTxnCommands2.stringifyValues(data), rs);
}
-}
+
+ @Test public void testExportPartitionedOrcWithOutColumnStats() throws
Exception {
+
+ runStatementOnDriver("drop table if exists T");
+ runStatementOnDriver("drop table if exists Tstage");
+ runStatementOnDriver("create table T (a int, b int) partitioned by (p int)
stored"
+ + " as orc tblproperties('transactional'='true')");
+ //Tstage is the target table
+ runStatementOnDriver("create table Tstage (a int, b int) partitioned by (p
int) stored"
+ + " as orc tblproperties('transactional'='true')");
+ //this creates an ORC data file with correct schema under table root
+ runStatementOnDriver("insert into Tstage
values(1,2,10),(3,4,11),(5,6,12)");
+ final int[][] rows = { { 3 } };
+ //now we have an archive with 3 partitions
+ runStatementOnDriver("export table Tstage to '" + getWarehouseDir() +
"/1'");
+
+ //load T
+ runStatementOnDriver("import table T from '" + getWarehouseDir() + "/1'");
+ List<String> rsProperties = runStatementOnDriver("show tblproperties T");
Review comment:
Can we also check the partition properties?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 468235)
Time Spent: 20m (was: 10m)
> Reset Columns stats in Export Statement
> ---------------------------------------
>
> Key: HIVE-23887
> URL: https://issues.apache.org/jira/browse/HIVE-23887
> Project: Hive
> Issue Type: Bug
> Components: Import/Export, Statistics
> Reporter: Ashish Sharma
> Assignee: Ashish Sharma
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> While doing "export table db.table to '/import/table' " entire column stat is
> getting exported which lead to columns stats corruption in import table.
> Reset columns stats while export to force Imported to recalculate the Columns
> stats
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
