sankarh commented on a change in pull request #1050:
URL: https://github.com/apache/hive/pull/1050#discussion_r465626819
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
##########
@@ -1493,6 +1497,39 @@ public static String getPartitionName(Path tablePath,
Path partitionPath, Set<St
return result;
}
+ public static String getNormalisedPartitionValue(String partitionValue,
String type) {
+
+ LOG.debug("Converting '" + partitionValue + "' to type: '" + type + "'.");
+
+ if (type.equalsIgnoreCase("tinyint")
+ || type.equalsIgnoreCase("smallint")
+ || type.equalsIgnoreCase("int")){
+ return Integer.toString(Integer.parseInt(partitionValue));
+ } else if (type.equalsIgnoreCase("bigint")){
+ return Long.toString(Long.parseLong(partitionValue));
+ } else if (type.equalsIgnoreCase("float")){
+ return Float.toString(Float.parseFloat(partitionValue));
+ } else if (type.equalsIgnoreCase("double")){
+ return Double.toString(Double.parseDouble(partitionValue));
+ } else if (type.startsWith("decimal")){
+ // Decimal datatypes are stored like decimal(10,10)
+ return new
BigDecimal(partitionValue).stripTrailingZeros().toPlainString();
+ }
+ return partitionValue;
Review comment:
As a follow-up, do we need to validate date, timestamp values before add
partitions? Are we checking it during inserts or add partition command?
##########
File path: ql/src/test/queries/clientpositive/msck_repair_7.q
##########
@@ -0,0 +1,63 @@
+DROP TABLE IF EXISTS repairtable_n7_1;
+DROP TABLE IF EXISTS repairtable_n7_2;
+DROP TABLE IF EXISTS repairtable_n7_3;
+DROP TABLE IF EXISTS repairtable_n7_4;
+DROP TABLE IF EXISTS repairtable_n7_5;
+DROP TABLE IF EXISTS repairtable_n7_6;
+DROP TABLE IF EXISTS repairtable_n7_7;
+
+CREATE EXTERNAL TABLE repairtable_n7_1(key INT) PARTITIONED BY (p1 TINYINT)
stored as ORC location
'${system:test.tmp.dir}/apps/hive/warehouse/test.db/repairtable_n7_1/';
+CREATE EXTERNAL TABLE repairtable_n7_2(key INT) PARTITIONED BY (p1 SMALLINT)
stored as ORC location
'${system:test.tmp.dir}/apps/hive/warehouse/test.db/repairtable_n7_2/';
+CREATE EXTERNAL TABLE repairtable_n7_3(key INT) PARTITIONED BY (p1 INT) stored
as ORC location
'${system:test.tmp.dir}/apps/hive/warehouse/test.db/repairtable_n7_3/';
+CREATE EXTERNAL TABLE repairtable_n7_4(key INT) PARTITIONED BY (p1 BIGINT)
stored as ORC location
'${system:test.tmp.dir}/apps/hive/warehouse/test.db/repairtable_n7_4/';
+CREATE EXTERNAL TABLE repairtable_n7_5(key INT) PARTITIONED BY (p1 FLOAT)
stored as ORC location
'${system:test.tmp.dir}/apps/hive/warehouse/test.db/repairtable_n7_5/';
+CREATE EXTERNAL TABLE repairtable_n7_6(key INT) PARTITIONED BY (p1 DOUBLE)
stored as ORC location
'${system:test.tmp.dir}/apps/hive/warehouse/test.db/repairtable_n7_6/';
+CREATE EXTERNAL TABLE repairtable_n7_7(key INT) PARTITIONED BY (p1
DECIMAL(10,10)) stored as ORC location
'${system:test.tmp.dir}/apps/hive/warehouse/test.db/repairtable_n7_7/';
Review comment:
Have another table with STRING type column where the 0 shouldn't be
truncated by MSCK repair.
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
##########
@@ -1493,6 +1497,39 @@ public static String getPartitionName(Path tablePath,
Path partitionPath, Set<St
return result;
}
+ public static String getNormalisedPartitionValue(String partitionValue,
String type) {
+
+ LOG.debug("Converting '" + partitionValue + "' to type: '" + type + "'.");
+
+ if (type.equalsIgnoreCase("tinyint")
+ || type.equalsIgnoreCase("smallint")
+ || type.equalsIgnoreCase("int")){
+ return Integer.toString(Integer.parseInt(partitionValue));
+ } else if (type.equalsIgnoreCase("bigint")){
+ return Long.toString(Long.parseLong(partitionValue));
+ } else if (type.equalsIgnoreCase("float")){
+ return Float.toString(Float.parseFloat(partitionValue));
+ } else if (type.equalsIgnoreCase("double")){
+ return Double.toString(Double.parseDouble(partitionValue));
+ } else if (type.startsWith("decimal")){
+ // Decimal datatypes are stored like decimal(10,10)
+ return new
BigDecimal(partitionValue).stripTrailingZeros().toPlainString();
+ }
+ return partitionValue;
+ }
+
+ public static Map<String, String> getPartitionColtoTypeMap(List<FieldSchema>
partitionCols) {
+ Map<String, String> typeMap = new HashMap<>();
+
+ if (!(partitionCols == null || partitionCols.isEmpty())) {
Review comment:
Avoid outer negation and use if ((partitionCols != null) &&
!partitionCols.isEmpty()). Even isEmpty need not be checked here as below loop
won't execute if it is empty.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
