weimingdiit commented on code in PR #8245:
URL: https://github.com/apache/hudi/pull/8245#discussion_r1145638109
##########
pom.xml:
##########
@@ -117,7 +117,7 @@
<joda.version>2.9.9</joda.version>
<hadoop.version>2.10.1</hadoop.version>
<hive.groupid>org.apache.hive</hive.groupid>
- <hive.version>2.3.1</hive.version>
+ <hive.version>2.3.4</hive.version>
Review Comment:
The hive version is updated from 2.3.1 to 2.3.4.
The bug of MetaStoreUtils, 【Long newStat = long.parselong
(newPart.getParameters().get(stat));】
Perhaps newPart does not contain 'stat', in which case parseLong will report
an error.
`
error log:2023-03-20T16:40:43.1619654Z Caused by:
java.lang.NumberFormatException: null
2023-03-20T16:40:43.1620230Z at
java.lang.Long.parseLong(Long.java:552)
2023-03-20T16:40:43.1620799Z at
java.lang.Long.parseLong(Long.java:631)
2023-03-20T16:40:43.1621465Z at
org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
2023-03-20T16:40:43.1622273Z at
org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
2023-03-20T16:40:43.1623187Z at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3833)
`
`
in 2.3.4
static boolean isFastStatsSame(Partition oldPart, Partition newPart) {
// requires to calculate stats if new and old have different fast stats
if ((oldPart != null) && (oldPart.getParameters() != null)) {
for (String stat : StatsSetupConst.fastStats) {
if (oldPart.getParameters().containsKey(stat)) {
Long oldStat = Long.parseLong(oldPart.getParameters().get(stat));
String newStat = newPart.getParameters().get(stat);
if (newStat == null || !oldStat.equals(Long.parseLong(newStat))) {
return false;
}
} else {
return false;
}
}
return true;
}
return false;
}
in 2.3.1
static boolean isFastStatsSame(Partition oldPart, Partition newPart) {
// requires to calculate stats if new and old have different fast stats
if ((oldPart != null) && (oldPart.getParameters() != null)) {
for (String stat : StatsSetupConst.fastStats) {
if (oldPart.getParameters().containsKey(stat)) {
Long oldStat = Long.parseLong(oldPart.getParameters().get(stat));
Long newStat = Long.parseLong(newPart.getParameters().get(stat));
if (!oldStat.equals(newStat)) {
return false;
}
} else {
return false;
}
}
return true;
}
return false;
}
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
