haospotai commented on issue #828: Synchronizing to hive partition is incorrect URL: https://github.com/apache/incubator-hudi/issues/828#issuecomment-568149302 I am using hudi-0.5.0 # here the raw data ```json{"partitionpath": "2019/12/18","ts":"2019/12/19","name":"Michael","uuid":"d6e42d99-a447-43f4-94f4-ee61f1e5a6c3"} {"partitionpath": "2019/12/18","ts":"2019/12/18","name":"Andy", "age":30,"uuid":"d6e42d99-a447-43f4-94f4-ee61f1e5a6c4"} {"partitionpath": "2019/12/18","ts":"2019/12/18","name":"Justin", "age":19,"uuid":"d6e42d99-a447-43f4-94f4-ee61f1e5a6c8"} ``` ## but when use run.sync.sh tool - Adding partitions 1 to table synctable Exception in thread "main" org.apache.hudi.hive.HoodieHiveSyncException: Failed to sync partitions for table synctable at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:172) at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:107) at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:67) at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:192) Caused by: java.lang.IllegalArgumentException: Partition key parts [] does not match with partition values [2019-12-18]. Check partition strategy. ```java /** * HDFS Path contain hive partition values for the keys it is partitioned on. This mapping is not straight forward and * requires a pluggable implementation to extract the partition value from HDFS path. * <p> * This implementation extracts datestr=yyyy-mm-dd from path of type /yyyy/mm/dd */ public class SlashEncodedDayPartitionValueExtractor implements PartitionValueExtractor { private transient DateTimeFormatter dtfOut; public SlashEncodedDayPartitionValueExtractor() { this.dtfOut = DateTimeFormat.forPattern("yyyy-MM-dd"); } private DateTimeFormatter getDtfOut() { if (dtfOut == null) { dtfOut = DateTimeFormat.forPattern("yyyy-MM-dd"); } return dtfOut; } @Override public List<String> extractPartitionValuesInPath(String partitionPath) { // partition path is expected to be in this format yyyy/mm/dd String[] splits = partitionPath.split("/"); if (splits.length != 3) { throw new IllegalArgumentException("Partition path " + partitionPath + " is not in the form yyyy/mm/dd "); } // Get the partition part and remove the / as well at the end int year = Integer.parseInt(splits[0]); int mm = Integer.parseInt(splits[1]); int dd = Integer.parseInt(splits[2]); DateTime dateTime = new DateTime(year, mm, dd, 0, 0); return Lists.newArrayList(getDtfOut().print(dateTime)); } ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
