lamber-ken commented on a change in pull request #1105: [HUDI-405] Fix sync no
hive partition at first time
URL: https://github.com/apache/incubator-hudi/pull/1105#discussion_r360702449
##########
File path: hudi-hive/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java
##########
@@ -632,24 +633,33 @@ public void close() {
if (!lastCommitTimeSynced.isPresent()) {
LOG.info("Last commit time synced is not known, listing all partitions
in " + syncConfig.basePath + ",FS :" + fs);
try {
- return FSUtils.getAllPartitionPaths(fs, syncConfig.basePath,
syncConfig.assumeDatePartitioning);
+
+ List<String> fsPartitions = FSUtils.getAllPartitionPaths(fs,
syncConfig.basePath, syncConfig.assumeDatePartitioning);
+ List<String> tlPartitions =
findPartitionsAfter(HOODIE_FIRST_COMMIT_TIME);
Review comment:
Hi @vinothchandar, I know what you mean. The problem the user is
experiencing is can not get any data at the first time, this solution is union
results from both `FSUtils.getAllPartitionPaths` and the first commit. We don't
need to care the `assumeDatePartitioning `, because the first commit is what
the user actually wrote.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
