jonashartwig commented on issue #5211: URL: https://github.com/apache/hudi/issues/5211#issuecomment-1313577663
Hi, I have a similar issue. We also partition our data by year, month, day and hour. That becomes quite some structure for few years. So we implemented a parameter based partition filter that pre-selects the time based partitions off of some config. So our partition pattern we then use to load hudi is something like this: `/data/path/col1=*/year=2022/month=11/day=14/hour=13/col2=*/col3=*`. Hudi then `org.apache.hudi.util.PathUtils#globPath` function to parse it. That function gets the base path of the input above `/data/path/col1=*/year=2022/month=11/day=14/hour=13/col2=*/` and iterates the contents until it finds the base dir, but the base dir is never found as =* then never actually becomes a folder. We use Hudi 0.8 before and are just updating to 0.12. Any advice= -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
