vinothchandar commented on a change in pull request #1229: [HUDI-535] Ensure
Compaction Plan is always written in .aux folder to avoid 0.5.0/0.5.1
reader-writer compatibility issues
URL: https://github.com/apache/incubator-hudi/pull/1229#discussion_r367471314
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
##########
@@ -279,18 +280,28 @@ private void deleteInstantFile(HoodieInstant instant) {
return readDataFromPath(detailPath);
}
+ public Option<byte[]> readCleanerInfoAsBytes(HoodieInstant instant) {
+ // Cleaner metadata are always stored only in timeline .hoodie
+ return readDataFromPath(new Path(metaClient.getMetaPath(),
instant.getFileName()));
+ }
+
//-----------------------------------------------------------------
// BEGIN - COMPACTION RELATED META-DATA MANAGEMENT.
//-----------------------------------------------------------------
- public Option<byte[]> readPlanAsBytes(HoodieInstant instant) {
- Path detailPath = null;
- if (metaClient.getTimelineLayoutVersion().isNullVersion()) {
- detailPath = new Path(metaClient.getMetaAuxiliaryPath(),
instant.getFileName());
- } else {
- detailPath = new Path(metaClient.getMetaPath(), instant.getFileName());
+ public Option<byte[]> readCompactionPlanAsBytes(HoodieInstant instant) {
+ try {
+ // This is going to be the common case in future when 0.5.1 is deployed.
Review comment:
but then in the meantime, every reader will be doing two RPCs for this
method right? I am thinking about flipping the order.. read from aux first,
then fallback to meta path...
This way, when we switch to writing in metapath, only the older readers will
incur this additional RPC... Does it make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
