slfan1989 commented on code in PR #13837:
URL: https://github.com/apache/iceberg/pull/13837#discussion_r2292598195
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java:
##########
@@ -304,12 +307,29 @@ private String rebuildMetadata() {
.collect(Collectors.toSet());
rewritePositionDeletes(deleteFiles);
+ ImmutableRewriteTablePath.Result.Builder builder =
+ ImmutableRewriteTablePath.Result.builder()
+ .stagingLocation(stagingDir)
+ .rewrittenDeleteFilesCount(deleteFiles.size())
+ .rewrittenManifestFilesCount(metaFiles.size())
+ .latestVersion(RewriteTablePathUtil.fileName(endVersionName));
+
+ // skip file list
+ if (skipFileList) {
+ return builder.build();
+ }
+
Set<Pair<String, String>> copyPlan = Sets.newHashSet();
copyPlan.addAll(rewriteVersionResult.copyPlan());
copyPlan.addAll(rewriteManifestListResult.copyPlan());
copyPlan.addAll(rewriteManifestResult.copyPlan());
+ String fileListLocation = saveFileList(copyPlan);
- return saveFileList(copyPlan);
+ return builder
+ .stagingLocation(stagingDir)
+ .fileListLocation(fileListLocation)
Review Comment:
@dramaticlly
I checked the code based on nastra’s suggestion and confirmed that his
analysis is correct. I also verified it locally: if fileListLocation is not
set, an exception will indeed be thrown.
```
Cannot build Result, some of required attributes are not set
[fileListLocation]
java.lang.IllegalStateException: Cannot build Result, some of required
attributes are not set [fileListLocation]
at
org.apache.iceberg.actions.ImmutableRewriteTablePath$Result$Builder.build(ImmutableRewriteTablePath.java:514)
```
I’ve been thinking—if fileListLocation is empty, can we simply treat it as
`N/A`? My idea is to assign `N/A` directly during the check, without changing
the original definition. Do you think this approach would work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
