RussellSpitzer commented on code in PR #4902:
URL: https://github.com/apache/iceberg/pull/4902#discussion_r916372563
##########
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java:
##########
@@ -140,36 +140,55 @@ private RewriteDataFiles checkAndApplyOptions(InternalRow
args, RewriteDataFiles
return action.options(options);
}
- private RewriteDataFiles checkAndApplyStrategy(RewriteDataFiles action,
String strategy, SortOrder sortOrder) {
+ private RewriteDataFiles checkAndApplyStrategy(
+ RewriteDataFiles action,
+ String strategy,
+ String sortOrderString,
+ Schema schema) {
+ List<ExtendedParser.RawOrderField> zOrderFields = Lists.newArrayList();
+ List<ExtendedParser.RawOrderField> sortOrderFields = Lists.newArrayList();
+ if (sortOrderString != null) {
+ ExtendedParser.parseSortOrder(spark(), sortOrderString).forEach(field ->
{
+ if (field.term() instanceof Zorder) {
+ zOrderFields.add(field);
+ } else {
+ sortOrderFields.add(field);
+ }
+ });
+
+ if (!zOrderFields.isEmpty() && !sortOrderFields.isEmpty()) {
+ // TODO: we need to allow this in future when SparkAction has handling
for this.
+ throw new IllegalArgumentException("Both SortOrder and Zorder is
configured: " + sortOrderString);
Review Comment:
Perhaps, "Cannot mix identity sort columns and a Zorder sort expression"?
Since I'm not sure an end user would know what we are referring to here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
