[GitHub] [beam] amaliujia commented on a change in pull request #12843: [BEAM-10895] Support UNNEST an (possibly nested) array field of an struct column
amaliujia commented on a change in pull request #12843: URL: https://github.com/apache/beam/pull/12843#discussion_r488992286 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnnestRel.java ## @@ -111,31 +112,41 @@ public RelWriter explainTerms(RelWriter pw) { Schema joinedSchema = CalciteUtils.toSchema(getRowType()); return outer - .apply(ParDo.of(new UnnestFn(joinedSchema, unnestIndex))) + .apply(ParDo.of(new UnnestFn(joinedSchema, unnestIndices))) .setRowSchema(joinedSchema); } } private static class UnnestFn extends DoFn { private final Schema outputSchema; -private final int unnestIndex; +private final List unnestIndices; -private UnnestFn(Schema outputSchema, int unnestIndex) { +private UnnestFn(Schema outputSchema, List unnestIndices) { this.outputSchema = outputSchema; - this.unnestIndex = unnestIndex; + this.unnestIndices = unnestIndices; } @ProcessElement public void process(@Element Row row, OutputReceiver out) { - - @Nullable Collection rawValues = row.getArray(unnestIndex); + Row rowWithArrayField = row; + Schema schemaWithArrayField = outputSchema; + for (int i = unnestIndices.size() - 1; i > 0; i--) { +rowWithArrayField = rowWithArrayField.getRow(unnestIndices.get(i)); Review comment: So it looks like guaranteed to have nested row except for the last index? If not better to add validation here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [beam] amaliujia commented on a change in pull request #12843: [BEAM-10895] Support UNNEST an (possibly nested) array field of an struct column
amaliujia commented on a change in pull request #12843: URL: https://github.com/apache/beam/pull/12843#discussion_r488992286 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnnestRel.java ## @@ -111,31 +112,41 @@ public RelWriter explainTerms(RelWriter pw) { Schema joinedSchema = CalciteUtils.toSchema(getRowType()); return outer - .apply(ParDo.of(new UnnestFn(joinedSchema, unnestIndex))) + .apply(ParDo.of(new UnnestFn(joinedSchema, unnestIndices))) .setRowSchema(joinedSchema); } } private static class UnnestFn extends DoFn { private final Schema outputSchema; -private final int unnestIndex; +private final List unnestIndices; -private UnnestFn(Schema outputSchema, int unnestIndex) { +private UnnestFn(Schema outputSchema, List unnestIndices) { this.outputSchema = outputSchema; - this.unnestIndex = unnestIndex; + this.unnestIndices = unnestIndices; } @ProcessElement public void process(@Element Row row, OutputReceiver out) { - - @Nullable Collection rawValues = row.getArray(unnestIndex); + Row rowWithArrayField = row; + Schema schemaWithArrayField = outputSchema; + for (int i = unnestIndices.size() - 1; i > 0; i--) { +rowWithArrayField = rowWithArrayField.getRow(unnestIndices.get(i)); Review comment: So it looks like guaranteed to have nested row except for the last index? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org