[
https://issues.apache.org/jira/browse/BEAM-11849?focusedWorklogId=562526&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-562526
]
ASF GitHub Bot logged work on BEAM-11849:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Mar/21 18:54
Start Date: 08/Mar/21 18:54
Worklog Time Spent: 10m
Work Description: ibzib commented on a change in pull request #14035:
URL: https://github.com/apache/beam/pull/14035#discussion_r589663990
##########
File path:
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -92,6 +93,14 @@
x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
new UdafImpl<>(new ArrayAgg.ArrayAggArray()));
+ public static final SqlOperator ARRAY_CONCAT_AGG_FN =
+ createUdafOperator(
+ "array_concat_agg",
+ x ->
+ createTypeFactory()
+
.createArrayType(createTypeFactory().createSqlType(SqlTypeName.BIGINT), -1),
Review comment:
You need to get the array element type from `x`. Otherwise this function
will only work for integers.
##########
File path:
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlDialectSpecTest.java
##########
@@ -3843,4 +3847,27 @@ public void testArrayAggZetasql() {
pipeline.run().waitUntilFinish(Duration.standardMinutes(PIPELINE_EXECUTION_WAITTIME_MINUTES));
}
+
+ @Test
+ public void testArrayConcatAggZetasql() {
+ String sql =
+ "WITH aggregate_example AS (SELECT [1,2] AS numbers UNION ALL SELECT
[3,4] AS numbers UNION ALL SELECT [5, 6] AS numbers) SELECT
ARRAY_CONCAT_AGG(numbers) AS count_to_six_agg FROM aggregate_example";
+
+ ZetaSQLQueryPlanner zetaSQLQueryPlanner = new ZetaSQLQueryPlanner(config);
+ BeamRelNode beamRelNode = zetaSQLQueryPlanner.convertToBeamRel(sql);
+ PCollection<Row> stream = BeamSqlRelUtils.toPCollection(pipeline,
beamRelNode);
+ PAssert.thatSingleton(stream)
+ .satisfies(
+ row -> {
+ Collection<Object> output = row.getArray("count_to_six_agg");
+ HashSet<Object> outputSet = new HashSet<Object>(output);
+
+ HashSet<Object> expectedOutputSet = Sets.newHashSet(1L, 2L, 3L,
4L, 5L, 6L);
+
+ assertThat("array_field", expectedOutputSet.equals(outputSet));
Review comment:
Use `containsInAnyOrder` instead of building and comparing sets. It's
simpler.
http://hamcrest.org/JavaHamcrest/javadoc/1.3/org/hamcrest/Matchers.html#containsInAnyOrder(T...)
##########
File path:
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlDialectSpecTest.java
##########
@@ -3843,4 +3847,27 @@ public void testArrayAggZetasql() {
pipeline.run().waitUntilFinish(Duration.standardMinutes(PIPELINE_EXECUTION_WAITTIME_MINUTES));
}
+
+ @Test
+ public void testArrayConcatAggZetasql() {
+ String sql =
+ "WITH aggregate_example AS (SELECT [1,2] AS numbers UNION ALL SELECT
[3,4] AS numbers UNION ALL SELECT [5, 6] AS numbers) SELECT
ARRAY_CONCAT_AGG(numbers) AS count_to_six_agg FROM aggregate_example";
Review comment:
Break this query string into multiple lines.
##########
File path:
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -266,7 +275,7 @@ public SqlSyntax getSyntax() {
};
}
- private static RelDataType createSqlType(SqlTypeName typeName, boolean
withNullability) {
+ public static RelDataType createSqlType(SqlTypeName typeName, boolean
withNullability) {
Review comment:
Why does this need to be public?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 562526)
Time Spent: 50m (was: 40m)
> Support ARRAY_CONCAT_AGG fn for Zetasql dialect
> -----------------------------------------------
>
> Key: BEAM-11849
> URL: https://issues.apache.org/jira/browse/BEAM-11849
> Project: Beam
> Issue Type: New Feature
> Components: dsl-sql-zetasql
> Reporter: Sonam Ramchand
> Priority: P2
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
