[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729407#comment-15729407
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/2919
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
> Fix For: 1.2.0
>
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729396#comment-15729396
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user twalthr commented on the issue:
https://github.com/apache/flink/pull/2919
Merging...
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729394#comment-15729394
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/2919
Thanks for the update.
+1 to merge.
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729389#comment-15729389
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user twalthr commented on the issue:
https://github.com/apache/flink/pull/2919
Thanks for reviewing my PR. I hope I addressed all your comments.
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726115#comment-15726115
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91109092
--- Diff: docs/dev/table_api.md ---
@@ -3378,9 +3472,32 @@ ROW (value [, value]* )
Creates a row from a list of values.
+-->
+
+
+
+{% highlight text %}
+array ‘[’ index ‘]’
+{% endhighlight %}
+
+
+Returns the element at a particular location in an array. The
index starts at 1.
--- End diff --
`location` -> `position`
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726107#comment-15726107
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91124107
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/schema/ArrayRelDataType.scala
---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.api.table.plan.schema
+
+import org.apache.calcite.rel.`type`.RelDataType
+import org.apache.calcite.sql.`type`.ArraySqlType
+import org.apache.flink.api.common.typeinfo.TypeInformation
+
+/**
+ * Flink distinguishes between primitive arrays (int[], double[], ...) and
+ * object arrays (Integer[], MyPojo[], ...). This custom type considers
the two cases.
--- End diff --
`considers` -> `supports` or `covers`?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726110#comment-15726110
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91107452
--- Diff: docs/dev/table_api.md ---
@@ -2027,6 +2034,50 @@ COMPOSITE.get(INT)
+
+
+{% highlight java %}
+ARRAY.at(INT)
+{% endhighlight %}
+
+
+Returns the element at a particular location in an array. The
index starts at 1.
--- End diff --
`location` -> `position`?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726113#comment-15726113
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r9043
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/scala/table/expressionDsl.scala
---
@@ -454,6 +455,29 @@ trait ImplicitExpressionOperations {
* into a flat representation where every subtype is a separate field.
*/
def flatten() = Flattening(expr)
+
+ /**
+* Accesses the element of an array based on an index (starting at 1).
+*
+* @param index position of the element (starting at 1)
+* @return value of the element
+*/
+ def at(index: Expression) = ArrayElementAt(expr, index)
+
+ /**
+* Returns the number of elements of an array.
+*
+* @return number of element
+*/
+ def cardinality() = ArrayCardinality(expr)
+
+ /**
+* Returns the sole element of an array. Returns null if the collection
is empty.
+* Throws an exception if the array has more than one element.
+*
+* @return first element of the array
--- End diff --
`the only element of an array with a single entry`?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726108#comment-15726108
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91110871
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/scala/table/expressionDsl.scala
---
@@ -454,6 +455,29 @@ trait ImplicitExpressionOperations {
* into a flat representation where every subtype is a separate field.
*/
def flatten() = Flattening(expr)
+
+ /**
+* Accesses the element of an array based on an index (starting at 1).
+*
+* @param index position of the element (starting at 1)
+* @return value of the element
+*/
+ def at(index: Expression) = ArrayElementAt(expr, index)
+
+ /**
+* Returns the number of elements of an array.
+*
+* @return number of element
+*/
+ def cardinality() = ArrayCardinality(expr)
+
+ /**
+* Returns the sole element of an array. Returns null if the collection
is empty.
--- End diff --
`of an array` -> `of an array with a single element.`
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726114#comment-15726114
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91124734
--- Diff:
flink-libraries/flink-table/src/test/scala/org/apache/flink/api/table/expressions/ArrayTypeTest.scala
---
@@ -0,0 +1,342 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.api.table.expressions
+
+import java.sql.Date
+
+import org.apache.flink.api.common.typeinfo.{PrimitiveArrayTypeInfo,
TypeInformation}
+import org.apache.flink.api.java.typeutils.ObjectArrayTypeInfo
+import org.apache.flink.api.scala.table._
+import org.apache.flink.api.table.expressions.utils.ExpressionTestBase
+import org.apache.flink.api.table.typeutils.RowTypeInfo
+import org.apache.flink.api.table.{Row, Types, ValidationException}
+import org.junit.Test
+
+class ArrayTypeTest extends ExpressionTestBase {
+
+ @Test(expected = classOf[ValidationException])
+ def testEmptyArraySql(): Unit = {
+testSqlApi("ARRAY[]", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testEmptyArrayTableApi(): Unit = {
+testTableApi("FAIL", "array()", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testNullArraySql(): Unit = {
+testSqlApi("ARRAY[NULL]", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testDifferentTypesArraySql(): Unit = {
+testSqlApi("ARRAY[1, TRUE]", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testDifferentTypesArrayTableApi(): Unit = {
+testTableApi("FAIL", "array(1, true)", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testUnsupportedComparison(): Unit = {
+testAllApis(
+ 'f2 <= 'f5.at(1),
+ "f2 <= f5.at(1)",
+ "f2 <= f5[1]",
+ "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testElementNonArray(): Unit = {
--- End diff --
test other APIs as well?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726116#comment-15726116
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91124760
--- Diff:
flink-libraries/flink-table/src/test/scala/org/apache/flink/api/table/expressions/ArrayTypeTest.scala
---
@@ -0,0 +1,342 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.api.table.expressions
+
+import java.sql.Date
+
+import org.apache.flink.api.common.typeinfo.{PrimitiveArrayTypeInfo,
TypeInformation}
+import org.apache.flink.api.java.typeutils.ObjectArrayTypeInfo
+import org.apache.flink.api.scala.table._
+import org.apache.flink.api.table.expressions.utils.ExpressionTestBase
+import org.apache.flink.api.table.typeutils.RowTypeInfo
+import org.apache.flink.api.table.{Row, Types, ValidationException}
+import org.junit.Test
+
+class ArrayTypeTest extends ExpressionTestBase {
+
+ @Test(expected = classOf[ValidationException])
+ def testEmptyArraySql(): Unit = {
+testSqlApi("ARRAY[]", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testEmptyArrayTableApi(): Unit = {
+testTableApi("FAIL", "array()", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testNullArraySql(): Unit = {
+testSqlApi("ARRAY[NULL]", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testDifferentTypesArraySql(): Unit = {
+testSqlApi("ARRAY[1, TRUE]", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testDifferentTypesArrayTableApi(): Unit = {
+testTableApi("FAIL", "array(1, true)", "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testUnsupportedComparison(): Unit = {
+testAllApis(
+ 'f2 <= 'f5.at(1),
+ "f2 <= f5.at(1)",
+ "f2 <= f5[1]",
+ "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testElementNonArray(): Unit = {
+testTableApi(
+ 'f0.element(),
+ "FAIL",
+ "FAIL")
+ }
+
+ @Test(expected = classOf[ValidationException])
+ def testCardinalityOnNonArray(): Unit = {
--- End diff --
test other APIs as well?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726119#comment-15726119
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91114685
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/codegen/CodeGenUtils.scala
---
@@ -189,6 +188,16 @@ object CodeGenUtils {
throw new CodeGenException("Interval expression type expected.")
}
+ def requireArray(genExpr: GeneratedExpression) =
+if (!TypeCheckUtils.isArray(genExpr.resultType)) {
+ throw new CodeGenException("Array expression type expected.")
+}
+
+ def requireInteger(genExpr: GeneratedExpression) =
+if (!TypeCheckUtils.isInteger(genExpr.resultType)) {
+ throw new CodeGenException("Array expression type expected.")
--- End diff --
`Array` -> `Integer`
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726112#comment-15726112
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91109035
--- Diff: docs/dev/table_api.md ---
@@ -2577,6 +2628,50 @@ COMPOSITE.get(INT)
+
+
+{% highlight scala %}
+ARRAY.at(INT)
+{% endhighlight %}
+
+
+Returns the element at a particular location in an array. The
index starts at 1.
--- End diff --
the comments in the Java section probably apply to Scala as well.
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726106#comment-15726106
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91123689
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/expressions/array.scala
---
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.api.table.expressions
+
+import org.apache.calcite.rex.RexNode
+import org.apache.calcite.sql.fun.SqlStdOperatorTable
+import org.apache.calcite.tools.RelBuilder
+import org.apache.flink.api.common.typeinfo.BasicTypeInfo.INT_TYPE_INFO
+import org.apache.flink.api.common.typeinfo.{BasicTypeInfo,
PrimitiveArrayTypeInfo, TypeInformation}
+import org.apache.flink.api.java.typeutils.ObjectArrayTypeInfo
+import org.apache.flink.api.table.FlinkRelBuilder
+import org.apache.flink.api.table.validate.{ValidationFailure,
ValidationResult, ValidationSuccess}
+
+import scala.collection.JavaConverters._
+
+case class ArrayConstructor(elements: Seq[Expression]) extends Expression {
+
+ override private[flink] def children: Seq[Expression] = elements
+
+ override private[flink] def toRexNode(implicit relBuilder: RelBuilder):
RexNode = {
+val relDataType = relBuilder
+ .asInstanceOf[FlinkRelBuilder]
+ .getTypeFactory
+ .createTypeFromTypeInfo(resultType)
+val values = elements.map(_.toRexNode).toList.asJava
+relBuilder
+ .getRexBuilder
+ .makeCall(relDataType, SqlStdOperatorTable.ARRAY_VALUE_CONSTRUCTOR,
values)
+ }
+
+ override def toString = s"array(${elements.mkString(", ")})"
+
+ override private[flink] def resultType =
ObjectArrayTypeInfo.getInfoFor(elements.head.resultType)
+
+ override private[flink] def validateInput(): ValidationResult = {
+if (elements.isEmpty) {
+ return ValidationFailure("Empty arrays are not supported yet.")
+}
+val elementType = elements.head.resultType
+if (!elements.forall(_.resultType == elementType)) {
+ ValidationFailure("Not all elements of the array have the same
type.")
+} else {
+ ValidationSuccess
+}
+ }
+}
+
+case class ArrayElementAt(array: Expression, index: Expression) extends
Expression {
+
+ override private[flink] def children: Seq[Expression] = Seq(array, index)
+
+ override private[flink] def toRexNode(implicit relBuilder: RelBuilder):
RexNode = {
+relBuilder
+ .getRexBuilder
+ .makeCall(SqlStdOperatorTable.ITEM, array.toRexNode, index.toRexNode)
+ }
+
+ override def toString = s"($array).at($index)"
+
+ override private[flink] def resultType = array.resultType match {
+case oati: ObjectArrayTypeInfo[_, _] => oati.getComponentInfo
+case pati: PrimitiveArrayTypeInfo[_] => pati.getComponentType
+ }
+
+ override private[flink] def validateInput(): ValidationResult = {
--- End diff --
Does it make sense to check that the index is `> 0` if it is a literal?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726118#comment-15726118
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91108867
--- Diff: docs/dev/table_api.md ---
@@ -2027,6 +2034,50 @@ COMPOSITE.get(INT)
+
+
+{% highlight java %}
+ARRAY.at(INT)
+{% endhighlight %}
+
+
+Returns the element at a particular location in an array. The
index starts at 1.
+
+
+
+
+
+{% highlight java %}
+array(ANY [, ANY ]*)
+{% endhighlight %}
+
+
+Creates an array from a list of values. The array will be an
array of objects (not primitives).
+
+
+
+
+
+{% highlight java %}
+ARRAY.cardinality()
+{% endhighlight %}
+
+
+Returns the number of elements of an array.
+
+
+
+
+
+{% highlight scala %}
+ARRAY.element()
+{% endhighlight %}
+
+
+Returns the sole element of an array. Returns null
if the collection is empty. Throws an exception if the array has more than one
element.
--- End diff --
I think "sole element" in this context is not very clear. I'd rephrase it
to something like: "Returns the sole element of an array with a single element."
`collection` -> `array`?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726117#comment-15726117
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91110730
--- Diff:
flink-libraries/flink-table/src/main/scala/org/apache/flink/api/scala/table/expressionDsl.scala
---
@@ -454,6 +455,29 @@ trait ImplicitExpressionOperations {
* into a flat representation where every subtype is a separate field.
*/
def flatten() = Flattening(expr)
+
+ /**
+* Accesses the element of an array based on an index (starting at 1).
+*
+* @param index position of the element (starting at 1)
+* @return value of the element
+*/
+ def at(index: Expression) = ArrayElementAt(expr, index)
+
+ /**
+* Returns the number of elements of an array.
+*
+* @return number of element
--- End diff --
`element` -> `elements`
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726111#comment-15726111
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91126562
--- Diff: tools/maven/scalastyle-config.xml ---
@@ -68,11 +68,11 @@
-
--- End diff --
Can we use a custom scalastyle for `flink-table` instead of modifying the
global one?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726109#comment-15726109
]
ASF GitHub Bot commented on FLINK-4554:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/2919#discussion_r91106487
--- Diff: docs/dev/table_api.md ---
@@ -1470,7 +1470,14 @@ The Table API is built on top of Flink's DataSet and
DataStream API. Internally,
| `Types.INTERVAL_MONTHS`| `INTERVAL YEAR TO MONTH`|
`java.lang.Integer`|
| `Types.INTERVAL_MILLIS`| `INTERVAL DAY TO SECOND(3)` | `java.lang.Long`
|
-Advanced types such as generic types, composite types (e.g. POJOs or
Tuples), and arrays can be fields of a row. Generic types and arrays are
treated as a black box within Table API and SQL yet. Composite types, however,
are fully supported types where fields of a composite type can be accessed
using the `.get()` operator in Table API and dot operator (e.g.
`MyTable.pojoColumn.myField`) in SQL. Composite types can also be flattened
using `.flatten()` in Table API or `MyTable.pojoColumn.*` in SQL.
+
+Advanced types such as generic types, composite types (e.g. POJOs or
Tuples), and array types (object or primitive arrays) can be fields of a row.
+
+Generic types are treated as a black box within Table API and SQL yet.
+
+Composite types, however, are fully supported types where fields of a
composite type can be accessed using the `.get()` operator in Table API and dot
operator (e.g. `MyTable.pojoColumn.myField`) in SQL. Composite types can also
be flattened using `.flatten()` in Table API or `MyTable.pojoColumn.*` in SQL.
+
+Array types can be access using the `myArray.at(1)` operator in Table API
and `myArray[1]` operator in SQL. Array literals can be created using `array(1,
2, 3)` in Table API and `ARRAY[1, 2, 3]` in SQL.
--- End diff --
`can be access` -> `can be accessed`
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712445#comment-15712445
]
ASF GitHub Bot commented on FLINK-4554:
---
GitHub user twalthr opened a pull request:
https://github.com/apache/flink/pull/2919
[FLINK-4554] [table] Add support for array types
This PR adds array support for the Table API and SQL. It adds support for
both primitive arrays and object arrays. It adds literals, element access,
comparison and the functions `ELEMENT` and `CARDINALITY`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/twalthr/flink FLINK-4554
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2919.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2919
commit 0a93866b9e2510f07f818c3ac34db439c5b52b42
Author: twalthr
Date: 2016-09-23T14:44:42Z
[FLINK-4554] [table] Add support for array types
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526259#comment-15526259
]
Timo Walther commented on FLINK-4554:
-
[~kirill-morozov-epam] thanks for your interest. Unfortunately, I already
started with this issue and will open a PR soon. Maybe you can pick something
else. There are many other Table API/SQL related issues open.
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525968#comment-15525968
]
Kirill Morozov commented on FLINK-4554:
---
[~twalthr], I'am sorry. Please, give me a chance to resolve this ticket. Thank
you very much in any case
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15468154#comment-15468154
]
Timo Walther commented on FLINK-4554:
-
Apache Calcite does the parsing for us and they strictly follow the SQL
standard. I don't know if they support this syntax.
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4554) Add support for array types
[
https://issues.apache.org/jira/browse/FLINK-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467918#comment-15467918
]
Konstantin Boudnik commented on FLINK-4554:
---
Do you think adding a support for ranges a-la Groovy style ie {{ARRAY[1..3]}}
should also be supported?
> Add support for array types
> ---
>
> Key: FLINK-4554
> URL: https://issues.apache.org/jira/browse/FLINK-4554
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Timo Walther
>
> Support creating arrays:
> {code}ARRAY[1, 2, 3]{code}
> Access array values:
> {code}myArray[3]{code}
> And operations like:
> {{UNNEST, UNNEST WITH ORDINALITY, CARDINALITY}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
