[GitHub] [flink] flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration
flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration URL: https://github.com/apache/flink/pull/8303#issuecomment-511684151 ## CI report: * 6a7ca58b4a04f6dce250045e021702e67e82b893 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119421914) * 4d38a8df0d59734c4b2386689a2f17b9f2b44b12 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441376) * 9c14836f8639e98d58cf7bb32e38b938b3843994 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119577044) * 76186776c5620598a19234245bbd05dfdfb1c62c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120113740) * 628ca7b316ad3968c90192a47a84dd01f26e2578 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122381349) * d204a725ff3c8a046cbd1b84e34d9e3ae8aafeac : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123620485) * 143efadbdb6c4681569d5b412a175edfb1633b85 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123637809) * b78b64a82ed2a9a92886095ec42f06d5082ad830 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123671219) * 5145a0b9d6b320456bb971d96b9cc47707c8fd28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125476639) * 0d4d944c28c59ca1caa6c453c347ec786b40d245 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125762588) * 91552c3804f5e96cc573e6ed48756f2b54c037d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125844084) * fbf6d2850b6aa7c303981e6f5b24b0da0956b820 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126086321) * 45282a2139ec3c7196ba10b6fb829b0ec275 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126350641) * cc515180d99fa35b482c61c2e264e77760e55dc5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126354591) * b8341665cb9b8bf3559d93ea74715ddac3ca67b2 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127617990) * 462a8cf7286f63afcfb5cecbce37df232d3f8ec4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127623432) * 51397b3c71d8dd4549eba379b2b873d2e523e268 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127625943) * 60e035ab71050b367fcf5411953a75a8a3b32a75 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127672092) * 978c0723caaf584fe91b59cc9aabbd77815907c8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129038982) * 5784c98687a35b6437a3948992f1302f8e849c5d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129143114) * 841b65e4522c78c55c644f1aaf9c81c2d2c9efd7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129318099) * 016e9a5a75871e467a70dee62d92183c2e7f3023 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129391415) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work
flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work URL: https://github.com/apache/flink/pull/9766#issuecomment-534879961 ## CI report: * 8fffbbc4bc605ccedad7412ac2292d3488261fa3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129046428) * 120f12f13987874b1738da7db1aa370f4335830a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129105662) * 8d3ff9abe90e8e23b2bd1ec12cd99dc7444f6a8d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129205704) * 2c449c28158d24a2a65c9b01f1636d8b544d29ee : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129214214) * 4e78ce4c1b10167455191ac37b615f9116bed348 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129218842) * c01756b0ca85868cc3b21b7d96e2eae8e9b76082 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129222850) * 0aa5803255b33a3575c87dc5e8b7c6916de812a7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129243177) * 1877dc719551e2a10bb03e99d2cacb6e35ad9826 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129388963) * 7bf5b29144488851c10dcbafa88e7bfacc34147a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129393475) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-13314) Correct resultType of some PlannerExpression when operands contains DecimalTypeInfo or BigDecimalTypeInfo in Blink planner
[ https://issues.apache.org/jira/browse/FLINK-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] godfrey he closed FLINK-13314. -- > Correct resultType of some PlannerExpression when operands contains > DecimalTypeInfo or BigDecimalTypeInfo in Blink planner > -- > > Key: FLINK-13314 > URL: https://issues.apache.org/jira/browse/FLINK-13314 > Project: Flink > Issue Type: Bug > Components: Table SQL / Planner >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Blocker > Labels: pull-request-available > Fix For: 1.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Correct resultType of the following PlannerExpression when operands contains > DecimalTypeInfo or BigDecimalTypeInfo in Blink planner: > Minus/plus/Div/Mul/Ceil/Floor/Round > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-12810) Support to run a TableAPI query like 'table.select('a, 'b, 'c)' in Blink runner
[
https://issues.apache.org/jira/browse/FLINK-12810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he closed FLINK-12810.
--
> Support to run a TableAPI query like 'table.select('a, 'b, 'c)' in Blink
> runner
> ---
>
> Key: FLINK-12810
> URL: https://issues.apache.org/jira/browse/FLINK-12810
> Project: Flink
> Issue Type: Task
> Components: Table SQL / Planner
>Reporter: Jing Zhang
>Assignee: Jing Zhang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Support to run a TableAPI query like 'table.select('a, 'b, 'c)', including:
> 1. add RexNodeConverter to convert Expression to RexNode
> 2. Introduce RexPlannerExpression to wraps a RexNode, which is similar to
> RexPlannerExpression in flink-planner module.
> 3. add QueryOperationConverter to convert QueryOperation tree to RelNode
> 4. Introduce PlannerQueryOperation to wraps a RelNode in QueryOperation,
> which is similar to PlannerQueryOperation in flink-planner module.
> 5. add OperationTreeBuilder and relative classes(ExpressionResolver,
> TableReferenceLookup, FieldReferenceLoopup, ExpandColumnFunctionsRule,
> LookupCallByNameRule, ResolverRule, StarReferenceFlattingRule,
> VerifyNoUnresolvedExpressionRule), which is almost copy from flink-planner
> module to blink-planner module, except that we remove dependency on
> PlannerExpression. **Note: This is temporal solution, since
> OperationTreeBuilder will be moved to api module finally.**
> 6. add implicit conversion in expressionDsl
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-12348) Use TableConfig in API module to replace TableConfig in blink-planner-module
[ https://issues.apache.org/jira/browse/FLINK-12348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] godfrey he closed FLINK-12348. -- Fixed in 1.9.0: 428c7049882cfc482562b69d1a367cdb9b0eec72 > Use TableConfig in API module to replace TableConfig in blink-planner-module > > > Key: FLINK-12348 > URL: https://issues.apache.org/jira/browse/FLINK-12348 > Project: Flink > Issue Type: Task > Components: Table SQL / Planner >Reporter: Jing Zhang >Assignee: Jing Zhang >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Since TableConfig already moved to API module in > [FLINK-11067|https://issues.apache.org/jira/browse/FLINK-11067], TableConfig > in blink-planner-module should not exist anymore. The issue aims to remove > the TableConfig in blink-planner-module, use TableConfig in API module > instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work
flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work URL: https://github.com/apache/flink/pull/9766#issuecomment-534879961 ## CI report: * 8fffbbc4bc605ccedad7412ac2292d3488261fa3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129046428) * 120f12f13987874b1738da7db1aa370f4335830a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129105662) * 8d3ff9abe90e8e23b2bd1ec12cd99dc7444f6a8d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129205704) * 2c449c28158d24a2a65c9b01f1636d8b544d29ee : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129214214) * 4e78ce4c1b10167455191ac37b615f9116bed348 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129218842) * c01756b0ca85868cc3b21b7d96e2eae8e9b76082 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129222850) * 0aa5803255b33a3575c87dc5e8b7c6916de812a7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129243177) * 1877dc719551e2a10bb03e99d2cacb6e35ad9826 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129388963) * 7bf5b29144488851c10dcbafa88e7bfacc34147a : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] dianfu commented on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work
dianfu commented on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work URL: https://github.com/apache/flink/pull/9766#issuecomment-535786174 The test failure of the Kafka test case has already been reported in https://issues.apache.org/jira/browse/FLINK-14235 and it's not related to this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] dianfu commented on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work
dianfu commented on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work URL: https://github.com/apache/flink/pull/9766#issuecomment-535785537 @hequn8128 @WeiZhong94 Thanks a lot for your review. Updated the PR accordingly. Looking forward to your feedback. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] dianfu commented on a change in pull request #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work
dianfu commented on a change in pull request #9766: [FLINK-14018][python] Add
Python building blocks to make sure the basic functionality of Python
ScalarFunction could work
URL: https://github.com/apache/flink/pull/9766#discussion_r328912615
##
File path:
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/plan/nodes/CommonPythonCalc.scala
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.table.plan.nodes
+
+import org.apache.calcite.rex.{RexCall, RexInputRef, RexNode}
+import org.apache.flink.table.functions.FunctionLanguage
+import org.apache.flink.table.functions.python.{PythonFunction,
PythonFunctionInfo, SimplePythonFunction}
+import org.apache.flink.table.functions.utils.ScalarSqlFunction
+
+import scala.collection.JavaConversions._
+import scala.collection.mutable
+
+trait CommonPythonCalc {
+
+ private[flink] def extractPythonScalarFunctionInfos(
+ rexCalls: Array[RexCall]): (Array[Int], Array[PythonFunctionInfo]) = {
+// using LinkedHashMap to keep the insert order
+val inputNodes = new mutable.LinkedHashMap[RexNode, Integer]()
+val pythonFunctionInfos = rexCalls.map(createPythonScalarFunctionInfo(_,
inputNodes))
+
+val udfInputOffsets = inputNodes.toArray.sortBy(_._2).map(_._1).map {
+ case inputRef: RexInputRef => inputRef.getIndex
+}
+(udfInputOffsets, pythonFunctionInfos)
+ }
+
+ private[flink] def createPythonScalarFunctionInfo(
+ rexCall: RexCall,
+ inputNodes: mutable.Map[RexNode, Integer]): PythonFunctionInfo =
rexCall.getOperator match {
+case sfc: ScalarSqlFunction if sfc.getScalarFunction.getLanguage ==
FunctionLanguage.PYTHON =>
+ val inputs = new mutable.ArrayBuffer[AnyRef]()
+ rexCall.getOperands.foreach {
+case pythonRexCall: RexCall if
pythonRexCall.getOperator.asInstanceOf[ScalarSqlFunction]
+ .getScalarFunction.getLanguage == FunctionLanguage.PYTHON =>
+ // Continuous Python UDFs can be chained together
+ val argPythonInfo = createPythonScalarFunctionInfo(pythonRexCall,
inputNodes)
+ inputs.append(argPythonInfo)
+
+case argNode: RexNode =>
Review comment:
Good catch! Python UDF with constant parameters will be supported in
FLINK-14208. Will throw an exception here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration
flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration URL: https://github.com/apache/flink/pull/8303#issuecomment-511684151 ## CI report: * 6a7ca58b4a04f6dce250045e021702e67e82b893 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119421914) * 4d38a8df0d59734c4b2386689a2f17b9f2b44b12 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441376) * 9c14836f8639e98d58cf7bb32e38b938b3843994 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119577044) * 76186776c5620598a19234245bbd05dfdfb1c62c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120113740) * 628ca7b316ad3968c90192a47a84dd01f26e2578 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122381349) * d204a725ff3c8a046cbd1b84e34d9e3ae8aafeac : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123620485) * 143efadbdb6c4681569d5b412a175edfb1633b85 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123637809) * b78b64a82ed2a9a92886095ec42f06d5082ad830 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123671219) * 5145a0b9d6b320456bb971d96b9cc47707c8fd28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125476639) * 0d4d944c28c59ca1caa6c453c347ec786b40d245 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125762588) * 91552c3804f5e96cc573e6ed48756f2b54c037d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125844084) * fbf6d2850b6aa7c303981e6f5b24b0da0956b820 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126086321) * 45282a2139ec3c7196ba10b6fb829b0ec275 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126350641) * cc515180d99fa35b482c61c2e264e77760e55dc5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126354591) * b8341665cb9b8bf3559d93ea74715ddac3ca67b2 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127617990) * 462a8cf7286f63afcfb5cecbce37df232d3f8ec4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127623432) * 51397b3c71d8dd4549eba379b2b873d2e523e268 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127625943) * 60e035ab71050b367fcf5411953a75a8a3b32a75 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127672092) * 978c0723caaf584fe91b59cc9aabbd77815907c8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129038982) * 5784c98687a35b6437a3948992f1302f8e849c5d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129143114) * 841b65e4522c78c55c644f1aaf9c81c2d2c9efd7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129318099) * 016e9a5a75871e467a70dee62d92183c2e7f3023 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129391415) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14208) Optimize Python UDFs with parameters of constant values
[ https://issues.apache.org/jira/browse/FLINK-14208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-14208: Description: We need support Python UDFs with parameters of constant values. It should be noticed that the constant parameters are not needed to be transferred between the Java operator and the Python worker. (was: We need optimize Python UDFs with parameters of constant values. For example, the constant parameter isn't needed to be transferred between the Java operator and the Python worker.) > Optimize Python UDFs with parameters of constant values > --- > > Key: FLINK-14208 > URL: https://issues.apache.org/jira/browse/FLINK-14208 > Project: Flink > Issue Type: Sub-task > Components: API / Python >Reporter: Dian Fu >Priority: Major > Fix For: 1.10.0 > > > We need support Python UDFs with parameters of constant values. It should be > noticed that the constant parameters are not needed to be transferred between > the Java operator and the Python worker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese
flinkbot edited a comment on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese URL: https://github.com/apache/flink/pull/9749#issuecomment-534149758 ## CI report: * 141eb6db531af4b12a9506489f280f180c02a408 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128770115) * 0b96f066d276e4a483785f01335d36876e0e971c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128887957) * 5d7e1626653293ea20f935829d553de3b3e421fe : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129282782) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-11711) Add table and column stats
[ https://issues.apache.org/jira/browse/FLINK-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] godfrey he closed FLINK-11711. -- Resolution: Fixed TableStats was moved into flink-table-common in FLINK-11516 > Add table and column stats > -- > > Key: FLINK-11711 > URL: https://issues.apache.org/jira/browse/FLINK-11711 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Planner >Reporter: godfrey he >Assignee: godfrey he >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > We define two structure mode to hold statistics > 1. TableStats: statistics for table level, contains 2 elements: > rowCount: Long // the number of row count of table > colStats: Map[String, ColumnStats] // map each column to its ColumnStats > 2. ColumnStats: statistics for column level, contains 6 elements: > ndv: Long // number of distinct values > nullCount: Long // number of null values > avgLen: Double // average length of column values > maxLen: Integer // max length of column values > max: Any // max value of column values > min: Any // min value of column values -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-11711) Add table and column stats
[ https://issues.apache.org/jira/browse/FLINK-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939114#comment-16939114 ] godfrey he commented on FLINK-11711: I will close this issue and will open a new issue to support any type for max/min value and other stat type like histogram > Add table and column stats > -- > > Key: FLINK-11711 > URL: https://issues.apache.org/jira/browse/FLINK-11711 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Planner >Reporter: godfrey he >Assignee: godfrey he >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > We define two structure mode to hold statistics > 1. TableStats: statistics for table level, contains 2 elements: > rowCount: Long // the number of row count of table > colStats: Map[String, ColumnStats] // map each column to its ColumnStats > 2. ColumnStats: statistics for column level, contains 6 elements: > ndv: Long // number of distinct values > nullCount: Long // number of null values > avgLen: Double // average length of column values > maxLen: Integer // max length of column values > max: Any // max value of column values > min: Any // min value of column values -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration
flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration URL: https://github.com/apache/flink/pull/8303#issuecomment-511684151 ## CI report: * 6a7ca58b4a04f6dce250045e021702e67e82b893 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119421914) * 4d38a8df0d59734c4b2386689a2f17b9f2b44b12 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441376) * 9c14836f8639e98d58cf7bb32e38b938b3843994 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119577044) * 76186776c5620598a19234245bbd05dfdfb1c62c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120113740) * 628ca7b316ad3968c90192a47a84dd01f26e2578 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122381349) * d204a725ff3c8a046cbd1b84e34d9e3ae8aafeac : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123620485) * 143efadbdb6c4681569d5b412a175edfb1633b85 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123637809) * b78b64a82ed2a9a92886095ec42f06d5082ad830 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123671219) * 5145a0b9d6b320456bb971d96b9cc47707c8fd28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125476639) * 0d4d944c28c59ca1caa6c453c347ec786b40d245 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125762588) * 91552c3804f5e96cc573e6ed48756f2b54c037d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125844084) * fbf6d2850b6aa7c303981e6f5b24b0da0956b820 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126086321) * 45282a2139ec3c7196ba10b6fb829b0ec275 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126350641) * cc515180d99fa35b482c61c2e264e77760e55dc5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126354591) * b8341665cb9b8bf3559d93ea74715ddac3ca67b2 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127617990) * 462a8cf7286f63afcfb5cecbce37df232d3f8ec4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127623432) * 51397b3c71d8dd4549eba379b2b873d2e523e268 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127625943) * 60e035ab71050b367fcf5411953a75a8a3b32a75 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127672092) * 978c0723caaf584fe91b59cc9aabbd77815907c8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129038982) * 5784c98687a35b6437a3948992f1302f8e849c5d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129143114) * 841b65e4522c78c55c644f1aaf9c81c2d2c9efd7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129318099) * 016e9a5a75871e467a70dee62d92183c2e7f3023 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes
flinkbot edited a comment on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes URL: https://github.com/apache/flink/pull/9777#issuecomment-53537 ## CI report: * 8408299abf3e5f421ed2867ffa5629171c47af6b : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129227886) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-12263) Remove SINGLE_VALUE aggregate function from physical plan
[
https://issues.apache.org/jira/browse/FLINK-12263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee closed FLINK-12263.
Resolution: Invalid
> Remove SINGLE_VALUE aggregate function from physical plan
> -
>
> Key: FLINK-12263
> URL: https://issues.apache.org/jira/browse/FLINK-12263
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
>
> SINGLE_VALUE is an aggregate function which only accepts one row, and throws
> exception when received more than one row.
>
> For example:
> {code:sql}
> SELECT a2, SUM(a1) FROM A GROUP BY a2 HAVING SUM(a1) > (SELECT SUM(a1) * 0.1
> FROM A)
> {code}
> will get a physical plan contains SINGLE_VALUE:
> {code:sql}
> +- NestedLoopJoin(joinType=[InnerJoin], where=[>(EXPR$1, $f0)], select=[a2,
> EXPR$1, $f0], build=[right], singleRowJoin=[true])
>:- HashAggregate(isMerge=[true], groupBy=[a2], select=[a2,
> Final_SUM(sum$0) AS EXPR$1])
>: +- Exchange(distribution=[hash[a2]])
>: +- LocalHashAggregate(groupBy=[a2], select=[a2, Partial_SUM(a1) AS
> sum$0])
>:+- TableSourceScan(table=[[A, source: [TestTableSource(a1,
> a2)]]], fields=[a1, a2])
>+- Exchange(distribution=[broadcast])
> +- HashAggregate(isMerge=[true], select=[Final_SINGLE_VALUE(value$0,
> count$1) AS $f0])
> +- Exchange(distribution=[single])
> +- LocalHashAggregate(select=[Partial_SINGLE_VALUE(EXPR$0) AS
> (value$0, count$1)])
>+- Calc(select=[*($f0, 0.1) AS EXPR$0])
> +- HashAggregate(isMerge=[true], select=[Final_SUM(sum$0)
> AS $f0])
> +- Exchange(distribution=[single])
> +- LocalHashAggregate(select=[Partial_SUM(a1) AS
> sum$0])
>+- Calc(select=[a1])
> +- TableSourceScan(table=[[A, source:
> [TestTableSource(a1, a2)]]], fields=[a1, a2])
> {code}
> But SINGLE_VALUE is a bit wired in physical plan because the logical plan can
> make sure there is only one input row. Moreover it it also introduces
> additional overhead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-12263) Remove SINGLE_VALUE aggregate function from physical plan
[
https://issues.apache.org/jira/browse/FLINK-12263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939104#comment-16939104
]
Jingsong Lee commented on FLINK-12263:
--
SINGLE_VALUE is useful for "select * from A where a = (select a from B)". So we
can not remove it.
> Remove SINGLE_VALUE aggregate function from physical plan
> -
>
> Key: FLINK-12263
> URL: https://issues.apache.org/jira/browse/FLINK-12263
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
>
> SINGLE_VALUE is an aggregate function which only accepts one row, and throws
> exception when received more than one row.
>
> For example:
> {code:sql}
> SELECT a2, SUM(a1) FROM A GROUP BY a2 HAVING SUM(a1) > (SELECT SUM(a1) * 0.1
> FROM A)
> {code}
> will get a physical plan contains SINGLE_VALUE:
> {code:sql}
> +- NestedLoopJoin(joinType=[InnerJoin], where=[>(EXPR$1, $f0)], select=[a2,
> EXPR$1, $f0], build=[right], singleRowJoin=[true])
>:- HashAggregate(isMerge=[true], groupBy=[a2], select=[a2,
> Final_SUM(sum$0) AS EXPR$1])
>: +- Exchange(distribution=[hash[a2]])
>: +- LocalHashAggregate(groupBy=[a2], select=[a2, Partial_SUM(a1) AS
> sum$0])
>:+- TableSourceScan(table=[[A, source: [TestTableSource(a1,
> a2)]]], fields=[a1, a2])
>+- Exchange(distribution=[broadcast])
> +- HashAggregate(isMerge=[true], select=[Final_SINGLE_VALUE(value$0,
> count$1) AS $f0])
> +- Exchange(distribution=[single])
> +- LocalHashAggregate(select=[Partial_SINGLE_VALUE(EXPR$0) AS
> (value$0, count$1)])
>+- Calc(select=[*($f0, 0.1) AS EXPR$0])
> +- HashAggregate(isMerge=[true], select=[Final_SUM(sum$0)
> AS $f0])
> +- Exchange(distribution=[single])
> +- LocalHashAggregate(select=[Partial_SUM(a1) AS
> sum$0])
>+- Calc(select=[a1])
> +- TableSourceScan(table=[[A, source:
> [TestTableSource(a1, a2)]]], fields=[a1, a2])
> {code}
> But SINGLE_VALUE is a bit wired in physical plan because the logical plan can
> make sure there is only one input row. Moreover it it also introduces
> additional overhead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work
flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work URL: https://github.com/apache/flink/pull/9766#issuecomment-534879961 ## CI report: * 8fffbbc4bc605ccedad7412ac2292d3488261fa3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129046428) * 120f12f13987874b1738da7db1aa370f4335830a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129105662) * 8d3ff9abe90e8e23b2bd1ec12cd99dc7444f6a8d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129205704) * 2c449c28158d24a2a65c9b01f1636d8b544d29ee : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129214214) * 4e78ce4c1b10167455191ac37b615f9116bed348 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129218842) * c01756b0ca85868cc3b21b7d96e2eae8e9b76082 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129222850) * 0aa5803255b33a3575c87dc5e8b7c6916de812a7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129243177) * 1877dc719551e2a10bb03e99d2cacb6e35ad9826 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129388963) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14241) Add ARM installation step for flink e2e container test
wangxiyuan created FLINK-14241: -- Summary: Add ARM installation step for flink e2e container test Key: FLINK-14241 URL: https://issues.apache.org/jira/browse/FLINK-14241 Project: Flink Issue Type: Sub-task Components: Tests Affects Versions: 1.9.0 Reporter: wangxiyuan Fix For: 2.0.0 Flink e2e container test runs under docker, docker-compose and kubernets. But the way Flink using to install the package doesn't work on ARM. # docker-compose doesn't have ARM release bin file # minikube doesn't support ARM. So we should add the new step for ARM arch. # install docker-compose by `pip install` # use kubeadm instead of minikube -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work
flinkbot edited a comment on issue #9766: [FLINK-14018][python] Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work URL: https://github.com/apache/flink/pull/9766#issuecomment-534879961 ## CI report: * 8fffbbc4bc605ccedad7412ac2292d3488261fa3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129046428) * 120f12f13987874b1738da7db1aa370f4335830a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129105662) * 8d3ff9abe90e8e23b2bd1ec12cd99dc7444f6a8d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129205704) * 2c449c28158d24a2a65c9b01f1636d8b544d29ee : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129214214) * 4e78ce4c1b10167455191ac37b615f9116bed348 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129218842) * c01756b0ca85868cc3b21b7d96e2eae8e9b76082 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129222850) * 0aa5803255b33a3575c87dc5e8b7c6916de812a7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129243177) * 1877dc719551e2a10bb03e99d2cacb6e35ad9826 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese
flinkbot edited a comment on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese URL: https://github.com/apache/flink/pull/9749#issuecomment-534149758 ## CI report: * 141eb6db531af4b12a9506489f280f180c02a408 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128770115) * 0b96f066d276e4a483785f01335d36876e0e971c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128887957) * 5d7e1626653293ea20f935829d553de3b3e421fe : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129282782) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-10672) Task stuck while writing output to flink
[
https://issues.apache.org/jira/browse/FLINK-10672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939094#comment-16939094
]
Yun Gao commented on FLINK-10672:
-
Hi [~ibzib], very sorry for not very familiar with Beam, but I see from the
[stack log|[^jstack_66985.log]] that there is a GRPC server thread which is
also writing data. So what is the relationship between the tasks and the GRPC
？Are tasks also communicate with GRPC and do GRPC might cause cycle between
tasks ?
> Task stuck while writing output to flink
>
>
> Key: FLINK-10672
> URL: https://issues.apache.org/jira/browse/FLINK-10672
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.5.4
> Environment: OS: Debuan rodente 4.17
> Flink version: 1.5.4
> ||Key||Value||
> |jobmanager.heap.mb|1024|
> |jobmanager.rpc.address|localhost|
> |jobmanager.rpc.port|6123|
> |metrics.reporter.jmx.class|org.apache.flink.metrics.jmx.JMXReporter|
> |metrics.reporter.jmx.port|9250-9260|
> |metrics.reporters|jmx|
> |parallelism.default|1|
> |rest.port|8081|
> |taskmanager.heap.mb|1024|
> |taskmanager.numberOfTaskSlots|1|
> |web.tmpdir|/tmp/flink-web-bdb73d6c-5b9e-47b5-9ebf-eed0a7c82c26|
>
> h1. Overview
> ||Data Port||All Slots||Free Slots||CPU Cores||Physical Memory||JVM Heap
> Size||Flink Managed Memory||
> |43501|1|0|12|62.9 GB|922 MB|642 MB|
> h1. Memory
> h2. JVM (Heap/Non-Heap)
> ||Type||Committed||Used||Maximum||
> |Heap|922 MB|575 MB|922 MB|
> |Non-Heap|68.8 MB|64.3 MB|-1 B|
> |Total|991 MB|639 MB|922 MB|
> h2. Outside JVM
> ||Type||Count||Used||Capacity||
> |Direct|3,292|105 MB|105 MB|
> |Mapped|0|0 B|0 B|
> h1. Network
> h2. Memory Segments
> ||Type||Count||
> |Available|3,194|
> |Total|3,278|
> h1. Garbage Collection
> ||Collector||Count||Time||
> |G1_Young_Generation|13|336|
> |G1_Old_Generation|1|21|
>Reporter: Ankur Goenka
>Assignee: Yun Gao
>Priority: Major
> Labels: beam
> Attachments: 1uruvakHxBu.png, 3aDKQ24WvKk.png, Po89UGDn58V.png,
> jmx_dump.json, jmx_dump_detailed.json, jstack_129827.log, jstack_163822.log,
> jstack_66985.log
>
>
> I am running a fairly complex pipleline with 200+ task.
> The pipeline works fine with small data (order of 10kb input) but gets stuck
> with a slightly larger data (300kb input).
>
> The task gets stuck while writing the output toFlink, more specifically it
> gets stuck while requesting memory segment in local buffer pool. The Task
> manager UI shows that it has enough memory and memory segments to work with.
> The relevant stack trace is
> {quote}"grpc-default-executor-0" #138 daemon prio=5 os_prio=0
> tid=0x7fedb0163800 nid=0x30b7f in Object.wait() [0x7fedb4f9]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at (C/C++) 0x7fef201c7dae (Unknown Source)
> at (C/C++) 0x7fef1f2aea07 (Unknown Source)
> at (C/C++) 0x7fef1f241cd3 (Unknown Source)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xf6d56450> (a java.util.ArrayDeque)
> at
> org.apache.flink.runtime.io.network.buffer.LocalBufferPool.requestMemorySegment(LocalBufferPool.java:247)
> - locked <0xf6d56450> (a java.util.ArrayDeque)
> at
> org.apache.flink.runtime.io.network.buffer.LocalBufferPool.requestBufferBuilderBlocking(LocalBufferPool.java:204)
> at
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.requestNewBufferBuilder(RecordWriter.java:213)
> at
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.sendToTarget(RecordWriter.java:144)
> at
> org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:107)
> at
> org.apache.flink.runtime.operators.shipping.OutputCollector.collect(OutputCollector.java:65)
> at
> org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
> at
> org.apache.beam.runners.flink.translation.functions.FlinkExecutableStagePruningFunction.flatMap(FlinkExecutableStagePruningFunction.java:42)
> at
> org.apache.beam.runners.flink.translation.functions.FlinkExecutableStagePruningFunction.flatMap(FlinkExecutableStagePruningFunction.java:26)
> at
> org.apache.flink.runtime.operators.chaining.ChainedFlatMapDriver.collect(ChainedFlatMapDriver.java:80)
> at
> org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
> at
> org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction$MyDataReceiver.accept(FlinkExecutableStageFunction.java:230)
> - locked <0xf6a60bd0> (a java.lang.Object)
> at
> org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.accept(BeamFnDataInboundObserver.java:81)
> at
> org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.accept(BeamFnDataInboundObserver.java:32)
> at
>
[jira] [Assigned] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young reassigned FLINK-11491: -- Assignee: Leonard Xu (was: Kurt Young) > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-11491: --- Component/s: Table SQL / Planner > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: Timo Walther >Assignee: Leonard Xu >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939092#comment-16939092 ] Kurt Young commented on FLINK-11491: Sure and thanks [~Leonard Xu] > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Timo Walther >Assignee: Kurt Young >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] gaofeilong198810 commented on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese
gaofeilong198810 commented on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese URL: https://github.com/apache/flink/pull/9749#issuecomment-535772055 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-11493) Finalize the Blink SQL merging efforts
[ https://issues.apache.org/jira/browse/FLINK-11493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-11493: --- Fix Version/s: 1.10.0 > Finalize the Blink SQL merging efforts > -- > > Key: FLINK-11493 > URL: https://issues.apache.org/jira/browse/FLINK-11493 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Timo Walther >Assignee: Kurt Young >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue is an umbrella issue for tasks related to the finalization of the > Blink SQL merging efforts. The goal is to make the separate Blink branch > obsolete (at least from a Blink SQL perspective). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939091#comment-16939091 ] Leonard Xu commented on FLINK-11491: [~ykt836] I‘d like to improve this，cause I’m doing relate things now. could you kindly assign this to me? > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Timo Walther >Assignee: Kurt Young >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-11492) Support the full Blink SQL runtime
[ https://issues.apache.org/jira/browse/FLINK-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-11492: --- Fix Version/s: 1.10.0 > Support the full Blink SQL runtime > -- > > Key: FLINK-11492 > URL: https://issues.apache.org/jira/browse/FLINK-11492 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Timo Walther >Assignee: Kurt Young >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue is an umbrella issue for tasks related to the execution of batch > and streaming operations with a performance that is comparable to the > performance of Blink's SQL runtime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes
flinkbot edited a comment on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes URL: https://github.com/apache/flink/pull/9777#issuecomment-53537 ## CI report: * 8408299abf3e5f421ed2867ffa5629171c47af6b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129227886) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-11491) Support all TPC-DS queries
[ https://issues.apache.org/jira/browse/FLINK-11491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-11491: --- Fix Version/s: 1.10.0 > Support all TPC-DS queries > -- > > Key: FLINK-11491 > URL: https://issues.apache.org/jira/browse/FLINK-11491 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Timo Walther >Assignee: Kurt Young >Priority: Major > Fix For: 1.10.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue describes the goal of supporting all TPC-DS queries on a unified > runtime for batch and streaming. > Operations might not be executed with the full performance until changes in > other Flink core components have taken place. > This includes external catalog support and Hive integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-11490) Add an initial Blink SQL batch runtime
[ https://issues.apache.org/jira/browse/FLINK-11490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young updated FLINK-11490: --- Fix Version/s: 1.9.0 > Add an initial Blink SQL batch runtime > -- > > Key: FLINK-11490 > URL: https://issues.apache.org/jira/browse/FLINK-11490 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Runtime >Reporter: Timo Walther >Assignee: Jingsong Lee >Priority: Major > Fix For: 1.9.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue is an umbrella issue for tasks related to the merging of Blink > batch runtime features. The goal is to provide minimum viable product (MVP) > to batch users. > An exact list of batch features, their properties, and dependencies needs to > be defined. > The type system might not have been reworked at this stage. Operations might > not be executed with the full performance until changes in other Flink core > components have taken place. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] yanghua commented on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes
yanghua commented on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes URL: https://github.com/apache/flink/pull/9777#issuecomment-535767736 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes
yanghua commented on issue #9777: [FLINK-13143] Remove CheckpointExceptionHandler relevant classes URL: https://github.com/apache/flink/pull/9777#issuecomment-535767706 @pnowojski Thanks. It seems the `run travis` command did not take effects. Try again now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on issue #9776: [HOTFIX][coordination] remove the invalid parameter in handleCheckpointSuccess method
yanghua commented on issue #9776: [HOTFIX][coordination] remove the invalid parameter in handleCheckpointSuccess method URL: https://github.com/apache/flink/pull/9776#issuecomment-535766282 @lamber-ken Good catch! thanks, I will fix it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14240) The configuration added via TableConfig#configuration should be added to GlobalJobParameters for the legacy planner
Dian Fu created FLINK-14240: --- Summary: The configuration added via TableConfig#configuration should be added to GlobalJobParameters for the legacy planner Key: FLINK-14240 URL: https://issues.apache.org/jira/browse/FLINK-14240 Project: Flink Issue Type: Bug Components: Table SQL / Planner Reporter: Dian Fu Fix For: 1.10.0 TableConfig#configuration has been added in FLINK-12348 which allows users to add configurations to the underlying configuration. The added configuration will be added to GlobalJobParameters for blink planner. This should also be supported for the legacy planner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9712: [FLINK-13656] [sql-parser][table-planner][table-planner-blink] Bump Calcite dependency to 1.21.0
flinkbot edited a comment on issue #9712: [FLINK-13656] [sql-parser][table-planner][table-planner-blink] Bump Calcite dependency to 1.21.0 URL: https://github.com/apache/flink/pull/9712#issuecomment-532984287 ## CI report: * f2b7710f65f478342de389c8e099799287ddf3f9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128285030) * 19872fd562f606e9738f5b5a77f7a1d3e93a39b9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128302786) * 211f984c5380d61cf78f94e8ddcb4576f5c3f751 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128442800) * ed7ead402b73f1d5958f13c050dcda4b22dbe40a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128451919) * 33e4d2ef7f0db97a5fad81b5ef4542a299f805e6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128453915) * 2f413575dd750406c2dfe2f04b93283ca629fbc9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128456492) * 367beca92524d27a0fdbc918e8c080bb7957be3a : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129033648) * 2a3224847cee17db6416a4b86dfa3d1e394aca32 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129209349) * b0d6644397e20b880a926efc8eb488bd23c817c3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129217082) * 4c42a9eecfe5bc6d3a0a259e2d57a99234e2bd35 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129367661) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14239) Emitting the max watermark in StreamSource#run may cause it to arrive the downstream early
Haibo Sun created FLINK-14239:
-
Summary: Emitting the max watermark in StreamSource#run may cause
it to arrive the downstream early
Key: FLINK-14239
URL: https://issues.apache.org/jira/browse/FLINK-14239
Project: Flink
Issue Type: Bug
Reporter: Haibo Sun
Fix For: 1.10.0
For {{Source}}, the max watermark is emitted in {{StreamSource#run}} currently.
If some records are also output in {{close}} of {{RichSourceFunction}}, then
the max watermark will reach the downstream operator before these records.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] KarmaGYZ commented on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese
KarmaGYZ commented on issue #9749: [FLINK-14115][docs-zh] Translate DataStream Code Walkthrough to Chinese URL: https://github.com/apache/flink/pull/9749#issuecomment-535759374 @gaofeilong198810 Yes, but the indentation in EN version seems useless. I don't think we need to sync with EN version in this point. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-10705) Rework Flink Web Dashboard
[ https://issues.apache.org/jira/browse/FLINK-10705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939072#comment-16939072 ] Yadong Xie commented on FLINK-10705: updated: FLIP-75: Flink Web UI Improvement Proposal [https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit#] > Rework Flink Web Dashboard > -- > > Key: FLINK-10705 > URL: https://issues.apache.org/jira/browse/FLINK-10705 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.6.2, 1.8.0 >Reporter: Fabian Wollert >Assignee: Yadong Xie >Priority: Major > Labels: pull-request-available > Attachments: 3rdpartylicenses.txt, image-2018-10-29-09-17-24-115.png, > snapshot.jpeg > > Time Spent: 20m > Remaining Estimate: 0h > > The Flink Dashboard is very simple currently and should get updated. This is > the umbrella ticket for other tickets regarding this. Please check the > sub-tickets for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-11490) Add an initial Blink SQL batch runtime
[ https://issues.apache.org/jira/browse/FLINK-11490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee closed FLINK-11490. Resolution: Fixed > Add an initial Blink SQL batch runtime > -- > > Key: FLINK-11490 > URL: https://issues.apache.org/jira/browse/FLINK-11490 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Runtime >Reporter: Timo Walther >Assignee: Jingsong Lee >Priority: Major > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue is an umbrella issue for tasks related to the merging of Blink > batch runtime features. The goal is to provide minimum viable product (MVP) > to batch users. > An exact list of batch features, their properties, and dependencies needs to > be defined. > The type system might not have been reworked at this stage. Operations might > not be executed with the full performance until changes in other Flink core > components have taken place. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9712: [FLINK-13656] [sql-parser][table-planner][table-planner-blink] Bump Calcite dependency to 1.21.0
flinkbot edited a comment on issue #9712: [FLINK-13656] [sql-parser][table-planner][table-planner-blink] Bump Calcite dependency to 1.21.0 URL: https://github.com/apache/flink/pull/9712#issuecomment-532984287 ## CI report: * f2b7710f65f478342de389c8e099799287ddf3f9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128285030) * 19872fd562f606e9738f5b5a77f7a1d3e93a39b9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128302786) * 211f984c5380d61cf78f94e8ddcb4576f5c3f751 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128442800) * ed7ead402b73f1d5958f13c050dcda4b22dbe40a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128451919) * 33e4d2ef7f0db97a5fad81b5ef4542a299f805e6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128453915) * 2f413575dd750406c2dfe2f04b93283ca629fbc9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128456492) * 367beca92524d27a0fdbc918e8c080bb7957be3a : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129033648) * 2a3224847cee17db6416a4b86dfa3d1e394aca32 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129209349) * b0d6644397e20b880a926efc8eb488bd23c817c3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129217082) * 4c42a9eecfe5bc6d3a0a259e2d57a99234e2bd35 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-11489) Add an initial Blink SQL streaming runtime
[ https://issues.apache.org/jira/browse/FLINK-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu closed FLINK-11489. --- Fix Version/s: 1.9.0 Resolution: Fixed All the sub-tasks has been merged in 1.9.0. > Add an initial Blink SQL streaming runtime > -- > > Key: FLINK-11489 > URL: https://issues.apache.org/jira/browse/FLINK-11489 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Runtime >Reporter: Timo Walther >Assignee: Jark Wu >Priority: Major > Fix For: 1.9.0 > > > A more detailed description can be found in > [FLIP-32|https://cwiki.apache.org/confluence/display/FLINK/FLIP-32%3A+Restructure+flink-table+for+future+contributions]. > This issue is an umbrella issue for tasks related to the merging of Blink > streaming runtime features. The goal is to provide minimum viable product > (MVP) to streaming users. > An exact list of streaming features, their properties, and dependencies needs > to be defined. > The type system might not have been reworked at this stage. Operations might > not be executed with the full performance until changes in other Flink core > components have taken place. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-11993) Introduce partitionable filesystem sink
[ https://issues.apache.org/jira/browse/FLINK-11993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu closed FLINK-11993. --- Resolution: Duplicate > Introduce partitionable filesystem sink > --- > > Key: FLINK-11993 > URL: https://issues.apache.org/jira/browse/FLINK-11993 > Project: Flink > Issue Type: Task > Components: Table SQL / Ecosystem >Reporter: Jing Zhang >Priority: Major > > Introduce partitionable filesystem sink, > 1. Add partition trait for filesystem connector > 2. All the filesystem formats can be declared as partitioned through new DDL > grammar. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14230) Change the endInput call of the downstream operator to after the upstream operator closes
[ https://issues.apache.org/jira/browse/FLINK-14230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Sun updated FLINK-14230: -- Summary: Change the endInput call of the downstream operator to after the upstream operator closes (was: Change the endinput call of the downstream operator to after the upstream operator closes) > Change the endInput call of the downstream operator to after the upstream > operator closes > -- > > Key: FLINK-14230 > URL: https://issues.apache.org/jira/browse/FLINK-14230 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Task >Reporter: Haibo Sun >Assignee: Haibo Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] Mrart removed a comment on issue #9773: [FLINK-14210][metrics]support connect timeout and write timeout confi…
Mrart removed a comment on issue #9773: [FLINK-14210][metrics]support connect timeout and write timeout confi… URL: https://github.com/apache/flink/pull/9773#issuecomment-535409349 @1u0 Could you help me review the other PR （https://github.com/apache/flink/pull/9542） This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Mrart removed a comment on issue #9542: [FLINK-13873][metrics] Change the column family as tags for influxdb …
Mrart removed a comment on issue #9542: [FLINK-13873][metrics] Change the column family as tags for influxdb … URL: https://github.com/apache/flink/pull/9542#issuecomment-534871400 @zentol Could you help review again，I have add A config option。I am sorry for delay。 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14175) Upgrade KPL version in flink-connector-kinesis to fix application OOM
[ https://issues.apache.org/jira/browse/FLINK-14175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhilasha Seth updated FLINK-14175: --- Description: The [KPL version|https://github.com/apache/flink/blob/release-1.9/flink-connectors/flink-connector-kinesis/pom.xml#L38] (0.12.9) used by flink-connector-kinesis in the affected Flink versions has a thread leak bug that causes applications to run out of memory after frequent restarts: KPL Issue - [https://github.com/awslabs/amazon-kinesis-producer/issues/224] Fix - [https://github.com/awslabs/amazon-kinesis-producer/pull/225/files] Upgrading KPL to 0.12.10 or higher is necessary to avoid this issue. The recommended version to upgrade would be the latest (0.13.1) Note that KPL version in Flink 1.10.0 has been updated to the latest version (0.13.1): https://issues.apache.org/jira/browse/FLINK-12847 was: The [KPL version|https://github.com/apache/flink/blob/release-1.9/flink-connectors/flink-connector-kinesis/pom.xml#L38] currently in use (0.12.9) has a thread leak bug that causes applications to run out of memory after frequent restarts: KPL Issue - [https://github.com/awslabs/amazon-kinesis-producer/issues/224] Fix - [https://github.com/awslabs/amazon-kinesis-producer/pull/225/files] Upgrading KPL to 0.12.10 or higher is necessary to avoid this issue. > Upgrade KPL version in flink-connector-kinesis to fix application OOM > - > > Key: FLINK-14175 > URL: https://issues.apache.org/jira/browse/FLINK-14175 > Project: Flink > Issue Type: Bug > Components: Connectors / Kinesis >Affects Versions: 1.6.3, 1.6.4, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0 > Environment: [link title|http://example.com][link > title|http://example.com] >Reporter: Abhilasha Seth >Priority: Major > > The [KPL > version|https://github.com/apache/flink/blob/release-1.9/flink-connectors/flink-connector-kinesis/pom.xml#L38] > (0.12.9) used by flink-connector-kinesis in the affected Flink versions has > a thread leak bug that causes applications to run out of memory after > frequent restarts: > KPL Issue - [https://github.com/awslabs/amazon-kinesis-producer/issues/224] > Fix - [https://github.com/awslabs/amazon-kinesis-producer/pull/225/files] > Upgrading KPL to 0.12.10 or higher is necessary to avoid this issue. The > recommended version to upgrade would be the latest (0.13.1) > Note that KPL version in Flink 1.10.0 has been updated to the latest version > (0.13.1): https://issues.apache.org/jira/browse/FLINK-12847 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-14197) Increasing trend for state size of keyed stream using ProcessWindowFunction with ProcessingTimeSessionWindows
[
https://issues.apache.org/jira/browse/FLINK-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938961#comment-16938961
]
Kenneth William Krugler commented on FLINK-14197:
-
I would suggest posting to the mailing list, as you'll (a) get input from the
larger Flink community, and (b) the discussion is more visible and thus useful
to other users.
> Increasing trend for state size of keyed stream using ProcessWindowFunction
> with ProcessingTimeSessionWindows
> -
>
> Key: FLINK-14197
> URL: https://issues.apache.org/jira/browse/FLINK-14197
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / State Backends
>Affects Versions: 1.9.0
> Environment: Tested with:
> * Local Flink Mini Cluster running from IDE
> * Flink standalone cluster run in docker
>Reporter: Oliver Kostera
>Priority: Major
>
> I'm using *ProcessWindowFunction* in a keyed stream with the following
> definition:
> {code:java}
> final SingleOutputStreamOperator processWindowFunctionStream
> =
>
> keyedStream.window(ProcessingTimeSessionWindows.withGap(Time.milliseconds(100)))
> .process(new
> CustomProcessWindowFunction()).uid(PROCESS_WINDOW_FUNCTION_OPERATOR_ID)
> .name("Process window function");
> {code}
> My checkpointing configuration is set to use RocksDB state backend with
> incremental checkpointing and EXACTLY_ONCE mode.
> In a runtime I noticed that even though data ingestion is static - same keys
> and frequency of messages the size of the process window operator keeps
> increasing. I tried to reproduce it with minimal similar setup here:
> https://github.com/loliver1234/flink-process-window-function and was
> successful to do so.
> Testing conditions:
> - RabbitMQ source with Exactly-once guarantee and 65k prefetch count
> - RabbitMQ sink to collect messages
> - Simple ProcessWindowFunction that only pass messages through
> - Stream time characteristic set to TimeCharacteristic.ProcessingTime
> Testing scenario:
> - Start flink job and check initial state size - State Size: 127 KB
> - Start sending messages, 1000 same unique keys every 1s (they are not
> falling into defined time window gap set to 100ms, each message should create
> new window)
> - State of the process window operator keeps increasing - after 1mln messages
> state ended up to be around 2mb
> - Stop sending messages and wait till rabbit queue is fully consumed and few
> checkpoints go by
> - Was expected to see state size to decrease to base value but it stayed at
> 2mb
> - Continue to send messages with the same keys and state kept increasing
> trend.
> What I checked:
> - Registration and deregistration of timestamps set for time windows - each
> registration matched its deregistration
> - Checked that in fact there are no window merges
> - Tried custom Trigger disabling window merges and setting onProcessingTime
> trigger to TriggerResult.FIRE_AND_PURGE - same state behavior
> On staging environment, we noticed that state for that operator keeps
> increasing indefinitely, after some months reaching even 1,5gb for 100k
> unique keys
> Flink commit id: 9c32ed9
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9772: Flink 14199: Only use dedicated/named classes for mailbox letters.
flinkbot edited a comment on issue #9772: Flink 14199: Only use dedicated/named classes for mailbox letters. URL: https://github.com/apache/flink/pull/9772#issuecomment-534982677 ## CI report: * d5e85b7a984024ea1aee2ce8bdf527d89c21e88e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129089795) * cb97c18a494ce9306a830a95f395e606a4a1a15e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129326891) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration
flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration URL: https://github.com/apache/flink/pull/8303#issuecomment-511684151 ## CI report: * 6a7ca58b4a04f6dce250045e021702e67e82b893 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119421914) * 4d38a8df0d59734c4b2386689a2f17b9f2b44b12 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441376) * 9c14836f8639e98d58cf7bb32e38b938b3843994 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119577044) * 76186776c5620598a19234245bbd05dfdfb1c62c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120113740) * 628ca7b316ad3968c90192a47a84dd01f26e2578 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122381349) * d204a725ff3c8a046cbd1b84e34d9e3ae8aafeac : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123620485) * 143efadbdb6c4681569d5b412a175edfb1633b85 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123637809) * b78b64a82ed2a9a92886095ec42f06d5082ad830 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123671219) * 5145a0b9d6b320456bb971d96b9cc47707c8fd28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125476639) * 0d4d944c28c59ca1caa6c453c347ec786b40d245 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125762588) * 91552c3804f5e96cc573e6ed48756f2b54c037d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125844084) * fbf6d2850b6aa7c303981e6f5b24b0da0956b820 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126086321) * 45282a2139ec3c7196ba10b6fb829b0ec275 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126350641) * cc515180d99fa35b482c61c2e264e77760e55dc5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126354591) * b8341665cb9b8bf3559d93ea74715ddac3ca67b2 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127617990) * 462a8cf7286f63afcfb5cecbce37df232d3f8ec4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127623432) * 51397b3c71d8dd4549eba379b2b873d2e523e268 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127625943) * 60e035ab71050b367fcf5411953a75a8a3b32a75 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127672092) * 978c0723caaf584fe91b59cc9aabbd77815907c8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129038982) * 5784c98687a35b6437a3948992f1302f8e849c5d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129143114) * 841b65e4522c78c55c644f1aaf9c81c2d2c9efd7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129318099) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9772: Flink 14199: Only use dedicated/named classes for mailbox letters.
flinkbot edited a comment on issue #9772: Flink 14199: Only use dedicated/named classes for mailbox letters. URL: https://github.com/apache/flink/pull/9772#issuecomment-534982677 ## CI report: * d5e85b7a984024ea1aee2ce8bdf527d89c21e88e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129089795) * cb97c18a494ce9306a830a95f395e606a4a1a15e : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration
flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration URL: https://github.com/apache/flink/pull/8303#issuecomment-511684151 ## CI report: * 6a7ca58b4a04f6dce250045e021702e67e82b893 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119421914) * 4d38a8df0d59734c4b2386689a2f17b9f2b44b12 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441376) * 9c14836f8639e98d58cf7bb32e38b938b3843994 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119577044) * 76186776c5620598a19234245bbd05dfdfb1c62c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120113740) * 628ca7b316ad3968c90192a47a84dd01f26e2578 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122381349) * d204a725ff3c8a046cbd1b84e34d9e3ae8aafeac : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123620485) * 143efadbdb6c4681569d5b412a175edfb1633b85 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123637809) * b78b64a82ed2a9a92886095ec42f06d5082ad830 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123671219) * 5145a0b9d6b320456bb971d96b9cc47707c8fd28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125476639) * 0d4d944c28c59ca1caa6c453c347ec786b40d245 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125762588) * 91552c3804f5e96cc573e6ed48756f2b54c037d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125844084) * fbf6d2850b6aa7c303981e6f5b24b0da0956b820 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126086321) * 45282a2139ec3c7196ba10b6fb829b0ec275 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126350641) * cc515180d99fa35b482c61c2e264e77760e55dc5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126354591) * b8341665cb9b8bf3559d93ea74715ddac3ca67b2 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127617990) * 462a8cf7286f63afcfb5cecbce37df232d3f8ec4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127623432) * 51397b3c71d8dd4549eba379b2b873d2e523e268 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127625943) * 60e035ab71050b367fcf5411953a75a8a3b32a75 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127672092) * 978c0723caaf584fe91b59cc9aabbd77815907c8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129038982) * 5784c98687a35b6437a3948992f1302f8e849c5d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129143114) * 841b65e4522c78c55c644f1aaf9c81c2d2c9efd7 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129318099) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration
flinkbot edited a comment on issue #8303: [FLINK-12343] [flink-yarn] add file replication config for yarn configuration URL: https://github.com/apache/flink/pull/8303#issuecomment-511684151 ## CI report: * 6a7ca58b4a04f6dce250045e021702e67e82b893 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119421914) * 4d38a8df0d59734c4b2386689a2f17b9f2b44b12 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441376) * 9c14836f8639e98d58cf7bb32e38b938b3843994 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119577044) * 76186776c5620598a19234245bbd05dfdfb1c62c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120113740) * 628ca7b316ad3968c90192a47a84dd01f26e2578 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122381349) * d204a725ff3c8a046cbd1b84e34d9e3ae8aafeac : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123620485) * 143efadbdb6c4681569d5b412a175edfb1633b85 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123637809) * b78b64a82ed2a9a92886095ec42f06d5082ad830 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123671219) * 5145a0b9d6b320456bb971d96b9cc47707c8fd28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125476639) * 0d4d944c28c59ca1caa6c453c347ec786b40d245 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125762588) * 91552c3804f5e96cc573e6ed48756f2b54c037d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125844084) * fbf6d2850b6aa7c303981e6f5b24b0da0956b820 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126086321) * 45282a2139ec3c7196ba10b6fb829b0ec275 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126350641) * cc515180d99fa35b482c61c2e264e77760e55dc5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/126354591) * b8341665cb9b8bf3559d93ea74715ddac3ca67b2 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127617990) * 462a8cf7286f63afcfb5cecbce37df232d3f8ec4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127623432) * 51397b3c71d8dd4549eba379b2b873d2e523e268 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127625943) * 60e035ab71050b367fcf5411953a75a8a3b32a75 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127672092) * 978c0723caaf584fe91b59cc9aabbd77815907c8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129038982) * 5784c98687a35b6437a3948992f1302f8e849c5d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129143114) * 841b65e4522c78c55c644f1aaf9c81c2d2c9efd7 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11
flinkbot edited a comment on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11 URL: https://github.com/apache/flink/pull/9775#issuecomment-535327571 ## CI report: * 7ca40f4410862da362c16e5a9410ca692854451f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129209325) * 18531c05a028f98d44a072705392b3c966493416 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306707) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14186) Nightly e2e-misc-jdk11 fails complaining unrecognized VM option
[
https://issues.apache.org/jira/browse/FLINK-14186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938825#comment-16938825
]
Yun Gao commented on FLINK-14186:
-
With the discussion under the [PR|[https://github.com/apache/flink/pull/9775]],
we should indeed skip the ES 2.3.5 cases when running with JDK 11. The ES 2.3.5
cases is used in three scripts and only _tools/travis/splits/split_misc.sh_ is
tested with JDK 11, thus we skip the tests if we found it is running with JDK
11.
> Nightly e2e-misc-jdk11 fails complaining unrecognized VM option
> ---
>
> Key: FLINK-14186
> URL: https://issues.apache.org/jira/browse/FLINK-14186
> Project: Flink
> Issue Type: Bug
> Components: Tests
>Reporter: Yu Li
>Assignee: Yun Gao
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As titled, this failure reproduces stably in recent nightly runs, such as
> https://api.travis-ci.org/v3/job/588201921/log.txt . From the log we could
> see message like below:
> {noformat}
> Unrecognized VM option 'UseParNewGC'
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-14186) Nightly e2e-misc-jdk11 fails complaining unrecognized VM option
[
https://issues.apache.org/jira/browse/FLINK-14186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938235#comment-16938235
]
Yun Gao edited comment on FLINK-14186 at 9/26/19 5:19 PM:
--
The reason of this issue is that -XX:+UseParNewGC [has been removed from the
JDK11|[https://bugs.openjdk.java.net/browse/JDK-8151084]] as the garbage
collector algorithms have evolved. However, Elastic Search 2.3.5 has
configured this option in its startup script. Therefore, the Elastic Search
2.3.5 will fail to start on JDK11, which cause the test fail.
To fix this problem, we can set environment variable
JAVA_OPTS="-XX:+IgnoreUnrecognizedVMOptions" when starting the Elastic Search
cluster. This environment variable parameter is provided by Elastic Search
2.3.5 script and IgnoreUnrecognizedVMOptions let JVM ignores unrecognized
options instead throw an error.
The Elastic Search 5.x and 6.x have removed -XX:+UseParNewGC from the default
JVM config and thus they should not met this issue. Besides, in these two
version JAVA_OPTS is deprecated and ES_JAVA_OPTS is used instead. Since the two
version currently do not have problem, I think we do not need to also configure
ES_JAVA_OPTS now.
For reference, the code involved in Flink is:
# e2e - misc - jdk11 configured in .travis.yml have finally called
_tools/travis/splits/split_misc.sh_
# _split_misc.sh_ have three ElasticSearch related tests. They all finally
called flink-end-to-end-tests/test-scripts/test_streaming_elasticsearch.sh.
# test_streaming_elasticsearch.sh further called the function
setup_elasticsearch included from
flink-end-to-end-tests/test-scripts/elasticsearch-common.sh
# setup_elasticsearch downloads the specified ElasticSearch distribution
package and unzip it. Then it calls "_$elasticsearchDir/bin/elasticsearch &"_
to start the _ _ElasticSearch_ _ cluster. We should then modify the startup
environment here.
was (Author: gaoyunhaii):
The reason of this issue is that -XX:+UseParNewGC [has been removed from the
JDK11|[https://bugs.openjdk.java.net/browse/JDK-8151084]] as the garbage
collector algorithms have evolved. However, Elastic Search 2.3.5 has
configured this option in its startup script. Therefore, the Elastic Search
2.3.5 will fail to start on JDK11, which cause the test fail.
To fix this problem, we can set environment variable
JAVA_OPTS="-XX:+IgnoreUnrecognizedVMOptions" when starting the Elastic Search
cluster. This environment variable parameter is provided by Elastic Search
2.3.5 script and IgnoreUnrecognizedVMOptions let JVM ignores unrecognized
options instead throw an error.
The Elastic Search 5.x and 6.x have removed -XX:+UseParNewGC from the default
JVM config and thus they should not met this issue. Besides, in these two
version JAVA_OPTS is deprecated and ES_JAVA_OPTS is used instead. Since the two
version currently do not have problem, I think we do not need to also configure
ES_JAVA_OPTS now.
For reference, the code involved in Flink is:
# flink-end-to-end-tests/run-nightly-tests.sh have three ElasticSearch related
tests. They all finally called
flink-end-to-end-tests/test-scripts/test_streaming_elasticsearch.sh.
# test_streaming_elasticsearch.sh further called the function
setup_elasticsearch included from
flink-end-to-end-tests/test-scripts/elasticsearch-common.sh
# setup_elasticsearch downloads the specified ElasticSearch distribution
package and unzip it. Then it calls "_$elasticsearchDir/bin/elasticsearch &"_
to start the __ ElasticSearch __ cluster. We should then modify the startup
environment here.
> Nightly e2e-misc-jdk11 fails complaining unrecognized VM option
> ---
>
> Key: FLINK-14186
> URL: https://issues.apache.org/jira/browse/FLINK-14186
> Project: Flink
> Issue Type: Bug
> Components: Tests
>Reporter: Yu Li
>Assignee: Yun Gao
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As titled, this failure reproduces stably in recent nightly runs, such as
> https://api.travis-ci.org/v3/job/588201921/log.txt . From the log we could
> see message like below:
> {noformat}
> Unrecognized VM option 'UseParNewGC'
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] gaoyunhaii commented on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11
gaoyunhaii commented on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11 URL: https://github.com/apache/flink/pull/9775#issuecomment-535602029 Hi @tillrohrmann , I have updated the PR to skip the ES 2.3.5 when running with JDK11. I have manually [triggered the tests ](https://travis-ci.org/gaoyunhaii/flink/jobs/590039818) on Elastic Search and the ES 2.3.5 case was skipped indeed (The [test script](https://github.com/gaoyunhaii/flink/blob/fix_es_tests_2_run_nightly/tools/travis/splits/split_misc.sh) is modified to only run with Elastic Search cases, or the test will exceed 50 min and will be terminated by Travis). Very thanks @tillrohrmann and @TisonKun for the review and suggestions! :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9412: [FLINK-13675][ml] Add class of Vector Slice Mapper
flinkbot edited a comment on issue #9412: [FLINK-13675][ml] Add class of Vector Slice Mapper URL: https://github.com/apache/flink/pull/9412#issuecomment-519965864 ## CI report: * e9ff64826d561117cbd984df4201d15e0523b091 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122613554) * 8e513594aaea05df91f6d2dd17088c4adb2fd1e9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125894095) * 55e76e5d450f112e1500036c79cbd4743fcd5889 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129307107) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9410: [FLINK-13673][ml] Add class of Vector Normalize Mapper
flinkbot edited a comment on issue #9410: [FLINK-13673][ml] Add class of Vector Normalize Mapper URL: https://github.com/apache/flink/pull/9410#issuecomment-519963431 ## CI report: * 06b26ce69e490794f3bcda79766f8edc9d6e48ce : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122612649) * a5e7c30d865ea6ab35644743c9477a940ec9de65 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893914) * e08c7de638ed3770aa4b6808d68704b7672830b3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129307038) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9411: [FLINK-13674][ml] Add class of Vector Size Hint Mapper
flinkbot edited a comment on issue #9411: [FLINK-13674][ml] Add class of Vector Size Hint Mapper URL: https://github.com/apache/flink/pull/9411#issuecomment-519965855 ## CI report: * 4af38a946a55d1d6b68d7d249f87a65239c947f7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122613586) * 5e9e66fcf02dc111a807661502761e1a67a043eb : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893971) * 9cffa68773935aeb4fc7f92c87cafa940ea57944 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129307069) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9408: [FLINK-13671][ml] Add class for VectorEleWiseProductMapper
flinkbot edited a comment on issue #9408: [FLINK-13671][ml] Add class for VectorEleWiseProductMapper URL: https://github.com/apache/flink/pull/9408#issuecomment-519961164 ## CI report: * fbf9704fe2812d28d7a6b9ca6272a260e91089b3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122611669) * a2091168929daf552e6e839c9ef179c983192fcf : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893825) * b14008db52a3171fc39b7863f440741a9c536039 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306952) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9405: [FLINK-13668][ml] Add abstract classes for three typical scenarios of (Flat)Mapper
flinkbot edited a comment on issue #9405: [FLINK-13668][ml] Add abstract classes for three typical scenarios of (Flat)Mapper URL: https://github.com/apache/flink/pull/9405#issuecomment-519901201 ## CI report: * c1fedc4de6a92bc2c5f84d0e0c3038f29fd8944f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122585906) * 7f235ecf56417da32c78bb5d064c9d833e1d5261 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893682) * 35b7c49d3f0961f28e71ebcc4fa446dd2a4b1d88 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306881) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14238) ParquetPojoInputFormat logging bug
[
https://issues.apache.org/jira/browse/FLINK-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Lonergan updated FLINK-14238:
--
Issue Type: Bug (was: Improvement)
> ParquetPojoInputFormat logging bug
> --
>
> Key: FLINK-14238
> URL: https://issues.apache.org/jira/browse/FLINK-14238
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.9.0
>Reporter: John Lonergan
>Priority: Minor
>
> Line 64: LOG.error("Fields number is %d", getFieldNames().length);
> Generates a bunch of spurious error logging.
> It is a coding error
> - should probably have been trace but ...
> - is actually a coding error as "%d" isn't a valid format for the logger so
> we never see the length
> Recommend - This line should be deleted.
> https://github.com/apache/flink/blob/d015ce7a3b86a529f4db79ed8ac8dbe28c62d6b8/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/ParquetPojoInputFormat.java#L64
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-14238) ParquetPojoInputFormat logging bug
John Lonergan created FLINK-14238:
-
Summary: ParquetPojoInputFormat logging bug
Key: FLINK-14238
URL: https://issues.apache.org/jira/browse/FLINK-14238
Project: Flink
Issue Type: Improvement
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Affects Versions: 1.9.0
Reporter: John Lonergan
Line 64: LOG.error("Fields number is %d", getFieldNames().length);
Generates a bunch of spurious error logging.
It is a coding error
- should probably have been trace but ...
- is actually a coding error as "%d" isn't a valid format for the logger so we
never see the length
Recommend - This line should be deleted.
https://github.com/apache/flink/blob/d015ce7a3b86a529f4db79ed8ac8dbe28c62d6b8/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/ParquetPojoInputFormat.java#L64
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9780: [FLINK-14042] [flink-table-planner] Fix TemporalTable row schema always inferred as nullable
flinkbot edited a comment on issue #9780: [FLINK-14042] [flink-table-planner] Fix TemporalTable row schema always inferred as nullable URL: https://github.com/apache/flink/pull/9780#issuecomment-535571919 ## CI report: * 8c0e3b4b94421c9f5ac4cfa470d0859e88471f38 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129303252) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9412: [FLINK-13675][ml] Add class of Vector Slice Mapper
flinkbot edited a comment on issue #9412: [FLINK-13675][ml] Add class of Vector Slice Mapper URL: https://github.com/apache/flink/pull/9412#issuecomment-519965864 ## CI report: * e9ff64826d561117cbd984df4201d15e0523b091 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122613554) * 8e513594aaea05df91f6d2dd17088c4adb2fd1e9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125894095) * 55e76e5d450f112e1500036c79cbd4743fcd5889 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129307107) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9409: [FLINK-13672][ml] Add class for VectorInteractionMapper
flinkbot edited a comment on issue #9409: [FLINK-13672][ml] Add class for VectorInteractionMapper URL: https://github.com/apache/flink/pull/9409#issuecomment-519961173 ## CI report: * 9b5e1620eb70dd95996ec3e8b7bebe1f38ac89a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122611642) * 1d402f4ed866db5e9bbd35496949f8fdc8197c5e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893871) * c95ee221f9ba6098a7f9e70b66c46a6667a44d8c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306996) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9411: [FLINK-13674][ml] Add class of Vector Size Hint Mapper
flinkbot edited a comment on issue #9411: [FLINK-13674][ml] Add class of Vector Size Hint Mapper URL: https://github.com/apache/flink/pull/9411#issuecomment-519965855 ## CI report: * 4af38a946a55d1d6b68d7d249f87a65239c947f7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122613586) * 5e9e66fcf02dc111a807661502761e1a67a043eb : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893971) * 9cffa68773935aeb4fc7f92c87cafa940ea57944 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129307069) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9410: [FLINK-13673][ml] Add class of Vector Normalize Mapper
flinkbot edited a comment on issue #9410: [FLINK-13673][ml] Add class of Vector Normalize Mapper URL: https://github.com/apache/flink/pull/9410#issuecomment-519963431 ## CI report: * 06b26ce69e490794f3bcda79766f8edc9d6e48ce : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122612649) * a5e7c30d865ea6ab35644743c9477a940ec9de65 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893914) * e08c7de638ed3770aa4b6808d68704b7672830b3 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129307038) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9408: [FLINK-13671][ml] Add class for VectorEleWiseProductMapper
flinkbot edited a comment on issue #9408: [FLINK-13671][ml] Add class for VectorEleWiseProductMapper URL: https://github.com/apache/flink/pull/9408#issuecomment-519961164 ## CI report: * fbf9704fe2812d28d7a6b9ca6272a260e91089b3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122611669) * a2091168929daf552e6e839c9ef179c983192fcf : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893825) * b14008db52a3171fc39b7863f440741a9c536039 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129306952) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9407: [FLINK-13670][ml] Add class for VectorAssemblerMapper
flinkbot edited a comment on issue #9407: [FLINK-13670][ml] Add class for VectorAssemblerMapper URL: https://github.com/apache/flink/pull/9407#issuecomment-51995 ## CI report: * bf69f9f25f8a62a494f530e7cba75879ac002412 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122610736) * e92b935e874524c967a9b0296833872c602d0a33 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893754) * 40621b28dc8bd9098d56e53b8893824e82205614 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306927) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9405: [FLINK-13668][ml] Add abstract classes for three typical scenarios of (Flat)Mapper
flinkbot edited a comment on issue #9405: [FLINK-13668][ml] Add abstract classes for three typical scenarios of (Flat)Mapper URL: https://github.com/apache/flink/pull/9405#issuecomment-519901201 ## CI report: * c1fedc4de6a92bc2c5f84d0e0c3038f29fd8944f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122585906) * 7f235ecf56417da32c78bb5d064c9d833e1d5261 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893682) * 35b7c49d3f0961f28e71ebcc4fa446dd2a4b1d88 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129306881) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9578: [FLINK-13924][ml] Add summarizer and summary for sparse vector and dense vector.
flinkbot edited a comment on issue #9578: [FLINK-13924][ml] Add summarizer and summary for sparse vector and dense vector. URL: https://github.com/apache/flink/pull/9578#issuecomment-526650851 ## CI report: * 8da3386df1a24f8e5a26a2314f1f5c283f3e8a09 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125254089) * 3f1efc6a08bd46620f2581ebba8028e13b2abb08 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306832) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9577: [FLINK-13923][ml] Add summarizer and summary for table.
flinkbot edited a comment on issue #9577: [FLINK-13923][ml] Add summarizer and summary for table. URL: https://github.com/apache/flink/pull/9577#issuecomment-526650805 ## CI report: * 99cc6a255384d08ee885f86f9f8578581354d38a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/125254035) * d4bb82f7b0029e2d8588f557762b5f3bec27a59d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306783) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9574: [FLINK-13918][ml] Add class for TokenizerMapper.
flinkbot edited a comment on issue #9574: [FLINK-13918][ml] Add class for TokenizerMapper. URL: https://github.com/apache/flink/pull/9574#issuecomment-526599548 ## CI report: * 960f4fef06446118e7ccb225754aef0620f6983f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125232602) * 7abeec172e04bdcd37e643c4eddc1ffa61f82643 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306741) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11
flinkbot edited a comment on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11 URL: https://github.com/apache/flink/pull/9775#issuecomment-535327571 ## CI report: * 7ca40f4410862da362c16e5a9410ca692854451f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129209325) * 18531c05a028f98d44a072705392b3c966493416 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129306707) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9501: [FLINK-12697] [State Backends] Support on-disk state storage for spill-able heap backend
flinkbot edited a comment on issue #9501: [FLINK-12697] [State Backends] Support on-disk state storage for spill-able heap backend URL: https://github.com/apache/flink/pull/9501#issuecomment-523769032 ## CI report: * 6107a00876759dcef075592c8bfa56764f66b6fa : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124167653) * 4263f76529de2d80aa3554ee91c206fc0f9152e5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124179375) * befcc101f408baca6bb2bf83eea14cb4a9f496bd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124863326) * df676fa3d969772b8995538adc147bbb86c36189 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124881729) * 24328ec47f13f4a445f3c431ea6025a368aaedc1 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125238415) * 38d3ce4f2e78ec98aceea6a9d1f958f903ae5f3f : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128211309) * cfbb8387223123f660bc5d5b1f02ec36616b6c11 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128339902) * 2819c8dd4ede0cb088ac20ba3715e8d8bafb124e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129069417) * 1cf8f18b890dcbb0e19870cb4be6efc1a600fea5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129082047) * 20e2c58d7ba5b385a6b6e898a49b8ce9a0c81243 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129230744) * 8af6229634471ccd064c02b71fc60bc5262c8ab1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129306669) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9733: [FLINK-14154][ml] Add the class for multivariate Gaussian Distribution.
flinkbot edited a comment on issue #9733: [FLINK-14154][ml] Add the class for multivariate Gaussian Distribution. URL: https://github.com/apache/flink/pull/9733#issuecomment-533547601 ## CI report: * ea3d0cd6da22ebd2846bf0f3a2fa1d9f21964675 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128511616) * ccc157ea824dbb72054b73e8edbb1b9a79656bd5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129299120) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add an implementation of pipeline's api
xuyang1706 commented on a change in pull request #9184: [FLINK-13339][ml] Add
an implementation of pipeline's api
URL: https://github.com/apache/flink/pull/9184#discussion_r328711228
##
File path:
flink-ml-parent/flink-ml-lib/src/main/java/org/apache/flink/ml/common/AlgoOperator.java
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.flink.ml.common;
+
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.ml.api.misc.param.Params;
+import org.apache.flink.ml.api.misc.param.WithParams;
+import org.apache.flink.ml.batchoperator.source.TableSourceBatchOp;
+import org.apache.flink.ml.params.shared.HasMLEnvironmentId;
+import org.apache.flink.ml.streamoperator.source.TableSourceStreamOp;
+import org.apache.flink.table.api.Table;
+import org.apache.flink.table.api.TableSchema;
+import org.apache.flink.table.api.internal.TableImpl;
+import org.apache.flink.table.api.java.StreamTableEnvironment;
+
+import java.io.Serializable;
+
+/**
+ * Base class for algorithm operators.
+ *
+ * Hold a Table as its output.
+ *
+ * @param The class type of the {@link AlgoOperator} implementation itself
+ */
+public abstract class AlgoOperator>
+ implements WithParams, HasMLEnvironmentId, Serializable {
+
+ /**
+* Params for algorithms.
+*/
+ private Params params;
+
+ /**
+* The table held by operator.
+*/
+ private Table output = null;
+
+ /**
+* The side outputs of operator that be similar to the stream's side
outputs.
+*/
+ private Table[] sideOutputs = null;
+
+ /**
+* Construct the operator with empty Params.
+*/
+ protected AlgoOperator() {
Review comment:
Thanks, we have the `setOutput` and can set the output in any method of the
operator
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9412: [FLINK-13675][ml] Add class of Vector Slice Mapper
flinkbot edited a comment on issue #9412: [FLINK-13675][ml] Add class of Vector Slice Mapper URL: https://github.com/apache/flink/pull/9412#issuecomment-519965864 ## CI report: * e9ff64826d561117cbd984df4201d15e0523b091 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122613554) * 8e513594aaea05df91f6d2dd17088c4adb2fd1e9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125894095) * 55e76e5d450f112e1500036c79cbd4743fcd5889 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9411: [FLINK-13674][ml] Add class of Vector Size Hint Mapper
flinkbot edited a comment on issue #9411: [FLINK-13674][ml] Add class of Vector Size Hint Mapper URL: https://github.com/apache/flink/pull/9411#issuecomment-519965855 ## CI report: * 4af38a946a55d1d6b68d7d249f87a65239c947f7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122613586) * 5e9e66fcf02dc111a807661502761e1a67a043eb : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893971) * 9cffa68773935aeb4fc7f92c87cafa940ea57944 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9409: [FLINK-13672][ml] Add class for VectorInteractionMapper
flinkbot edited a comment on issue #9409: [FLINK-13672][ml] Add class for VectorInteractionMapper URL: https://github.com/apache/flink/pull/9409#issuecomment-519961173 ## CI report: * 9b5e1620eb70dd95996ec3e8b7bebe1f38ac89a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122611642) * 1d402f4ed866db5e9bbd35496949f8fdc8197c5e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893871) * c95ee221f9ba6098a7f9e70b66c46a6667a44d8c : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9408: [FLINK-13671][ml] Add class for VectorEleWiseProductMapper
flinkbot edited a comment on issue #9408: [FLINK-13671][ml] Add class for VectorEleWiseProductMapper URL: https://github.com/apache/flink/pull/9408#issuecomment-519961164 ## CI report: * fbf9704fe2812d28d7a6b9ca6272a260e91089b3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122611669) * a2091168929daf552e6e839c9ef179c983192fcf : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893825) * b14008db52a3171fc39b7863f440741a9c536039 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9410: [FLINK-13673][ml] Add class of Vector Normalize Mapper
flinkbot edited a comment on issue #9410: [FLINK-13673][ml] Add class of Vector Normalize Mapper URL: https://github.com/apache/flink/pull/9410#issuecomment-519963431 ## CI report: * 06b26ce69e490794f3bcda79766f8edc9d6e48ce : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122612649) * a5e7c30d865ea6ab35644743c9477a940ec9de65 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893914) * e08c7de638ed3770aa4b6808d68704b7672830b3 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9405: [FLINK-13668][ml] Add abstract classes for three typical scenarios of (Flat)Mapper
flinkbot edited a comment on issue #9405: [FLINK-13668][ml] Add abstract classes for three typical scenarios of (Flat)Mapper URL: https://github.com/apache/flink/pull/9405#issuecomment-519901201 ## CI report: * c1fedc4de6a92bc2c5f84d0e0c3038f29fd8944f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122585906) * 7f235ecf56417da32c78bb5d064c9d833e1d5261 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893682) * 35b7c49d3f0961f28e71ebcc4fa446dd2a4b1d88 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9407: [FLINK-13670][ml] Add class for VectorAssemblerMapper
flinkbot edited a comment on issue #9407: [FLINK-13670][ml] Add class for VectorAssemblerMapper URL: https://github.com/apache/flink/pull/9407#issuecomment-51995 ## CI report: * bf69f9f25f8a62a494f530e7cba75879ac002412 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122610736) * e92b935e874524c967a9b0296833872c602d0a33 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125893754) * 40621b28dc8bd9098d56e53b8893824e82205614 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9574: [FLINK-13918][ml] Add class for TokenizerMapper.
flinkbot edited a comment on issue #9574: [FLINK-13918][ml] Add class for TokenizerMapper. URL: https://github.com/apache/flink/pull/9574#issuecomment-526599548 ## CI report: * 960f4fef06446118e7ccb225754aef0620f6983f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125232602) * 7abeec172e04bdcd37e643c4eddc1ffa61f82643 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9577: [FLINK-13923][ml] Add summarizer and summary for table.
flinkbot edited a comment on issue #9577: [FLINK-13923][ml] Add summarizer and summary for table. URL: https://github.com/apache/flink/pull/9577#issuecomment-526650805 ## CI report: * 99cc6a255384d08ee885f86f9f8578581354d38a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/125254035) * d4bb82f7b0029e2d8588f557762b5f3bec27a59d : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9572: [FLINK-13916][ml] Add class for NGramMapper.
flinkbot edited a comment on issue #9572: [FLINK-13916][ml] Add class for NGramMapper. URL: https://github.com/apache/flink/pull/9572#issuecomment-526595476 ## CI report: * 3fc2e05b18798cd3f755e1887356fc9f8ea2af60 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125230840) * 4911303981d06bbea0d2deed79acc9b025a4d4c0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129303127) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9578: [FLINK-13924][ml] Add summarizer and summary for sparse vector and dense vector.
flinkbot edited a comment on issue #9578: [FLINK-13924][ml] Add summarizer and summary for sparse vector and dense vector. URL: https://github.com/apache/flink/pull/9578#issuecomment-526650851 ## CI report: * 8da3386df1a24f8e5a26a2314f1f5c283f3e8a09 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125254089) * 3f1efc6a08bd46620f2581ebba8028e13b2abb08 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9573: [FLINK-13917][ml] Add class for RegexTokenizerMapper.
flinkbot edited a comment on issue #9573: [FLINK-13917][ml] Add class for RegexTokenizerMapper. URL: https://github.com/apache/flink/pull/9573#issuecomment-526595511 ## CI report: * 1849728c5e62cf8796c24f480c7ebed40c416647 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125230870) * 244be2c6520ad462422496e55fd7cb6130e6c82f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129303148) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9780: [FLINK-14042] [flink-table-planner] Fix TemporalTable row schema always inferred as nullable
flinkbot edited a comment on issue #9780: [FLINK-14042] [flink-table-planner] Fix TemporalTable row schema always inferred as nullable URL: https://github.com/apache/flink/pull/9780#issuecomment-535571919 ## CI report: * 8c0e3b4b94421c9f5ac4cfa470d0859e88471f38 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129303252) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9522: [FLINK-13834][ml] Add class for PolynomialExpansionMapper.
flinkbot edited a comment on issue #9522: [FLINK-13834][ml] Add class for PolynomialExpansionMapper. URL: https://github.com/apache/flink/pull/9522#issuecomment-524304491 ## CI report: * 299364c49caf5e7be1f3a2bd1ae57c066804f938 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124369531) * ea642277d1a93aacc619a5d91015b089af3792ba : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129302974) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11
flinkbot edited a comment on issue #9775: [FLINK-14186][e2e] Skip Elastic Search 2.3.5 case when running with JDK11 URL: https://github.com/apache/flink/pull/9775#issuecomment-535327571 ## CI report: * 7ca40f4410862da362c16e5a9410ca692854451f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129209325) * 18531c05a028f98d44a072705392b3c966493416 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9769: [FLINK-14203][ml] add an abstract class for mappers with rich model.
flinkbot edited a comment on issue #9769: [FLINK-14203][ml] add an abstract class for mappers with rich model. URL: https://github.com/apache/flink/pull/9769#issuecomment-534932470 ## CI report: * a69f4c99a1a61595c3a75acb56300e190b5adb65 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129069456) * 5901a029799fd55dd1a08e667cf991a41a90eddf : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129302905) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9501: [FLINK-12697] [State Backends] Support on-disk state storage for spill-able heap backend
flinkbot edited a comment on issue #9501: [FLINK-12697] [State Backends] Support on-disk state storage for spill-able heap backend URL: https://github.com/apache/flink/pull/9501#issuecomment-523769032 ## CI report: * 6107a00876759dcef075592c8bfa56764f66b6fa : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124167653) * 4263f76529de2d80aa3554ee91c206fc0f9152e5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124179375) * befcc101f408baca6bb2bf83eea14cb4a9f496bd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124863326) * df676fa3d969772b8995538adc147bbb86c36189 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124881729) * 24328ec47f13f4a445f3c431ea6025a368aaedc1 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125238415) * 38d3ce4f2e78ec98aceea6a9d1f958f903ae5f3f : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128211309) * cfbb8387223123f660bc5d5b1f02ec36616b6c11 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/128339902) * 2819c8dd4ede0cb088ac20ba3715e8d8bafb124e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129069417) * 1cf8f18b890dcbb0e19870cb4be6efc1a600fea5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129082047) * 20e2c58d7ba5b385a6b6e898a49b8ce9a0c81243 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129230744) * 8af6229634471ccd064c02b71fc60bc5262c8ab1 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] TisonKun commented on issue #4844: [FLINK-7844] [ckPt] Fail unacknowledged pending checkpoints for fine grained recovery
TisonKun commented on issue #4844: [FLINK-7844] [ckPt] Fail unacknowledged pending checkpoints for fine grained recovery URL: https://github.com/apache/flink/pull/4844#issuecomment-535577290 Thanks a lot for your information Till! Noted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14237) No need to rename shipped Flink jar
[
https://issues.apache.org/jira/browse/FLINK-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zili Chen updated FLINK-14237:
--
Description:
Currently, when we ship Flink jar configured by -yj, we always rename it as
{{flink.jar}}. It seems a redundant operation since we can always use the exact
name of the real jar. It also causes some confusion to our users who should not
be required to know about Flink internal implementation that they configure a
specific Flink jar(said {{flink-private-version-suffix.jar}}) but cannot find
it on YARN container, because it is now {{flink.jar}}.
CC [~trohrmann]
was:
Currently, when we ship Flink jar configured by -yj, we always rename it as
{{flink.jar}}. It seems a redundant operation since we can always use the exact
name of the real jar. It also cause some confusion to our users who should not
be required to know the detail of Flink that they configure a specific Flink
jar(said {{flink-private-version-suffix.jar}}) but cannot find it on YARN
container, because it is now {{flink.jar}}.
CC [~trohrmann]
> No need to rename shipped Flink jar
> ---
>
> Key: FLINK-14237
> URL: https://issues.apache.org/jira/browse/FLINK-14237
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Reporter: Zili Chen
>Assignee: Zili Chen
>Priority: Major
> Fix For: 1.10.0
>
>
> Currently, when we ship Flink jar configured by -yj, we always rename it as
> {{flink.jar}}. It seems a redundant operation since we can always use the
> exact name of the real jar. It also causes some confusion to our users who
> should not be required to know about Flink internal implementation that they
> configure a specific Flink jar(said {{flink-private-version-suffix.jar}}) but
> cannot find it on YARN container, because it is now {{flink.jar}}.
> CC [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
