[jira] [Resolved] (FLINK-14226) Subset of nightly tests fail due to No output has been received in the last 10m0s
[
https://issues.apache.org/jira/browse/FLINK-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann resolved FLINK-14226.
---
Resolution: Duplicate
This problem has been fixed as part of FLINK-14178
> Subset of nightly tests fail due to No output has been received in the last
> 10m0s
> -
>
> Key: FLINK-14226
> URL: https://issues.apache.org/jira/browse/FLINK-14226
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Gary Yao
>Priority: Blocker
>
> https://travis-ci.org/apache/flink/builds/589469198
> https://api.travis-ci.org/v3/job/589469225/log.txt
> {noformat}
> 19:51:07.028 [INFO] --- maven-shade-plugin:3.2.1:shade (shade-flink) @
> flink-elasticsearch6-test ---
> 19:51:07.038 [INFO] Excluding
> org.apache.flink:flink-connector-elasticsearch6_2.11:jar:1.10-SNAPSHOT from
> the shaded jar.
> 19:51:07.045 [INFO] Excluding
> org.apache.flink:flink-connector-elasticsearch-base_2.11:jar:1.10-SNAPSHOT
> from the shaded jar.
> 19:51:07.046 [INFO] Excluding
> org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:6.3.1 from
> the shaded jar.
> 19:51:07.046 [INFO] Excluding org.elasticsearch:elasticsearch:jar:6.3.1 from
> the shaded jar.
> 19:51:07.046 [INFO] Excluding org.elasticsearch:elasticsearch-core:jar:6.3.1
> from the shaded jar.
> 19:51:07.046 [INFO] Excluding
> org.elasticsearch:elasticsearch-secure-sm:jar:6.3.1 from the shaded jar.
> 19:51:07.046 [INFO] Excluding
> org.elasticsearch:elasticsearch-x-content:jar:6.3.1 from the shaded jar.
> 19:51:07.047 [INFO] Excluding org.yaml:snakeyaml:jar:1.17 from the shaded jar.
> 19:51:07.047 [INFO] Excluding
> com.fasterxml.jackson.core:jackson-core:jar:2.8.10 from the shaded jar.
> 19:51:07.047 [INFO] Excluding
> com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.8.10 from the
> shaded jar.
> 19:51:07.047 [INFO] Excluding
> com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.8.10 from the
> shaded jar.
> 19:51:07.047 [INFO] Excluding
> com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.8.10 from the
> shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-core:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding
> org.apache.lucene:lucene-analyzers-common:jar:7.3.1 from the shaded jar.
> 19:51:07.047 [INFO] Excluding
> org.apache.lucene:lucene-backward-codecs:jar:7.3.1 from the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-grouping:jar:7.3.1
> from the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-highlighter:jar:7.3.1
> from the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-join:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-memory:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-misc:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-queries:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-queryparser:jar:7.3.1
> from the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-sandbox:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-spatial:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding
> org.apache.lucene:lucene-spatial-extras:jar:7.3.1 from the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-spatial3d:jar:7.3.1
> from the shaded jar.
> 19:51:07.047 [INFO] Excluding org.apache.lucene:lucene-suggest:jar:7.3.1 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding org.elasticsearch:elasticsearch-cli:jar:6.3.1
> from the shaded jar.
> 19:51:07.047 [INFO] Excluding net.sf.jopt-simple:jopt-simple:jar:5.0.2 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding com.carrotsearch:hppc:jar:0.7.1 from the shaded
> jar.
> 19:51:07.047 [INFO] Excluding joda-time:joda-time:jar:2.5 from the shaded jar.
> 19:51:07.047 [INFO] Excluding com.tdunning:t-digest:jar:3.2 from the shaded
> jar.
> 19:51:07.047 [INFO] Excluding org.hdrhistogram:HdrHistogram:jar:2.1.9 from
> the shaded jar.
> 19:51:07.047 [INFO] Excluding org.elasticsearch:jna:jar:4.5.1 from the shaded
> jar.
> 19:51:07.047 [INFO] Excluding
> org.elasticsearch.client:elasticsearch-rest-client:jar:6.3.1 from the shaded
> jar.
> 19:51:07.047 [INFO] Excluding org.apache.httpcomponents:httpclient:jar:4.5.3
> from the shaded jar.
> 19:51:07.048 [INFO] Excluding org.apache.httpcomponents:httpcore:jar:4.4.6
> from the shaded jar.
> 19:51:07.048 [INFO] Excluding
> org.apache.httpcomponents:httpasyncclient:jar:4.1.2 from the shaded jar.
> 19:51:07.048 [INFO] Excluding
> org.apache.httpcomponents:httpcore-nio:jar:4.4.5 from
[jira] [Resolved] (FLINK-14178) maven-shade-plugin 3.2.1 doesn't work on ARM for Flink
[ https://issues.apache.org/jira/browse/FLINK-14178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann resolved FLINK-14178. --- Resolution: Fixed Fixed via 81038ff1ec0f809f19677a469447053d489f964c > maven-shade-plugin 3.2.1 doesn't work on ARM for Flink > -- > > Key: FLINK-14178 > URL: https://issues.apache.org/jira/browse/FLINK-14178 > Project: Flink > Issue Type: Sub-task > Components: Build System >Affects Versions: 1.10.0 >Reporter: wangxiyuan >Assignee: Dian Fu >Priority: Minor > Labels: pull-request-available > Fix For: 1.10.0 > > Attachments: debug.log > > Time Spent: 20m > Remaining Estimate: 0h > > recently, maven-shade-plugin is bumped from 3.0.0 to 3.2.1 by the > [commit|https://github.com/apache/flink/commit/e7216eebc846a69272c21375af0f4db8009c2e3e]. > While with my test locally on ARM, The Flink build process will be jammed. > After debugging, I found there is an infinite loop. > Downgrade maven-shade-plugin to 3.1.0 can solve this problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] tillrohrmann closed pull request #9817: [FLINK-14178] Downgrade maven-shade-plugin to 3.1.0
tillrohrmann closed pull request #9817: [FLINK-14178] Downgrade maven-shade-plugin to 3.1.0 URL: https://github.com/apache/flink/pull/9817 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14178) maven-shade-plugin 3.2.1 doesn't work on ARM for Flink
[ https://issues.apache.org/jira/browse/FLINK-14178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann updated FLINK-14178: -- Fix Version/s: (was: 2.0.0) 1.10.0 > maven-shade-plugin 3.2.1 doesn't work on ARM for Flink > -- > > Key: FLINK-14178 > URL: https://issues.apache.org/jira/browse/FLINK-14178 > Project: Flink > Issue Type: Sub-task > Components: Build System >Affects Versions: 1.10.0 >Reporter: wangxiyuan >Assignee: Dian Fu >Priority: Minor > Labels: pull-request-available > Fix For: 1.10.0 > > Attachments: debug.log > > Time Spent: 20m > Remaining Estimate: 0h > > recently, maven-shade-plugin is bumped from 3.0.0 to 3.2.1 by the > [commit|https://github.com/apache/flink/commit/e7216eebc846a69272c21375af0f4db8009c2e3e]. > While with my test locally on ARM, The Flink build process will be jammed. > After debugging, I found there is an infinite loop. > Downgrade maven-shade-plugin to 3.1.0 can solve this problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14178) maven-shade-plugin 3.2.1 doesn't work on ARM for Flink
[ https://issues.apache.org/jira/browse/FLINK-14178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann updated FLINK-14178: -- Affects Version/s: (was: 2.0.0) 1.10.0 > maven-shade-plugin 3.2.1 doesn't work on ARM for Flink > -- > > Key: FLINK-14178 > URL: https://issues.apache.org/jira/browse/FLINK-14178 > Project: Flink > Issue Type: Sub-task > Components: Build System >Affects Versions: 1.10.0 >Reporter: wangxiyuan >Assignee: Dian Fu >Priority: Minor > Labels: pull-request-available > Fix For: 2.0.0 > > Attachments: debug.log > > Time Spent: 20m > Remaining Estimate: 0h > > recently, maven-shade-plugin is bumped from 3.0.0 to 3.2.1 by the > [commit|https://github.com/apache/flink/commit/e7216eebc846a69272c21375af0f4db8009c2e3e]. > While with my test locally on ARM, The Flink build process will be jammed. > After debugging, I found there is an infinite loop. > Downgrade maven-shade-plugin to 3.1.0 can solve this problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) * 348504e69b23eb6ef2d2a0f190884edfb3405469 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129787647) * 12c37a76c6125f957da15fd5e676053d6ab43e28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129792126) * 89bf2b1f95fe8431b447ee16aa8219afeeb2f847 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129802895) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9815: [FLINK-14117][docs-zh] Translate changes on index page to Chinese
flinkbot edited a comment on issue #9815: [FLINK-14117][docs-zh] Translate changes on index page to Chinese URL: https://github.com/apache/flink/pull/9815#issuecomment-536343016 ## CI report: * 615acdb2511760c55f8831934f710678a8962acc : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129610971) * 38894aa30a82d5763ad8137e176ac7d78ee13178 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129652562) * 4b35c7d8ce5d86f030037b924a943f857d7f8a01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129684998) * 9099b3a2a220c0d289899f05cea6714fc281d800 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129802867) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Mrart edited a comment on issue #9773: [FLINK-14210][metrics]support connect timeout and write timeout confi…
Mrart edited a comment on issue #9773: [FLINK-14210][metrics]support connect timeout and write timeout confi… URL: https://github.com/apache/flink/pull/9773#issuecomment-536870094 @rmetzger Could you help review this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Mrart commented on issue #9773: [FLINK-14210][metrics]support connect timeout and write timeout confi…
Mrart commented on issue #9773: [FLINK-14210][metrics]support connect timeout and write timeout confi… URL: https://github.com/apache/flink/pull/9773#issuecomment-536870094 @rmetzger Could you help approval this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) * 348504e69b23eb6ef2d2a0f190884edfb3405469 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129787647) * 12c37a76c6125f957da15fd5e676053d6ab43e28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129792126) * 89bf2b1f95fe8431b447ee16aa8219afeeb2f847 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9815: [FLINK-14117][docs-zh] Translate changes on index page to Chinese
flinkbot edited a comment on issue #9815: [FLINK-14117][docs-zh] Translate changes on index page to Chinese URL: https://github.com/apache/flink/pull/9815#issuecomment-536343016 ## CI report: * 615acdb2511760c55f8831934f710678a8962acc : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129610971) * 38894aa30a82d5763ad8137e176ac7d78ee13178 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129652562) * 4b35c7d8ce5d86f030037b924a943f857d7f8a01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129684998) * 9099b3a2a220c0d289899f05cea6714fc281d800 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration
flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration URL: https://github.com/apache/flink/pull/9762#issuecomment-534689957 ## CI report: * cefac57542dbd85a6693f6a1b7c8c90d91e85d30 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128980426) * f04ce4e35191731da657f170c6949ba70d87b6e3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129575680) * 0250869c17414f0c0047b34fdf6a6ad4b57bf017 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129576444) * d2d2ea63507f9e4f798ac96169e4ca11921e4954 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579078) * dac3959dc4798b80edf9636b727dbeeb222c0a1f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129797705) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
walterddr commented on a change in pull request #8468:
[FLINK-12399][table][table-planner] Fix FilterableTableSource does not change
after applyPredicate
URL: https://github.com/apache/flink/pull/8468#discussion_r329875995
##
File path:
flink-connectors/flink-orc/src/main/java/org/apache/flink/orc/OrcTableSource.java
##
@@ -213,7 +213,8 @@ public boolean isFilterPushedDown() {
@Override
public String explainSource() {
- return "OrcFile[path=" + path + ", schema=" + orcSchema + ",
filter=" + predicateString() + "]";
+ return "OrcFile[path=" + path + ", schema=" + orcSchema + ",
filter=" + predicateString()
Review comment:
had to change this override since in fact the OrcTableSource does not
explain the pushed down predicate.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
walterddr commented on a change in pull request #8468:
[FLINK-12399][table][table-planner] Fix FilterableTableSource does not change
after applyPredicate
URL: https://github.com/apache/flink/pull/8468#discussion_r329876013
##
File path:
flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/ParquetTableSource.java
##
@@ -223,7 +224,8 @@ public TableSchema getTableSchema() {
@Override
public String explainSource() {
return "ParquetFile[path=" + path + ", schema=" + parquetSchema
+ ", filter=" + predicateString()
- + ", typeInfo=" + typeInfo + "]";
+ + ", typeInfo=" + typeInfo + ", selectedFields=" +
Arrays.toString(selectedFields)
Review comment:
same here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
walterddr commented on a change in pull request #8468:
[FLINK-12399][table][table-planner] Fix FilterableTableSource does not change
after applyPredicate
URL: https://github.com/apache/flink/pull/8468#discussion_r329876088
##
File path:
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/plan/rules/logical/PushFilterIntoTableSourceScanRule.scala
##
@@ -83,6 +84,14 @@ class PushFilterIntoTableSourceScanRule extends RelOptRule(
val newTableSource = filterableSource.applyPredicate(remainingPredicates)
+if (remainingPredicates.size() > 0
+ &&
newTableSource.asInstanceOf[FilterableTableSource[_]].isFilterPushedDown
+ &&
newTableSource.explainSource().equals(scan.tableSource.explainSource())) {
+ throw new TableException("Failed to push filter into table source! "
Review comment:
throwing `TableException` here. do you think we should make this change a
soft one to only generate a warning and still let the planner to go through?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-9395) Outer Joins with array types on the outer join input fail
[
https://issues.apache.org/jira/browse/FLINK-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941521#comment-16941521
]
Ken Geis commented on FLINK-9395:
-
Good news. This worked for me using the Blink planner.
> Outer Joins with array types on the outer join input fail
> -
>
> Key: FLINK-9395
> URL: https://issues.apache.org/jira/browse/FLINK-9395
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Reporter: Ken Geis
>Priority: Major
> Attachments: JoinTest.java
>
>
> Given a table {{a}} with a single column {{id}}, the following query
> {code:sql}
> SELECT * FROM (SELECT id, ARRAY[id] AS b FROM a) b
> LEFT OUTER JOIN a AS b ON a.id = b.id
> {code}
> fails with the error:
> {noformat}
> org.apache.flink.api.common.InvalidProgramException: Selected sort key is not
> a sortable type
> at
> org.apache.flink.api.java.operators.SortPartitionOperator.ensureSortableKey(SortPartitionOperator.java:145)
> at
> org.apache.flink.api.java.operators.SortPartitionOperator.sortPartition(SortPartitionOperator.java:111)
> at
> org.apache.flink.table.plan.nodes.dataset.DataSetJoin$$anonfun$partitionAndSort$1.apply(DataSetJoin.scala:466)
> at
> org.apache.flink.table.plan.nodes.dataset.DataSetJoin$$anonfun$partitionAndSort$1.apply(DataSetJoin.scala:465)
> at
> scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:57)
> at
> scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:66)
> at
> scala.collection.mutable.ArrayOps$ofInt.foldLeft(ArrayOps.scala:234)
> at
> org.apache.flink.table.plan.nodes.dataset.DataSetJoin.partitionAndSort(DataSetJoin.scala:465)
> at
> org.apache.flink.table.plan.nodes.dataset.DataSetJoin.addLeftOuterJoin(DataSetJoin.scala:268)
> at
> org.apache.flink.table.plan.nodes.dataset.DataSetJoin.translateToPlan(DataSetJoin.scala:176)
> at
> org.apache.flink.table.api.BatchTableEnvironment.translate(BatchTableEnvironment.scala:399)
> at
> org.apache.flink.table.api.BatchTableEnvironment.translate(BatchTableEnvironment.scala:378)
> at
> org.apache.flink.table.api.java.BatchTableEnvironment.toDataSet(BatchTableEnvironment.scala:146)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] walterddr commented on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
walterddr commented on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-536859787 @KurtYoung sorry for the delay. I just implemented a way to safe check to prevent users from utilizing a pushdown table source without overriding and explaining the predicator pushdown in the `explainSource()` API. please kindly take a look when you have time. The throw exception might've been too much of a harsh term. I can change it to warning and let the program pass if necessary, please let me know what do you think of this approach. thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration
flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration URL: https://github.com/apache/flink/pull/9762#issuecomment-534689957 ## CI report: * cefac57542dbd85a6693f6a1b7c8c90d91e85d30 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128980426) * f04ce4e35191731da657f170c6949ba70d87b6e3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129575680) * 0250869c17414f0c0047b34fdf6a6ad4b57bf017 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129576444) * d2d2ea63507f9e4f798ac96169e4ca11921e4954 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579078) * dac3959dc4798b80edf9636b727dbeeb222c0a1f : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129797705) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration
flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration URL: https://github.com/apache/flink/pull/9762#issuecomment-534689957 ## CI report: * cefac57542dbd85a6693f6a1b7c8c90d91e85d30 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128980426) * f04ce4e35191731da657f170c6949ba70d87b6e3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129575680) * 0250869c17414f0c0047b34fdf6a6ad4b57bf017 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129576444) * d2d2ea63507f9e4f798ac96169e4ca11921e4954 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579078) * dac3959dc4798b80edf9636b727dbeeb222c0a1f : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) * 348504e69b23eb6ef2d2a0f190884edfb3405469 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129787647) * 12c37a76c6125f957da15fd5e676053d6ab43e28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129792126) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) * 348504e69b23eb6ef2d2a0f190884edfb3405469 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129787647) * 12c37a76c6125f957da15fd5e676053d6ab43e28 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129792126) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhengcanbin commented on issue #9781: [FLINK-14098] Support multiple statements for TableEnvironment.
zhengcanbin commented on issue #9781: [FLINK-14098] Support multiple statements for TableEnvironment. URL: https://github.com/apache/flink/pull/9781#issuecomment-536836014 > One general question from my side: Shall we also allow multiline statements in `sqlQuery()`? In theory we could allow that and just ensure that the last statement is a query statement. Similarly we need to ensure that there is no query statement in `sqlUpdate()`. What is JDBC doing in those cases? Thanks, @twalthr In JDBC, `executeXXX` just accepts one statement, `executeUpdate` allows DDL/DML and throws an exception when executing DQL, conversely, `executeQuery` supports DQL while DDL/DML is not allowed. IMO, it also makes sense to support multi statements in `sqlQuery()`, DDL/DML are allowed if a query is the last statement, also, multiple queries should not be allowed. But it seems the boundary is not very clear between these two interfaces after we apply these changes, both of them allow DDL/DML, would such behavior cause confusion for users to some extent? Actually I tend to not change the behavior of `sqlQuery()`, so `sqlQuery()` is only for read side and `sqlUpdate()` for write side. Thoughts? @wuchong @dawidwys This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) * 348504e69b23eb6ef2d2a0f190884edfb3405469 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129787647) * 12c37a76c6125f957da15fd5e676053d6ab43e28 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] TisonKun commented on a change in pull request #9810: [FLINK-14284] Add shut down future to Dispatcher
TisonKun commented on a change in pull request #9810: [FLINK-14284] Add shut
down future to Dispatcher
URL: https://github.com/apache/flink/pull/9810#discussion_r329858169
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java
##
@@ -603,7 +616,7 @@ private JobManagerRunner
startJobManagerRunner(JobManagerRunner jobManagerRunner
@Override
public CompletableFuture shutDownCluster() {
- closeAsync();
+ shutDownFuture.complete(ApplicationStatus.SUCCEEDED);
Review comment:
And follow the previous pull request we actually complete with
`ApplicationStatus.UNKNOWN` in this case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] TisonKun commented on a change in pull request #9810: [FLINK-14284] Add shut down future to Dispatcher
TisonKun commented on a change in pull request #9810: [FLINK-14284] Add shut
down future to Dispatcher
URL: https://github.com/apache/flink/pull/9810#discussion_r329857622
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java
##
@@ -603,7 +616,7 @@ private JobManagerRunner
startJobManagerRunner(JobManagerRunner jobManagerRunner
@Override
public CompletableFuture shutDownCluster() {
- closeAsync();
+ shutDownFuture.complete(ApplicationStatus.SUCCEEDED);
Review comment:
So we complete `shutDownFuture` here and cluster entrypoint takes care of
the close of the whole cluster?
Another concern is `ApplicationStatus.SUCCEEDED` v.s.
`ApplicationStatus.CANCEL` since, IIRC, shutDownCluster is called via REST so
it means user shutdown the cluster intendedly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] TisonKun commented on a change in pull request #9810: [FLINK-14284] Add shut down future to Dispatcher
TisonKun commented on a change in pull request #9810: [FLINK-14284] Add shut
down future to Dispatcher
URL: https://github.com/apache/flink/pull/9810#discussion_r329857622
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java
##
@@ -603,7 +616,7 @@ private JobManagerRunner
startJobManagerRunner(JobManagerRunner jobManagerRunner
@Override
public CompletableFuture shutDownCluster() {
- closeAsync();
+ shutDownFuture.complete(ApplicationStatus.SUCCEEDED);
Review comment:
So we complete `shutDownFuture` here and cluster entrypoint takes care of
the close of the whole cluster?
Another concern is `ApplicationStatus.SUCCEEDED` v.s.
`ApplicationStatus.CANCELED` since, IIRC, shutDownCluster is called via REST so
it means user shutdown the cluster intendedly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] TisonKun commented on a change in pull request #9809: [FLINK-14282] Simplify DispatcherResourceManagerComponent hierarchy
TisonKun commented on a change in pull request #9809: [FLINK-14282] Simplify
DispatcherResourceManagerComponent hierarchy
URL: https://github.com/apache/flink/pull/9809#discussion_r329855229
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/component/DefaultDispatcherResourceManagerComponentFactory.java
##
@@ -71,24 +71,23 @@
/**
* Abstract class which implements the creation of the {@link
DispatcherResourceManagerComponent} components.
*
- * @param type of the {@link DispatcherRunner}
* @param type of the {@link RestfulGateway} given to the {@link
WebMonitorEndpoint}
*/
-public abstract class AbstractDispatcherResourceManagerComponentFactory implements
DispatcherResourceManagerComponentFactory {
+public class DefaultDispatcherResourceManagerComponentFactory implements DispatcherResourceManagerComponentFactory {
Review comment:
I don't think one should instance an
`DefaultDispatcherResourceManagerComponentFactory`. How about keep it
`abstract`? Both names are OK for me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] TisonKun commented on a change in pull request #9809: [FLINK-14282] Simplify DispatcherResourceManagerComponent hierarchy
TisonKun commented on a change in pull request #9809: [FLINK-14282] Simplify
DispatcherResourceManagerComponent hierarchy
URL: https://github.com/apache/flink/pull/9809#discussion_r329855229
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/component/DefaultDispatcherResourceManagerComponentFactory.java
##
@@ -71,24 +71,23 @@
/**
* Abstract class which implements the creation of the {@link
DispatcherResourceManagerComponent} components.
*
- * @param type of the {@link DispatcherRunner}
* @param type of the {@link RestfulGateway} given to the {@link
WebMonitorEndpoint}
*/
-public abstract class AbstractDispatcherResourceManagerComponentFactory implements
DispatcherResourceManagerComponentFactory {
+public class DefaultDispatcherResourceManagerComponentFactory implements DispatcherResourceManagerComponentFactory {
Review comment:
I don't think one should instance an
{{DefaultDispatcherResourceManagerComponentFactory}}. How about keep it
`abstract`? Both names are OK for me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-14282) Simplify DispatcherResourceManagerComponent
[
https://issues.apache.org/jira/browse/FLINK-14282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941460#comment-16941460
]
Zili Chen commented on FLINK-14282:
---
Well I misunderstand so we still have multiple {{Dispatcher}} s. But this clean
up is still valid :P
> Simplify DispatcherResourceManagerComponent
> ---
>
> Key: FLINK-14282
> URL: https://issues.apache.org/jira/browse/FLINK-14282
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With the completion of the FLINK-14281 it is now possible to encapsulate the
> shutdown logic of the {{MiniDispatcher}} within the {{DispatcherRunner}}.
> Consequently, it is no longer necessary to have separate
> {{DispatcherResourceManagerComponent}} implementations. I suggest to remove
> the special case implementations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14282) Simplify DispatcherResourceManagerComponent
[
https://issues.apache.org/jira/browse/FLINK-14282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941459#comment-16941459
]
Zili Chen commented on FLINK-14282:
---
Nice to have! The difference between {{JobCluster}} and {{SessionCluster}} is
better to exist in {{ClutserEntrypoint}} level, not in {{Dispatcher}} level.
> Simplify DispatcherResourceManagerComponent
> ---
>
> Key: FLINK-14282
> URL: https://issues.apache.org/jira/browse/FLINK-14282
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With the completion of the FLINK-14281 it is now possible to encapsulate the
> shutdown logic of the {{MiniDispatcher}} within the {{DispatcherRunner}}.
> Consequently, it is no longer necessary to have separate
> {{DispatcherResourceManagerComponent}} implementations. I suggest to remove
> the special case implementations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) * 348504e69b23eb6ef2d2a0f190884edfb3405469 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129787647) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) * 348504e69b23eb6ef2d2a0f190884edfb3405469 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781330) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124398771) * 6208617ff2d84bef7efaa7ee7cf96cba00031d88 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127279857) * 33997c30f049e32a22cd6caa0427568a52d25e63 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127650128) * 9366993839d7a63193b59d838e840b8e6e9e6679 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129781312) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog
flinkbot edited a comment on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog URL: https://github.com/apache/flink/pull/9822#issuecomment-536774896 ## CI report: * 611ecdea461d9c202e0fd3ad0b33f24e8a4db061 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129772949) * cc357e4d5365aa98db6ebb8a3c1a821080bb202e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129775325) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) * ce3e9599c4e4ef59fad56cc4625e8dafcc2d804e : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate
flinkbot edited a comment on issue #8468: [FLINK-12399][table][table-planner] Fix FilterableTableSource does not change after applyPredicate URL: https://github.com/apache/flink/pull/8468#issuecomment-524372146 ## CI report: * baae1632aabac35e6e08b402065857c4d67491f2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124398771) * 6208617ff2d84bef7efaa7ee7cf96cba00031d88 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127279857) * 33997c30f049e32a22cd6caa0427568a52d25e63 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/127650128) * 9366993839d7a63193b59d838e840b8e6e9e6679 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] dianfu commented on issue #9653: [FLINK-14014][python] Introduce PythonScalarFunctionRunner to handle the communication with Python worker for Python ScalarFunction execution
dianfu commented on issue #9653: [FLINK-14014][python] Introduce PythonScalarFunctionRunner to handle the communication with Python worker for Python ScalarFunction execution URL: https://github.com/apache/flink/pull/9653#issuecomment-536794706 @tillrohrmann Very appreciated for your suggestions. That makes sense to me. Will be more careful for this kind of changes next time, e.g. doing it in a separate PR and making sure end-to-end test passed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] dianfu commented on a change in pull request #9748: [FLINK-14016][python][flink-table-planner] Introduce DataStreamPythonCalc for Python function execution
dianfu commented on a change in pull request #9748:
[FLINK-14016][python][flink-table-planner] Introduce DataStreamPythonCalc for
Python function execution
URL: https://github.com/apache/flink/pull/9748#discussion_r329827390
##
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/functions/FunctionDefinition.java
##
@@ -40,6 +40,13 @@
*/
FunctionKind getKind();
+ /**
+* Returns the language of function this definition describes.
+*/
+ default FunctionLanguage getLanguage() {
Review comment:
@twalthr Thanks a lot for your comments. Appreciated! Just share my thoughts
as following:
1. `Python ScalarFunction` also extends `ScalarFunction` and it will
override `getLanguage`. If it's declared as final, there is no way to override
it any more. This can be seen from the implementation of
[PythonFunctionCodeGenerator](https://github.com/apache/flink/blob/fc4a1df1ad48224ae846fbbdeeb21b4b7b164e02/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/codegen/PythonFunctionCodeGenerator.scala#L74).
2. Actually users could write a user-defined Python `ScalarFunction` with
the `ScalarFunction` interface if they want(In this case, they need to override
the getLanguage interface).
[PythonFunctionCodeGenerator](https://github.com/apache/flink/blob/fc4a1df1ad48224ae846fbbdeeb21b4b7b164e02/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/codegen/PythonFunctionCodeGenerator.scala#L74)
could be seen as an example how to do that, e.g. they need just implement the
`PythonFunction` interface and wrap the serialized Python function inside it.
Does it make sense to you? Looking forward to your feedback.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] bowenli86 commented on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog
bowenli86 commented on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog URL: https://github.com/apache/flink/pull/9822#issuecomment-536792601 @xuefuz @lirui-apache @zjuwangg This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] bowenli86 commented on issue #9689: [FLINK-7151] add a basic function ddl
bowenli86 commented on issue #9689: [FLINK-7151] add a basic function ddl URL: https://github.com/apache/flink/pull/9689#issuecomment-536792400 Tests are failing. Can you first them first? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog
flinkbot edited a comment on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog URL: https://github.com/apache/flink/pull/9822#issuecomment-536774896 ## CI report: * 611ecdea461d9c202e0fd3ad0b33f24e8a4db061 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129772949) * cc357e4d5365aa98db6ebb8a3c1a821080bb202e : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129775325) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot edited a comment on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129775346) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot commented on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536781542 ## CI report: * 2b5531b068338321eb04cdf71714eae0d088ec0e : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog
flinkbot edited a comment on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog URL: https://github.com/apache/flink/pull/9822#issuecomment-536774896 ## CI report: * 611ecdea461d9c202e0fd3ad0b33f24e8a4db061 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129772949) * cc357e4d5365aa98db6ebb8a3c1a821080bb202e : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14273) when User-Defined Aggregate Functions(UDAF) parameters are inconsistent with the definition, the error reporting is confusing
[
https://issues.apache.org/jira/browse/FLINK-14273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941378#comment-16941378
]
hailong wang commented on FLINK-14273:
--
Thank you. Jark Wu, I had submit a PR, I appreciate if you have a look.
> when User-Defined Aggregate Functions(UDAF) parameters are inconsistent with
> the definition, the error reporting is confusing
> -
>
> Key: FLINK-14273
> URL: https://issues.apache.org/jira/browse/FLINK-14273
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
>Affects Versions: 1.9.0
>Reporter: hailong wang
>Assignee: hailong wang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When UDAF parameters are inconsistent with the definition of accumulate
> method, all arguments to the accumulate method are listed in the error. But
> the first argument of accumulate is accumulator, users don't have to care
> when using SQL.
> For example:
> {code:java}
> INSERT INTO Orders SELECT name, USERUDAF(id, name) FROM Orders GROUP BY
> TUMBLE(rowTime, interval '10' second ), id, name
> {code}
> USERUDAF is a User-Defined Aggregate Functions, and accumulate is defined as
> follow:
> {code:java}
> public void accumulate(Long acc, String a) {……}
> {code}
> At present, error is as follows:
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Given parameters
> of function do not match any signature.
> Actual: (java.lang.Integer, java.lang.String)
> Expected: (java.lang.Integer, java.lang.String)
> {code}
> This error will mislead users, and the expected errors are as follows :
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Given parameters
> of function do not match any signature.
> Actual: (java.lang.Integer, java.lang.String)
> Expected: (java.lang.String){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
flinkbot commented on issue #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823#issuecomment-536777025 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 2b5531b068338321eb04cdf71714eae0d088ec0e (Mon Sep 30 22:23:05 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14273) when User-Defined Aggregate Functions(UDAF) parameters are inconsistent with the definition, the error reporting is confusing
[
https://issues.apache.org/jira/browse/FLINK-14273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-14273:
---
Labels: pull-request-available (was: )
> when User-Defined Aggregate Functions(UDAF) parameters are inconsistent with
> the definition, the error reporting is confusing
> -
>
> Key: FLINK-14273
> URL: https://issues.apache.org/jira/browse/FLINK-14273
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
>Affects Versions: 1.9.0
>Reporter: hailong wang
>Assignee: hailong wang
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.10.0
>
>
> When UDAF parameters are inconsistent with the definition of accumulate
> method, all arguments to the accumulate method are listed in the error. But
> the first argument of accumulate is accumulator, users don't have to care
> when using SQL.
> For example:
> {code:java}
> INSERT INTO Orders SELECT name, USERUDAF(id, name) FROM Orders GROUP BY
> TUMBLE(rowTime, interval '10' second ), id, name
> {code}
> USERUDAF is a User-Defined Aggregate Functions, and accumulate is defined as
> follow:
> {code:java}
> public void accumulate(Long acc, String a) {……}
> {code}
> At present, error is as follows:
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Given parameters
> of function do not match any signature.
> Actual: (java.lang.Integer, java.lang.String)
> Expected: (java.lang.Integer, java.lang.String)
> {code}
> This error will mislead users, and the expected errors are as follows :
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Given parameters
> of function do not match any signature.
> Actual: (java.lang.Integer, java.lang.String)
> Expected: (java.lang.String){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wangxlong opened a new pull request #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction
wangxlong opened a new pull request #9823: [FLINK-14273][Table]Add accumulator type to actual type list in AggSqlFunction URL: https://github.com/apache/flink/pull/9823 ## What is the purpose of the change Add accumulator type to actual type list in AggSqlFunction according to issue FLINK-14273 ## Brief change log - Add a method to Prints the first signature of methods with given name in a class and one signature consisting of DataType. - Use the above method to actual type list when ValidationException ## Verifying this change This change is a trivial work according to existing methods without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog
flinkbot commented on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog URL: https://github.com/apache/flink/pull/9822#issuecomment-536774896 ## CI report: * 611ecdea461d9c202e0fd3ad0b33f24e8a4db061 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-11632) Make TaskManager automatic bind address picking more explicit (by default) and more configurable
[
https://issues.apache.org/jira/browse/FLINK-11632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941373#comment-16941373
]
Tim commented on FLINK-11632:
-
Which version of Flink is this fix available in? Thanks;
> Make TaskManager automatic bind address picking more explicit (by default)
> and more configurable
>
>
> Key: FLINK-11632
> URL: https://issues.apache.org/jira/browse/FLINK-11632
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination, Runtime / Network
>Reporter: Alex
>Assignee: Alex
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, there is an optional {{taskmanager.host}} configuration option in
> {{flink-conf.yaml}} that allows users of Flink to "statically" pre-define
> what should be a bind address for TaskManager to listen on (note: it's also
> possible to override this option by passing corresponding command line option
> to Flink).
> In case when the option is not set, TaskManager would try [heuristically pick
> up a bind
> address|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java#L421-L442].
> The resulting address (hostname) is used to advertise different service
> endpoints (running in TM) to the JobManager. Also it would be resolved to an
> {{[InetAddress|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerRunner.java#L359]}}
> later that used as binding address for TMs inner node communication.
> This proposal is to minimize usage of heuristics (by default) by introducing
> a new configuration option (for example, {{taskmanager.host.bind-policy}})
> with possible values:
> * {{"hostname"}} - default, use TM's host's name ({{==
> InetAddress.getLocalHost().getHostName()}};
> * {{"ip"}} - use TM's host's ip address ({{==
> InetAddress.getLocalHost().getHostAddress()}});
> * {{"auto-detect-hostname"}} - use the heuristics based detection mechanism.
> *Note:* the configuration key and values could be named better and open for
> proposals.
> *Note 2:* in the future, the configuration option _may_ require to be
> extended to allow choosing some specific network interface, or preference of
> ipv6 vs ipv4.
> h3. Rationale
> [The heuristics
> mechanism|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/net/ConnectionUtils.java#L364-L475]
> tries to establish a probe connection to {{jobmanager.rpc.address}} from
> different network interface addresses.
> In case of parallel setups (when JM and multiple TMs start simultaneously,
> in parallel), this depends on timing, assigned network ip addresses and may
> end up with "non-uniform" address bindings of TMs (some may be "lucky" to
> pick up non default network interface, some would fallback to
> {{InetAddress.getLocalHost().getHostName()}}. At the end, it's less obvious
> and transparent which binding address a TM picks up.
> In practice, it's possible that in majority of cases (in well setup
> environments) the heuristics mechanism returns a result that matches
> {{InetAddress.getLocalHost()}}. The proposal is to stick with this more
> simpler and explicit binding (by default), avoiding non-determinism of
> heuristics.
> The old mechanism is kept available, in case if it is useful in some setups.
> But would require explicit configuration setting.
> Additionally, this proposal extends "auto configuration" option by allowing
> users to choose the host's ip address (instead of hostname). This may be
> convenient in situations where the TMs' machines are not necessary reachable
> via DNS (for example in a Kubernetes setup).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] dianfu commented on issue #9817: [FLINK-14178] Downgrade maven-shade-plugin to 3.1.0
dianfu commented on issue #9817: [FLINK-14178] Downgrade maven-shade-plugin to 3.1.0 URL: https://github.com/apache/flink/pull/9817#issuecomment-536772915 @tillrohrmann Thanks a lot for sharing the link. The end-to-end tests have passed from the link you shared and so downgrade to 3.1.1 should work. There is also no "No output has been received in the last 10m0s" error any more from the end-to-end tests I triggered for 3.1.0: https://travis-ci.org/dianfu/flink/builds/591410893. There are a few tests failed because of killed after running 50 mins. I have re-triggered it: https://travis-ci.org/dianfu/flink/builds/591715903 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] dianfu edited a comment on issue #9817: [FLINK-14178] Downgrade maven-shade-plugin to 3.1.0
dianfu edited a comment on issue #9817: [FLINK-14178] Downgrade maven-shade-plugin to 3.1.0 URL: https://github.com/apache/flink/pull/9817#issuecomment-536772915 @tillrohrmann Thanks a lot for sharing the link. The end-to-end tests have passed from the link you shared and so downgrade to 3.1.1 should work. There is also no "No output has been received in the last 10m0s" error any more from the end-to-end tests I triggered for 3.1.0: https://travis-ci.org/dianfu/flink/builds/591410893. There are a few tests failed because of killed after running 50 mins. I have re-triggered it: https://travis-ci.org/dianfu/flink/builds/591715903 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog
flinkbot commented on issue #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog URL: https://github.com/apache/flink/pull/9822#issuecomment-536770082 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 611ecdea461d9c202e0fd3ad0b33f24e8a4db061 (Mon Sep 30 21:58:44 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14301) add documentation for temp and temp system functions and new function resolution order
Bowen Li created FLINK-14301: Summary: add documentation for temp and temp system functions and new function resolution order Key: FLINK-14301 URL: https://issues.apache.org/jira/browse/FLINK-14301 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Bowen Li Assignee: Bowen Li Fix For: 1.10.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14216) introduce temp system functions to FunctionCatalog
[ https://issues.apache.org/jira/browse/FLINK-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-14216: --- Labels: pull-request-available (was: ) > introduce temp system functions to FunctionCatalog > -- > > Key: FLINK-14216 > URL: https://issues.apache.org/jira/browse/FLINK-14216 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Bowen Li >Assignee: Bowen Li >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] bowenli86 opened a new pull request #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog
bowenli86 opened a new pull request #9822: [FLINK-14216][table] introduce temp system functions to FunctionCatalog URL: https://github.com/apache/flink/pull/9822 ## What is the purpose of the change adapt existing APIs to the introduction of temporary system functions according to FLIP-57 ## Brief change log - renamed `registerXxxFunction()` to `registerTemporarySystemXxxFunction()` - renamed variable `userFunctions` in FunctionCatalog to `tempSystemFuncitons` ## Verifying this change This change is already covered by existing tests ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (yes) - If yes, how is the feature documented? (docs / JavaDocs) docs will be added separately This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-14201) SQL supports redis sink connector
[
https://issues.apache.org/jira/browse/FLINK-14201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941331#comment-16941331
]
Xiaochuan Yu edited comment on FLINK-14201 at 9/30/19 9:36 PM:
---
{quote}We can support the TableSink and TableSource for redis in bahir-flink
{quote}
[~jark], I'm interested in contributing to Flink and have been looking through
the issue tracker for a while for a first task that is well-defined.
Would this be a reasonable first task to work on for me?
Do we want something similar to the CassandraAppendTableSink but with a user
specifying RedisCommandDescription(or RedisMapper) instead of CQL template?
was (Author: xcyu):
{quote}We can support the TableSink and TableSource for redis in bahir-flink
{quote}
[~jark], I'm interested in contributing to Flink and have been looking through
the issue tracker for a while for a first task that is well-defined.
Would this be a reasonable first task to work on for me?
Do we want something similar to the CassandraAppendTableSink but with a user
specifying RedisCommandDescription instead of CQL template?
> SQL supports redis sink connector
> -
>
> Key: FLINK-14201
> URL: https://issues.apache.org/jira/browse/FLINK-14201
> Project: Flink
> Issue Type: Wish
> Components: Connectors / Common
>Reporter: Zijie Lu
>Priority: Minor
>
> Can SQL supports the connector type of redis? I think in many cases we need
> to store the result in the redis.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14201) SQL supports redis sink connector
[
https://issues.apache.org/jira/browse/FLINK-14201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941331#comment-16941331
]
Xiaochuan Yu commented on FLINK-14201:
--
{quote}We can support the TableSink and TableSource for redis in bahir-flink
{quote}
[~jark], I'm interested in contributing to Flink and have been looking through
the issue tracker for a while for a first task that is well-defined.
Would this be a reasonable first task to work on for me?
Do we want something similar to the CassandraAppendTableSink but with a user
specifying RedisCommandDescription instead of CQL template?
> SQL supports redis sink connector
> -
>
> Key: FLINK-14201
> URL: https://issues.apache.org/jira/browse/FLINK-14201
> Project: Flink
> Issue Type: Wish
> Components: Connectors / Common
>Reporter: Zijie Lu
>Priority: Minor
>
> Can SQL supports the connector type of redis? I think in many cases we need
> to store the result in the redis.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14123) Change taskmanager.memory.fraction default value to 0.6
[
https://issues.apache.org/jira/browse/FLINK-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941325#comment-16941325
]
Stephan Ewen commented on FLINK-14123:
--
[~xintongsong][~azagrebin] Given that in the future, we compute the amount of
managed memory prior to starting the internal services, it probably makes sense
to lower the fraction anyways.
It would also be good to have good experience on the parallel GC, which seems
to be still the best GC, throughput wise, for batch processing.
What do you think about this change here?
> Change taskmanager.memory.fraction default value to 0.6
> ---
>
> Key: FLINK-14123
> URL: https://issues.apache.org/jira/browse/FLINK-14123
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Affects Versions: 1.9.0
>Reporter: liupengcheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, we are testing flink batch task, such as terasort, however, it
> started only awhile then it failed due to OOM.
>
> {code:java}
> org.apache.flink.client.program.ProgramInvocationException: Job failed.
> (JobID: a807e1d635bd4471ceea4282477f8850)
> at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326)
> at
> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62)
> at
> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:539)
> at
> com.github.ehiggs.spark.terasort.FlinkTeraSort$.main(FlinkTeraSort.scala:89)
> at
> com.github.ehiggs.spark.terasort.FlinkTeraSort.main(FlinkTeraSort.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:604)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:466)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1007)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1080)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1080)
> Caused by: org.apache.flink.runtime.client.JobExecutionException: Job
> execution failed.
> at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
> at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259)
> ... 23 more
> Caused by: java.lang.RuntimeException: Error obtaining the sorted input:
> Thread 'SortMerger Reading Thread' terminated due to an exception: GC
> overhead limit exceeded
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650)
> at
> org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1109)
> at org.apache.flink.runtime.operators.NoOpDriver.run(NoOpDriver.java:82)
> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:504)
> at
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated
> due to an exception: GC overhead limit exceeded
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831)
> Caused by:
[GitHub] [flink] flinkbot edited a comment on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient
flinkbot edited a comment on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient URL: https://github.com/apache/flink/pull/9820#issuecomment-536452582 ## CI report: * ff3ef7ae21616fb0295e3bdc53fa349c8f136a4b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129646465) * 654524c3508c89d08d5039a451ba425a5a2e3a41 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129666135) * 7f462e2cad4ca8866abccda492b53007a45b3407 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129712387) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout
flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout URL: https://github.com/apache/flink/pull/9794#issuecomment-536174033 ## CI report: * f9fbec1b59a8519c83bfb3dff4f6f1d9a785350b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129544195) * 5c98fa4029661182bfe847fbdd471900965f2bc1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129702720) * c7065d2746df83be426ed7b0c2171d51516d0f8b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129716759) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9787: [FLINK-14252][coordination] Encapsulate Dispatcher services in DispatcherServices
flinkbot edited a comment on issue #9787: [FLINK-14252][coordination] Encapsulate Dispatcher services in DispatcherServices URL: https://github.com/apache/flink/pull/9787#issuecomment-535981579 ## CI report: * 872b66ca40938834c13f34d32b3d340064c1bf95 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129466049) * 55a959a0d3fec886b6d83373add537c4f7d613ab : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129589016) * 65eaae5c87fd5a41a8499bf58f67a53d62a59115 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129707366) * 501c04ea87ee9736e808772db44a61ad57206d78 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129712366) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14300) StreamTask#invoke leaks threads if OperatorChain fails to be constructed
[ https://issues.apache.org/jira/browse/FLINK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcos Klein updated FLINK-14300: - Summary: StreamTask#invoke leaks threads if OperatorChain fails to be constructed (was: org.apache.flink.streaming.runtime.tasks.StreamTask#invoke leaks threads if org.apache.flink.streaming.runtime.tasks.OperatorChain fails to be constructed) > StreamTask#invoke leaks threads if OperatorChain fails to be constructed > > > Key: FLINK-14300 > URL: https://issues.apache.org/jira/browse/FLINK-14300 > Project: Flink > Issue Type: Bug > Components: Runtime / Task >Affects Versions: 1.8.1, 1.8.2, 1.9.0 >Reporter: Marcos Klein >Priority: Minor > Attachments: thread-leak-patch.diff > > > In the *StreamTask#invoke* method if an exception occurs during the > allocation of the > [operatorChain|[https://github.com/apache/flink/blob/release-1.9.0/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L370]] > class, the [exception > handling|[https://github.com/apache/flink/blob/release-1.9.0/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L485-L491]] > fails to cleanup the threads allocated as *StreamTask#recordWriters*. This > causes threads to leak as flink attempts to continually restart and fail for > the same cause. > > An example cause is a deserialization issue on a custom operator from a > checkpoint. > > Attached is a suggested fix for the master branch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14300) org.apache.flink.streaming.runtime.tasks.StreamTask#invoke leaks threads if org.apache.flink.streaming.runtime.tasks.OperatorChain fails to be constructed
Marcos Klein created FLINK-14300: Summary: org.apache.flink.streaming.runtime.tasks.StreamTask#invoke leaks threads if org.apache.flink.streaming.runtime.tasks.OperatorChain fails to be constructed Key: FLINK-14300 URL: https://issues.apache.org/jira/browse/FLINK-14300 Project: Flink Issue Type: Bug Components: Runtime / Task Affects Versions: 1.9.0, 1.8.2, 1.8.1 Reporter: Marcos Klein Attachments: thread-leak-patch.diff In the *StreamTask#invoke* method if an exception occurs during the allocation of the [operatorChain|[https://github.com/apache/flink/blob/release-1.9.0/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L370]] class, the [exception handling|[https://github.com/apache/flink/blob/release-1.9.0/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L485-L491]] fails to cleanup the threads allocated as *StreamTask#recordWriters*. This causes threads to leak as flink attempts to continually restart and fail for the same cause. An example cause is a deserialization issue on a custom operator from a checkpoint. Attached is a suggested fix for the master branch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9812: [FLINK-14286] Remove Akka specific parsing from LeaderConnectionInfo
flinkbot edited a comment on issue #9812: [FLINK-14286] Remove Akka specific parsing from LeaderConnectionInfo URL: https://github.com/apache/flink/pull/9812#issuecomment-536322371 ## CI report: * c1ca1131b02a2c555cd8386fd07aa1cfccbab161 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129600831) * da85c13656e6cf02fcadbadf50df0c201f07970c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129707797) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9811: [FLINK-14285] Remove generics from Dispatcher factories
flinkbot edited a comment on issue #9811: [FLINK-14285] Remove generics from Dispatcher factories URL: https://github.com/apache/flink/pull/9811#issuecomment-536320588 ## CI report: * 658cddd4424505c1a904a926ad6e626abbd85cd7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129600090) * 5b4df7e3bd41053dcce049edf420ed584a42b98a : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129707753) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9821: [FLINK-14298] Replace LeaderContender#getAddress with #getDescription
flinkbot edited a comment on issue #9821: [FLINK-14298] Replace LeaderContender#getAddress with #getDescription URL: https://github.com/apache/flink/pull/9821#issuecomment-536618253 ## CI report: * 9ca9df3eaa22965444373a3b6142798ca5559f50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129712419) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9810: [FLINK-14284] Add shut down future to Dispatcher
flinkbot edited a comment on issue #9810: [FLINK-14284] Add shut down future to Dispatcher URL: https://github.com/apache/flink/pull/9810#issuecomment-536320582 ## CI report: * 3ea0f47e7dd76ec115db4ef583b416685107604b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129600085) * e73e7e0a72952be963e29226bbb9b94487dafde6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129707704) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9809: [FLINK-14282] Simplify DispatcherResourceManagerComponent hierarchy
flinkbot edited a comment on issue #9809: [FLINK-14282] Simplify DispatcherResourceManagerComponent hierarchy URL: https://github.com/apache/flink/pull/9809#issuecomment-536318695 ## CI report: * 531cd688ceb25b544833025d9b556a0d686b29c4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129599298) * a3ba53aa778e777d8f38750eb6a5e5147fd21654 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129707662) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9807: [FLINK-14280] Introduce DispatcherRunner
flinkbot edited a comment on issue #9807: [FLINK-14280] Introduce DispatcherRunner URL: https://github.com/apache/flink/pull/9807#issuecomment-536314902 ## CI report: * 9ef8fe986e4b06d6ae8512e5051933458363511c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129597654) * 748954650809fd2fa18572d1c925d3b647a55906 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129707578) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9790: [FLINK-14261] Add PermanentlyFencedRpcEndpoint
flinkbot edited a comment on issue #9790: [FLINK-14261] Add PermanentlyFencedRpcEndpoint URL: https://github.com/apache/flink/pull/9790#issuecomment-535998099 ## CI report: * f1bcc25d3af40fc482018fb8459bd8b942b15f68 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129474398) * 87ad063945f80e3070f8c9906a2eb89a91e2017b : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129707483) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout
flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout URL: https://github.com/apache/flink/pull/9794#issuecomment-536174033 ## CI report: * f9fbec1b59a8519c83bfb3dff4f6f1d9a785350b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129544195) * 5c98fa4029661182bfe847fbdd471900965f2bc1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129702720) * c7065d2746df83be426ed7b0c2171d51516d0f8b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129716759) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9789: [FLINK-14260] Remove Mockito usage from TestingJobManagerRunnerFactory
flinkbot edited a comment on issue #9789: [FLINK-14260] Remove Mockito usage from TestingJobManagerRunnerFactory URL: https://github.com/apache/flink/pull/9789#issuecomment-535998019 ## CI report: * 8e00c88ee19da724ffd4dcf0a6bfea732141d2e9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129474364) * b6eac19eed03b58cd286c6a62c32e2a4ab669a28 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129707475) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on a change in pull request #9720: [FLINK-13025] Elasticsearch 7.x support
yanghua commented on a change in pull request #9720: [FLINK-13025] Elasticsearch 7.x support URL: https://github.com/apache/flink/pull/9720#discussion_r329662780 ## File path: flink-connectors/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE ## @@ -0,0 +1,45 @@ +flink-sql-connector-elasticsearch7 Review comment: > Quite some of the dependencies are test dependencies or transitive dependencies of test dependencies, I don't think they should be added here. Sorry, I do not know we should exclude test dependencies before adding missing dependencied into the notice file. Will remove them when we can decide excluded dependencied list. Besides the license issue, I want to know whether there is any other issue need to be processed? If there is, you can point out so that I can fixed it and at the same time we can wait reply from @twalthr . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14299) Factor status and system metrics out of JobManagerMetricGroup
Till Rohrmann created FLINK-14299:
-
Summary: Factor status and system metrics out of
JobManagerMetricGroup
Key: FLINK-14299
URL: https://issues.apache.org/jira/browse/FLINK-14299
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination, Runtime / Metrics
Affects Versions: 1.10.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Fix For: 1.10.0
At the moment, we use the {{JobManagerMetricGroup}} to not only register
{{Dispatcher}} specific metrics but also process specific metrics such as CPU,
threads, memory, etc. Due to this fact, it is not possible to close the
{{JobManagerMetricGroup}} when the life time of the {{Dispatcher}} terminates.
In order to do this, I suggest to introduce a new {{ProcessMetricGroup}} which
is used to register the process specific metrics.
In order to guarantee backwards compatibility, I suggest to use the same scope
format as {{SCOPE_NAMING_JM}} and then appending {{.Status}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient
flinkbot edited a comment on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient URL: https://github.com/apache/flink/pull/9820#issuecomment-536452582 ## CI report: * ff3ef7ae21616fb0295e3bdc53fa349c8f136a4b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129646465) * 654524c3508c89d08d5039a451ba425a5a2e3a41 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129666135) * 7f462e2cad4ca8866abccda492b53007a45b3407 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129712387) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9821: [FLINK-14298] Replace LeaderContender#getAddress with #getDescription
flinkbot edited a comment on issue #9821: [FLINK-14298] Replace LeaderContender#getAddress with #getDescription URL: https://github.com/apache/flink/pull/9821#issuecomment-536618253 ## CI report: * 9ca9df3eaa22965444373a3b6142798ca5559f50 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129712419) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout
flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout URL: https://github.com/apache/flink/pull/9794#issuecomment-536174033 ## CI report: * f9fbec1b59a8519c83bfb3dff4f6f1d9a785350b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129544195) * 5c98fa4029661182bfe847fbdd471900965f2bc1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129702720) * c7065d2746df83be426ed7b0c2171d51516d0f8b : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9786: [FLINK-14251] Add FutureUtils#forward utility
flinkbot edited a comment on issue #9786: [FLINK-14251] Add FutureUtils#forward utility URL: https://github.com/apache/flink/pull/9786#issuecomment-535968595 ## CI report: * 426cd9dc6c60f5582848fa52f2fa5f76182c8125 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129462607) * a5b734301cfacd1126fce46a1e70d0707f3f8a77 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129702675) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9787: [FLINK-14252][coordination] Encapsulate Dispatcher services in DispatcherServices
flinkbot edited a comment on issue #9787: [FLINK-14252][coordination] Encapsulate Dispatcher services in DispatcherServices URL: https://github.com/apache/flink/pull/9787#issuecomment-535981579 ## CI report: * 872b66ca40938834c13f34d32b3d340064c1bf95 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129466049) * 55a959a0d3fec886b6d83373add537c4f7d613ab : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129589016) * 65eaae5c87fd5a41a8499bf58f67a53d62a59115 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/129707366) * 501c04ea87ee9736e808772db44a61ad57206d78 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129712366) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration
flinkbot edited a comment on issue #9762: !IGNORE! ZK 3.5.5 shaded migration URL: https://github.com/apache/flink/pull/9762#issuecomment-534689957 ## CI report: * cefac57542dbd85a6693f6a1b7c8c90d91e85d30 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/128980426) * f04ce4e35191731da657f170c6949ba70d87b6e3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129575680) * 0250869c17414f0c0047b34fdf6a6ad4b57bf017 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129576444) * d2d2ea63507f9e4f798ac96169e4ca11921e4954 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129579078) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] aljoscha commented on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient
aljoscha commented on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient URL: https://github.com/apache/flink/pull/9820#issuecomment-536623553 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9821: [FLINK-14298] Replace LeaderContender#getAddress with #getDescription
flinkbot commented on issue #9821: [FLINK-14298] Replace LeaderContender#getAddress with #getDescription URL: https://github.com/apache/flink/pull/9821#issuecomment-536618253 ## CI report: * 9ca9df3eaa22965444373a3b6142798ca5559f50 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient
flinkbot edited a comment on issue #9820: [FLINK-14290] Decouple plan translation from job execution/ClusterClient URL: https://github.com/apache/flink/pull/9820#issuecomment-536452582 ## CI report: * ff3ef7ae21616fb0295e3bdc53fa349c8f136a4b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129646465) * 654524c3508c89d08d5039a451ba425a5a2e3a41 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129666135) * 7f462e2cad4ca8866abccda492b53007a45b3407 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhuzhurk commented on a change in pull request #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout
zhuzhurk commented on a change in pull request #9794: [FLINK-14247][runtime]
Use NoResourceAvailableException to wrap TimeoutException on slot allocation
timeout
URL: https://github.com/apache/flink/pull/9794#discussion_r329645418
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/DefaultScheduler.java
##
@@ -362,12 +365,22 @@ private static void propagateIfNonNull(final Throwable
throwable) {
.registerProducedPartitions(logicalSlot.getTaskManagerLocation(),
sendScheduleOrUpdateConsumerMessage);
executionVertex.tryAssignResource(logicalSlot);
} else {
- handleTaskFailure(executionVertexId, throwable);
+ handleTaskFailure(executionVertexId,
maybeWrapWithNoResourceAvailableException(throwable));
}
return null;
};
}
+ private Throwable maybeWrapWithNoResourceAvailableException(final
Throwable failure) {
+ final Throwable strippedThrowable =
ExceptionUtils.stripCompletionException(failure);
+ if (strippedThrowable instanceof TimeoutException) {
+ return new NoResourceAvailableException("Could not
allocate the required slot within slot request timeout. " +
Review comment:
Ok. The cause is included.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9813: [FLINK-14287] Decouple leader address from LeaderContender
flinkbot edited a comment on issue #9813: [FLINK-14287] Decouple leader address from LeaderContender URL: https://github.com/apache/flink/pull/9813#issuecomment-536324105 ## CI report: * b7a7e7b6dc8a16d23097911aa98852e2cf4d9c00 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129601641) * 3b8410b82ec366014551e35fb8791c6ad623faf9 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129707833) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9811: [FLINK-14285] Remove generics from Dispatcher factories
flinkbot edited a comment on issue #9811: [FLINK-14285] Remove generics from Dispatcher factories URL: https://github.com/apache/flink/pull/9811#issuecomment-536320588 ## CI report: * 658cddd4424505c1a904a926ad6e626abbd85cd7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129600090) * 5b4df7e3bd41053dcce049edf420ed584a42b98a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707753) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9810: [FLINK-14284] Add shut down future to Dispatcher
flinkbot edited a comment on issue #9810: [FLINK-14284] Add shut down future to Dispatcher URL: https://github.com/apache/flink/pull/9810#issuecomment-536320582 ## CI report: * 3ea0f47e7dd76ec115db4ef583b416685107604b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129600085) * e73e7e0a72952be963e29226bbb9b94487dafde6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707704) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9812: [FLINK-14286] Remove Akka specific parsing from LeaderConnectionInfo
flinkbot edited a comment on issue #9812: [FLINK-14286] Remove Akka specific parsing from LeaderConnectionInfo URL: https://github.com/apache/flink/pull/9812#issuecomment-536322371 ## CI report: * c1ca1131b02a2c555cd8386fd07aa1cfccbab161 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129600831) * da85c13656e6cf02fcadbadf50df0c201f07970c : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707797) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhuzhurk commented on a change in pull request #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout
zhuzhurk commented on a change in pull request #9794: [FLINK-14247][runtime]
Use NoResourceAvailableException to wrap TimeoutException on slot allocation
timeout
URL: https://github.com/apache/flink/pull/9794#discussion_r329645078
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/utils/SimpleSlotProvider.java
##
@@ -113,7 +114,11 @@ public SimpleSlotProvider(JobID jobId, int numSlots,
TaskManagerGateway taskMana
return
CompletableFuture.completedFuture(result);
}
else {
- return FutureUtils.completedExceptionally(new
NoResourceAvailableException());
+ if (allowQueued) {
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9809: [FLINK-14282] Simplify DispatcherResourceManagerComponent hierarchy
flinkbot edited a comment on issue #9809: [FLINK-14282] Simplify DispatcherResourceManagerComponent hierarchy URL: https://github.com/apache/flink/pull/9809#issuecomment-536318695 ## CI report: * 531cd688ceb25b544833025d9b556a0d686b29c4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129599298) * a3ba53aa778e777d8f38750eb6a5e5147fd21654 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707662) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9807: [FLINK-14280] Introduce DispatcherRunner
flinkbot edited a comment on issue #9807: [FLINK-14280] Introduce DispatcherRunner URL: https://github.com/apache/flink/pull/9807#issuecomment-536314902 ## CI report: * 9ef8fe986e4b06d6ae8512e5051933458363511c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129597654) * 748954650809fd2fa18572d1c925d3b647a55906 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707578) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9808: [FLINK-14281] Add DispatcherRunner#getShutDownFuture
flinkbot edited a comment on issue #9808: [FLINK-14281] Add DispatcherRunner#getShutDownFuture URL: https://github.com/apache/flink/pull/9808#issuecomment-536316924 ## CI report: * 02c93d5d7d69324c33461040fec5c4014eb2b8d6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129598507) * 618f898c4a74c7f35f66ba7ab37e08cf8d66581e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129707614) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9806: [FLINK-14277][tests] Upgrade InMemoryJobGraphStore to TestingJobGraphStore
flinkbot edited a comment on issue #9806: [FLINK-14277][tests] Upgrade InMemoryJobGraphStore to TestingJobGraphStore URL: https://github.com/apache/flink/pull/9806#issuecomment-536297732 ## CI report: * 195ce512365e9a2421ff00424a60242666bcf28f : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129590933) * 40047602faf287063110b3f315b9bed734360290 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129707521) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout
flinkbot edited a comment on issue #9794: [FLINK-14247][runtime] Use NoResourceAvailableException to wrap TimeoutException on slot allocation timeout URL: https://github.com/apache/flink/pull/9794#issuecomment-536174033 ## CI report: * f9fbec1b59a8519c83bfb3dff4f6f1d9a785350b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129544195) * 5c98fa4029661182bfe847fbdd471900965f2bc1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129702720) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9789: [FLINK-14260] Remove Mockito usage from TestingJobManagerRunnerFactory
flinkbot edited a comment on issue #9789: [FLINK-14260] Remove Mockito usage from TestingJobManagerRunnerFactory URL: https://github.com/apache/flink/pull/9789#issuecomment-535998019 ## CI report: * 8e00c88ee19da724ffd4dcf0a6bfea732141d2e9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129474364) * b6eac19eed03b58cd286c6a62c32e2a4ab669a28 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707475) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9790: [FLINK-14261] Add PermanentlyFencedRpcEndpoint
flinkbot edited a comment on issue #9790: [FLINK-14261] Add PermanentlyFencedRpcEndpoint URL: https://github.com/apache/flink/pull/9790#issuecomment-535998099 ## CI report: * f1bcc25d3af40fc482018fb8459bd8b942b15f68 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129474398) * 87ad063945f80e3070f8c9906a2eb89a91e2017b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707483) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9788: [FLINK-14259] Introduce JobManagerRunner interface
flinkbot edited a comment on issue #9788: [FLINK-14259] Introduce JobManagerRunner interface URL: https://github.com/apache/flink/pull/9788#issuecomment-535997974 ## CI report: * f0ef95eb93dd1becb31008367bb1b724f834fe25 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129474347) * 7cdb9038651812e334dfe307d74074748887b335 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129590280) * c3d883ef5d6c141ae4c8ba2564c5037ca6394af1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129707397) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9787: [FLINK-14252][coordination] Encapsulate Dispatcher services in DispatcherServices
flinkbot edited a comment on issue #9787: [FLINK-14252][coordination] Encapsulate Dispatcher services in DispatcherServices URL: https://github.com/apache/flink/pull/9787#issuecomment-535981579 ## CI report: * 872b66ca40938834c13f34d32b3d340064c1bf95 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129466049) * 55a959a0d3fec886b6d83373add537c4f7d613ab : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129589016) * 65eaae5c87fd5a41a8499bf58f67a53d62a59115 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/129707366) * 501c04ea87ee9736e808772db44a61ad57206d78 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
