[jira] [Updated] (FLINK-14370) KafkaProducerAtLeastOnceITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink fails on Travis
[
https://issues.apache.org/jira/browse/FLINK-14370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hequn Cheng updated FLINK-14370:
Fix Version/s: (was: 1.8.2)
1.8.3
> KafkaProducerAtLeastOnceITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink
> fails on Travis
> ---
>
> Key: FLINK-14370
> URL: https://issues.apache.org/jira/browse/FLINK-14370
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: Jiangjie Qin
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.10.0, 1.8.3, 1.9.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The
> {{KafkaProducerAtLeastOnceITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink}}
> fails on Travis with
> {code}
> Test
> testOneToOneAtLeastOnceRegularSink(org.apache.flink.streaming.connectors.kafka.KafkaProducerAtLeastOnceITCase)
> failed with:
> java.lang.AssertionError: Job should fail!
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaProducerTestBase.testOneToOneAtLeastOnce(KafkaProducerTestBase.java:280)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaProducerTestBase.testOneToOneAtLeastOnceRegularSink(KafkaProducerTestBase.java:206)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
> https://api.travis-ci.com/v3/job/244297223/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-14988) Table dependency shading fails
[
https://issues.apache.org/jira/browse/FLINK-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Wysakowicz closed FLINK-14988.
Resolution: Duplicate
> Table dependency shading fails
> --
>
> Key: FLINK-14988
> URL: https://issues.apache.org/jira/browse/FLINK-14988
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Affects Versions: 1.9.1
>Reporter: Chesnay Schepler
>Priority: Blocker
>
> https://api.travis-ci.org/v3/job/618168734/log.txt
> {code}
> ==
> Running 'Dependency shading of table modules test'
> ==
> TEST_DATA_DIR:
> /home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-39933628613
> Flink dist directory:
> /home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java/target/flink-table-api-java-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala/target/flink-table-api-scala_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java-bridge/target/flink-table-api-java-bridge_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala-bridge/target/flink-table-api-scala-bridge_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner/target/flink-table-planner_2.11-1.9-SNAPSHOT.jar
> jar.
> Failure: There are unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner-blink/target/flink-table-planner-blink_2.11-1.9-SNAPSHOT.jar
> jar: -> com.esotericsoftware.kryo not found
> [FAIL] Test script contains errors.
> Checking for errors...
> No errors in log files.
> Checking for exceptions...
> No exceptions in log files.
> Checking for non-empty .out files...
> grep:
> /home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT/log/*.out:
> No such file or directory
> No non-empty .out files.
> [FAIL] 'Dependency shading of table modules test' failed after 0 minutes and
> 27 seconds! Test exited with exit code 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-14235) Kafka010ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator fails on travis
[
https://issues.apache.org/jira/browse/FLINK-14235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hequn Cheng updated FLINK-14235:
Fix Version/s: 1.8.3
> Kafka010ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator
> fails on travis
> --
>
> Key: FLINK-14235
> URL: https://issues.apache.org/jira/browse/FLINK-14235
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.10.0
>Reporter: Piotr Nowojski
>Assignee: Piotr Nowojski
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.10.0, 1.8.3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Fails with following message:
> {noformat}
> Expected to contain all of: <[0]>, but was: <[]>
> {noformat}
> with the following stack trace:
> {noformat}
> Test
> testOneToOneAtLeastOnceCustomOperator(org.apache.flink.streaming.connectors.kafka.Kafka010ProducerITCase)
> failed with:
> java.lang.AssertionError: Expected to contain all of: <[0]>, but was: <[]>
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaTestBase.assertAtLeastOnceForTopic(KafkaTestBase.java:235)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaProducerTestBase.testOneToOneAtLeastOnce(KafkaProducerTestBase.java:289)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator(KafkaProducerTestBase.java:214)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {noformat}
> https://api.travis-ci.com/v3/job/239300010/log.txt
> This might be related to https://issues.apache.org/jira/browse/FLINK-14224
> but the failure message is different (here it's data loss, there the job has
> failed)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14235) Kafka010ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator fails on travis
[
https://issues.apache.org/jira/browse/FLINK-14235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984795#comment-16984795
]
Hequn Cheng commented on FLINK-14235:
-
Resolved in 1.8.3 via e0387a8007707ab29795e3aa3794ad279eaaeaf9
in master via c31e44e5402c5fd7deb10b83534740ac7f66d0f8
> Kafka010ProducerITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator
> fails on travis
> --
>
> Key: FLINK-14235
> URL: https://issues.apache.org/jira/browse/FLINK-14235
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.10.0
>Reporter: Piotr Nowojski
>Assignee: Piotr Nowojski
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Fails with following message:
> {noformat}
> Expected to contain all of: <[0]>, but was: <[]>
> {noformat}
> with the following stack trace:
> {noformat}
> Test
> testOneToOneAtLeastOnceCustomOperator(org.apache.flink.streaming.connectors.kafka.Kafka010ProducerITCase)
> failed with:
> java.lang.AssertionError: Expected to contain all of: <[0]>, but was: <[]>
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaTestBase.assertAtLeastOnceForTopic(KafkaTestBase.java:235)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaProducerTestBase.testOneToOneAtLeastOnce(KafkaProducerTestBase.java:289)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator(KafkaProducerTestBase.java:214)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {noformat}
> https://api.travis-ci.com/v3/job/239300010/log.txt
> This might be related to https://issues.apache.org/jira/browse/FLINK-14224
> but the failure message is different (here it's data loss, there the job has
> failed)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10356: [FLINK-14902][Connector-JDBC]JDBCTableSource Supports AsyncLookupFunction
flinkbot commented on issue #10356: [FLINK-14902][Connector-JDBC]JDBCTableSource Supports AsyncLookupFunction URL: https://github.com/apache/flink/pull/10356#issuecomment-559693220 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit f8a737240c2cee7e20d7cda06e2ea6fe0b232c00 (Fri Nov 29 07:47:31 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14902) JDBCTableSource support AsyncLookupFunction
[ https://issues.apache.org/jira/browse/FLINK-14902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-14902: --- Labels: pull-request-available (was: ) > JDBCTableSource support AsyncLookupFunction > --- > > Key: FLINK-14902 > URL: https://issues.apache.org/jira/browse/FLINK-14902 > Project: Flink > Issue Type: Improvement > Components: Connectors / JDBC >Affects Versions: 1.9.0 >Reporter: hailong wang >Assignee: hailong wang >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > > JDBCTableSource support AsyncLookupFunction -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] wangxlong opened a new pull request #10356: [FLINK-14902][Connector-JDBC]JDBCTableSource Supports AsyncLookupFunction
wangxlong opened a new pull request #10356: [FLINK-14902][Connector-JDBC]JDBCTableSource Supports AsyncLookupFunction URL: https://github.com/apache/flink/pull/10356 ## What is the purpose of the change JDBCTableSource supports AsyncLookupFunction for temporal table join ## Brief change log 1. Add JDBCConnection to wrap Connection and PreparedStatement. 2.Add ConnectionManager to manage JDBCConnection. 3.Share JDBCLookupOptions with JDBCLookupFunction and JDBCAsyncLookupFunction 4.ShareJDBCLookupBuilder with JDBCLookupFunction and JDBCAsyncLookupFunction 5.Add JDBCAsyncLookupFunction to support Async lookup JDBC. ## Verifying this change This change added tests and can be verified as follows: 1.JDBCLookupFunctionITCase#test ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] kl0u commented on issue #10346: [FLINK-14972] Make Remote(Stream)Environment use Executors.
kl0u commented on issue #10346: [FLINK-14972] Make Remote(Stream)Environment use Executors. URL: https://github.com/apache/flink/pull/10346#issuecomment-559691884 I addressed the comments. Let me know if now it is good to merge @TisonKun and @aljoscha This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] kl0u commented on a change in pull request #10346: [FLINK-14972] Make Remote(Stream)Environment use Executors.
kl0u commented on a change in pull request #10346: [FLINK-14972] Make
Remote(Stream)Environment use Executors.
URL: https://github.com/apache/flink/pull/10346#discussion_r352009886
##
File path:
flink-clients/src/main/java/org/apache/flink/client/deployment/executors/RemoteExecutor.java
##
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.client.deployment.executors;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.core.execution.Executor;
+
+/**
+ * The {@link Executor} to be used when executing a job on an already running
cluster.
+ */
+@Internal
+public class RemoteExecutor extends StandaloneSessionClusterExecutor {
Review comment:
Good point @TisonKun ! Thanks for spotting this. I will address that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-14813) Expose the new mechanism implemented in FLINK-14472 as a "is back-pressured" metric
[
https://issues.apache.org/jira/browse/FLINK-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984783#comment-16984783
]
lining commented on FLINK-14813:
FLINK-14815 is blocked by this JIRA, the previous link was wrong.
> Expose the new mechanism implemented in FLINK-14472 as a "is back-pressured"
> metric
> ---
>
> Key: FLINK-14813
> URL: https://issues.apache.org/jira/browse/FLINK-14813
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Network, Runtime / REST
>Reporter: lining
>Assignee: lining
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> { "id": "0.Shuffle.BackPressure.isBackPressured", "value": "true" }
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10256: [FLINK-14467][runtime] Let MesosJobClusterEntrypoint use user code class loader
flinkbot edited a comment on issue #10256: [FLINK-14467][runtime] Let MesosJobClusterEntrypoint use user code class loader URL: https://github.com/apache/flink/pull/10256#issuecomment-92815 ## CI report: * cb019fdc2ef071edc76f9d93ced50551605a4cae : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/137230882) * 3b3b1746da9e29cc162152fa39ed2ce330b3ba18 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-14956) MemoryMappedBoundedData Compressed Buffer Slicer
[ https://issues.apache.org/jira/browse/FLINK-14956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang closed FLINK-14956. Resolution: Duplicate > MemoryMappedBoundedData Compressed Buffer Slicer > > > Key: FLINK-14956 > URL: https://issues.apache.org/jira/browse/FLINK-14956 > Project: Flink > Issue Type: Improvement > Components: Runtime / Network >Reporter: Nicholas Jiang >Priority: Minor > Attachments: CPU-IO.png, Compress-Read.png, Compress-Write.png > > > MemoryMappedBoundedData, implementation of BoundedData simply through > ByteBuffers backed by memory, uses CompressedBufferSlicer which is > implementation of BoundedData.Reader to slice next buffer with uncompress. > CompressedBufferSlicer reads BoundedData by LZ4SafeDecompressor decompressing > byte buffer.When FileChannelMemoryMappedBoundedData tries to write buffer, > this uses LZ4Compressor to compress buffer to improve I/O performance. > Compress read process： > !Compress-Read.png|width=556,height=251! > Compress write process： > !Compress-Write.png|width=278,height=261! > CPU/IO performance comparsion chart： > !CPU-IO.png|width=416,height=312! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-14956) MemoryMappedBoundedData Compressed Buffer Slicer
[ https://issues.apache.org/jira/browse/FLINK-14956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984777#comment-16984777 ] zhijiang commented on FLINK-14956: -- I would close this duplicated issue to focus on FLINK-14845 only. > MemoryMappedBoundedData Compressed Buffer Slicer > > > Key: FLINK-14956 > URL: https://issues.apache.org/jira/browse/FLINK-14956 > Project: Flink > Issue Type: Improvement > Components: Runtime / Network >Reporter: Nicholas Jiang >Priority: Minor > Attachments: CPU-IO.png, Compress-Read.png, Compress-Write.png > > > MemoryMappedBoundedData, implementation of BoundedData simply through > ByteBuffers backed by memory, uses CompressedBufferSlicer which is > implementation of BoundedData.Reader to slice next buffer with uncompress. > CompressedBufferSlicer reads BoundedData by LZ4SafeDecompressor decompressing > byte buffer.When FileChannelMemoryMappedBoundedData tries to write buffer, > this uses LZ4Compressor to compress buffer to improve I/O performance. > Compress read process： > !Compress-Read.png|width=556,height=251! > Compress write process： > !Compress-Write.png|width=278,height=261! > CPU/IO performance comparsion chart： > !CPU-IO.png|width=416,height=312! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13550) Support for CPU FlameGraphs in new web UI
[ https://issues.apache.org/jira/browse/FLINK-13550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984769#comment-16984769 ] David Moravek commented on FLINK-13550: --- Hi [~xintongsong], we already use this in our internal build and I'm definitely planning to contribute this back. It'd be helpful if you could assign the issue to me. I'll try to send an initial PR with the rest endpoint within next week. > Support for CPU FlameGraphs in new web UI > - > > Key: FLINK-13550 > URL: https://issues.apache.org/jira/browse/FLINK-13550 > Project: Flink > Issue Type: New Feature > Components: Runtime / REST, Runtime / Web Frontend >Reporter: David Moravek >Priority: Major > > For a better insight into a running job, it would be useful to have ability > to render a CPU flame graph for a particular job vertex. > Flink already has a stack-trace sampling mechanism in-place, so it should be > straightforward to implement. > This should be done by implementing a new endpoint in REST API, which would > sample the stack-trace the same way as current BackPressureTracker does, only > with a different sampling rate and length of sampling. > [Here|https://www.youtube.com/watch?v=GUNDehj9z9o] is a little demo of the > feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API
flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-558081118 ## CI report: * aadc1cfef92eec54d86efbf39f50d91afda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138022323) * daf85a75b9c24918058b8bfe09416b2828bd02a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160048) * 7fe9b0e6c496482990905b9cec4389c8cbb8930a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138161977) * fe4dfd5ebd0508fc24dcfcb55aab4b1c99cd6bd3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138175553) * 7d98e670ac6e917eb2c121493ba0b5925240899d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138375416) * 08b96bbcc026a2feb1b257000a553521f4304d93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138380708) * 880bec8f2cc3097edeb0985e04db02d8fd205a2f : UNKNOWN * 16fd227a34688708f0c42fb3fc8ce2aad32555bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138401580) * 2fb0f5b45b8f6dd236cd7298b5644fb9767b90c4 : UNKNOWN * bd5087fd703553fbbc190260aeba03b595ba0c93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606066) * ca1cab6fd8f080cee0697fca6be7d29ec2d6b790 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/138653335) * 6d8f1afa7026a6bbab7a3d024fc67b8588e8f522 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138654905) * 7221e66b915522b5aa11a9b41c0f9e4056c8199b : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138659160) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14989) Improve efficiency by bitwise operation.
[ https://issues.apache.org/jira/browse/FLINK-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984744#comment-16984744 ] Zhenghua Gao commented on FLINK-14989: -- I'm afraid we should not do that. Modern compiler will optimize this away for you. And as [~TsReaper] mentioned above: you could do a benchmark to measure the benefit of the alternatives you try. > Improve efficiency by bitwise operation. > > > Key: FLINK-14989 > URL: https://issues.apache.org/jira/browse/FLINK-14989 > Project: Flink > Issue Type: Improvement > Components: Table SQL / Runtime >Reporter: Zhang Ziqiang >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > To improve performance, I replaced "/" with ">>", and replaced "*" with "<<". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13595) KafkaITCase.testBigRecordJob fails on Travis
[
https://issues.apache.org/jira/browse/FLINK-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984741#comment-16984741
]
Dian Fu commented on FLINK-13595:
-
Another instance: [https://api.travis-ci.org/v3/job/618270925/log.txt]
> KafkaITCase.testBigRecordJob fails on Travis
>
>
> Key: FLINK-13595
> URL: https://issues.apache.org/jira/browse/FLINK-13595
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Tests
>Affects Versions: 1.9.0
>Reporter: Till Rohrmann
>Assignee: Jiangjie Qin
>Priority: Critical
> Labels: test-stability
> Fix For: 1.10.0
>
>
> The {{KafkaITCase.testBigRecordJob}} failed with a {{TestTimedOutException}}
> on Travis.
> {code}
> Test
> testBigRecordJob(org.apache.flink.streaming.connectors.kafka.KafkaITCase)
> failed with:
> org.junit.runners.model.TestTimedOutException: test timed out after 6
> milliseconds
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1252)
> at java.lang.Thread.join(Thread.java:1326)
> at
> org.apache.kafka.clients.admin.KafkaAdminClient.close(KafkaAdminClient.java:476)
> at org.apache.kafka.clients.admin.AdminClient.close(AdminClient.java:92)
> at org.apache.kafka.clients.admin.AdminClient.close(AdminClient.java:75)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.deleteTestTopic(KafkaTestEnvironmentImpl.java:153)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaTestBase.deleteTestTopic(KafkaTestBase.java:204)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaConsumerTestBase.runBigRecordTestTopology(KafkaConsumerTestBase.java:1336)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaITCase.testBigRecordJob(KafkaITCase.java:121)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> https://api.travis-ci.org/v3/job/568176170/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation.
flinkbot edited a comment on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation. URL: https://github.com/apache/flink/pull/10355#issuecomment-559652719 ## CI report: * 5e8e90e2e98172438ded5aa1cd88d5a47b650339 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138657715) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API
flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-558081118 ## CI report: * aadc1cfef92eec54d86efbf39f50d91afda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138022323) * daf85a75b9c24918058b8bfe09416b2828bd02a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160048) * 7fe9b0e6c496482990905b9cec4389c8cbb8930a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138161977) * fe4dfd5ebd0508fc24dcfcb55aab4b1c99cd6bd3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138175553) * 7d98e670ac6e917eb2c121493ba0b5925240899d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138375416) * 08b96bbcc026a2feb1b257000a553521f4304d93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138380708) * 880bec8f2cc3097edeb0985e04db02d8fd205a2f : UNKNOWN * 16fd227a34688708f0c42fb3fc8ce2aad32555bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138401580) * 2fb0f5b45b8f6dd236cd7298b5644fb9767b90c4 : UNKNOWN * bd5087fd703553fbbc190260aeba03b595ba0c93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606066) * ca1cab6fd8f080cee0697fca6be7d29ec2d6b790 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/138653335) * 6d8f1afa7026a6bbab7a3d024fc67b8588e8f522 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138654905) * 7221e66b915522b5aa11a9b41c0f9e4056c8199b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138659160) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation.
flinkbot edited a comment on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation. URL: https://github.com/apache/flink/pull/10355#issuecomment-559652719 ## CI report: * 5e8e90e2e98172438ded5aa1cd88d5a47b650339 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138657715) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API
flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-558081118 ## CI report: * aadc1cfef92eec54d86efbf39f50d91afda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138022323) * daf85a75b9c24918058b8bfe09416b2828bd02a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160048) * 7fe9b0e6c496482990905b9cec4389c8cbb8930a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138161977) * fe4dfd5ebd0508fc24dcfcb55aab4b1c99cd6bd3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138175553) * 7d98e670ac6e917eb2c121493ba0b5925240899d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138375416) * 08b96bbcc026a2feb1b257000a553521f4304d93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138380708) * 880bec8f2cc3097edeb0985e04db02d8fd205a2f : UNKNOWN * 16fd227a34688708f0c42fb3fc8ce2aad32555bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138401580) * 2fb0f5b45b8f6dd236cd7298b5644fb9767b90c4 : UNKNOWN * bd5087fd703553fbbc190260aeba03b595ba0c93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606066) * ca1cab6fd8f080cee0697fca6be7d29ec2d6b790 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/138653335) * 6d8f1afa7026a6bbab7a3d024fc67b8588e8f522 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138654905) * 7221e66b915522b5aa11a9b41c0f9e4056c8199b : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
flinkbot edited a comment on issue #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource URL: https://github.com/apache/flink/pull/10354#issuecomment-559648653 ## CI report: * 39f094776ea236e08b18c471c36fc784d8e85bbc : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138656317) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation.
flinkbot commented on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation. URL: https://github.com/apache/flink/pull/10355#issuecomment-559652719 ## CI report: * 5e8e90e2e98172438ded5aa1cd88d5a47b650339 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation.
flinkbot commented on issue #10355: [FLINK-14989][table] Improve efficiency by bitwise operation. URL: https://github.com/apache/flink/pull/10355#issuecomment-559651151 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5e8e90e2e98172438ded5aa1cd88d5a47b650339 (Fri Nov 29 03:53:14 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14989).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhangzqit opened a new pull request #10355: [FLINK-14989][table] Improve efficiency by bitwise operation.
zhangzqit opened a new pull request #10355: [FLINK-14989][table] Improve efficiency by bitwise operation. URL: https://github.com/apache/flink/pull/10355 ## What is the purpose of the change *Improve efficiency by bitwise operation.* ## Brief change log - *To improve performance, I replaced "/" with ">>", and replaced "*" with "<<".* ## Verifying this change This change is a trivial rework / code optimization without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14989) Improve efficiency by bitwise operation.
[ https://issues.apache.org/jira/browse/FLINK-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-14989: --- Labels: pull-request-available (was: ) > Improve efficiency by bitwise operation. > > > Key: FLINK-14989 > URL: https://issues.apache.org/jira/browse/FLINK-14989 > Project: Flink > Issue Type: Improvement > Components: Table SQL / Runtime >Reporter: Zhang Ziqiang >Priority: Minor > Labels: pull-request-available > > To improve performance, I replaced "/" with ">>", and replaced "*" with "<<". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14990) Add bit functions to Blink planner
hailong wang created FLINK-14990: Summary: Add bit functions to Blink planner Key: FLINK-14990 URL: https://issues.apache.org/jira/browse/FLINK-14990 Project: Flink Issue Type: Improvement Components: Table SQL / API Affects Versions: 1.9.1 Reporter: hailong wang Fix For: 1.10.0 Add bit to functions Blink planner which be useful. Bit functions include：bitAnd、bitNot、bitOr 、bitXor -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
flinkbot commented on issue #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource URL: https://github.com/apache/flink/pull/10354#issuecomment-559648653 ## CI report: * 39f094776ea236e08b18c471c36fc784d8e85bbc : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API
flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-558081118 ## CI report: * aadc1cfef92eec54d86efbf39f50d91afda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138022323) * daf85a75b9c24918058b8bfe09416b2828bd02a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160048) * 7fe9b0e6c496482990905b9cec4389c8cbb8930a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138161977) * fe4dfd5ebd0508fc24dcfcb55aab4b1c99cd6bd3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138175553) * 7d98e670ac6e917eb2c121493ba0b5925240899d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138375416) * 08b96bbcc026a2feb1b257000a553521f4304d93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138380708) * 880bec8f2cc3097edeb0985e04db02d8fd205a2f : UNKNOWN * 16fd227a34688708f0c42fb3fc8ce2aad32555bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138401580) * 2fb0f5b45b8f6dd236cd7298b5644fb9767b90c4 : UNKNOWN * bd5087fd703553fbbc190260aeba03b595ba0c93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606066) * ca1cab6fd8f080cee0697fca6be7d29ec2d6b790 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/138653335) * 6d8f1afa7026a6bbab7a3d024fc67b8588e8f522 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138654905) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] guoweiM commented on a change in pull request #10256: [FLINK-14467][runtime] Let MesosJobClusterEntrypoint use user code class loader
guoweiM commented on a change in pull request #10256: [FLINK-14467][runtime]
Let MesosJobClusterEntrypoint use user code class loader
URL: https://github.com/apache/flink/pull/10256#discussion_r351968587
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/overlays/UserLibOverlay.java
##
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.clusterframework.overlays;
+
+import org.apache.flink.configuration.ConfigConstants;
+import org.apache.flink.core.fs.Path;
+import org.apache.flink.runtime.clusterframework.ContainerSpecification;
+
+import javax.annotation.Nullable;
+
+import java.io.File;
+import java.io.IOException;
+
+/**
+ * Overlays the user library into a container.
+ * The following directory and files in the directory are copied to the
container if it exists:
+ * - {@link ConfigConstants#DEFAULT_FLINK_USR_LIB_DIR}/
+ */
+public class UserLibOverlay extends AbstractContainerOverlay {
+
+ @Nullable
+ private final File usrLibDirectory;
+
+ private UserLibOverlay(@Nullable File usrLibDirectory) {
+ this.usrLibDirectory = usrLibDirectory;
+ }
+
+ @Override
+ public void configure(ContainerSpecification container) throws
IOException {
+ if (usrLibDirectory != null) {
+ addPathRecursively(usrLibDirectory, new
Path(FlinkDistributionOverlay.getTargetRoot().getName()), container);
Review comment:
I got it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
flinkbot commented on issue #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource URL: https://github.com/apache/flink/pull/10354#issuecomment-559645834 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 39f094776ea236e08b18c471c36fc784d8e85bbc (Fri Nov 29 03:15:10 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14729).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14729) Multi-topics consuming from KafkaTableSource
[ https://issues.apache.org/jira/browse/FLINK-14729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984720#comment-16984720 ] Fang Peng Cheng commented on FLINK-14729: - new PR, https://github.com/apache/flink/pull/10354 > Multi-topics consuming from KafkaTableSource > > > Key: FLINK-14729 > URL: https://issues.apache.org/jira/browse/FLINK-14729 > Project: Flink > Issue Type: New Feature > Components: Connectors / Kafka >Reporter: Leo Zhang >Priority: Major > Labels: features, pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Hi, all. I propose a new functionality of KafkaTableSource which can consume > multiple topics at the same time. > *Design plan* > * Add a new constructor in KafkaTableSource which accepts topics with List > type as one parameter. > * Modify the existed one which only accepts one topic as string type to call > the proposed one to finish the instantiation. That is to say, wrap this topic > in a list and pass it to the multi-topics-consuming constructor. > * Modify the overridden method createKafkaConsumer in KafkaTableSource to > pass topics as List instead of String. > * Replace the field topic with topics as List type in KafkaTableSourceBase > and modify every place using topic with topics. So we just need to modify the > constructor KafkaTableSourceBase, method getDataStream, and equals and > hashCode. > *Test plan* > There is less to do as KafkaTableSource is based on FlinkKafkaConsumer which > already supports consuming multiple topics and is tested well. Of course, we > can easily add further more tests if needed. > > So what's your opinion? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] fangpengcheng95 opened a new pull request #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
fangpengcheng95 opened a new pull request #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource URL: https://github.com/apache/flink/pull/10354 ## What is the purpose of the change propose a new functionality of KafkaTableSource which can consume multiple topics at the same time. ## Brief change log Design plan Add a new constructor in KafkaTableSource which accepts topics with List type as one parameter. Modify the existed one which only accepts one topic as string type to call the proposed one to finish the instantiation. That is to say, wrap this topic in a list and pass it to the multi-topics-consuming constructor. Modify the overridden method createKafkaConsumer in KafkaTableSource to pass topics as List instead of String. Replace the field topic with topics as List type in KafkaTableSourceBase and modify every place using topic with topics. So we just need to modify the constructor KafkaTableSourceBase, method getDataStream, and equals and hashCode. ## Verifying this change Test plan There is less to do as KafkaTableSource is based on FlinkKafkaConsumer which already supports consuming multiple topics and is tested well. Of course, we can easily add further more tests if needed. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14989) Improve efficiency by bitwise operation.
[ https://issues.apache.org/jira/browse/FLINK-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984718#comment-16984718 ] Caizhi Weng commented on FLINK-14989: - Hi Ziqiang, Thanks for the JIRA. Which part are you going to replace multiplication and division with bitwise operators? Have you measured the performance before and after modification and what's the result? > Improve efficiency by bitwise operation. > > > Key: FLINK-14989 > URL: https://issues.apache.org/jira/browse/FLINK-14989 > Project: Flink > Issue Type: Improvement > Components: Table SQL / Runtime >Reporter: Zhang Ziqiang >Priority: Minor > > To improve performance, I replaced "/" with ">>", and replaced "*" with "<<". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-14946) Retraction infer would result in bad plan under corner case in blink planner
[
https://issues.apache.org/jira/browse/FLINK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984717#comment-16984717
]
lincoln lee edited comment on FLINK-14946 at 11/29/19 3:03 AM:
---
[~jinyu.zj] I think your two solutions can be done separately and parallel.
For the 1st, we can submit an issue to Calcite community to confirm if the
garbage collection behaviors right in such a use case.
For the 2nd, I agree with you that switch the match order from *BOTTOM_UP* to
*TOP_DOWN* becase it's align with the propagation sequence on the retraction
inference phase.
was (Author: lincoln.86xy):
[~jinyu.zj] I think your two solutions can be done separately and parallel.
For the 1st, we can submit an issue to Calcite community to confirm if the
garbage collection behaviors right in such a use case.
For the 2nd, I agree with you that switch the match order from
\{code}BOTTOM_UP\{code} to \{code}TOP_DOWN\{code} becase it's align with the
propagation sequence on the retraction inference phase.
> Retraction infer would result in bad plan under corner case in blink planner
>
>
> Key: FLINK-14946
> URL: https://issues.apache.org/jira/browse/FLINK-14946
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Jing Zhang
>Priority: Major
> Attachments: RetractionRules1Test.scala,
> image-2019-11-26-14-54-34-797.png, screenshot-1.png, screenshot-2.png,
> screenshot-3.png, screenshot-4.png, screenshot-5.png, screenshot-6.png
>
>
> Retractions rule would result in bad plan under some case, I simplify the
> case like the following sql, complete test case could be found in attachments.
> {code:scala}
> val join_sql =
> """
> |SELECT
> | ll.a AS a,
> | ll.b AS b,
> | cnt
> |FROM (
> | SELECT a, b, COUNT(c) AS cnt FROM l GROUP BY a, b
> |) ll
> |JOIN (
> | SELECT a, b FROM r GROUP BY a, b
> |) rr ON
> |(ll.a = rr.a AND ll.b = rr.b)
> """.stripMargin !image-2019-11-26-14-52-52-824.png!
> val sqlQuery =
> s"""
> |SELECT a, b_1, SUM(cnt) AS cnt
> |FROM (
> | SELECT *, b AS b_1 FROM (${join_sql})
> | UNION ALL
> | SELECT *, 'SEA' AS b_1 FROM (${join_sql})
> |) AS total_result
> |GROUP BY a, b_1
> """.stripMargin
> {code}
> The plan is :
> !image-2019-11-26-14-54-34-797.png!
> After retraction infer, we expect two join node in the above plan has
> `AccRetract` asAccMode. However, AccMode of Join1 is right, accMode of Join2
> is unexpected.
> I find the `SetAccModeRule` never apply to Join2 because before actually
> apply `SetAccModeRule` to Join2, HepPlanner would check if the vertex belongs
> to DAG or not, and the result is false. So HepPlanner will not apply
> `SetAccModeRule` to Join2.
> !screenshot-1.png!
>
> Here is detailed follow-up process:
> 1. Join2 matches `SetUpdatesAsRetractionRule`, results in an equivalent Join
> node (called Join2') with new children which has UpdateAsRetractionTrait with
> true flag
> 2. New right child of Join2, which is Exchange, matches
> `SetUpdatesAsRetractionRule`, results in an equivalent Exchange node (called
> Exchange-right') with new inputs which has UpdateAsRetractionTrait with true
> flag
> 3. New left child of Join2 matches `SetUpdatesAsRetractionRule`, similar as
> step2, generate an equivalent node called (called Exchange-left')
> 4. Join1 matches `SetUpdatesAsRetractionRule`, results in an equivalent Join
> node (called Join1'), similar as step1
> 5. New right child of Join1, which is Exchange, matches
> `SetUpdatesAsRetractionRule`, results in an equivalent Exchange node with new
> inputs which has UpdateAsRetractionTrait with true flag, *however, HepPlanner
> find digest of new Exchange is same as that Exchange-right' in step2*,
> HepPlanner will not create new vertex, but reuse the vertex which contain
> Exchange-right'
> 6. New left child of Join1, which is Exchange, matches
> `SetUpdatesAsRetractionRule`, results in an equivalent Exchange node with
> new inputs which has UpdateAsRetractionTrait with true flag, *however,
> HepPlanner find digest of new Exchange is same as that Exchange-left' in
> step3*, HepPlanner will not create new vertex, but reuse the vertex which
> contain Exchange-left'. Besides, HepPlanner would replace inputs of parent of
> Exchange, (namely Join1) from old Exchange to new Exchange in
> `contractVertices` methods.
> !screenshot-2.png!
> !screenshot-3.png!
> In `updateVertex`, Join1' would put newKey and the vertex to
> `mapDigestToVertex`. However Join1' digest is exactly same as
[GitHub] [flink] flinkbot edited a comment on issue #10353: [docs] remove surplus semicolon in Utils
flinkbot edited a comment on issue #10353: [docs] remove surplus semicolon in Utils URL: https://github.com/apache/flink/pull/10353#issuecomment-559639650 ## CI report: * cf2b40b6d31449d3ec41b2b94a4685d64adf9777 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138653342) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14946) Retraction infer would result in bad plan under corner case in blink planner
[
https://issues.apache.org/jira/browse/FLINK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984717#comment-16984717
]
lincoln lee commented on FLINK-14946:
-
[~jinyu.zj] I think your two solutions can be done separately and parallel.
For the 1st, we can submit an issue to Calcite community to confirm if the
garbage collection behaviors right in such a use case.
For the 2nd, I agree with you that switch the match order from
\{code}BOTTOM_UP\{code} to \{code}TOP_DOWN\{code} becase it's align with the
propagation sequence on the retraction inference phase.
> Retraction infer would result in bad plan under corner case in blink planner
>
>
> Key: FLINK-14946
> URL: https://issues.apache.org/jira/browse/FLINK-14946
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Jing Zhang
>Priority: Major
> Attachments: RetractionRules1Test.scala,
> image-2019-11-26-14-54-34-797.png, screenshot-1.png, screenshot-2.png,
> screenshot-3.png, screenshot-4.png, screenshot-5.png, screenshot-6.png
>
>
> Retractions rule would result in bad plan under some case, I simplify the
> case like the following sql, complete test case could be found in attachments.
> {code:scala}
> val join_sql =
> """
> |SELECT
> | ll.a AS a,
> | ll.b AS b,
> | cnt
> |FROM (
> | SELECT a, b, COUNT(c) AS cnt FROM l GROUP BY a, b
> |) ll
> |JOIN (
> | SELECT a, b FROM r GROUP BY a, b
> |) rr ON
> |(ll.a = rr.a AND ll.b = rr.b)
> """.stripMargin !image-2019-11-26-14-52-52-824.png!
> val sqlQuery =
> s"""
> |SELECT a, b_1, SUM(cnt) AS cnt
> |FROM (
> | SELECT *, b AS b_1 FROM (${join_sql})
> | UNION ALL
> | SELECT *, 'SEA' AS b_1 FROM (${join_sql})
> |) AS total_result
> |GROUP BY a, b_1
> """.stripMargin
> {code}
> The plan is :
> !image-2019-11-26-14-54-34-797.png!
> After retraction infer, we expect two join node in the above plan has
> `AccRetract` asAccMode. However, AccMode of Join1 is right, accMode of Join2
> is unexpected.
> I find the `SetAccModeRule` never apply to Join2 because before actually
> apply `SetAccModeRule` to Join2, HepPlanner would check if the vertex belongs
> to DAG or not, and the result is false. So HepPlanner will not apply
> `SetAccModeRule` to Join2.
> !screenshot-1.png!
>
> Here is detailed follow-up process:
> 1. Join2 matches `SetUpdatesAsRetractionRule`, results in an equivalent Join
> node (called Join2') with new children which has UpdateAsRetractionTrait with
> true flag
> 2. New right child of Join2, which is Exchange, matches
> `SetUpdatesAsRetractionRule`, results in an equivalent Exchange node (called
> Exchange-right') with new inputs which has UpdateAsRetractionTrait with true
> flag
> 3. New left child of Join2 matches `SetUpdatesAsRetractionRule`, similar as
> step2, generate an equivalent node called (called Exchange-left')
> 4. Join1 matches `SetUpdatesAsRetractionRule`, results in an equivalent Join
> node (called Join1'), similar as step1
> 5. New right child of Join1, which is Exchange, matches
> `SetUpdatesAsRetractionRule`, results in an equivalent Exchange node with new
> inputs which has UpdateAsRetractionTrait with true flag, *however, HepPlanner
> find digest of new Exchange is same as that Exchange-right' in step2*,
> HepPlanner will not create new vertex, but reuse the vertex which contain
> Exchange-right'
> 6. New left child of Join1, which is Exchange, matches
> `SetUpdatesAsRetractionRule`, results in an equivalent Exchange node with
> new inputs which has UpdateAsRetractionTrait with true flag, *however,
> HepPlanner find digest of new Exchange is same as that Exchange-left' in
> step3*, HepPlanner will not create new vertex, but reuse the vertex which
> contain Exchange-left'. Besides, HepPlanner would replace inputs of parent of
> Exchange, (namely Join1) from old Exchange to new Exchange in
> `contractVertices` methods.
> !screenshot-2.png!
> !screenshot-3.png!
> In `updateVertex`, Join1' would put newKey and the vertex to
> `mapDigestToVertex`. However Join1' digest is exactly same as Join2' digest.
> Besides, `mapDigestToVertex` already contains same key with Vertex contains
> Join2'. *So the operation would replace the value of newKey in
> `mapDigestToVertex` from Join2' to Join1'*
> 7. Join1' matches `SetAccModeRule`, results in an equivalent Join node
> (called Join1'') with AccRetract as AccMode. After apply the rule, HepPlanner
> starts collectGarbage, Join1' would be added in sweepSet because it's not
> reachable from root, so
[jira] [Created] (FLINK-14989) Improve efficiency by bitwise operation.
Zhang Ziqiang created FLINK-14989: - Summary: Improve efficiency by bitwise operation. Key: FLINK-14989 URL: https://issues.apache.org/jira/browse/FLINK-14989 Project: Flink Issue Type: Improvement Components: Table SQL / Runtime Reporter: Zhang Ziqiang To improve performance, I replaced "/" with ">>", and replaced "*" with "<<". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API
flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-558081118 ## CI report: * aadc1cfef92eec54d86efbf39f50d91afda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138022323) * daf85a75b9c24918058b8bfe09416b2828bd02a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160048) * 7fe9b0e6c496482990905b9cec4389c8cbb8930a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138161977) * fe4dfd5ebd0508fc24dcfcb55aab4b1c99cd6bd3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138175553) * 7d98e670ac6e917eb2c121493ba0b5925240899d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138375416) * 08b96bbcc026a2feb1b257000a553521f4304d93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138380708) * 880bec8f2cc3097edeb0985e04db02d8fd205a2f : UNKNOWN * 16fd227a34688708f0c42fb3fc8ce2aad32555bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138401580) * 2fb0f5b45b8f6dd236cd7298b5644fb9767b90c4 : UNKNOWN * bd5087fd703553fbbc190260aeba03b595ba0c93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606066) * ca1cab6fd8f080cee0697fca6be7d29ec2d6b790 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138653335) * 6d8f1afa7026a6bbab7a3d024fc67b8588e8f522 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] TisonKun commented on issue #10311: [FLINK-14762][client] Enrich JobClient API
TisonKun commented on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-559643033 tison 10:48 AM I narrow the change set to only unwrap accumulator inside client codes. Here is the diff 6d8f1afa7026a6bbab7a3d024fc67b8588e8f522 So the remain concern from my side is about core variant of JobStatus . I will be ok if you can describe how we deal with these two JobStatus in the future. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] jinglining edited a comment on issue #10247: [FLINK-14813][runtime]Expose the new mechanism implemented in FLINK-1…
jinglining edited a comment on issue #10247: [FLINK-14813][runtime]Expose the new mechanism implemented in FLINK-1… URL: https://github.com/apache/flink/pull/10247#issuecomment-559642356 Could we create a new metric called BackPressMetrics for it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] jinglining edited a comment on issue #10247: [FLINK-14813][runtime]Expose the new mechanism implemented in FLINK-1…
jinglining edited a comment on issue #10247: [FLINK-14813][runtime]Expose the new mechanism implemented in FLINK-1… URL: https://github.com/apache/flink/pull/10247#issuecomment-559642356 Could we create a new metrics called BackPressMetrics for it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] jinglining commented on issue #10247: [FLINK-14813][runtime]Expose the new mechanism implemented in FLINK-1…
jinglining commented on issue #10247: [FLINK-14813][runtime]Expose the new mechanism implemented in FLINK-1… URL: https://github.com/apache/flink/pull/10247#issuecomment-559642356 could we create a new metrics called BackPressMetrics for it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351963262
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351963949
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
Review comment:
```suggestion
```
lowWatermark is no longer needed here
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351963262
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351962703
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984714#comment-16984714
]
Zhenghua Gao commented on FLINK-14987:
--
Yes. The JDBCTableSource can't support DataTypes.DECIMAL (which support
precision and scale) now. The reason is the JDBCTableSource use a deprecated
interface *getReturnType* for the result type, which is the old type system
style. The recommended way is to use *getProducedDataType* instead which use
the new type system on DataTypes.
[~jark] [~ykt836] Should we repair all connectors to support new type system in
1.10 ?
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Priority: Blocker
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351963149
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
-for (let i = 0; i < node.parallelism; i++) {
- listOfMetricId.push(`${i}.currentInputWatermark`);
-}
-return this.metricService.getMetrics(this.jobId, node.id,
listOfMetricId).pipe(
- map(metrics => {
-let minValue = NaN;
-const watermarks: { [index: string]: number } = {};
-for (const key in metrics.values) {
- const value = metrics.values[key];
- const subtaskIndex = key.replace('.currentInputWatermark', '');
- watermarks[subtaskIndex] = value;
- if (isNaN(minValue) || value < minValue) {
-minValue = value;
- }
-}
-if (!isNaN(minValue) && minValue > LONG_MIN_VALUE) {
- lowWatermark = minValue;
-} else {
- lowWatermark = NaN;
-}
+return this.metricService.getWatermarks(this.jobId, node.id).pipe(
+ map(result => {
+lowWatermark = result.lowWatermark
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351962827
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
-for (let i = 0; i < node.parallelism; i++) {
- listOfMetricId.push(`${i}.currentInputWatermark`);
-}
-return this.metricService.getMetrics(this.jobId, node.id,
listOfMetricId).pipe(
- map(metrics => {
-let minValue = NaN;
-const watermarks: { [index: string]: number } = {};
-for (const key in metrics.values) {
- const value = metrics.values[key];
- const subtaskIndex = key.replace('.currentInputWatermark', '');
- watermarks[subtaskIndex] = value;
- if (isNaN(minValue) || value < minValue) {
-minValue = value;
- }
-}
-if (!isNaN(minValue) && minValue > LONG_MIN_VALUE) {
- lowWatermark = minValue;
-} else {
- lowWatermark = NaN;
-}
+return this.metricService.getWatermarks(this.jobId, node.id).pipe(
+ map(result => {
+lowWatermark = result.lowWatermark
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351962926
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
-for (let i = 0; i < node.parallelism; i++) {
- listOfMetricId.push(`${i}.currentInputWatermark`);
-}
-return this.metricService.getMetrics(this.jobId, node.id,
listOfMetricId).pipe(
- map(metrics => {
-let minValue = NaN;
-const watermarks: { [index: string]: number } = {};
-for (const key in metrics.values) {
- const value = metrics.values[key];
- const subtaskIndex = key.replace('.currentInputWatermark', '');
- watermarks[subtaskIndex] = value;
- if (isNaN(minValue) || value < minValue) {
-minValue = value;
- }
-}
-if (!isNaN(minValue) && minValue > LONG_MIN_VALUE) {
- lowWatermark = minValue;
-} else {
- lowWatermark = NaN;
-}
+return this.metricService.getWatermarks(this.jobId, node.id).pipe(
+ map(result => {
+lowWatermark = result.lowWatermark
return { ...node, lowWatermark };
Review comment:
```suggestion
return { ...node, lowWatermark: result.lowWatermark };
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351962827
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
-for (let i = 0; i < node.parallelism; i++) {
- listOfMetricId.push(`${i}.currentInputWatermark`);
-}
-return this.metricService.getMetrics(this.jobId, node.id,
listOfMetricId).pipe(
- map(metrics => {
-let minValue = NaN;
-const watermarks: { [index: string]: number } = {};
-for (const key in metrics.values) {
- const value = metrics.values[key];
- const subtaskIndex = key.replace('.currentInputWatermark', '');
- watermarks[subtaskIndex] = value;
- if (isNaN(minValue) || value < minValue) {
-minValue = value;
- }
-}
-if (!isNaN(minValue) && minValue > LONG_MIN_VALUE) {
- lowWatermark = minValue;
-} else {
- lowWatermark = NaN;
-}
+return this.metricService.getWatermarks(this.jobId, node.id).pipe(
+ map(result => {
+lowWatermark = result.lowWatermark
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351960228
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
-for (let i = 0; i < node.parallelism; i++) {
- listOfMetricId.push(`${i}.currentInputWatermark`);
-}
-return this.metricService.getMetrics(this.jobId, node.id,
listOfMetricId).pipe(
- map(metrics => {
-let minValue = NaN;
-const watermarks: { [index: string]: number } = {};
-for (const key in metrics.values) {
- const value = metrics.values[key];
- const subtaskIndex = key.replace('.currentInputWatermark', '');
- watermarks[subtaskIndex] = value;
- if (isNaN(minValue) || value < minValue) {
-minValue = value;
- }
-}
-if (!isNaN(minValue) && minValue > LONG_MIN_VALUE) {
- lowWatermark = minValue;
-} else {
- lowWatermark = NaN;
-}
+return this.metricService.getWatermarks(this.jobId, node.id).pipe(
+ map(result => {
+lowWatermark = result.lowWatermark
Review comment:
miss 'semicolon' here
```suggestion
lowWatermark = result.lowWatermark;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351962703
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10353: [docs] remove surplus semicolon in Utils
flinkbot commented on issue #10353: [docs] remove surplus semicolon in Utils URL: https://github.com/apache/flink/pull/10353#issuecomment-559639650 ## CI report: * cf2b40b6d31449d3ec41b2b94a4685d64adf9777 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API
flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-558081118 ## CI report: * aadc1cfef92eec54d86efbf39f50d91afda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138022323) * daf85a75b9c24918058b8bfe09416b2828bd02a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160048) * 7fe9b0e6c496482990905b9cec4389c8cbb8930a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138161977) * fe4dfd5ebd0508fc24dcfcb55aab4b1c99cd6bd3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138175553) * 7d98e670ac6e917eb2c121493ba0b5925240899d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138375416) * 08b96bbcc026a2feb1b257000a553521f4304d93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138380708) * 880bec8f2cc3097edeb0985e04db02d8fd205a2f : UNKNOWN * 16fd227a34688708f0c42fb3fc8ce2aad32555bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138401580) * 2fb0f5b45b8f6dd236cd7298b5644fb9767b90c4 : UNKNOWN * bd5087fd703553fbbc190260aeba03b595ba0c93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606066) * ca1cab6fd8f080cee0697fca6be7d29ec2d6b790 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351960228
##
File path:
flink-runtime-web/web-dashboard/src/app/pages/job/overview/job-overview.component.ts
##
@@ -66,28 +65,10 @@ export class JobOverviewComponent implements OnInit,
OnDestroy {
mergeWithWatermarks(nodes: NodesItemCorrectInterface[]):
Observable {
return forkJoin(
nodes.map(node => {
-const listOfMetricId = [];
let lowWatermark = NaN;
-for (let i = 0; i < node.parallelism; i++) {
- listOfMetricId.push(`${i}.currentInputWatermark`);
-}
-return this.metricService.getMetrics(this.jobId, node.id,
listOfMetricId).pipe(
- map(metrics => {
-let minValue = NaN;
-const watermarks: { [index: string]: number } = {};
-for (const key in metrics.values) {
- const value = metrics.values[key];
- const subtaskIndex = key.replace('.currentInputWatermark', '');
- watermarks[subtaskIndex] = value;
- if (isNaN(minValue) || value < minValue) {
-minValue = value;
- }
-}
-if (!isNaN(minValue) && minValue > LONG_MIN_VALUE) {
- lowWatermark = minValue;
-} else {
- lowWatermark = NaN;
-}
+return this.metricService.getWatermarks(this.jobId, node.id).pipe(
+ map(result => {
+lowWatermark = result.lowWatermark
Review comment:
miss 'semicolon' here
```suggestion
lowWatermark = result.lowWatermark;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10353: [docs] remove surplus semicolon in Utils
flinkbot commented on issue #10353: [docs] remove surplus semicolon in Utils URL: https://github.com/apache/flink/pull/10353#issuecomment-559636956 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit cf2b40b6d31449d3ec41b2b94a4685d64adf9777 (Fri Nov 29 02:15:00 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **Invalid pull request title: No valid Jira ID provided** Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] fangpengcheng95 opened a new pull request #10353: [docs] remove surplus semicolon in Utils
fangpengcheng95 opened a new pull request #10353: [docs] remove surplus semicolon in Utils URL: https://github.com/apache/flink/pull/10353 ## What is the purpose of the change remove surplus semicolon in Utils ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add dedicated watermarks metric retrieval endpoint
vthinkxie commented on a change in pull request #10238: [FLINK-8949] Add
dedicated watermarks metric retrieval endpoint
URL: https://github.com/apache/flink/pull/10238#discussion_r351959019
##
File path: flink-runtime-web/web-dashboard/src/app/services/metrics.service.ts
##
@@ -79,36 +79,37 @@ export class MetricsService {
}
/**
- * Get watermarks data
+ * Gets the watermarks for a given vertex id.
* @param jobId
* @param vertexId
- * @param parallelism
*/
- getWatermarks(jobId: string, vertexId: string, parallelism: number) {
-const listOfMetricName = new Array(parallelism).fill(0).map((_, index) =>
`${index}.currentInputWatermark`);
-return this.getMetrics(jobId, vertexId, listOfMetricName).pipe(
- map(metrics => {
-let minValue = NaN;
-let lowWatermark = NaN;
-const watermarks: { [id: string]: number } = {};
-const ref = metrics.values;
-for (const key in ref) {
- const value = ref[key];
- const subTaskIndex = key.replace('.currentInputWatermark', '');
- watermarks[subTaskIndex] = value;
- if (isNaN(minValue) || value < minValue) {
-minValue = value;
+ getWatermarks(jobId: string, vertexId: string) {
+return this.httpClient
+ .get>(
+`${BASE_URL}/jobs/${jobId}/vertices/${vertexId}/watermarks`
+ )
+ .pipe(
+map(arr => {
+ let minValue = NaN;
+ let lowWatermark = NaN;
+ const watermarks: { [id: string]: number } = {};
+ arr.forEach(item => {
+const value = parseInt(item.value, 10);
Review comment:
is it a desire to drop the decimal here, if not I suggest to use
`+item.value`
```suggestion
const value = +item.value;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] Rishi55 commented on issue #9468: [FLINK-13689] [Connectors/ElasticSearch] Fix thread leak when elasticsearch6 rest high level cli…
Rishi55 commented on issue #9468: [FLINK-13689] [Connectors/ElasticSearch] Fix thread leak when elasticsearch6 rest high level cli… URL: https://github.com/apache/flink/pull/9468#issuecomment-559636145 > @Rishi55 Any updates on this? @aljoscha I need some help in how to call ping(). I have been contacting my friends but couldn't get anything. Can you please point me to someone who can help me? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] TisonKun commented on issue #10311: [FLINK-14762][client] Enrich JobClient API
TisonKun commented on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-559635706 cherry-pick from slack channel. feel free to react wherever you like. >Sorry but when rebasing I cannot convince myself about why we introduce a flink-core variant of `JobStatus`? `ClusterClient` will return runtime `JobStatus` while `JobClient` returns `JobStatus`. It doesn’t make sense to me for introducing such different. >Runtime version `JobStatus` doesn’t depend on anything inside runtime but a self-contained enum. Shall we add it into `o.a.f.api.common`? Different from ClosureCleaner which could be used by connectors I think `JobStatus` is previously totally internal concept that should not breaks user setups and dependencies if we move it. >I’ve pushed a set of commits that we all agree on. The remain problem is about `getJobStatus` and `getAccumulator` > >for `getJobStatus` the main concern is about where `JobStatus` stays and whether we introduce a variant of `JobStatus`. My opinion is above. for `getAccumulator` the main concern is about whether Flink does unpack job for the user. I think we can do so, but maybe in another pass of pull request so that we firstly move forward this set under consensus. So my idea is that we commit this set of commit as part 1 of FLINK-14762 and I start a new pull request refactor `getAccumulator` and then implement its `JobClient` interface. While let’s align about `JobStatus` . Another coin about `JobStatus` is that we already display this sort of status on WebUI so it is reasonable to be core/common api(at least it is effectively user-facing). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] fangpengcheng95 closed pull request #10343: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
fangpengcheng95 closed pull request #10343: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource URL: https://github.com/apache/flink/pull/10343 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10343: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
flinkbot edited a comment on issue #10343: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource URL: https://github.com/apache/flink/pull/10343#issuecomment-559320006 ## CI report: * 42fbaee718ac9509938b448b80f1b78ba3d7cd52 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138507502) * 0b765573aec7a60ed6ba17a10757ebfe6970edb3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138512992) * 28e8bc19d98e9cbad3fcaf02c4344c1dea468134 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138533256) * 31a5d3deff6c5d1c8eff5ac6367523a3716d7f5b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138649363) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10343: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
flinkbot edited a comment on issue #10343: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource URL: https://github.com/apache/flink/pull/10343#issuecomment-559320006 ## CI report: * 42fbaee718ac9509938b448b80f1b78ba3d7cd52 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138507502) * 0b765573aec7a60ed6ba17a10757ebfe6970edb3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138512992) * 28e8bc19d98e9cbad3fcaf02c4344c1dea468134 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138533256) * 31a5d3deff6c5d1c8eff5ac6367523a3716d7f5b : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler
flinkbot edited a comment on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler URL: https://github.com/apache/flink/pull/10114#issuecomment-551014940 ## CI report: * c436371614d7db5dfc8c97e01a4594c3139715db : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/135418856) * d2f3a108547945633de80938684aa9c750534269 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/135432444) * dcb4596a1246234038c1f327d9e858b7c654966e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135445805) * 3bf7b045474d4bf278eb416491c32c9fb3c218a9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135495382) * 90152b07e795bc931c37b4baf2092ba0ee9219f1 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135649425) * 1657e4dd1f3ecbfa2956552efd401641c94a56aa : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135963847) * 31de348cd4fc6f1c41cdd94853c1e5ecb8206b4c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136101528) * 641fae6542a62c3ba329e628f69387d53b95b0b6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136421792) * 639a9c20d8bb35d7fb7fb48511fe62feac619fb9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136517722) * 8c1235c8b5a1de6a974ea40f0287680d5c1f5f46 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136568224) * 5c508e3be15e685c067cfddfd2450b60c1fcfba7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136696854) * 72b9c027dd87b09477958fc2d077933491cb0d4d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136738729) * 44b7e4785aac200e4fc63659d2376a78f566a517 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137159954) * 2b73f20851eed3026e7881c8ae162e2e309848ee : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137238368) * 48bec2c450e8dd14313c58d206d3af0571f45972 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137361133) * e67ebd1eb36cadd4a10e5494168f19074d0640be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137394691) * 14e06e47a8675be77c07f79fe28cc5c2122005ce : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/137413258) * 99795cfb3fb946c478f4e86c4b53b508e9e42092 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137439435) * aa3aaea5dc747bfe4c06b7f41d1809280526dff6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137534672) * 93a48786992d6f8ac6f27608fcbf86cd2a2e0d5b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137590164) * 4091ee1e5d61018c60e279a04ff3d39d67c3544b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137725590) * 3a8e1edc7c7a53c030558c5a45867d20d9c90ee3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137886191) * faf9849aea036ba44162bb6c32ab832bee657f57 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138628854) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions
flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions URL: https://github.com/apache/flink/pull/10345#issuecomment-559421343 ## CI report: * 5206399001512006f4b3d7663e7b5be8ea02a4a2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138541983) * cb4089dec82717cdc6ad6e78b2dbf4d0b03e57d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138580753) * 18e1d269e688e6f39fd02cf409316776b24e8601 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606098) * 8924109bc101b39c6057d44aa14224cc12215b7c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138625034) * 1922068e5dd92138fb2cd37c225ae0e5c6a5284f : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138628834) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler
flinkbot edited a comment on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler URL: https://github.com/apache/flink/pull/10114#issuecomment-551014940 ## CI report: * c436371614d7db5dfc8c97e01a4594c3139715db : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/135418856) * d2f3a108547945633de80938684aa9c750534269 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/135432444) * dcb4596a1246234038c1f327d9e858b7c654966e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135445805) * 3bf7b045474d4bf278eb416491c32c9fb3c218a9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135495382) * 90152b07e795bc931c37b4baf2092ba0ee9219f1 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135649425) * 1657e4dd1f3ecbfa2956552efd401641c94a56aa : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135963847) * 31de348cd4fc6f1c41cdd94853c1e5ecb8206b4c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136101528) * 641fae6542a62c3ba329e628f69387d53b95b0b6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136421792) * 639a9c20d8bb35d7fb7fb48511fe62feac619fb9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136517722) * 8c1235c8b5a1de6a974ea40f0287680d5c1f5f46 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136568224) * 5c508e3be15e685c067cfddfd2450b60c1fcfba7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136696854) * 72b9c027dd87b09477958fc2d077933491cb0d4d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136738729) * 44b7e4785aac200e4fc63659d2376a78f566a517 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137159954) * 2b73f20851eed3026e7881c8ae162e2e309848ee : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137238368) * 48bec2c450e8dd14313c58d206d3af0571f45972 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137361133) * e67ebd1eb36cadd4a10e5494168f19074d0640be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137394691) * 14e06e47a8675be77c07f79fe28cc5c2122005ce : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/137413258) * 99795cfb3fb946c478f4e86c4b53b508e9e42092 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137439435) * aa3aaea5dc747bfe4c06b7f41d1809280526dff6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137534672) * 93a48786992d6f8ac6f27608fcbf86cd2a2e0d5b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137590164) * 4091ee1e5d61018c60e279a04ff3d39d67c3544b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137725590) * 3a8e1edc7c7a53c030558c5a45867d20d9c90ee3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137886191) * faf9849aea036ba44162bb6c32ab832bee657f57 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138628854) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions
flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions URL: https://github.com/apache/flink/pull/10345#issuecomment-559421343 ## CI report: * 5206399001512006f4b3d7663e7b5be8ea02a4a2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138541983) * cb4089dec82717cdc6ad6e78b2dbf4d0b03e57d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138580753) * 18e1d269e688e6f39fd02cf409316776b24e8601 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606098) * 8924109bc101b39c6057d44aa14224cc12215b7c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138625034) * 1922068e5dd92138fb2cd37c225ae0e5c6a5284f : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138628834) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] GJL commented on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler
GJL commented on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler URL: https://github.com/apache/flink/pull/10114#issuecomment-559608641 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler
flinkbot edited a comment on issue #10114: [FLINK -14651][runtime] Enable DefaultScheduler URL: https://github.com/apache/flink/pull/10114#issuecomment-551014940 ## CI report: * c436371614d7db5dfc8c97e01a4594c3139715db : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/135418856) * d2f3a108547945633de80938684aa9c750534269 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/135432444) * dcb4596a1246234038c1f327d9e858b7c654966e : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135445805) * 3bf7b045474d4bf278eb416491c32c9fb3c218a9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135495382) * 90152b07e795bc931c37b4baf2092ba0ee9219f1 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135649425) * 1657e4dd1f3ecbfa2956552efd401641c94a56aa : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/135963847) * 31de348cd4fc6f1c41cdd94853c1e5ecb8206b4c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136101528) * 641fae6542a62c3ba329e628f69387d53b95b0b6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136421792) * 639a9c20d8bb35d7fb7fb48511fe62feac619fb9 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136517722) * 8c1235c8b5a1de6a974ea40f0287680d5c1f5f46 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/136568224) * 5c508e3be15e685c067cfddfd2450b60c1fcfba7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136696854) * 72b9c027dd87b09477958fc2d077933491cb0d4d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136738729) * 44b7e4785aac200e4fc63659d2376a78f566a517 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137159954) * 2b73f20851eed3026e7881c8ae162e2e309848ee : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137238368) * 48bec2c450e8dd14313c58d206d3af0571f45972 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137361133) * e67ebd1eb36cadd4a10e5494168f19074d0640be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137394691) * 14e06e47a8675be77c07f79fe28cc5c2122005ce : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/137413258) * 99795cfb3fb946c478f4e86c4b53b508e9e42092 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137439435) * aa3aaea5dc747bfe4c06b7f41d1809280526dff6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137534672) * 93a48786992d6f8ac6f27608fcbf86cd2a2e0d5b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137590164) * 4091ee1e5d61018c60e279a04ff3d39d67c3544b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137725590) * 3a8e1edc7c7a53c030558c5a45867d20d9c90ee3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137886191) * faf9849aea036ba44162bb6c32ab832bee657f57 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions
flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions URL: https://github.com/apache/flink/pull/10345#issuecomment-559421343 ## CI report: * 5206399001512006f4b3d7663e7b5be8ea02a4a2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138541983) * cb4089dec82717cdc6ad6e78b2dbf4d0b03e57d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138580753) * 18e1d269e688e6f39fd02cf409316776b24e8601 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606098) * 8924109bc101b39c6057d44aa14224cc12215b7c : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138625034) * 1922068e5dd92138fb2cd37c225ae0e5c6a5284f : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14988) Table dependency shading fails
[
https://issues.apache.org/jira/browse/FLINK-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-14988:
-
Affects Version/s: (was: 1.10.0)
1.9.1
> Table dependency shading fails
> --
>
> Key: FLINK-14988
> URL: https://issues.apache.org/jira/browse/FLINK-14988
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Affects Versions: 1.9.1
>Reporter: Chesnay Schepler
>Priority: Blocker
>
> https://api.travis-ci.org/v3/job/618168734/log.txt
> {code}
> ==
> Running 'Dependency shading of table modules test'
> ==
> TEST_DATA_DIR:
> /home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-39933628613
> Flink dist directory:
> /home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java/target/flink-table-api-java-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala/target/flink-table-api-scala_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java-bridge/target/flink-table-api-java-bridge_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala-bridge/target/flink-table-api-scala-bridge_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner/target/flink-table-planner_2.11-1.9-SNAPSHOT.jar
> jar.
> Failure: There are unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner-blink/target/flink-table-planner-blink_2.11-1.9-SNAPSHOT.jar
> jar: -> com.esotericsoftware.kryo not found
> [FAIL] Test script contains errors.
> Checking for errors...
> No errors in log files.
> Checking for exceptions...
> No exceptions in log files.
> Checking for non-empty .out files...
> grep:
> /home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT/log/*.out:
> No such file or directory
> No non-empty .out files.
> [FAIL] 'Dependency shading of table modules test' failed after 0 minutes and
> 27 seconds! Test exited with exit code 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-14988) Table dependency shading fails
Chesnay Schepler created FLINK-14988:
Summary: Table dependency shading fails
Key: FLINK-14988
URL: https://issues.apache.org/jira/browse/FLINK-14988
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Affects Versions: 1.10.0
Reporter: Chesnay Schepler
https://api.travis-ci.org/v3/job/618168734/log.txt
{code}
==
Running 'Dependency shading of table modules test'
==
TEST_DATA_DIR:
/home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-39933628613
Flink dist directory:
/home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT
Success: There are no unwanted dependencies in the
/home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java/target/flink-table-api-java-1.9-SNAPSHOT.jar
jar.
Success: There are no unwanted dependencies in the
/home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala/target/flink-table-api-scala_2.11-1.9-SNAPSHOT.jar
jar.
Success: There are no unwanted dependencies in the
/home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java-bridge/target/flink-table-api-java-bridge_2.11-1.9-SNAPSHOT.jar
jar.
Success: There are no unwanted dependencies in the
/home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala-bridge/target/flink-table-api-scala-bridge_2.11-1.9-SNAPSHOT.jar
jar.
Success: There are no unwanted dependencies in the
/home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner/target/flink-table-planner_2.11-1.9-SNAPSHOT.jar
jar.
Failure: There are unwanted dependencies in the
/home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner-blink/target/flink-table-planner-blink_2.11-1.9-SNAPSHOT.jar
jar: -> com.esotericsoftware.kryo not found
[FAIL] Test script contains errors.
Checking for errors...
No errors in log files.
Checking for exceptions...
No exceptions in log files.
Checking for non-empty .out files...
grep:
/home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT/log/*.out:
No such file or directory
No non-empty .out files.
[FAIL] 'Dependency shading of table modules test' failed after 0 minutes and 27
seconds! Test exited with exit code 1
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-14988) Table dependency shading fails
[
https://issues.apache.org/jira/browse/FLINK-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-14988:
-
Priority: Blocker (was: Major)
> Table dependency shading fails
> --
>
> Key: FLINK-14988
> URL: https://issues.apache.org/jira/browse/FLINK-14988
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Affects Versions: 1.10.0
>Reporter: Chesnay Schepler
>Priority: Blocker
>
> https://api.travis-ci.org/v3/job/618168734/log.txt
> {code}
> ==
> Running 'Dependency shading of table modules test'
> ==
> TEST_DATA_DIR:
> /home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-39933628613
> Flink dist directory:
> /home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java/target/flink-table-api-java-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala/target/flink-table-api-scala_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-java-bridge/target/flink-table-api-java-bridge_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-api-scala-bridge/target/flink-table-api-scala-bridge_2.11-1.9-SNAPSHOT.jar
> jar.
> Success: There are no unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner/target/flink-table-planner_2.11-1.9-SNAPSHOT.jar
> jar.
> Failure: There are unwanted dependencies in the
> /home/travis/build/apache/flink/flink-end-to-end-tests/../flink-table/flink-table-planner-blink/target/flink-table-planner-blink_2.11-1.9-SNAPSHOT.jar
> jar: -> com.esotericsoftware.kryo not found
> [FAIL] Test script contains errors.
> Checking for errors...
> No errors in log files.
> Checking for exceptions...
> No exceptions in log files.
> Checking for non-empty .out files...
> grep:
> /home/travis/build/apache/flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT/log/*.out:
> No such file or directory
> No non-empty .out files.
> [FAIL] 'Dependency shading of table modules test' failed after 0 minutes and
> 27 seconds! Test exited with exit code 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] GJL closed pull request #10352: [WIP][FLINK-14651]
GJL closed pull request #10352: [WIP][FLINK-14651] URL: https://github.com/apache/flink/pull/10352 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions
flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions URL: https://github.com/apache/flink/pull/10345#issuecomment-559421343 ## CI report: * 5206399001512006f4b3d7663e7b5be8ea02a4a2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138541983) * cb4089dec82717cdc6ad6e78b2dbf4d0b03e57d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138580753) * 18e1d269e688e6f39fd02cf409316776b24e8601 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606098) * 8924109bc101b39c6057d44aa14224cc12215b7c : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138625034) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-14859) Avoid leaking unassigned Slot in DefaultScheduler when Deployment is outdated
[
https://issues.apache.org/jira/browse/FLINK-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16978581#comment-16978581
]
Gary Yao edited comment on FLINK-14859 at 11/28/19 8:46 PM:
master:
beba9dc0a48ca2256d5b871ef96885811273bf0e
30447531f78d9feac12630e5c966ac9f81681381
was (Author: gjy):
master: beba9dc0a48ca2256d5b871ef96885811273bf0e
> Avoid leaking unassigned Slot in DefaultScheduler when Deployment is outdated
> -
>
> Key: FLINK-14859
> URL: https://issues.apache.org/jira/browse/FLINK-14859
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Gary Yao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In {{DefaultScheduler#assignResourceOrHandleError()}}, if the deployment is
> outdated, we should release the possibly acquired {{LogicalSlot}} so that we
> do not leak resources.
> Below is an example to illustrate how slot leak is currently possible:
> # Vertices A1, A2, A3 are scheduled in a batch.
> # A2 acquires a slot. A1, A3 do not.
> # A1 fails due to slot allocation timeout and triggers failover
> ({{DefaultScheduler#cancelTasksAsync}})
> # A2 is canceled first and its returned slot is assigned to A3, which
> triggers {{DefaultScheduler#assignResourceOrHandleError}} of A3.
> However, A3 is not canceled yet but it is outdated because
> {{executionVertexVersioner#recordVertexModifications}} was already invoked
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-14859) Avoid leaking unassigned Slot in DefaultScheduler when Deployment is outdated
[
https://issues.apache.org/jira/browse/FLINK-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Yao closed FLINK-14859.
Resolution: Fixed
> Avoid leaking unassigned Slot in DefaultScheduler when Deployment is outdated
> -
>
> Key: FLINK-14859
> URL: https://issues.apache.org/jira/browse/FLINK-14859
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Gary Yao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In {{DefaultScheduler#assignResourceOrHandleError()}}, if the deployment is
> outdated, we should release the possibly acquired {{LogicalSlot}} so that we
> do not leak resources.
> Below is an example to illustrate how slot leak is currently possible:
> # Vertices A1, A2, A3 are scheduled in a batch.
> # A2 acquires a slot. A1, A3 do not.
> # A1 fails due to slot allocation timeout and triggers failover
> ({{DefaultScheduler#cancelTasksAsync}})
> # A2 is canceled first and its returned slot is assigned to A3, which
> triggers {{DefaultScheduler#assignResourceOrHandleError}} of A3.
> However, A3 is not canceled yet but it is outdated because
> {{executionVertexVersioner#recordVertexModifications}} was already invoked
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] GJL closed pull request #10351: [FLINK-14859][runtime] Remove wrong checkState
GJL closed pull request #10351: [FLINK-14859][runtime] Remove wrong checkState URL: https://github.com/apache/flink/pull/10351 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10193: [FLINK-13938][yarn] Use pre-uploaded flink binary to accelerate flink submission
flinkbot edited a comment on issue #10193: [FLINK-13938][yarn] Use pre-uploaded flink binary to accelerate flink submission URL: https://github.com/apache/flink/pull/10193#issuecomment-553881527 ## CI report: * e3ac83fe02a7583159184772ff4b4341fa65f827 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136517817) * eefbec6756be60a27698d275a1b94bef7cd0c1e2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136636043) * 19a83ead105c951505dbafb0280fa2d25132c9a0 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136645898) * dd2b911c850a56e3d6aa4a3c7e16b30431977bf5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/136651764) * 06b368d9fbd88eabf71391fc1662b4d8a626d43c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137165694) * d4b77c8aab32cdeb11806fdd45ea88141051a157 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137317107) * 53b86608c1d008c53112b34c634ccf96419cd921 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137343147) * 2969fb4fb3afc8c331415c1ca478b05f3cb47b47 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137394777) * edccba4a6db80772f9494f5631e1bc6a340d6586 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/137697818) * a109168bc5582fad8bbd3dade6f30990931583b5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138010243) * 6600e07db82467eb3ce41e6d1c8032c9bcdd9751 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138072264) * b3155b18b290b31df0fc5e6bdf29ef421bf68373 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160080) * 5350fff0d5479bd2015de2e61895a4da06aece47 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138181974) * 3f24f8ce1b43bb832ca1987323be3717ed7bc6bd : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138197839) * ef0a6430c6e026f134905b40e9c79915ffc87ee3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138387359) * 74d293dad137877a8a6966f8d9ceb8a72da79898 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138591690) * 29ff6192e49efafe70f835aca6230035ca17455b : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606137) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] rkhachatryan commented on a change in pull request #10345: [FLINK-12484][runtime] synchronize all mailbox actions
rkhachatryan commented on a change in pull request #10345:
[FLINK-12484][runtime] synchronize all mailbox actions
URL: https://github.com/apache/flink/pull/10345#discussion_r351909177
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/mailbox/MailboxExecutorImpl.java
##
@@ -37,9 +39,12 @@
private final int priority;
- public MailboxExecutorImpl(@Nonnull TaskMailbox mailbox, int priority) {
+ public final ExecutionDecorator executionDecorator;
+
+ public MailboxExecutorImpl(@Nonnull TaskMailbox mailbox, int priority,
ExecutionDecorator executionDecorator) {
Review comment:
Not all wrapped actions are `Mails`, so we will end up either wrapping them
too into `Mail` or having different approaches to execute `Mails` and regular
actions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10351: [FLINK-14859][runtime] Remove wrong checkState
flinkbot edited a comment on issue #10351: [FLINK-14859][runtime] Remove wrong checkState URL: https://github.com/apache/flink/pull/10351#issuecomment-559560938 ## CI report: * 5d7ba9db7d9923e8ecca91b46c600ac7630f0327 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606117) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10352: [WIP][FLINK-14651]
flinkbot edited a comment on issue #10352: [WIP][FLINK-14651] URL: https://github.com/apache/flink/pull/10352#issuecomment-559568529 ## CI report: * cf7f3a6cf7db44ef19e351ebe6654500a96017cc : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/138609890) * 07ab4e89c7c296b49e8c2a242be539b0ee9176d0 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138620636) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions
flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions URL: https://github.com/apache/flink/pull/10345#issuecomment-559421343 ## CI report: * 5206399001512006f4b3d7663e7b5be8ea02a4a2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138541983) * cb4089dec82717cdc6ad6e78b2dbf4d0b03e57d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138580753) * 18e1d269e688e6f39fd02cf409316776b24e8601 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606098) * 8924109bc101b39c6057d44aa14224cc12215b7c : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10352: [WIP][FLINK-14651]
flinkbot edited a comment on issue #10352: [WIP][FLINK-14651] URL: https://github.com/apache/flink/pull/10352#issuecomment-559568529 ## CI report: * cf7f3a6cf7db44ef19e351ebe6654500a96017cc : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/138609890) * 07ab4e89c7c296b49e8c2a242be539b0ee9176d0 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138620636) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions
flinkbot edited a comment on issue #10345: [FLINK-12484][runtime] synchronize all mailbox actions URL: https://github.com/apache/flink/pull/10345#issuecomment-559421343 ## CI report: * 5206399001512006f4b3d7663e7b5be8ea02a4a2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138541983) * cb4089dec82717cdc6ad6e78b2dbf4d0b03e57d4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138580753) * 18e1d269e688e6f39fd02cf409316776b24e8601 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606098) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API
flinkbot edited a comment on issue #10311: [FLINK-14762][client] Enrich JobClient API URL: https://github.com/apache/flink/pull/10311#issuecomment-558081118 ## CI report: * aadc1cfef92eec54d86efbf39f50d91afda6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138022323) * daf85a75b9c24918058b8bfe09416b2828bd02a5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138160048) * 7fe9b0e6c496482990905b9cec4389c8cbb8930a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138161977) * fe4dfd5ebd0508fc24dcfcb55aab4b1c99cd6bd3 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138175553) * 7d98e670ac6e917eb2c121493ba0b5925240899d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/138375416) * 08b96bbcc026a2feb1b257000a553521f4304d93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138380708) * 880bec8f2cc3097edeb0985e04db02d8fd205a2f : UNKNOWN * 16fd227a34688708f0c42fb3fc8ce2aad32555bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138401580) * 2fb0f5b45b8f6dd236cd7298b5644fb9767b90c4 : UNKNOWN * bd5087fd703553fbbc190260aeba03b595ba0c93 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138606066) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] rkhachatryan commented on a change in pull request #10345: [FLINK-12484][runtime] synchronize all mailbox actions
rkhachatryan commented on a change in pull request #10345: [FLINK-12484][runtime] synchronize all mailbox actions URL: https://github.com/apache/flink/pull/10345#discussion_r351914910 ## File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java ## @@ -262,7 +262,7 @@ protected StreamTask( this.accumulatorMap = getEnvironment().getAccumulatorRegistry().getUserMap(); this.recordWriter = createRecordWriterDelegate(configuration, environment); this.executionDecorator = Preconditions.checkNotNull(executionDecorator); - this.mailboxProcessor = new MailboxProcessor(this::processInput); + this.mailboxProcessor = new MailboxProcessor(this::processInput, this.executionDecorator); Review comment: Inlining `invokeProcessingTimeCallback` and removing sync inside. As of `performCheckpoint()`, I don't think it's safe, because it's accessible via public methods. I think we should check the difference in benchmarks first. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] rkhachatryan commented on a change in pull request #10345: [FLINK-12484][runtime] synchronize all mailbox actions
rkhachatryan commented on a change in pull request #10345:
[FLINK-12484][runtime] synchronize all mailbox actions
URL: https://github.com/apache/flink/pull/10345#discussion_r351909719
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/ExecutionDecorator.java
##
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.streaming.runtime.tasks.mailbox.Mail;
+import org.apache.flink.util.function.ThrowingRunnable;
+
+import java.util.concurrent.Callable;
+
+/**
+ * Wraps execution of a {@link Runnable}, {@link ThrowingRunnable}, {@link
Callable}, or {@link Mail}.
+ * Intended to customize execution in sub-types fo {@link
org.apache.flink.streaming.runtime.tasks.StreamTask StreamTask},
+ * e.g. synchronization in {@link
org.apache.flink.streaming.runtime.tasks.SourceStreamTask SourceStreamTask}.
+ */
+@Internal
+public interface ExecutionDecorator {
+ void run(Runnable runnable);
+
+void runThrowing(ThrowingRunnable runnable)
throws E;
+
+R call(Callable callable) throws Exception;
+
+ void dispatch(Mail mail);
+
+ ExecutionDecorator NO_OP = new ExecutionDecorator() {
+ @Override
+ public void run(Runnable runnable) {
+ runnable.run();
+ }
+
+ @Override
+ public void
runThrowing(ThrowingRunnable runnable) throws E {
+ runnable.run();
+ }
+
+ @Override
+ public R call(Callable callable) throws Exception {
+ return callable.call();
+ }
+
+ @Override
+ public void dispatch(Mail mail) {
+ mail.run();
+ }
+ };
+
+ /**
+* Returns an ExecutionDecorator that synchronizes each invocation.
+*/
+ static SynchronizedExecutionDecorator syncExecutionDecorator() {
+ return syncExecutionDecorator(new Object());
+ }
+
+ /**
+* Returns an ExecutionDecorator that synchronizes each invocation on a
given object.
+*/
+ static SynchronizedExecutionDecorator syncExecutionDecorator(Object
mutex) {
+ return new SynchronizedExecutionDecorator(mutex);
+ }
+
+ /**
+* A {@link ExecutionDecorator} that synchronizes every operation on
the provided mutex.
+*/
+ class SynchronizedExecutionDecorator implements ExecutionDecorator {
Review comment:
On the other hand, `checkpointLock` is confusing for newcomers:) As it is
used not only for checkpoints.
And the method `getCheckpointLock()` is still present. So I think we can
just add Javadoc for now, and mention this in a later commit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10350: [FLINK-14957] Remove deprecated -yst option
flinkbot edited a comment on issue #10350: [FLINK-14957] Remove deprecated -yst option URL: https://github.com/apache/flink/pull/10350#issuecomment-559552706 ## CI report: * 91798df536b1f681f2c81a228545d9ba09d58e8b : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138601652) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10352: [WIP][FLINK-14651]
flinkbot edited a comment on issue #10352: [WIP][FLINK-14651] URL: https://github.com/apache/flink/pull/10352#issuecomment-559568529 ## CI report: * cf7f3a6cf7db44ef19e351ebe6654500a96017cc : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/138609890) * 07ab4e89c7c296b49e8c2a242be539b0ee9176d0 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10349: [FLINK-14833] [runtime] Remove hierachy of SnapshotStrategySynchronicityBehavior in HeapSnapshotStrategy
flinkbot edited a comment on issue #10349: [FLINK-14833] [runtime] Remove hierachy of SnapshotStrategySynchronicityBehavior in HeapSnapshotStrategy URL: https://github.com/apache/flink/pull/10349#issuecomment-559552672 ## CI report: * fdb6aea8d100865946dbee7840647e1314cb8241 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138601616) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] rkhachatryan commented on a change in pull request #10345: [FLINK-12484][runtime] synchronize all mailbox actions
rkhachatryan commented on a change in pull request #10345:
[FLINK-12484][runtime] synchronize all mailbox actions
URL: https://github.com/apache/flink/pull/10345#discussion_r351909719
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/ExecutionDecorator.java
##
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.streaming.runtime.tasks.mailbox.Mail;
+import org.apache.flink.util.function.ThrowingRunnable;
+
+import java.util.concurrent.Callable;
+
+/**
+ * Wraps execution of a {@link Runnable}, {@link ThrowingRunnable}, {@link
Callable}, or {@link Mail}.
+ * Intended to customize execution in sub-types fo {@link
org.apache.flink.streaming.runtime.tasks.StreamTask StreamTask},
+ * e.g. synchronization in {@link
org.apache.flink.streaming.runtime.tasks.SourceStreamTask SourceStreamTask}.
+ */
+@Internal
+public interface ExecutionDecorator {
+ void run(Runnable runnable);
+
+void runThrowing(ThrowingRunnable runnable)
throws E;
+
+R call(Callable callable) throws Exception;
+
+ void dispatch(Mail mail);
+
+ ExecutionDecorator NO_OP = new ExecutionDecorator() {
+ @Override
+ public void run(Runnable runnable) {
+ runnable.run();
+ }
+
+ @Override
+ public void
runThrowing(ThrowingRunnable runnable) throws E {
+ runnable.run();
+ }
+
+ @Override
+ public R call(Callable callable) throws Exception {
+ return callable.call();
+ }
+
+ @Override
+ public void dispatch(Mail mail) {
+ mail.run();
+ }
+ };
+
+ /**
+* Returns an ExecutionDecorator that synchronizes each invocation.
+*/
+ static SynchronizedExecutionDecorator syncExecutionDecorator() {
+ return syncExecutionDecorator(new Object());
+ }
+
+ /**
+* Returns an ExecutionDecorator that synchronizes each invocation on a
given object.
+*/
+ static SynchronizedExecutionDecorator syncExecutionDecorator(Object
mutex) {
+ return new SynchronizedExecutionDecorator(mutex);
+ }
+
+ /**
+* A {@link ExecutionDecorator} that synchronizes every operation on
the provided mutex.
+*/
+ class SynchronizedExecutionDecorator implements ExecutionDecorator {
Review comment:
On the other hand, `checkpointLock` is confusing for newcomers:) As it is
used not only for checkpoints.
I think we can just add Javadoc.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] rkhachatryan commented on a change in pull request #10345: [FLINK-12484][runtime] synchronize all mailbox actions
rkhachatryan commented on a change in pull request #10345:
[FLINK-12484][runtime] synchronize all mailbox actions
URL: https://github.com/apache/flink/pull/10345#discussion_r351909177
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/mailbox/MailboxExecutorImpl.java
##
@@ -37,9 +39,12 @@
private final int priority;
- public MailboxExecutorImpl(@Nonnull TaskMailbox mailbox, int priority) {
+ public final ExecutionDecorator executionDecorator;
+
+ public MailboxExecutorImpl(@Nonnull TaskMailbox mailbox, int priority,
ExecutionDecorator executionDecorator) {
Review comment:
Not all wrapped actions aren't `Mails`, so we will end up either wrapping
them too into `Mail` or having different approaches to execute `Mails` and
regular actions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10341: [FLINK-14974][runtime] Calculate managed memory fractions with BigDecimal and round down it properly
flinkbot edited a comment on issue #10341: [FLINK-14974][runtime] Calculate managed memory fractions with BigDecimal and round down it properly URL: https://github.com/apache/flink/pull/10341#issuecomment-559181765 ## CI report: * a4045089cf8d671a4160c09e60cb3fece7fbd881 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138455849) * 362346f7189522300eacc6be35b76dcffdcbf300 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138601590) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10339: [FLINK-14976][cassandra] Release semaphore on all Throwable's in send()
flinkbot edited a comment on issue #10339: [FLINK-14976][cassandra] Release semaphore on all Throwable's in send() URL: https://github.com/apache/flink/pull/10339#issuecomment-559115891 ## CI report: * 65f4f8552770a2e1fd05c604a5cc074f32487c74 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/138425013) * 8efd77647534f028e8549ac3a210447b78e8d6de : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138439684) * 10e4a64ff821f17d5b34cab8339cf18df4c56775 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/138597065) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14910) disableAutoGeneratedUIDs fails on keyBy
[
https://issues.apache.org/jira/browse/FLINK-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984586#comment-16984586
]
Dawid Wysakowicz commented on FLINK-14910:
--
I think an easy fix would be to perform the check introduced in FLINK-11653
only for {{PhysicalTransformation}}. We create {{StreamOperators}} only for
those. We can set uids for all of them.
Will prepare a fix shortly.
> disableAutoGeneratedUIDs fails on keyBy
> ---
>
> Key: FLINK-14910
> URL: https://issues.apache.org/jira/browse/FLINK-14910
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.9.0
>Reporter: William Cheng
>Assignee: Dawid Wysakowicz
>Priority: Major
>
> There doesn't seem to be a way to add a UID to the Partition operator created
> by KeyBy, causing `disableAutoGeneratedUIDs` to fail.
>
> Here's a simple test case that will reproduce the issue:
> {noformat}
> @Test
> public void testFailedUID() throws Exception {
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> env.getConfig().disableAutoGeneratedUIDs();
> DataStream data = env.fromCollection(Arrays.asList("1", "2",
> "3")).uid("source-uid");
> data.keyBy(i -> i)
> .map(i -> i).uid("map-uid");
> env.execute();
> }{noformat}
> {noformat}
> testFailedUID(twitch.creatoranalytics.sessions.StreamingJobTest) Time
> elapsed: 0.008 sec <<< ERROR!
> java.lang.IllegalStateException: Auto generated UIDs have been disabled but
> no UID or hash has been assigned to operator Partition
> {noformat}
>
> This passes if the keyBy is removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-14669) All hadoop-2.4.1 related nightly end-to-end tests failed on travis
[
https://issues.apache.org/jira/browse/FLINK-14669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler closed FLINK-14669.
Resolution: Duplicate
FLINK-13745 was always just a band-aid; there's currently nothing we can do.
> All hadoop-2.4.1 related nightly end-to-end tests failed on travis
> --
>
> Key: FLINK-14669
> URL: https://issues.apache.org/jira/browse/FLINK-14669
> Project: Flink
> Issue Type: Bug
> Components: Tests
>Affects Versions: 1.10.0
>Reporter: Yu Li
>Priority: Blocker
> Labels: test-stability
> Attachments: image-2019-11-08-15-02-31-268.png
>
>
> As titled, all hadoop 2.4.1 tests failed in build
> [https://travis-ci.org/apache/flink/builds/608709634]
> !image-2019-11-08-15-02-31-268.png|width=609,height=205,vspace=2!
> From the log it seems to be timed out when downloading dependencies
> {noformat}
> /home/travis/flink_cache/40913/flink/docs/concepts/runtime.zh.md
> /home/travis/flink_cache/40913/flink/docs/_config_dev_en.yml
> /home/travis/flink_cache/40913/\n...[0m
> [32;1mchanges detected, packing new archive[0m
> [32;1muploading
> master/cache--linux-xenial-98cdbf5c3ae4db3a919a6366e5a96e33ecd8f19b8853a50e8f545bd43fc8164e--jdk-openjdk8.tgz[0m
> [32;1mcache uploaded[0m
> travis_time:end:1b949da8:start=1573163710718943370,finish=1573163798721501362,duration=88002557992,event=cache
> [0Ktravis_fold:end:cache.2
> [0K
> Done. Your build exited with 1.
> {noformat}
> [https://api.travis-ci.org/v3/job/608709640/log.txt]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14835) Make `org.apache.flink.configuration.Configuration` support method chaining
[
https://issues.apache.org/jira/browse/FLINK-14835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984580#comment-16984580
]
Chesnay Schepler commented on FLINK-14835:
--
Starting from 1.10 you can use {{Configuration#set(ConfigOption, T)}} in
this way (added in FLINK-14493).
Beyond that I don't think there's anything we can do.
> Make `org.apache.flink.configuration.Configuration` support method chaining
> ---
>
> Key: FLINK-14835
> URL: https://issues.apache.org/jira/browse/FLINK-14835
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Affects Versions: 1.9.1
>Reporter: Victor Wong
>Priority: Major
>
> *Goal:*
> To make the following code examples work in production, which is very handy
> for users to set a couple of configurations:
> {code:java}
> // instantiate table environment
> TableEnvironment tEnv = ...tEnv.getConfig()// access high-level
> configuration
> .getConfiguration() // set low-level key-value options
> .setString("table.exec.mini-batch.enabled", "true") // enable mini-batch
> optimization
> .setString("table.exec.mini-batch.allow-latency", "5 s") // use 5 seconds
> to buffer input records
> .setString("table.exec.mini-batch.size", "5000"); // the maximum number of
> records can be buffered by each aggregate operator task
> {code}
>
> *Suggestion:*
> Currently, the return type of `setXXX` method is "void", we can make it
> return `Configuration` itself to support method chaining.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
