[GitHub] [flink] Mrart commented on a change in pull request #10510: [FLINK-15176][container] Add '--job-classname' to flink-container 'jo…
Mrart commented on a change in pull request #10510: [FLINK-15176][container]
Add '--job-classname' to flink-container 'jo…
URL: https://github.com/apache/flink/pull/10510#discussion_r368212686
##
File path: flink-container/kubernetes/job-cluster-job.yaml.template
##
@@ -31,7 +31,7 @@ spec:
containers:
- name: flink-job-cluster
image: ${FLINK_IMAGE_NAME}
-args: ["job-cluster", "-Djobmanager.rpc.address=flink-job-cluster",
+args: ["job-cluster", "--job-classname=${JOB_CLASSNAME}",
"-Djobmanager.rpc.address=flink-job-cluster",
Review comment:
I know how to scan the main class, I think we need a more suitable methed to
tell user to add the main class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] Mrart commented on a change in pull request #10510: [FLINK-15176][container] Add '--job-classname' to flink-container 'jo…
Mrart commented on a change in pull request #10510: [FLINK-15176][container]
Add '--job-classname' to flink-container 'jo…
URL: https://github.com/apache/flink/pull/10510#discussion_r368212620
##
File path: flink-container/kubernetes/job-cluster-job.yaml.template
##
@@ -31,7 +31,7 @@ spec:
containers:
- name: flink-job-cluster
image: ${FLINK_IMAGE_NAME}
-args: ["job-cluster", "-Djobmanager.rpc.address=flink-job-cluster",
+args: ["job-cluster", "--job-classname=${JOB_CLASSNAME}",
"-Djobmanager.rpc.address=flink-job-cluster",
Review comment:
I think It write into docment will more suitable.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] Mrart commented on a change in pull request #10510: [FLINK-15176][container] Add '--job-classname' to flink-container 'jo…
Mrart commented on a change in pull request #10510: [FLINK-15176][container]
Add '--job-classname' to flink-container 'jo…
URL: https://github.com/apache/flink/pull/10510#discussion_r368212620
##
File path: flink-container/kubernetes/job-cluster-job.yaml.template
##
@@ -31,7 +31,7 @@ spec:
containers:
- name: flink-job-cluster
image: ${FLINK_IMAGE_NAME}
-args: ["job-cluster", "-Djobmanager.rpc.address=flink-job-cluster",
+args: ["job-cluster", "--job-classname=${JOB_CLASSNAME}",
"-Djobmanager.rpc.address=flink-job-cluster",
Review comment:
I think It write into docment will more suitable.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability.
flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability. URL: https://github.com/apache/flink/pull/10785#issuecomment-571482425 ## CI report: * bfb11ff827395ce52de3c19567ec5fb60334e4d5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143356049) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4153) * 65aaf3db7340f912c5fe9e54cb065c38b7c88c0d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144898098) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4424) * 9ac1d96d3592069c274833f5b2ef03c02cda2316 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/145031962) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4453) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend
flinkbot edited a comment on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend URL: https://github.com/apache/flink/pull/10893#issuecomment-575859797 ## CI report: * 2a07b12e4f943499075f42960fc84dd445df6f3c Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/145041173) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4454) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend
flinkbot edited a comment on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend URL: https://github.com/apache/flink/pull/10893#issuecomment-575859797 ## CI report: * 2a07b12e4f943499075f42960fc84dd445df6f3c Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/145041173) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4454) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability.
flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability. URL: https://github.com/apache/flink/pull/10785#issuecomment-571482425 ## CI report: * bfb11ff827395ce52de3c19567ec5fb60334e4d5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143356049) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4153) * 65aaf3db7340f912c5fe9e54cb065c38b7c88c0d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144898098) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4424) * 9ac1d96d3592069c274833f5b2ef03c02cda2316 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/145031962) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4453) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend
flinkbot commented on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend URL: https://github.com/apache/flink/pull/10893#issuecomment-575859797 ## CI report: * 2a07b12e4f943499075f42960fc84dd445df6f3c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability.
flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability. URL: https://github.com/apache/flink/pull/10785#issuecomment-571482425 ## CI report: * bfb11ff827395ce52de3c19567ec5fb60334e4d5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143356049) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4153) * 65aaf3db7340f912c5fe9e54cb065c38b7c88c0d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144898098) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4424) * 9ac1d96d3592069c274833f5b2ef03c02cda2316 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/145031962) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4453) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15638) releasing/create_release_branch.sh does not set version in flink-python/pyflink/version.py
[
https://issues.apache.org/jira/browse/FLINK-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018475#comment-17018475
]
Hequn Cheng commented on FLINK-15638:
-
Fixed
in 1.10.0 via 1416b0f9dac37c11c4a17c60a15747b25c6b8e3e
in 1.11.0 via 1ffa2990d5fdb32e9100bd7a18b6dae1630563a5
> releasing/create_release_branch.sh does not set version in
> flink-python/pyflink/version.py
> --
>
> Key: FLINK-15638
> URL: https://issues.apache.org/jira/browse/FLINK-15638
> Project: Flink
> Issue Type: Bug
> Components: Release System
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Hequn Cheng
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{releasing/create_release_branch.sh}} does not set the version in
> {{flink-python/pyflink/version.py}}. Currently the version.py contains:
> {noformat}
> __version__ = "1.10.dev0"
> {noformat}
> {{setup.py}} will replace .dev0 with -SNAPSHOT and tries to find the
> respective flink distribution in the flink-dist/target, which will not exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-15638) releasing/create_release_branch.sh does not set version in flink-python/pyflink/version.py
[
https://issues.apache.org/jira/browse/FLINK-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hequn Cheng closed FLINK-15638.
---
Resolution: Fixed
> releasing/create_release_branch.sh does not set version in
> flink-python/pyflink/version.py
> --
>
> Key: FLINK-15638
> URL: https://issues.apache.org/jira/browse/FLINK-15638
> Project: Flink
> Issue Type: Bug
> Components: Release System
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Hequn Cheng
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{releasing/create_release_branch.sh}} does not set the version in
> {{flink-python/pyflink/version.py}}. Currently the version.py contains:
> {noformat}
> __version__ = "1.10.dev0"
> {noformat}
> {{setup.py}} will replace .dev0 with -SNAPSHOT and tries to find the
> respective flink distribution in the flink-dist/target, which will not exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend
flinkbot commented on issue #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend URL: https://github.com/apache/flink/pull/10893#issuecomment-575858109 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 2a07b12e4f943499075f42960fc84dd445df6f3c (Sat Jan 18 02:32:30 UTC 2020) **Warnings:** * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-15637).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15637) For RocksDBStateBackend, make RocksDB the default store for timers
[
https://issues.apache.org/jira/browse/FLINK-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-15637:
---
Labels: pull-request-available (was: )
> For RocksDBStateBackend, make RocksDB the default store for timers
> --
>
> Key: FLINK-15637
> URL: https://issues.apache.org/jira/browse/FLINK-15637
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Reporter: Stephan Ewen
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
>
> Set the {{state.backend.rocksdb.timer-service.factory}} to {{ROCKSDB}} by
> default. Also ensure that the programmatic default value becomes the same.
> We need to update the performance tuning guide to mention this.
>
> We need to update the release notes to mention this.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] klion26 opened a new pull request #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend
klion26 opened a new pull request #10893: [FLINK-15637][state backends] Make RocksDB the default store for timers when using RocksDBStateBackend URL: https://github.com/apache/flink/pull/10893 ## What is the purpose of the change Make RocksDB the default store for timers when using `RocksDBStateBackend`. ## Verifying this change This change is already covered by existing tests, such as *(please describe tests)*. - RocksDBStateBackendConfigTest.java ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10892: [FLINK-15638][release][python] Change version of pyflink to the release version when creating release branch
flinkbot commented on issue #10892: [FLINK-15638][release][python] Change version of pyflink to the release version when creating release branch URL: https://github.com/apache/flink/pull/10892#issuecomment-575857387 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 18828e5ab6b10835cace7c209799ef4982a00f41 (Sat Jan 18 02:22:54 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] hequn8128 merged pull request #10892: [FLINK-15638][release][python] Change version of pyflink to the release version when creating release branch
hequn8128 merged pull request #10892: [FLINK-15638][release][python] Change version of pyflink to the release version when creating release branch URL: https://github.com/apache/flink/pull/10892 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15638) releasing/create_release_branch.sh does not set version in flink-python/pyflink/version.py
[
https://issues.apache.org/jira/browse/FLINK-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-15638:
---
Labels: pull-request-available (was: )
> releasing/create_release_branch.sh does not set version in
> flink-python/pyflink/version.py
> --
>
> Key: FLINK-15638
> URL: https://issues.apache.org/jira/browse/FLINK-15638
> Project: Flink
> Issue Type: Bug
> Components: Release System
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Hequn Cheng
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
>
> {{releasing/create_release_branch.sh}} does not set the version in
> {{flink-python/pyflink/version.py}}. Currently the version.py contains:
> {noformat}
> __version__ = "1.10.dev0"
> {noformat}
> {{setup.py}} will replace .dev0 with -SNAPSHOT and tries to find the
> respective flink distribution in the flink-dist/target, which will not exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] hequn8128 opened a new pull request #10892: [FLINK-15638][release][python] Change version of pyflink to the release version when creating release branch
hequn8128 opened a new pull request #10892: [FLINK-15638][release][python] Change version of pyflink to the release version when creating release branch URL: https://github.com/apache/flink/pull/10892 ## What is the purpose of the change This pull request sets the version in flink-python/pyflink/version.py when creating release branch. ## Brief change log - Sets the version in flink-python/pyflink/version.py when creating release branch ## Verifying this change - This change is a trivial rework / code cleanup without any test coverage. - Test manually by creating the release branch and install pyflink on local. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability.
flinkbot edited a comment on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability. URL: https://github.com/apache/flink/pull/10785#issuecomment-571482425 ## CI report: * bfb11ff827395ce52de3c19567ec5fb60334e4d5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143356049) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4153) * 65aaf3db7340f912c5fe9e54cb065c38b7c88c0d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144898098) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4424) * 9ac1d96d3592069c274833f5b2ef03c02cda2316 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability.
xintongsong commented on issue #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability. URL: https://github.com/apache/flink/pull/10785#issuecomment-575856631 @tillrohrmann, thanks for the review. I've addressed your comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #10785: [FLINK-15375][core] Improve MemorySize to print / parse with better readability.
xintongsong commented on a change in pull request #10785: [FLINK-15375][core]
Improve MemorySize to print / parse with better readability.
URL: https://github.com/apache/flink/pull/10785#discussion_r368198923
##
File path:
flink-core/src/main/java/org/apache/flink/configuration/MemorySize.java
##
@@ -158,6 +162,38 @@ private String formatToString() {
highestIntegerUnit.getUnits()[1]);
}
+ public String toHumanReadableString() {
+ if (humanReadableStr == null) {
+ humanReadableStr = formatToHumanReadableString();
+ }
+
+ return humanReadableStr;
+ }
+
+ private String formatToHumanReadableString() {
+ MemoryUnit highestUnit = IntStream.range(0,
ORDERED_UNITS.size())
+ .sequential()
+ .filter(idx -> bytes >
ORDERED_UNITS.get(idx).getMultiplier())
+ .boxed()
+ .max(Comparator.naturalOrder())
+ .map(idx -> ORDERED_UNITS.get(idx))
+ .orElse(BYTES);
+
+ if (highestUnit == BYTES) {
+ return String.format(
+ "%d %s",
+ bytes,
+ BYTES.getUnits()[1]);
+ } else {
+ double approximate = 1.0 * bytes /
highestUnit.getMultiplier();
+ return String.format(
+ "%.3f%s (%d)",
Review comment:
I thought people could easily understand that the value in the parentheses
are they exact bytes. But true, it would be better to explicitly add the unit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Comment Edited] (FLINK-15447) Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
[
https://issues.apache.org/jira/browse/FLINK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018462#comment-17018462
]
Victor Wong edited comment on FLINK-15447 at 1/18/20 1:41 AM:
--
[~rongr] Thanks for your reply!
_Could you elaborate what does it mean by Flink-YARN default the value to /tmp?
I am guessing you mean JVM default the value to /tmp ?_
---
Yes, I mean that Flink-YARN is using the default value of JVM, which is "/tmp".
_So far the only place I can see in flink yarn code utilizing this key is .._
---
The third-party dependencies might utilize this key as well, as [~lzljs3620320]
mentioned.
My intention is similar to how [~xymaqingxiang] mentioned that setting the
value to a tmp directory under the working directory of Yarn container.
was (Author: victor-wong):
[~rongr] Thanks for your reply!
_Could you elaborate what does it mean by Flink-YARN default the value to /tmp?
I am guessing you mean JVM default the value to /tmp ?_
---
Yes, I mean that Flink-YARN is using the default value of JVM, which is "/tmp".
_So far the only place I can see in flink yarn code utilizing this key is:
https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L899_
---
The third-party dependencies might utilize this key as well, as [~lzljs3620320]
mentioned.
My intention is similar to how [~xymaqingxiang] mentioned that setting the
value to a tmp directory under the working directory of Yarn container.
> Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
> ---
>
> Key: FLINK-15447
> URL: https://issues.apache.org/jira/browse/FLINK-15447
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Affects Versions: 1.9.1
>Reporter: Victor Wong
>Priority: Major
>
> Currently, when running Flink on Yarn, the "java.io.tmpdir" property is set
> to the default value, which is "/tmp".
>
> Sometimes we ran into exceptions caused by a full "/tmp" directory, which
> would not be cleaned automatically after applications finished.
> I think we can set "java.io.tmpdir" to "PWD/tmp" directory, or
> something similar. "PWD" will be replaced with the true working
> directory of JM/TM by Yarn, which will be cleaned automatically.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-15447) Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
[
https://issues.apache.org/jira/browse/FLINK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018462#comment-17018462
]
Victor Wong edited comment on FLINK-15447 at 1/18/20 1:40 AM:
--
[~rongr] Thanks for your reply!
_Could you elaborate what does it mean by Flink-YARN default the value to /tmp?
I am guessing you mean JVM default the value to /tmp ?_
---
Yes, I mean that Flink-YARN is using the default value of JVM, which is "/tmp".
_So far the only place I can see in flink yarn code utilizing this key is:
https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L899_
---
The third-party dependencies might utilize this key as well, as [~lzljs3620320]
mentioned.
My intention is similar to how [~xymaqingxiang] mentioned that setting the
value to a tmp directory under the working directory of Yarn container.
was (Author: victor-wong):
[~rongr] Thanks for your reply!
_Could you elaborate what does it mean by Flink-YARN default the value to /tmp?
I am guessing you mean JVM default the value to /tmp ?
_
---
Yes, I mean that Flink-YARN is using the default value of JVM, which is "/tmp".
_So far the only place I can see in flink yarn code utilizing this key is:
https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L899
_
---
The third-party dependencies might utilize this key as well, as [~lzljs3620320]
mentioned.
My intention is similar to how [~xymaqingxiang] mentioned that setting the
value to a tmp directory under the working directory of Yarn container.
> Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
> ---
>
> Key: FLINK-15447
> URL: https://issues.apache.org/jira/browse/FLINK-15447
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Affects Versions: 1.9.1
>Reporter: Victor Wong
>Priority: Major
>
> Currently, when running Flink on Yarn, the "java.io.tmpdir" property is set
> to the default value, which is "/tmp".
>
> Sometimes we ran into exceptions caused by a full "/tmp" directory, which
> would not be cleaned automatically after applications finished.
> I think we can set "java.io.tmpdir" to "PWD/tmp" directory, or
> something similar. "PWD" will be replaced with the true working
> directory of JM/TM by Yarn, which will be cleaned automatically.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-15447) Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
[
https://issues.apache.org/jira/browse/FLINK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018462#comment-17018462
]
Victor Wong edited comment on FLINK-15447 at 1/18/20 1:40 AM:
--
[~rongr] Thanks for your reply!
_Could you elaborate what does it mean by Flink-YARN default the value to /tmp?
I am guessing you mean JVM default the value to /tmp ?_
---
Yes, I mean that Flink-YARN is using the default value of JVM, which is "/tmp".
_So far the only place I can see in flink yarn code utilizing this key is:
https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L899_
---
The third-party dependencies might utilize this key as well, as [~lzljs3620320]
mentioned.
My intention is similar to how [~xymaqingxiang] mentioned that setting the
value to a tmp directory under the working directory of Yarn container.
was (Author: victor-wong):
[~rongr] Thanks for your reply!
_Could you elaborate what does it mean by Flink-YARN default the value to /tmp?
I am guessing you mean JVM default the value to /tmp ?_
---
Yes, I mean that Flink-YARN is using the default value of JVM, which is "/tmp".
_So far the only place I can see in flink yarn code utilizing this key is:
https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L899_
---
The third-party dependencies might utilize this key as well, as [~lzljs3620320]
mentioned.
My intention is similar to how [~xymaqingxiang] mentioned that setting the
value to a tmp directory under the working directory of Yarn container.
> Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
> ---
>
> Key: FLINK-15447
> URL: https://issues.apache.org/jira/browse/FLINK-15447
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Affects Versions: 1.9.1
>Reporter: Victor Wong
>Priority: Major
>
> Currently, when running Flink on Yarn, the "java.io.tmpdir" property is set
> to the default value, which is "/tmp".
>
> Sometimes we ran into exceptions caused by a full "/tmp" directory, which
> would not be cleaned automatically after applications finished.
> I think we can set "java.io.tmpdir" to "PWD/tmp" directory, or
> something similar. "PWD" will be replaced with the true working
> directory of JM/TM by Yarn, which will be cleaned automatically.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15447) Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
[
https://issues.apache.org/jira/browse/FLINK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018462#comment-17018462
]
Victor Wong commented on FLINK-15447:
-
[~rongr] Thanks for your reply!
_Could you elaborate what does it mean by Flink-YARN default the value to /tmp?
I am guessing you mean JVM default the value to /tmp ?
_
---
Yes, I mean that Flink-YARN is using the default value of JVM, which is "/tmp".
_So far the only place I can see in flink yarn code utilizing this key is:
https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L899
_
---
The third-party dependencies might utilize this key as well, as [~lzljs3620320]
mentioned.
My intention is similar to how [~xymaqingxiang] mentioned that setting the
value to a tmp directory under the working directory of Yarn container.
> Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
> ---
>
> Key: FLINK-15447
> URL: https://issues.apache.org/jira/browse/FLINK-15447
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Affects Versions: 1.9.1
>Reporter: Victor Wong
>Priority: Major
>
> Currently, when running Flink on Yarn, the "java.io.tmpdir" property is set
> to the default value, which is "/tmp".
>
> Sometimes we ran into exceptions caused by a full "/tmp" directory, which
> would not be cleaned automatically after applications finished.
> I think we can set "java.io.tmpdir" to "PWD/tmp" directory, or
> something similar. "PWD" will be replaced with the true working
> directory of JM/TM by Yarn, which will be cleaned automatically.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-15616) Move boot error messages from python-udf-boot.log to taskmanager's log file
[
https://issues.apache.org/jira/browse/FLINK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hequn Cheng closed FLINK-15616.
---
Resolution: Resolved
> Move boot error messages from python-udf-boot.log to taskmanager's log file
> ---
>
> Key: FLINK-15616
> URL: https://issues.apache.org/jira/browse/FLINK-15616
> Project: Flink
> Issue Type: Improvement
> Components: API / Python
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Previously, the boot error messages are printed in the log file under
> FLINK_LOG_DIR, i.e.,
> {{"$FLINK_LOG_DIR/flink-$USER-python-udf-boot-$HOSTNAME.log"}}. This
> additional file is very hard to locate for users, so it would be better to
> print the error messages directly into the taskmanager log file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15616) Move boot error messages from python-udf-boot.log to taskmanager's log file
[
https://issues.apache.org/jira/browse/FLINK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hequn Cheng updated FLINK-15616:
Fix Version/s: 1.11.0
> Move boot error messages from python-udf-boot.log to taskmanager's log file
> ---
>
> Key: FLINK-15616
> URL: https://issues.apache.org/jira/browse/FLINK-15616
> Project: Flink
> Issue Type: Improvement
> Components: API / Python
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Previously, the boot error messages are printed in the log file under
> FLINK_LOG_DIR, i.e.,
> {{"$FLINK_LOG_DIR/flink-$USER-python-udf-boot-$HOSTNAME.log"}}. This
> additional file is very hard to locate for users, so it would be better to
> print the error messages directly into the taskmanager log file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15616) Move boot error messages from python-udf-boot.log to taskmanager's log file
[
https://issues.apache.org/jira/browse/FLINK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018454#comment-17018454
]
Hequn Cheng commented on FLINK-15616:
-
Resolved in 1.11.0 via 4c63ec4dd3d078c2af2077c3187de9b63784f1b7
> Move boot error messages from python-udf-boot.log to taskmanager's log file
> ---
>
> Key: FLINK-15616
> URL: https://issues.apache.org/jira/browse/FLINK-15616
> Project: Flink
> Issue Type: Improvement
> Components: API / Python
>Reporter: Hequn Cheng
>Assignee: Hequn Cheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Previously, the boot error messages are printed in the log file under
> FLINK_LOG_DIR, i.e.,
> {{"$FLINK_LOG_DIR/flink-$USER-python-udf-boot-$HOSTNAME.log"}}. This
> additional file is very hard to locate for users, so it would be better to
> print the error messages directly into the taskmanager log file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] hequn8128 closed pull request #10870: [FLINK-15616][python] Move boot error messages from python-udf-boot.log to taskmanager's log file
hequn8128 closed pull request #10870: [FLINK-15616][python] Move boot error messages from python-udf-boot.log to taskmanager's log file URL: https://github.com/apache/flink/pull/10870 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] curcur commented on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
curcur commented on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-575845406 Thanks for reviewing and merging! @zentol This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] curcur commented on a change in pull request #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
curcur commented on a change in pull request #10832:
[FLINK-14163][runtime]Enforce synchronous registration of
Execution#producedPartitions
URL: https://github.com/apache/flink/pull/10832#discussion_r368188645
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/ExecutionTest.java
##
@@ -540,6 +551,46 @@ public void testSlotReleaseAtomicallyReleasesExecution()
throws Exception {
});
}
+ /**
+* Tests that producedPartitions are registered synchronously under an
asynchronous interface.
+*/
+ @Test(expected = IllegalStateException.class)
Review comment:
I see, thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144983581) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4451) * 3e0617bd355a1e8081ebb9ce615365cb8296dbf2 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/145014752) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4452) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144983581) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4451) * 3e0617bd355a1e8081ebb9ce615365cb8296dbf2 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/145014752) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4452) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144983581) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4451) * 3e0617bd355a1e8081ebb9ce615365cb8296dbf2 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/145014752) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4452) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144983581) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4451) * 3e0617bd355a1e8081ebb9ce615365cb8296dbf2 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (FLINK-15638) releasing/create_release_branch.sh does not set version in flink-python/pyflink/version.py
[
https://issues.apache.org/jira/browse/FLINK-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sunjincheng reassigned FLINK-15638:
---
Assignee: Hequn Cheng (was: sunjincheng)
> releasing/create_release_branch.sh does not set version in
> flink-python/pyflink/version.py
> --
>
> Key: FLINK-15638
> URL: https://issues.apache.org/jira/browse/FLINK-15638
> Project: Flink
> Issue Type: Bug
> Components: Release System
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Hequn Cheng
>Priority: Blocker
> Fix For: 1.10.0
>
>
> {{releasing/create_release_branch.sh}} does not set the version in
> {{flink-python/pyflink/version.py}}. Currently the version.py contains:
> {noformat}
> __version__ = "1.10.dev0"
> {noformat}
> {{setup.py}} will replace .dev0 with -SNAPSHOT and tries to find the
> respective flink distribution in the flink-dist/target, which will not exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-15638) releasing/create_release_branch.sh does not set version in flink-python/pyflink/version.py
[
https://issues.apache.org/jira/browse/FLINK-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sunjincheng reassigned FLINK-15638:
---
Assignee: sunjincheng
> releasing/create_release_branch.sh does not set version in
> flink-python/pyflink/version.py
> --
>
> Key: FLINK-15638
> URL: https://issues.apache.org/jira/browse/FLINK-15638
> Project: Flink
> Issue Type: Bug
> Components: Release System
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: sunjincheng
>Priority: Blocker
> Fix For: 1.10.0
>
>
> {{releasing/create_release_branch.sh}} does not set the version in
> {{flink-python/pyflink/version.py}}. Currently the version.py contains:
> {noformat}
> __version__ = "1.10.dev0"
> {noformat}
> {{setup.py}} will replace .dev0 with -SNAPSHOT and tries to find the
> respective flink distribution in the flink-dist/target, which will not exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-14703) Port the Kafka SQL related tests.
[ https://issues.apache.org/jira/browse/FLINK-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-14703. Fix Version/s: 1.11.0 Resolution: Fixed master: 5420f295c23aa819384da1a10016159042817846 > Port the Kafka SQL related tests. > - > > Key: FLINK-14703 > URL: https://issues.apache.org/jira/browse/FLINK-14703 > Project: Flink > Issue Type: Sub-task > Components: Build System, Connectors / Kafka >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Labels: pull-request-available > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Port the three BASH SQL Kafka test cases : > 1. test_sql_client_kafka.sh > 2. test_sql_client_kafka010.sh > 3. test_sql_client_kafka011.sh > The rough patch of Kafka SQL e2e tests is here: > https://github.com/openinx/flink/commit/24b7a6e34d1e504bc7a411781b43c996644c0910 > Once the framework pull request (https://github.com/apache/flink/pull/10106) > get merged, will create the PR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zentol commented on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests.
zentol commented on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests. URL: https://github.com/apache/flink/pull/10236#issuecomment-575799902 @openinx Quick update for you: I polished the PR a bit, and reverted the addition of the FileSystemResource. In hindsight it was a premature suggestion of mine; while we should have it at some point it needs to be more sophisticated to properly work our filesystem plugins. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zentol merged pull request #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests.
zentol merged pull request #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests. URL: https://github.com/apache/flink/pull/10236 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests.
flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests. URL: https://github.com/apache/flink/pull/10236#issuecomment-554979302 ## CI report: * 271d1809c4fa83e7b7655e941e632f6c6d7318ec Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/136980462) * 683c3f2040dfe8564d250631b04c7acf2e67b08d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/141347317) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3656) * c03bec56d09dc0bdc50583188df2505ba47b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144979682) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4450) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144983581) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4451) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests.
flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests. URL: https://github.com/apache/flink/pull/10236#issuecomment-554979302 ## CI report: * 271d1809c4fa83e7b7655e941e632f6c6d7318ec Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/136980462) * 683c3f2040dfe8564d250631b04c7acf2e67b08d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/141347317) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3656) * c03bec56d09dc0bdc50583188df2505ba47b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144979682) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4450) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144983581) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4451) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-5763) Make savepoints self-contained and relocatable
[
https://issues.apache.org/jira/browse/FLINK-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018243#comment-17018243
]
Stephan Ewen commented on FLINK-5763:
-
We should make this non breaking, with SavepointV3.
> Make savepoints self-contained and relocatable
> --
>
> Key: FLINK-5763
> URL: https://issues.apache.org/jira/browse/FLINK-5763
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Reporter: Ufuk Celebi
>Priority: Critical
> Labels: usability
> Fix For: 1.11.0
>
>
> After a user has triggered a savepoint, a single savepoint file will be
> returned as a handle to the savepoint. A savepoint to {{}} creates a
> savepoint file like {{/savepoint-}}.
> This file contains the metadata of the corresponding checkpoint, but not the
> actual program state. While this works well for short term management
> (pause-and-resume a job), it makes it hard to manage savepoints over longer
> periods of time.
> h4. Problems
> h5. Scattered Checkpoint Files
> For file system based checkpoints (FsStateBackend, RocksDBStateBackend) this
> results in the savepoint referencing files from the checkpoint directory
> (usually different than ). For users, it is virtually impossible to
> tell which checkpoint files belong to a savepoint and which are lingering
> around. This can easily lead to accidentally invalidating a savepoint by
> deleting checkpoint files.
> h5. Savepoints Not Relocatable
> Even if a user is able to figure out which checkpoint files belong to a
> savepoint, moving these files will invalidate the savepoint as well, because
> the metadata file references absolute file paths.
> h5. Forced to Use CLI for Disposal
> Because of the scattered files, the user is in practice forced to use Flink’s
> CLI to dispose a savepoint. This should be possible to handle in the scope of
> the user’s environment via a file system delete operation.
> h4. Proposal
> In order to solve the described problems, savepoints should contain all their
> state, both metadata and program state, inside a single directory.
> Furthermore the metadata must only hold relative references to the checkpoint
> files. This makes it obvious which files make up the state of a savepoint and
> it is possible to move savepoints around by moving the savepoint directory.
> h5. Desired File Layout
> Triggering a savepoint to {{}} creates a directory as follows:
> {code}
> /savepoint--
> +-- _metadata
> +-- data- [1 or more]
> {code}
> We include the JobID in the savepoint directory name in order to give some
> hints about which job a savepoint belongs to.
> h5. CLI
> - Trigger: When triggering a savepoint to {{}} the savepoint
> directory will be returned as the handle to the savepoint.
> - Restore: Users can restore by pointing to the directory or the _metadata
> file. The data files should be required to be in the same directory as the
> _metadata file.
> - Dispose: The disposal command should be deprecated and eventually removed.
> While deprecated, disposal can happen by specifying the directory or the
> _metadata file (same as restore).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144983581) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4451) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809 ## CI report: * 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem
flinkbot edited a comment on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem URL: https://github.com/apache/flink/pull/10890#issuecomment-575711704 ## CI report: * 42db346a226bf559c202c38affea6e9eef97dafe Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974119) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4448) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] bowenli86 commented on a change in pull request #10879: [FLINK-15595][table] Remove CoreMudule in ModuleManager
bowenli86 commented on a change in pull request #10879: [FLINK-15595][table] Remove CoreMudule in ModuleManager URL: https://github.com/apache/flink/pull/10879#discussion_r368074550 ## File path: docs/dev/table/functions/index.md ## @@ -84,11 +84,13 @@ The resolution order is: 1. Temporary catalog function 2. Catalog function -## Ambiguous Function Reference +### Ambiguous Function Reference The resolution order is: 1. Temporary system function -2. System function -3. Temporary catalog function, in the current catalog and current database of the session -4. Catalog function, in the current catalog and current database of the session +2. Temporary catalog function, in the current catalog and current database of the session Review comment: I'd prefer if we can still maintain the function resolution order defined in FLIP 57. Changing it would fundamentally change the behaviors, and would plant troubles for future releases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] bowenli86 commented on a change in pull request #10879: [FLINK-15595][table] Remove CoreMudule in ModuleManager
bowenli86 commented on a change in pull request #10879: [FLINK-15595][table] Remove CoreMudule in ModuleManager URL: https://github.com/apache/flink/pull/10879#discussion_r368074550 ## File path: docs/dev/table/functions/index.md ## @@ -84,11 +84,13 @@ The resolution order is: 1. Temporary catalog function 2. Catalog function -## Ambiguous Function Reference +### Ambiguous Function Reference The resolution order is: 1. Temporary system function -2. System function -3. Temporary catalog function, in the current catalog and current database of the session -4. Catalog function, in the current catalog and current database of the session +2. Temporary catalog function, in the current catalog and current database of the session Review comment: I think the solution should still maintain the function resolution order. Changing it would fundamentally change the behaviors, and would plant trouble for future releases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] bowenli86 commented on issue #10879: [FLINK-15595][table] Remove CoreMudule in ModuleManager
bowenli86 commented on issue #10879: [FLINK-15595][table] Remove CoreMudule in ModuleManager URL: https://github.com/apache/flink/pull/10879#issuecomment-575742781 @twalthr @dawidwys can you also take a look and share your thoughts? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] bowenli86 commented on a change in pull request #10879: [FLINK-15595][table] Remove CoreMudule in ModuleManager
bowenli86 commented on a change in pull request #10879: [FLINK-15595][table]
Remove CoreMudule in ModuleManager
URL: https://github.com/apache/flink/pull/10879#discussion_r368076119
##
File path: docs/dev/table/modules.md
##
@@ -101,8 +93,6 @@ The following types are supported out of the box.
{% highlight yaml %}
modules:
- - name: core
- type: core
Review comment:
if in 1.11 we add back core module, this would be a problem because sql
client requires users listing all modules in yaml file. So if users just
upgrade to 1.11 without putting core module here, sql client would assume users
are explicitly not loading core.
I'm not sure what a better solution can be though, thus I'm neutral to this
change
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests.
flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests. URL: https://github.com/apache/flink/pull/10236#issuecomment-554979302 ## CI report: * 271d1809c4fa83e7b7655e941e632f6c6d7318ec Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/136980462) * 683c3f2040dfe8564d250631b04c7acf2e67b08d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/141347317) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3656) * c03bec56d09dc0bdc50583188df2505ba47b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144979682) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4450) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
flinkbot edited a comment on issue #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options URL: https://github.com/apache/flink/pull/10748#issuecomment-570221103 ## CI report: * 38514c02c79ab0ddb7e80bb483b6034a40d4e199 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142877759) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4047) * 6c2eb4fc84da2322124459c60c8263325cd8b6f1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142994287) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4081) * c94dd550a738913273947aca381b5f290d618e81 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144536250) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4372) * f6a0b8e7a74209ec6a22942a28c763207c27d503 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368064219
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTable.java
##
@@ -356,7 +373,7 @@ public boolean markSlotActive(AllocationID allocationId)
throws SlotNotFoundExce
* @return True if the slot could be marked inactive
*/
public boolean markSlotInactive(AllocationID allocationId, Time
slotTimeout) throws SlotNotFoundException {
- checkInit();
+ checkStartedOrClosing();
Review comment:
Why do we allow the closing state here? I think we cannot do this because
this might introduce a potential race condition between the `timerService`
being stopped and registering a new timeout on it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368065079
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTable.java
##
@@ -462,7 +471,7 @@ public int freeSlot(AllocationID allocationId, Throwable
cause) throws SlotNotFo
* @return True if the timeout is valid; otherwise false
*/
public boolean isValidTimeout(AllocationID allocationId, UUID ticket) {
- checkInit();
+ checkStartedOrClosing();
Review comment:
Same here, this might allow a race condition between this method and the
stopping of the `TimerService`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368068307
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTable.java
##
@@ -686,7 +696,7 @@ public MemoryManager getTaskMemoryManager(AllocationID
allocationID) throws Slot
@Override
public void notifyTimeout(AllocationID key, UUID ticket) {
- checkInit();
+ checkStartedOrClosing();
Review comment:
I am not sure whether this always holds true. I could imagine the situation
in which the `Timeout` in `TimerService` fires and just before it calls
`notifyTimeout`, we finish closing the `TaskSlotTable`. In this situation we
would fail this check.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368067378
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTable.java
##
@@ -605,7 +615,7 @@ public boolean addTask(Task task) throws
SlotNotFoundException, SlotNotActiveExc
* @return The removed task if there is any for the given execution
attempt id; otherwise null
*/
public Task removeTask(ExecutionAttemptID executionAttemptID) {
- checkInit();
+ checkStartedOrClosing();
Review comment:
Are we sure that `removeTask` can only be called when the state is started
or closing? Can't it happen that we call `removeTask` if there is a late
`TaskExecutor#updateTaskExecutionState` call?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368071764
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskExecutorTest.java
##
@@ -701,24 +701,22 @@ public void invoke() throws Exception {
}
@Test
- public void testTaskInterruptionAndTerminationOnShutdown() throws
Exception {
- TaskExecutorTestingContext submissionContext =
createTaskExecutorTestingContext(1);
+ public void testTaskSlotTableTerminationOnShutdown() throws Exception {
+ CompletableFuture taskSlotTableClosingFuture = new
CompletableFuture<>();
+ TaskExecutorTestingContext submissionContext =
createTaskExecutorTestingContext(
+
StubTaskSlotTable.newBuilder().closeAsyncReturns(taskSlotTableClosingFuture).build());
+ final CompletableFuture taskExecutorTerminationFuture;
try {
-
submissionContext.startAllocateSlotAndSubmit(TestInterruptableInvokable.class);
- TestInterruptableInvokable.waitForStarted();
+ submissionContext.start();
} finally {
- submissionContext.taskExecutor.closeAsync();
+ taskExecutorTerminationFuture =
submissionContext.taskExecutor.closeAsync();
}
- // check task has been interrupted
- TestInterruptableInvokable.waitForInterrupted();
-
// check task executor is waiting for the task completion and
has not terminated yet
- final CompletableFuture taskExecutorTerminationFuture =
submissionContext.taskExecutor.getTerminationFuture();
assertThat(taskExecutorTerminationFuture.isDone(), is(false));
- // check task executor has exited after task completion
- TestInterruptableInvokable.finish();
+ // check task executor has exited after task slot table
termination
+ taskSlotTableClosingFuture.complete(null);
Review comment:
Nice work :-)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368059634
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerServices.java
##
@@ -180,12 +180,6 @@ public void shutDown() throws FlinkException {
exception = ExceptionUtils.firstOrSuppressed(e,
exception);
}
- try {
- taskSlotTable.stop();
- } catch (Exception e) {
- exception = ExceptionUtils.firstOrSuppressed(e,
exception);
- }
Review comment:
Not sure whether we should remove this as long as `taskSlotTable` stays part
of `TaskManagerServices`. I think it would be better to call
`taskSlotTable.close()` if we don't factor `taskSlotTable` out of
`TaskManagerServices`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368070093
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/testtasks/TestInterruptableInvokable.java
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.testtasks;
+
+import org.apache.flink.runtime.execution.Environment;
+import org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+
+/**
+ * Test invokable which completes the given future when interrupted (can be
used only once).
+ */
+public class TestInterruptableInvokable extends AbstractInvokable {
+ private static CompletableFuture interruptedFuture;
+ private static CompletableFuture startedFuture;
+ private static CompletableFuture doneFuture;
Review comment:
Could be `OneShotLatches` instead of `CompletableFutures`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r368061316
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlot.java
##
@@ -273,9 +271,37 @@ public String toString() {
}
@Override
- public void close() {
- verifyMemoryFreed();
- this.memoryManager.shutdown();
+ public CompletableFuture closeAsync() {
+ return closeAsync(new FlinkException("Closing the slot"));
+ }
+
+ /**
+* Close the task slot asynchronously.
+*
+* Slot is moved to {@link TaskSlotState#RELEASING} state and only
once.
+* If there are active tasks running in the slot then they are failed.
+* The future of all tasks terminated and slot cleaned up is initiated
only once and always returned
+* in case of multiple attempts to close the slot.
+*
+* @param cause cause of closing
+* @return future of all running task if any being done and slot
cleaned up.
+*/
+ public CompletableFuture closeAsync(Throwable cause) {
Review comment:
I would not make this method public and instead leave it package private.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10889: [Flink-15355][plugins]Enable s3 plugins under Java 11.
flinkbot edited a comment on issue #10889: [Flink-15355][plugins]Enable s3 plugins under Java 11. URL: https://github.com/apache/flink/pull/10889#issuecomment-575697892 ## CI report: * b352d9bce1e32923a1b3da9ab9b835d5aa6dd613 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144966421) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4447) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem
flinkbot edited a comment on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem URL: https://github.com/apache/flink/pull/10890#issuecomment-575711704 ## CI report: * 42db346a226bf559c202c38affea6e9eef97dafe Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974119) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4448) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809 ## CI report: * 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10850: [FLINK-15519][configuration] Preserve logs from BashJavaUtils and make them part of TM logs
flinkbot edited a comment on issue #10850: [FLINK-15519][configuration] Preserve logs from BashJavaUtils and make them part of TM logs URL: https://github.com/apache/flink/pull/10850#issuecomment-574076308 ## CI report: * f902fd5c6cc693bac289427094b53d9e4274 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144288064) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4320) * f3dc16d5020c2140439a6980cb7f2642218b8ad4 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144310946) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4330) * 84c3ca6cd1fc8f4a9edbed0c698d1c3f73a7f589 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144966288) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4446) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader
flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader URL: https://github.com/apache/flink/pull/10836#issuecomment-573438212 ## CI report: * 0e53d682360fe30462917c820c9aa866caa957b5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144062568) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4270) * c5927b76a270ceaf3ae6442826e582d97487c52d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144863310) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4415) * 4abd8a69ee904427937fb2a1df0c2c70743e7f86 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144966253) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4445) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15644) Add support for SQL query validation
[
https://issues.apache.org/jira/browse/FLINK-15644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018217#comment-17018217
]
Zhenqiu Huang commented on FLINK-15644:
---
It is a very useful feature. Looking forward to it.
> Add support for SQL query validation
> -
>
> Key: FLINK-15644
> URL: https://issues.apache.org/jira/browse/FLINK-15644
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
>Reporter: Fabian Hueske
>Priority: Major
>
> It would be good if the {{TableEnvironment}} would offer methods to check the
> validity of SQL queries. Such a method could be used by services (CLI query
> shells, notebooks, SQL UIs) that are backed by Flink and execute their
> queries on Flink.
> Validation should be available in two levels:
> # Validation of syntax and semantics: This includes parsing the query,
> checking the catalog for dbs, tables, fields, type checks for expressions and
> functions, etc. This will check if the query is a valid SQL query.
> # Validation that query is supported: Checks if Flink can execute the given
> query. Some syntactically and semantically valid SQL queries are not
> supported, esp. in a streaming context. This requires running the optimizer.
> If the optimizer generates an execution plan, the query can be executed. This
> check includes the first step and is more expensive.
> The reason for this separation is that the first check can be done much fast
> as it does not involve calling the optimizer. Hence, it would be suitable for
> fast checks in an interactive query editor. The second check might take more
> time (depending on the complexity of the query) and might not be suitable for
> rapid checks but only on explicit user request.
> Requirements:
> * validation does not modify the state of the {{TableEnvironment}}, i.e. it
> does not add plan operators
> * validation does not require connector dependencies
> * validation can identify the update mode of a continuous query result
> (append-only, upsert, retraction).
> Out of scope for this issue:
> * better error messages for unsupported features as suggested by FLINK-7217
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests.
flinkbot edited a comment on issue #10236: [FLINK-14703][e2e] Port the Kafka SQL related tests. URL: https://github.com/apache/flink/pull/10236#issuecomment-554979302 ## CI report: * 271d1809c4fa83e7b7655e941e632f6c6d7318ec Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/136980462) * 683c3f2040dfe8564d250631b04c7acf2e67b08d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/141347317) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3656) * c03bec56d09dc0bdc50583188df2505ba47b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15611) KafkaITCase.testOneToOneSources fails on Travis
[
https://issues.apache.org/jira/browse/FLINK-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018210#comment-17018210
]
Gary Yao commented on FLINK-15611:
--
Can we rule out that this isn't similar to FLINK-15490?
> KafkaITCase.testOneToOneSources fails on Travis
> ---
>
> Key: FLINK-15611
> URL: https://issues.apache.org/jira/browse/FLINK-15611
> Project: Flink
> Issue Type: Bug
>Reporter: Yangze Guo
>Priority: Blocker
> Fix For: 1.10.0
>
>
> {{The test KafkaITCase.testOneToOneSources failed on Travis.}}
> {code:java}
> 03:15:02,019 INFO
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl -
> Deleting topic scale-down-before-first-checkpoint
> 03:15:02,037 INFO
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase -
>
> Test
> testScaleDownBeforeFirstCheckpoint(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase)
> successfully run.
>
> 03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
>- -
> 03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
>- Shut down KafkaTestBase
> 03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
>- -
> 03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
>- -
> 03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
>- KafkaTestBase finished
> 03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
>- -
> 03:15:25.731 [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time
> elapsed: 245.845 s - in
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase
> 03:15:26.099 [INFO]
> 03:15:26.099 [INFO] Results:
> 03:15:26.099 [INFO]
> 03:15:26.099 [ERROR] Failures:
> 03:15:26.099 [ERROR]
> KafkaITCase.testOneToOneSources:97->KafkaConsumerTestBase.runOneToOneExactlyOnceTest:862
> Test failed: Job execution failed.
> {code}
> https://api.travis-ci.com/v3/job/276124537/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15623) Buildling flink-python with maven profile docs-and-source fails
[
https://issues.apache.org/jira/browse/FLINK-15623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Yao updated FLINK-15623:
-
Priority: Major (was: Critical)
> Buildling flink-python with maven profile docs-and-source fails
> ---
>
> Key: FLINK-15623
> URL: https://issues.apache.org/jira/browse/FLINK-15623
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.10.0
> Environment: rev: 91d96abe5f42bd088a326870b4885d79611fccb5
>Reporter: Gary Yao
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> *Description*
> Building flink-python with maven profile {{docs-and-source}} fails due to
> checkstyle violations.
> *How to reproduce*
> Running
> {noformat}
> mvn clean install -pl flink-python -Pdocs-and-source -DskipTests
> -DretryFailedDeploymentCount=10
> {noformat}
> should fail with the following error
> {noformat}
> [...]
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8343]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8344]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8345]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8346]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8347]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8348]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8349]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8350]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 18.046 s
> [INFO] Finished at: 2020-01-16T16:44:01+00:00
> [INFO] Final Memory: 158M/2826M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (validate) on
> project flink-python_2.11: You have 7603 Checkstyle violations. -> [Help 1]
> [ERROR]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15623) Buildling flink-python with maven profile docs-and-source fails
[
https://issues.apache.org/jira/browse/FLINK-15623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018205#comment-17018205
]
Gary Yao commented on FLINK-15623:
--
We can downgrade the priority.
> Buildling flink-python with maven profile docs-and-source fails
> ---
>
> Key: FLINK-15623
> URL: https://issues.apache.org/jira/browse/FLINK-15623
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.10.0
> Environment: rev: 91d96abe5f42bd088a326870b4885d79611fccb5
>Reporter: Gary Yao
>Assignee: Chesnay Schepler
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> *Description*
> Building flink-python with maven profile {{docs-and-source}} fails due to
> checkstyle violations.
> *How to reproduce*
> Running
> {noformat}
> mvn clean install -pl flink-python -Pdocs-and-source -DskipTests
> -DretryFailedDeploymentCount=10
> {noformat}
> should fail with the following error
> {noformat}
> [...]
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8343]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8344]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8345]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8346]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8347]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8348]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8349]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8350]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 18.046 s
> [INFO] Finished at: 2020-01-16T16:44:01+00:00
> [INFO] Final Memory: 158M/2826M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (validate) on
> project flink-python_2.11: You have 7603 Checkstyle violations. -> [Help 1]
> [ERROR]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15623) Buildling flink-python with maven profile docs-and-source fails
[
https://issues.apache.org/jira/browse/FLINK-15623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Yao updated FLINK-15623:
-
Priority: Critical (was: Blocker)
> Buildling flink-python with maven profile docs-and-source fails
> ---
>
> Key: FLINK-15623
> URL: https://issues.apache.org/jira/browse/FLINK-15623
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.10.0
> Environment: rev: 91d96abe5f42bd088a326870b4885d79611fccb5
>Reporter: Gary Yao
>Assignee: Chesnay Schepler
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> *Description*
> Building flink-python with maven profile {{docs-and-source}} fails due to
> checkstyle violations.
> *How to reproduce*
> Running
> {noformat}
> mvn clean install -pl flink-python -Pdocs-and-source -DskipTests
> -DretryFailedDeploymentCount=10
> {noformat}
> should fail with the following error
> {noformat}
> [...]
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8343]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8344]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8345]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8346]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8347]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8348]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8349]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [ERROR]
> generated-sources/org/apache/flink/fnexecution/v1/FlinkFnApi.java:[8350]
> (regexp) RegexpSinglelineJava: Line has leading space characters; indentation
> should be performed with tabs only.
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 18.046 s
> [INFO] Finished at: 2020-01-16T16:44:01+00:00
> [INFO] Final Memory: 158M/2826M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (validate) on
> project flink-python_2.11: You have 7603 Checkstyle violations. -> [Help 1]
> [ERROR]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809 ## CI report: * 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem
flinkbot edited a comment on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem URL: https://github.com/apache/flink/pull/10890#issuecomment-575711704 ## CI report: * 42db346a226bf559c202c38affea6e9eef97dafe Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144974119) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4448) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader
flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader URL: https://github.com/apache/flink/pull/10836#issuecomment-573438212 ## CI report: * 0e53d682360fe30462917c820c9aa866caa957b5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144062568) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4270) * c5927b76a270ceaf3ae6442826e582d97487c52d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144863310) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4415) * 4abd8a69ee904427937fb2a1df0c2c70743e7f86 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144966253) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4445) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-15644) Add support for SQL query validation
Fabian Hueske created FLINK-15644:
-
Summary: Add support for SQL query validation
Key: FLINK-15644
URL: https://issues.apache.org/jira/browse/FLINK-15644
Project: Flink
Issue Type: New Feature
Components: Table SQL / API
Reporter: Fabian Hueske
It would be good if the {{TableEnvironment}} would offer methods to check the
validity of SQL queries. Such a method could be used by services (CLI query
shells, notebooks, SQL UIs) that are backed by Flink and execute their queries
on Flink.
Validation should be available in two levels:
# Validation of syntax and semantics: This includes parsing the query,
checking the catalog for dbs, tables, fields, type checks for expressions and
functions, etc. This will check if the query is a valid SQL query.
# Validation that query is supported: Checks if Flink can execute the given
query. Some syntactically and semantically valid SQL queries are not supported,
esp. in a streaming context. This requires running the optimizer. If the
optimizer generates an execution plan, the query can be executed. This check
includes the first step and is more expensive.
The reason for this separation is that the first check can be done much fast as
it does not involve calling the optimizer. Hence, it would be suitable for fast
checks in an interactive query editor. The second check might take more time
(depending on the complexity of the query) and might not be suitable for rapid
checks but only on explicit user request.
Requirements:
* validation does not modify the state of the {{TableEnvironment}}, i.e. it
does not add plan operators
* validation does not require connector dependencies
* validation can identify the update mode of a continuous query result
(append-only, upsert, retraction).
Out of scope for this issue:
* better error messages for unsupported features as suggested by FLINK-7217
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-12122) Spread out tasks evenly across all available registered TaskManagers
[
https://issues.apache.org/jira/browse/FLINK-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018197#comment-17018197
]
Till Rohrmann commented on FLINK-12122:
---
I think it could help in this situation. What I am a bit unsure about is
whether one should really not start the execution of a job if the minimum
number {{TaskExecutors}} hasn't been reached yet.
> Spread out tasks evenly across all available registered TaskManagers
>
>
> Key: FLINK-12122
> URL: https://issues.apache.org/jira/browse/FLINK-12122
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.6.4, 1.7.2, 1.8.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.2, 1.10.0
>
> Attachments: image-2019-05-21-12-28-29-538.png,
> image-2019-05-21-13-02-50-251.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> With Flip-6, we changed the default behaviour how slots are assigned to
> {{TaskManages}}. Instead of evenly spreading it out over all registered
> {{TaskManagers}}, we randomly pick slots from {{TaskManagers}} with a
> tendency to first fill up a TM before using another one. This is a regression
> wrt the pre Flip-6 code.
> I suggest to change the behaviour so that we try to evenly distribute slots
> across all available {{TaskManagers}} by considering how many of their slots
> are already allocated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15419) Validate SQL syntax not need to depend on connector jar
[
https://issues.apache.org/jira/browse/FLINK-15419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske updated FLINK-15419:
--
Summary: Validate SQL syntax not need to depend on connector jar (was:
Add support for SQL query validation)
> Validate SQL syntax not need to depend on connector jar
>
>
> Key: FLINK-15419
> URL: https://issues.apache.org/jira/browse/FLINK-15419
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Kaibo Zhou
>Priority: Major
> Fix For: 1.11.0
>
>
> As a platform user, I want to integrate Flink SQL in my platform.
> The users will register Source/Sink Tables and Functions to catalog service
> through UI, and write SQL scripts on Web SQLEditor. I want to validate the
> SQL syntax and validate that all catalog objects exist (table, fields, UDFs).
> After some investigation, I decided to use the `tEnv.sqlUpdate/sqlQuery` API
> to do this.`SqlParser` and`FlinkSqlParserImpl` is not a good choice, as it
> will not read the catalog.
> The users have registered *Kafka* source/sink table in the catalog, so the
> validation logic will be:
> {code:java}
> TableEnvironment tableEnv =
> tEnv.registerCatalog(CATALOG_NAME, catalog);
> tEnv.useCatalog(CATALOG_NAME);
> tEnv.useDatabase(DB_NAME);
> tEnv.sqlUpdate("INSERT INTO sinkTable SELECT f1,f2 FROM sourceTable");
> or
> tEnv.sqlQuery("SELECT * FROM tableName")
> {code}
> It will through exception on Flink 1.9.0 because I do not have
> `flink-connector-kafka_2.11-1.9.0.jar` in my classpath.
> {code:java}
> org.apache.flink.table.api.ValidationException: SQL validation failed.
> findAndCreateTableSource
> failed.org.apache.flink.table.api.ValidationException: SQL validation failed.
> findAndCreateTableSource failed. at
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:125)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:82)
> at
> org.apache.flink.table.planner.delegation.PlannerBase.parse(PlannerBase.scala:132)
> at
> org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlUpdate(TableEnvironmentImpl.java:335)
> The following factories have been considered:
> org.apache.flink.formats.json.JsonRowFormatFactory
> org.apache.flink.table.planner.delegation.BlinkPlannerFactory
> org.apache.flink.table.planner.delegation.BlinkExecutorFactory
> org.apache.flink.table.catalog.GenericInMemoryCatalogFactory
> org.apache.flink.table.sources.CsvBatchTableSourceFactory
> org.apache.flink.table.sources.CsvAppendTableSourceFactory
> org.apache.flink.table.sinks.CsvBatchTableSinkFactory
> org.apache.flink.table.sinks.CsvAppendTableSinkFactory
> at
> org.apache.flink.table.factories.TableFactoryService.filterByContext(TableFactoryService.java:283)
> at
> org.apache.flink.table.factories.TableFactoryService.filter(TableFactoryService.java:191)
> at
> org.apache.flink.table.factories.TableFactoryService.findSingleInternal(TableFactoryService.java:144)
> at
> org.apache.flink.table.factories.TableFactoryService.find(TableFactoryService.java:97)
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:64)
> {code}
> For a platform provider, the user's SQL may depend on *ANY* connector or even
> a custom connector. It is complicated to do dynamic loading connector jar
> after parser the connector type in SQL. And this requires the users must
> upload their custom connector jar before doing a syntax check.
> I hope that Flink can provide a friendly way to verify the syntax of SQL
> whose tables/functions are already registered in the catalog, *NOT* need to
> depend on the jar of the connector. This makes it easier for SQL to be
> integrated by external platforms.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15419) Add support for SQL query validation
[
https://issues.apache.org/jira/browse/FLINK-15419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske updated FLINK-15419:
--
Summary: Add support for SQL query validation (was: Validate SQL syntax
not need to depend on connector jar)
> Add support for SQL query validation
>
>
> Key: FLINK-15419
> URL: https://issues.apache.org/jira/browse/FLINK-15419
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Kaibo Zhou
>Priority: Major
> Fix For: 1.11.0
>
>
> As a platform user, I want to integrate Flink SQL in my platform.
> The users will register Source/Sink Tables and Functions to catalog service
> through UI, and write SQL scripts on Web SQLEditor. I want to validate the
> SQL syntax and validate that all catalog objects exist (table, fields, UDFs).
> After some investigation, I decided to use the `tEnv.sqlUpdate/sqlQuery` API
> to do this.`SqlParser` and`FlinkSqlParserImpl` is not a good choice, as it
> will not read the catalog.
> The users have registered *Kafka* source/sink table in the catalog, so the
> validation logic will be:
> {code:java}
> TableEnvironment tableEnv =
> tEnv.registerCatalog(CATALOG_NAME, catalog);
> tEnv.useCatalog(CATALOG_NAME);
> tEnv.useDatabase(DB_NAME);
> tEnv.sqlUpdate("INSERT INTO sinkTable SELECT f1,f2 FROM sourceTable");
> or
> tEnv.sqlQuery("SELECT * FROM tableName")
> {code}
> It will through exception on Flink 1.9.0 because I do not have
> `flink-connector-kafka_2.11-1.9.0.jar` in my classpath.
> {code:java}
> org.apache.flink.table.api.ValidationException: SQL validation failed.
> findAndCreateTableSource
> failed.org.apache.flink.table.api.ValidationException: SQL validation failed.
> findAndCreateTableSource failed. at
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:125)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:82)
> at
> org.apache.flink.table.planner.delegation.PlannerBase.parse(PlannerBase.scala:132)
> at
> org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlUpdate(TableEnvironmentImpl.java:335)
> The following factories have been considered:
> org.apache.flink.formats.json.JsonRowFormatFactory
> org.apache.flink.table.planner.delegation.BlinkPlannerFactory
> org.apache.flink.table.planner.delegation.BlinkExecutorFactory
> org.apache.flink.table.catalog.GenericInMemoryCatalogFactory
> org.apache.flink.table.sources.CsvBatchTableSourceFactory
> org.apache.flink.table.sources.CsvAppendTableSourceFactory
> org.apache.flink.table.sinks.CsvBatchTableSinkFactory
> org.apache.flink.table.sinks.CsvAppendTableSinkFactory
> at
> org.apache.flink.table.factories.TableFactoryService.filterByContext(TableFactoryService.java:283)
> at
> org.apache.flink.table.factories.TableFactoryService.filter(TableFactoryService.java:191)
> at
> org.apache.flink.table.factories.TableFactoryService.findSingleInternal(TableFactoryService.java:144)
> at
> org.apache.flink.table.factories.TableFactoryService.find(TableFactoryService.java:97)
> at
> org.apache.flink.table.factories.TableFactoryUtil.findAndCreateTableSource(TableFactoryUtil.java:64)
> {code}
> For a platform provider, the user's SQL may depend on *ANY* connector or even
> a custom connector. It is complicated to do dynamic loading connector jar
> after parser the connector type in SQL. And this requires the users must
> upload their custom connector jar before doing a syntax check.
> I hope that Flink can provide a friendly way to verify the syntax of SQL
> whose tables/functions are already registered in the catalog, *NOT* need to
> depend on the jar of the connector. This makes it easier for SQL to be
> integrated by external platforms.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
flinkbot commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809 ## CI report: * 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem
flinkbot commented on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem URL: https://github.com/apache/flink/pull/10890#issuecomment-575711704 ## CI report: * 42db346a226bf559c202c38affea6e9eef97dafe UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10889: [Flink-15355][plugins]Enable s3 plugins under Java 11.
flinkbot edited a comment on issue #10889: [Flink-15355][plugins]Enable s3 plugins under Java 11. URL: https://github.com/apache/flink/pull/10889#issuecomment-575697892 ## CI report: * b352d9bce1e32923a1b3da9ab9b835d5aa6dd613 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144966421) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4447) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10850: [FLINK-15519][configuration] Preserve logs from BashJavaUtils and make them part of TM logs
flinkbot edited a comment on issue #10850: [FLINK-15519][configuration] Preserve logs from BashJavaUtils and make them part of TM logs URL: https://github.com/apache/flink/pull/10850#issuecomment-574076308 ## CI report: * f902fd5c6cc693bac289427094b53d9e4274 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144288064) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4320) * f3dc16d5020c2140439a6980cb7f2642218b8ad4 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144310946) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4330) * 84c3ca6cd1fc8f4a9edbed0c698d1c3f73a7f589 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144966288) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4446) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader
flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader URL: https://github.com/apache/flink/pull/10836#issuecomment-573438212 ## CI report: * 0e53d682360fe30462917c820c9aa866caa957b5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144062568) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4270) * c5927b76a270ceaf3ae6442826e582d97487c52d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144863310) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4415) * 4abd8a69ee904427937fb2a1df0c2c70743e7f86 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144966253) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4445) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (FLINK-15601) Remove useless constant field NUM_STOP_CALL_TRIES in Execution
[
https://issues.apache.org/jira/browse/FLINK-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann reassigned FLINK-15601:
-
Assignee: vinoyang
> Remove useless constant field NUM_STOP_CALL_TRIES in Execution
> --
>
> Key: FLINK-15601
> URL: https://issues.apache.org/jira/browse/FLINK-15601
> Project: Flink
> Issue Type: Wish
> Components: Runtime / Task
>Reporter: vinoyang
>Assignee: vinoyang
>Priority: Major
>
> Currently, the constant field {{NUM_STOP_CALL_TRIES}} in {{Execution}} is not
> been used. IMO, we can remove it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15601) Remove useless constant field NUM_STOP_CALL_TRIES in Execution
[
https://issues.apache.org/jira/browse/FLINK-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018179#comment-17018179
]
Till Rohrmann commented on FLINK-15601:
---
Sure, go ahead [~yanghua].
> Remove useless constant field NUM_STOP_CALL_TRIES in Execution
> --
>
> Key: FLINK-15601
> URL: https://issues.apache.org/jira/browse/FLINK-15601
> Project: Flink
> Issue Type: Wish
> Components: Runtime / Task
>Reporter: vinoyang
>Priority: Major
>
> Currently, the constant field {{NUM_STOP_CALL_TRIES}} in {{Execution}} is not
> been used. IMO, we can remove it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15447) Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
[
https://issues.apache.org/jira/browse/FLINK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018177#comment-17018177
]
Rong Rong commented on FLINK-15447:
---
as far as I can tell, {{java.io.tmpdir}} is controlled by SystemEnv and should
be set as a JVM launch param similar to how [~xymaqingxiang] mentioned.
Could you elaborate what does it mean by Flink-YARN default the value to
{{/tmp}}? I am guessing you mean JVM default the value to {{/tmp}} ?
* So far the only place I can see in flink yarn code utilizing this key is:
https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L899
Is your intention to have Flink override the JVM configuration internally and
ignore the system environment config?
> Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
> ---
>
> Key: FLINK-15447
> URL: https://issues.apache.org/jira/browse/FLINK-15447
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Affects Versions: 1.9.1
>Reporter: Victor Wong
>Priority: Major
>
> Currently, when running Flink on Yarn, the "java.io.tmpdir" property is set
> to the default value, which is "/tmp".
>
> Sometimes we ran into exceptions caused by a full "/tmp" directory, which
> would not be cleaned automatically after applications finished.
> I think we can set "java.io.tmpdir" to "PWD/tmp" directory, or
> something similar. "PWD" will be replaced with the true working
> directory of JM/TM by Yarn, which will be cleaned automatically.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-12362) Remove legacy container number config option for Flink on yarn
[
https://issues.apache.org/jira/browse/FLINK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018173#comment-17018173
]
Till Rohrmann commented on FLINK-12362:
---
At the moment there is no way to configure a fixed number of containers. Flink
will try to allocate as many containers as there are necessary to run the
submitted job. If you define for every operator a different slot sharing group,
then you would need to know how many operators there are in your job to
calculate the total number of required slots.
> Remove legacy container number config option for Flink on yarn
> --
>
> Key: FLINK-12362
> URL: https://issues.apache.org/jira/browse/FLINK-12362
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Reporter: vinoyang
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{-n}} config option for Flink on YARN has been deprecated and removed
> from documentation. However, there are still some legacy code exists in the
> codebase. We need to clean up them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15198) Remove deprecated mesos.resourcemanager.tasks.mem in 1.11
[ https://issues.apache.org/jira/browse/FLINK-15198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018171#comment-17018171 ] ptmagic commented on FLINK-15198: - [~azagrebin] Please take a look at this pr [https://github.com/apache/flink/pull/10890] ，If there is something wrong, please let me know > Remove deprecated mesos.resourcemanager.tasks.mem in 1.11 > - > > Key: FLINK-15198 > URL: https://issues.apache.org/jira/browse/FLINK-15198 > Project: Flink > Issue Type: Task > Components: Deployment / Mesos >Affects Versions: 1.11.0 >Reporter: Andrey Zagrebin >Assignee: ptmagic >Priority: Blocker > Labels: pull-request-available > Fix For: 1.11.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In FLINK-15082, we deprecated 'mesos.resourcemanager.tasks.mem' in favour of > the new unified option 'taskmanager.memory.total-process.size' from FLIP-49. > We should remove it now in 1.11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
flinkbot commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor URL: https://github.com/apache/flink/pull/10891#issuecomment-575699403 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b (Fri Jan 17 16:34:00 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15561) Improve Kerberos delegation token login
[ https://issues.apache.org/jira/browse/FLINK-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-15561: --- Labels: pull-request-available usability (was: usability) > Improve Kerberos delegation token login > > > Key: FLINK-15561 > URL: https://issues.apache.org/jira/browse/FLINK-15561 > Project: Flink > Issue Type: Bug > Components: Deployment / YARN >Reporter: Rong Rong >Assignee: Rong Rong >Priority: Major > Labels: pull-request-available, usability > Fix For: 1.11.0 > > > Currently the security HadoopModule handles delegation token login seems to > be not working. > Some improvements including: spawning a delegation token renewal thread. See: > [1] > https://github.com/apache/flink/blob/release-1.9/flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/HadoopModule.java#L84 > > and [2] > https://github.com/hanborq/hadoop/blob/master/src/core/org/apache/hadoop/security/UserGroupInformation.java#L538 > Another is to ensure delegation token is also a valid format of credential > when launching YARN context. See [1] > https://github.com/apache/flink/blob/master/flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java#L484 > and [2] > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/HadoopModule.java#L146 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem
flinkbot commented on issue #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem URL: https://github.com/apache/flink/pull/10890#issuecomment-575698484 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 42db346a226bf559c202c38affea6e9eef97dafe (Fri Jan 17 16:31:43 UTC 2020) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr opened a new pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
walterddr opened a new pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor URL: https://github.com/apache/flink/pull/10891 …ker.
[GitHub] [flink] tillrohrmann commented on a change in pull request #10748: [FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
tillrohrmann commented on a change in pull request #10748:
[FLINK-15458][conf][docs] Add support for whitelisting ambiguous options
URL: https://github.com/apache/flink/pull/10748#discussion_r36892
##
File path:
flink-docs/src/test/java/org/apache/flink/docs/configuration/ConfigOptionsDocsCompletenessITCase.java
##
@@ -139,22 +139,42 @@ private static void
compareDocumentedAndExistingOptions(Map {
- List documentedState =
documentedOptions.remove(key);
+ List documentedState =
documentedOptions.get(key);
for (ExistingOption supposedState : supposedStates) {
if (documentedState == null ||
documentedState.isEmpty()) {
// option is not documented at all
problems.add("Option " +
supposedState.key + " in " + supposedState.containingClass + " is not
documented.");
- } else if
(documentedState.stream().noneMatch(documentedOption ->
supposedState.defaultValue.equals(documentedOption.defaultValue))) {
- // default is outdated
- problems.add("Documented default of " +
supposedState.key + " in " + supposedState.containingClass +
- " is outdated. Expected: " +
supposedState.defaultValue);
- } else if
(documentedState.stream().noneMatch(documentedOption ->
supposedState.description.equals(documentedOption.description))) {
- // description is outdated
- problems.add("Documented description of
" + supposedState.key + " in " + supposedState.containingClass +
- " is outdated.");
} else {
- // the docs for this option are
up-to-date
+ final Iterator
candidates = documentedState.iterator();
+
+ boolean matchingDefaultFound = false;
+ boolean matchingDescriptionFound =
false;
+ while (candidates.hasNext()) {
+ DocumentedOption candidate =
candidates.next();
+ if
(supposedState.defaultValue.equals(candidate.defaultValue)) {
+ matchingDefaultFound =
true;
+ }
+ if
(supposedState.description.equals(candidate.description)) {
+
matchingDescriptionFound = true;
+ }
+ if (matchingDefaultFound &&
matchingDescriptionFound) {
+ // option is properly
documented
+ candidates.remove();
+ break;
+ }
+ }
Review comment:
I'm not sure whether I completely understand this logic here. Can't it
happen that we have two options where one fulfills the default value and the
other fulfills the description? In this case we would remove the second option
and not report any problems even though both of them should cause a problem.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10887: [FLINK-15150][tests] Prevent job from reaching terminal state
flinkbot edited a comment on issue #10887: [FLINK-15150][tests] Prevent job from reaching terminal state URL: https://github.com/apache/flink/pull/10887#issuecomment-575653696 ## CI report: * 7f91b3855bca5e7e2d2d9abf196cca861e122312 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144944259) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4440) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10889: [Flink-15355][plugins]Enable s3 plugins under Java 11.
flinkbot commented on issue #10889: [Flink-15355][plugins]Enable s3 plugins under Java 11. URL: https://github.com/apache/flink/pull/10889#issuecomment-575697892 ## CI report: * b352d9bce1e32923a1b3da9ab9b835d5aa6dd613 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] PengTaoWW opened a new pull request #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem
PengTaoWW opened a new pull request #10890: [FLINK-15198][Deployment / Mesos] remove mesos.resourcemanager.tasks.mem URL: https://github.com/apache/flink/pull/10890 fix https://issues.apache.org/jira/projects/FLINK/issues/FLINK-15198 ## What is the purpose of the change In FLINK-15082, we deprecated 'mesos.resourcemanager.tasks.mem' in favour of the new unified option 'taskmanager.memory.total-process.size' from FLIP-49. We should remove it now in 1.11. https://issues.apache.org/jira/projects/FLINK/issues/FLINK-15198?filter=allopenissues ## Brief change log Replaced mesos.resourcemanager.tasks.mem in code and comments to taskmanager.memory.total-process.size ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10850: [FLINK-15519][configuration] Preserve logs from BashJavaUtils and make them part of TM logs
flinkbot edited a comment on issue #10850: [FLINK-15519][configuration] Preserve logs from BashJavaUtils and make them part of TM logs URL: https://github.com/apache/flink/pull/10850#issuecomment-574076308 ## CI report: * f902fd5c6cc693bac289427094b53d9e4274 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144288064) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4320) * f3dc16d5020c2140439a6980cb7f2642218b8ad4 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144310946) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4330) * 84c3ca6cd1fc8f4a9edbed0c698d1c3f73a7f589 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader
flinkbot edited a comment on issue #10836: [FLINK-11589][Security] Support security module and context discovery via ServiceLoader URL: https://github.com/apache/flink/pull/10836#issuecomment-573438212 ## CI report: * 0e53d682360fe30462917c820c9aa866caa957b5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144062568) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4270) * c5927b76a270ceaf3ae6442826e582d97487c52d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144863310) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4415) * 4abd8a69ee904427937fb2a1df0c2c70743e7f86 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
