[jira] [Commented] (FLINK-18356) flink-table-planner Exit code 137 returned from process
[
https://issues.apache.org/jira/browse/FLINK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685744#comment-17685744
]
Matthias Pohl commented on FLINK-18356:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=45865=logs=a9db68b9-a7e0-54b6-0f98-010e0aff39e2=cdd32e0b-6047-565b-c58f-14054472f1be=12250
> flink-table-planner Exit code 137 returned from process
> ---
>
> Key: FLINK-18356
> URL: https://issues.apache.org/jira/browse/FLINK-18356
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Tests
>Affects Versions: 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, 1.17.0
>Reporter: Piotr Nowojski
>Priority: Critical
> Labels: pull-request-available, test-stability
> Attachments: 1234.jpg, app-profiling_4.gif,
> image-2023-01-11-22-21-57-784.png, image-2023-01-11-22-22-32-124.png
>
>
> {noformat}
> = test session starts
> ==
> platform linux -- Python 3.7.3, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
> cachedir: .tox/py37-cython/.pytest_cache
> rootdir: /__w/3/s/flink-python
> collected 568 items
> pyflink/common/tests/test_configuration.py ..[
> 1%]
> pyflink/common/tests/test_execution_config.py ...[
> 5%]
> pyflink/dataset/tests/test_execution_environment.py .
> ##[error]Exit code 137 returned from process: file name '/bin/docker',
> arguments 'exec -i -u 1002
> 97fc4e22522d2ced1f4d23096b8929045d083dd0a99a4233a8b20d0489e9bddb
> /__a/externals/node/bin/node /__w/_temp/containerHandlerInvoker.js'.
> Finishing: Test - python
> {noformat}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=3729=logs=9cada3cb-c1d3-5621-16da-0f718fb86602=8d78fe4f-d658-5c70-12f8-4921589024c3
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-18356) flink-table-planner Exit code 137 returned from process
[
https://issues.apache.org/jira/browse/FLINK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685745#comment-17685745
]
Matthias Pohl commented on FLINK-18356:
---
[~337361...@qq.com] did you manage to look into it?
> flink-table-planner Exit code 137 returned from process
> ---
>
> Key: FLINK-18356
> URL: https://issues.apache.org/jira/browse/FLINK-18356
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Tests
>Affects Versions: 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, 1.17.0
>Reporter: Piotr Nowojski
>Priority: Critical
> Labels: pull-request-available, test-stability
> Attachments: 1234.jpg, app-profiling_4.gif,
> image-2023-01-11-22-21-57-784.png, image-2023-01-11-22-22-32-124.png
>
>
> {noformat}
> = test session starts
> ==
> platform linux -- Python 3.7.3, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
> cachedir: .tox/py37-cython/.pytest_cache
> rootdir: /__w/3/s/flink-python
> collected 568 items
> pyflink/common/tests/test_configuration.py ..[
> 1%]
> pyflink/common/tests/test_execution_config.py ...[
> 5%]
> pyflink/dataset/tests/test_execution_environment.py .
> ##[error]Exit code 137 returned from process: file name '/bin/docker',
> arguments 'exec -i -u 1002
> 97fc4e22522d2ced1f4d23096b8929045d083dd0a99a4233a8b20d0489e9bddb
> /__a/externals/node/bin/node /__w/_temp/containerHandlerInvoker.js'.
> Finishing: Test - python
> {noformat}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=3729=logs=9cada3cb-c1d3-5621-16da-0f718fb86602=8d78fe4f-d658-5c70-12f8-4921589024c3
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (FLINK-30910) ApplicationDispatcherBootstrapTest.testApplicationIsStoppedWhenStoppingBootstrap fails with assertion
[
https://issues.apache.org/jira/browse/FLINK-30910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Pohl resolved FLINK-30910.
---
Fix Version/s: 1.17.0
1.15.4
1.16.2
1.18.0
Resolution: Fixed
master: a6de5f23a30fcd580f4d005ade0fb2eba0e901dc
1.17: 5ded7b320eff4fe7587d9eeb1fe5fa87e217718b
1.16: cb83c268df09426060d1d45cc7fde9a999637691
1.15: eaf35c80342ab0df2fd70ef730e9299521b5a9d4
> ApplicationDispatcherBootstrapTest.testApplicationIsStoppedWhenStoppingBootstrap
> fails with assertion
> -
>
> Key: FLINK-30910
> URL: https://issues.apache.org/jira/browse/FLINK-30910
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.17.0, 1.15.3, 1.16.1
>Reporter: Matthias Pohl
>Assignee: Matthias Pohl
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.17.0, 1.15.4, 1.16.2, 1.18.0
>
>
> A build failure in
> {{ApplicationDispatcherBootstrapTest.testApplicationIsStoppedWhenStoppingBootstrap}}:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=45722=logs=0da23115-68bb-5dcd-192c-bd4c8adebde1=24c3384f-1bcb-57b3-224f-51bf973bbee8=9831
> {code}
> Feb 05 01:13:44 [ERROR] Tests run: 30, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 5.174 s <<< FAILURE! - in
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrapTest
> Feb 05 01:13:44 [ERROR]
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrapTest.testApplicationIsStoppedWhenStoppingBootstrap
> Time elapsed: 2.026 s <<< FAILURE!
> Feb 05 01:13:44 org.opentest4j.AssertionFailedError:
> Feb 05 01:13:44
> Feb 05 01:13:44 Expecting value to be true but was false
> Feb 05 01:13:44 at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> Feb 05 01:13:44 at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> Feb 05 01:13:44 at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> Feb 05 01:13:44 at
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrapTest.testApplicationIsStoppedWhenStoppingBootstrap(ApplicationDispatcherBootstrapTest.java:361)
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] XComp merged pull request #21882: [BP-1.16][FLINK-30910][runtime] Making test wait for stop to be called before finishing the bootstrap operation (#21869)
XComp merged PR #21882: URL: https://github.com/apache/flink/pull/21882 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] XComp commented on pull request #21882: [BP-1.16][FLINK-30910][runtime] Making test wait for stop to be called before finishing the bootstrap operation (#21869)
XComp commented on PR #21882: URL: https://github.com/apache/flink/pull/21882#issuecomment-1422174383 The review of this PR happened in the parent PR https://github.com/apache/flink/pull/21869. No conflicts arose while cherry-picking the commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] XComp merged pull request #21881: [BP-1.17][FLINK-30910][runtime] Making test wait for stop to be called before finishing the bootstrap operation (#21869)
XComp merged PR #21881: URL: https://github.com/apache/flink/pull/21881 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] XComp commented on pull request #21881: [BP-1.17][FLINK-30910][runtime] Making test wait for stop to be called before finishing the bootstrap operation (#21869)
XComp commented on PR #21881: URL: https://github.com/apache/flink/pull/21881#issuecomment-1422174107 The review of this PR happened in the parent PR https://github.com/apache/flink/pull/21869. No conflicts arose while cherry-picking the commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] XComp commented on pull request #21881: [BP-1.17][FLINK-30910][runtime] Making test wait for stop to be called before finishing the bootstrap operation (#21869)
XComp commented on PR #21881: URL: https://github.com/apache/flink/pull/21881#issuecomment-1422173499 Failure is caused by FLINK-25813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-28440) EventTimeWindowCheckpointingITCase failed with restore
[
https://issues.apache.org/jira/browse/FLINK-28440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685737#comment-17685737
]
Matthias Pohl commented on FLINK-28440:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=45857=logs=a57e0635-3fad-5b08-57c7-a4142d7d6fa9=2ef0effc-1da1-50e5-c2bd-aab434b1c5b7=10292
> EventTimeWindowCheckpointingITCase failed with restore
> --
>
> Key: FLINK-28440
> URL: https://issues.apache.org/jira/browse/FLINK-28440
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / State Backends
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Huang Xingbo
>Assignee: Yanfei Lei
>Priority: Critical
> Labels: auto-deprioritized-critical, pull-request-available,
> test-stability
> Attachments: image-2023-02-01-00-51-54-506.png,
> image-2023-02-01-01-10-01-521.png, image-2023-02-01-01-19-12-182.png,
> image-2023-02-01-16-47-23-756.png, image-2023-02-01-16-57-43-889.png,
> image-2023-02-02-10-52-56-599.png, image-2023-02-03-10-09-07-586.png,
> image-2023-02-03-12-03-16-155.png, image-2023-02-03-12-03-56-614.png
>
>
> {code:java}
> Caused by: java.lang.Exception: Exception while creating
> StreamOperatorStateContext.
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:256)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:268)
> at
> org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:106)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:722)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:698)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:665)
> at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
> at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:904)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.flink.util.FlinkException: Could not restore keyed
> state backend for WindowOperator_0a448493b4782967b150582570326227_(2/4) from
> any of the 1 provided restore options.
> at
> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160)
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:353)
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:165)
> ... 11 more
> Caused by: java.lang.RuntimeException: java.io.FileNotFoundException:
> /tmp/junit1835099326935900400/junit1113650082510421526/52ee65b7-033f-4429-8ddd-adbe85e27ced
> (No such file or directory)
> at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
> at
> org.apache.flink.runtime.state.changelog.StateChangelogHandleStreamHandleReader$1.advance(StateChangelogHandleStreamHandleReader.java:87)
> at
> org.apache.flink.runtime.state.changelog.StateChangelogHandleStreamHandleReader$1.hasNext(StateChangelogHandleStreamHandleReader.java:69)
> at
> org.apache.flink.state.changelog.restore.ChangelogBackendRestoreOperation.readBackendHandle(ChangelogBackendRestoreOperation.java:96)
> at
> org.apache.flink.state.changelog.restore.ChangelogBackendRestoreOperation.restore(ChangelogBackendRestoreOperation.java:75)
> at
> org.apache.flink.state.changelog.ChangelogStateBackend.restore(ChangelogStateBackend.java:92)
> at
> org.apache.flink.state.changelog.AbstractChangelogStateBackend.createKeyedStateBackend(AbstractChangelogStateBackend.java:136)
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:336)
> at
> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
> at
> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
> ... 13 more
> Caused by: java.io.FileNotFoundException:
>
[GitHub] [flink] XComp merged pull request #21883: [BP-1.15][FLINK-30910][runtime] Making test wait for stop to be called before finishing the bootstrap operation (#21869)
XComp merged PR #21883: URL: https://github.com/apache/flink/pull/21883 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] XComp commented on pull request #21883: [BP-1.15][FLINK-30910][runtime] Making test wait for stop to be called before finishing the bootstrap operation (#21869)
XComp commented on PR #21883: URL: https://github.com/apache/flink/pull/21883#issuecomment-1422171708 The review of this PR happened in the parent PR #21869. No conflicts arose while cherry-picking the commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] AraiYuno opened a new pull request, #528: [config] increase the default ReadinessTimeout to 5 minutes
AraiYuno opened a new pull request, #528: URL: https://github.com/apache/flink-kubernetes-operator/pull/528 ## What is the purpose of the change This pull request increases the default value of `kubernetes.operator.deployment.readiness.timeout` to 5 minutes from 1 minute. Discussed with @gyfora that the current default value of 1 minute might be too short in many prod deployment scenarios. ## Brief change log - simply updated the current `kubernetes.operator.deployment.readiness.timeout` to 5 minutes. ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. -> quick config default value update. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changes to the `CustomResourceDescriptors`: no - Core observer or reconciler logic that is regularly executed: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21890: [FLINK-30860]Add document for hybrid shuffle with adaptive batch scheduler
flinkbot commented on PR #21890: URL: https://github.com/apache/flink/pull/21890#issuecomment-1422164243 ## CI report: * 8aeaf0001bf4c156427acc1202a060507871532e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-30860) Add document for hybrid shuffle with adaptive batch scheduler
[ https://issues.apache.org/jira/browse/FLINK-30860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-30860: --- Description: Add document for hybrid shuffle with adaptive batch scheduler and index spilling. > Add document for hybrid shuffle with adaptive batch scheduler > - > > Key: FLINK-30860 > URL: https://issues.apache.org/jira/browse/FLINK-30860 > Project: Flink > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.17.0 >Reporter: Weijie Guo >Assignee: Weijie Guo >Priority: Major > Labels: pull-request-available > Fix For: 1.17.0 > > > Add document for hybrid shuffle with adaptive batch scheduler and index > spilling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30860) Add document for hybrid shuffle with adaptive batch scheduler
[ https://issues.apache.org/jira/browse/FLINK-30860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-30860: --- Component/s: Documentation > Add document for hybrid shuffle with adaptive batch scheduler > - > > Key: FLINK-30860 > URL: https://issues.apache.org/jira/browse/FLINK-30860 > Project: Flink > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.17.0 >Reporter: Weijie Guo >Assignee: Weijie Guo >Priority: Major > Labels: pull-request-available > Fix For: 1.17.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30860) Add document for hybrid shuffle with adaptive batch scheduler
[ https://issues.apache.org/jira/browse/FLINK-30860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-30860: --- Labels: pull-request-available (was: ) > Add document for hybrid shuffle with adaptive batch scheduler > - > > Key: FLINK-30860 > URL: https://issues.apache.org/jira/browse/FLINK-30860 > Project: Flink > Issue Type: Improvement >Affects Versions: 1.17.0 >Reporter: Weijie Guo >Assignee: Weijie Guo >Priority: Major > Labels: pull-request-available > Fix For: 1.17.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] reswqa opened a new pull request, #21890: [FLINK-30860]Add document for hybrid shuffle with adaptive batch scheduler
reswqa opened a new pull request, #21890: URL: https://github.com/apache/flink/pull/21890 ## What is the purpose of the change *Add document for hybrid shuffle with adaptive batch scheduler* ## Brief change log - *Add document for hybrid shuffle with adaptive batch scheduler* - *Add document for hybrid shuffle index spilling* ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] chucheng92 commented on pull request #21442: [FLINK-30274] Upgrade commons-collections 3.x to commons-collections4
chucheng92 commented on PR #21442: URL: https://github.com/apache/flink/pull/21442#issuecomment-1422143809 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21889: [FLINK-29801][runtime] FLIP-274: Introduce metric group for OperatorCoordinator
flinkbot commented on PR #21889: URL: https://github.com/apache/flink/pull/21889#issuecomment-1422142949 ## CI report: * 91081f980a18127638ac86466497c3987841c44a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-30959) UNIX_TIMESTAMP's return value does not meet expected
Yunfeng Zhou created FLINK-30959:
Summary: UNIX_TIMESTAMP's return value does not meet expected
Key: FLINK-30959
URL: https://issues.apache.org/jira/browse/FLINK-30959
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Affects Versions: 1.15.2
Reporter: Yunfeng Zhou
When running the following pyflink program
{code:python}
import pandas as pd
from pyflink.datastream import StreamExecutionEnvironment, HashMapStateBackend
from pyflink.table import StreamTableEnvironment
if __name__ == "__main__":
input_data = pd.DataFrame(
[
["Alex", 100.0, "2022-01-01 08:00:00.001 +0800"],
["Emma", 400.0, "2022-01-01 00:00:00.003 +"],
["Alex", 200.0, "2022-01-01 08:00:00.005 +0800"],
["Emma", 300.0, "2022-01-01 00:00:00.007 +"],
["Jack", 500.0, "2022-01-01 08:00:00.009 +0800"],
["Alex", 450.0, "2022-01-01 00:00:00.011 +"],
],
columns=["name", "avg_cost", "time"],
)
env = StreamExecutionEnvironment.get_execution_environment()
env.set_state_backend(HashMapStateBackend())
t_env = StreamTableEnvironment.create(env)
input_table = t_env.from_pandas(input_data)
t_env.create_temporary_view("input_table", input_table)
time_format = "-MM-dd HH:mm:ss.SSS X"
output_table = t_env.sql_query(
f"SELECT *, UNIX_TIMESTAMP(`time`, '{time_format}') AS unix_time FROM
input_table"
)
output_table.execute().print()
{code}
The actual output is
{code}
+++++--+
| op | name | avg_cost |
time |unix_time |
+++++--+
| +I | Alex | 100.0 |
2022-01-01 08:00:00.001 +0800 | 1640995200 |
| +I | Emma | 400.0 |
2022-01-01 00:00:00.003 + | 1640995200 |
| +I | Alex | 200.0 |
2022-01-01 08:00:00.005 +0800 | 1640995200 |
| +I | Emma | 300.0 |
2022-01-01 00:00:00.007 + | 1640995200 |
| +I | Jack | 500.0 |
2022-01-01 08:00:00.009 +0800 | 1640995200 |
| +I | Alex | 450.0 |
2022-01-01 00:00:00.011 + | 1640995200 |
+++++--+
{code}
While the expected result is
{code:java}
+++++--+
| op | name | avg_cost |
time |unix_time |
+++++--+
| +I | Alex | 100.0 |
2022-01-01 08:00:00.001 +0800 | 1640995200 |
| +I | Emma | 400.0 |
2022-01-01 00:00:00.003 + | 1640966400 |
| +I | Alex | 200.0 |
2022-01-01 08:00:00.005 +0800 | 1640995200 |
| +I | Emma | 300.0 |
2022-01-01 00:00:00.007 + | 1640966400 |
| +I | Jack | 500.0 |
2022-01-01 08:00:00.009 +0800 | 1640995200 |
| +I | Alex | 450.0 |
2022-01-01 00:00:00.011 + | 1640966400 |
+++++--+
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] ruanhang1993 opened a new pull request, #21889: [FLINK-29801][runtime] FLIP-274: Introduce metric group for OperatorCoordinator
ruanhang1993 opened a new pull request, #21889: URL: https://github.com/apache/flink/pull/21889 ## What is the purpose of the change This pull request provides the operator coordinator metric group. ## Brief change log Please see the changes in the [FLIP-274](https://cwiki.apache.org/confluence/display/FLINK/FLIP-274%3A+Introduce+metric+group+for+OperatorCoordinator). ## Verifying this change This change added unit tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? not documented -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-30274) Upgrade commons-collections 3.x to commons-collections4
[ https://issues.apache.org/jira/browse/FLINK-30274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685717#comment-17685717 ] Ran Tao commented on FLINK-30274: - [~martijnvisser] Hi, Martijn I have updated the pr. > Upgrade commons-collections 3.x to commons-collections4 > --- > > Key: FLINK-30274 > URL: https://issues.apache.org/jira/browse/FLINK-30274 > Project: Flink > Issue Type: Technical Debt > Components: Build System >Affects Versions: 1.16.0 >Reporter: Ran Tao >Assignee: Ran Tao >Priority: Major > Labels: pull-request-available > Attachments: image-2022-12-02-16-40-22-172.png > > > First, Apache commons-collections 3.x is a Java 1.3 compatible version, and > it does not use Java 5 generics. Apache commons-collections4 4.4 is an > upgraded version of commons-collections and it built by Java 8. > The Apache Spark has same issue: [https://github.com/apache/spark/pull/35257] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] chucheng92 commented on pull request #21442: [FLINK-30274] Upgrade commons-collections 3.x to commons-collections4
chucheng92 commented on PR #21442: URL: https://github.com/apache/flink/pull/21442#issuecomment-1422114983 @MartijnVisser PTAL, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] chucheng92 commented on pull request #21442: [FLINK-30274] Upgrade commons-collections 3.x to commons-collections4
chucheng92 commented on PR #21442: URL: https://github.com/apache/flink/pull/21442#issuecomment-1422113508 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] chucheng92 commented on pull request #21442: [FLINK-30274] Upgrade commons-collections 3.x to commons-collections4
chucheng92 commented on PR #21442: URL: https://github.com/apache/flink/pull/21442#issuecomment-1422103023 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21888: [FLINK-30958][rest][docs] Fix REST API doc generation failure caused by JobClientHeartbeatHeaders
flinkbot commented on PR #21888: URL: https://github.com/apache/flink/pull/21888#issuecomment-1422097051 ## CI report: * 589d2ba936bda80a06ff546ee586760e83f9ec94 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-30958) Rest API doc generation failure caused by JobClientHeartbeatHeaders
[ https://issues.apache.org/jira/browse/FLINK-30958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-30958: --- Labels: pull-request-available (was: ) > Rest API doc generation failure caused by JobClientHeartbeatHeaders > --- > > Key: FLINK-30958 > URL: https://issues.apache.org/jira/browse/FLINK-30958 > Project: Flink > Issue Type: Bug > Components: Runtime / REST >Affects Versions: 1.17.0 >Reporter: Jane Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.17.0 > > > `JobClientHeartbeatHeaders` should override `operationId` since > `getHttpMethod` returns `POST`. Otherwise `UnsupportedOperationException` is > thrown at `OpenApiSpecGenerator` when generating the REST API doc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] LadyForest opened a new pull request, #21888: [FLINK-30958][rest][docs] Fix REST API doc generation failure caused by JobClientHeartbeatHeaders
LadyForest opened a new pull request, #21888: URL: https://github.com/apache/flink/pull/21888 ## What is the purpose of the change This pull request fixes the REST API doc generation failure caused by `JobClientHeartbeatHeaders`. `JobClientHeartbeatHeaders` should override `operationId` since `getHttpMethod` returns `POST`. Otherwise `UnsupportedOperationException` is thrown at `OpenApiSpecGenerator` when generating the REST API doc. ## Brief change log - Override `operationId` for `JobClientHeartbeatHeaders`. - Regenerate the doc by `mvn package -Dgenerate-rest-docs -pl flink-docs -am -nsu -DskipTests` ## Verifying this change The issue can be reproduced by rolling back the changes made on `JobClientHeartbeatHeaders` and running `mvn package -Dgenerate-rest-docs -pl flink-docs -am -nsu -DskipTests`. The fix can be verified by applying the changes and re-running the build command. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-30508) CliClientITCase.testSqlStatements failed with output not matched with expected
[
https://issues.apache.org/jira/browse/FLINK-30508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685703#comment-17685703
]
luoyuxia edited comment on FLINK-30508 at 2/8/23 6:35 AM:
--
Hi, the reason maybe ctas statement write data to files and rename files to
commit {_}asynchronously{_}, then the following statemt try to read the files
written, but the ctas statement may rename these files at same time, so the
exception "can not access file" as them have been renamed.
[~lsy] Could you please help fix as it's introuced by your changes?
was (Author: luoyuxia):
Hi, the reason maybe ctas statement write data to files and rename files to
commit {_}asynchronously{_}, then the following statemt try to read the files
written, but the ctas statement may rename these files at same time, so the
exception "can not access file" as them have been removed.
[~lsy] Could you please help fix as it's introuced by your changes?
> CliClientITCase.testSqlStatements failed with output not matched with expected
> --
>
> Key: FLINK-30508
> URL: https://issues.apache.org/jira/browse/FLINK-30508
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Qingsheng Ren
>Assignee: Shengkai Fang
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=44246=logs=a9db68b9-a7e0-54b6-0f98-010e0aff39e2=cdd32e0b-6047-565b-c58f-14054472f1be=14992
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-30958) Rest API doc generation failure caused by JobClientHeartbeatHeaders
Jane Chan created FLINK-30958: - Summary: Rest API doc generation failure caused by JobClientHeartbeatHeaders Key: FLINK-30958 URL: https://issues.apache.org/jira/browse/FLINK-30958 Project: Flink Issue Type: Bug Components: Runtime / REST Affects Versions: 1.17.0 Reporter: Jane Chan Fix For: 1.17.0 `JobClientHeartbeatHeaders` should override `operationId` since `getHttpMethod` returns `POST`. Otherwise `UnsupportedOperationException` is thrown at `OpenApiSpecGenerator` when generating the REST API doc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-30508) CliClientITCase.testSqlStatements failed with output not matched with expected
[
https://issues.apache.org/jira/browse/FLINK-30508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685703#comment-17685703
]
luoyuxia commented on FLINK-30508:
--
Hi, the reason maybe ctas statement write data to files and rename files to
commit {_}asynchronously{_}, then the following statemt try to read the files
written, but the ctas statement may rename these files at same time, so the
exception "can not access file" as them have been removed.
[~lsy] Could you please help fix as it's introuced by your changes?
> CliClientITCase.testSqlStatements failed with output not matched with expected
> --
>
> Key: FLINK-30508
> URL: https://issues.apache.org/jira/browse/FLINK-30508
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Qingsheng Ren
>Assignee: Shengkai Fang
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=44246=logs=a9db68b9-a7e0-54b6-0f98-010e0aff39e2=cdd32e0b-6047-565b-c58f-14054472f1be=14992
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] saLeox commented on pull request #21149: [FLINK-29527][formats/parquet] Make unknownFieldsIndices work for single ParquetReader
saLeox commented on PR #21149: URL: https://github.com/apache/flink/pull/21149#issuecomment-1422086364 @luoyuxia Thanks for your comment, it sounds make sense to have similar option to control whether if merge schema, I will try to add them later, and notify here if make any progress. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-30508) CliClientITCase.testSqlStatements failed with output not matched with expected
[ https://issues.apache.org/jira/browse/FLINK-30508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685699#comment-17685699 ] Shengkai Fang commented on FLINK-30508: --- I read recently failed tests are related to the CTAS syntax. It seems the query doesn't wait for the last INSERT INTO statement to finish. Could you share some thoughts about Hive behavior [~lsy] [~luoyuxia] > CliClientITCase.testSqlStatements failed with output not matched with expected > -- > > Key: FLINK-30508 > URL: https://issues.apache.org/jira/browse/FLINK-30508 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client >Affects Versions: 1.16.0, 1.17.0 >Reporter: Qingsheng Ren >Assignee: Shengkai Fang >Priority: Critical > Labels: test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=44246=logs=a9db68b9-a7e0-54b6-0f98-010e0aff39e2=cdd32e0b-6047-565b-c58f-14054472f1be=14992 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] gaborgsomogyi commented on pull request #21870: [FLINK-30928][tests] Change to eclipse-temurin due to openjdk deprecation
gaborgsomogyi commented on PR #21870: URL: https://github.com/apache/flink/pull/21870#issuecomment-1422055349 cc @gyfora -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-28326) ResultPartitionTest.testIdleAndBackPressuredTime failed with AssertError
[
https://issues.apache.org/jira/browse/FLINK-28326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingjie Cao updated FLINK-28326:
Fix Version/s: 1.16.2
> ResultPartitionTest.testIdleAndBackPressuredTime failed with AssertError
>
>
> Key: FLINK-28326
> URL: https://issues.apache.org/jira/browse/FLINK-28326
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Huang Xingbo
>Assignee: Weijie Guo
>Priority: Major
> Labels: pull-request-available, stale-assigned, test-stability
> Fix For: 1.17.0, 1.16.2
>
>
> {code:java}
> 2022-06-30T09:23:24.0469768Z Jun 30 09:23:24 [INFO]
> 2022-06-30T09:23:24.0470382Z Jun 30 09:23:24 [ERROR] Failures:
> 2022-06-30T09:23:24.0471581Z Jun 30 09:23:24 [ERROR]
> ResultPartitionTest.testIdleAndBackPressuredTime:414
> 2022-06-30T09:23:24.0472898Z Jun 30 09:23:24 Expected: a value greater than
> <0L>
> 2022-06-30T09:23:24.0474090Z Jun 30 09:23:24 but: <0L> was equal to <0L>
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37406=logs=0da23115-68bb-5dcd-192c-bd4c8adebde1=24c3384f-1bcb-57b3-224f-51bf973bbee8
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (FLINK-28326) ResultPartitionTest.testIdleAndBackPressuredTime failed with AssertError
[
https://issues.apache.org/jira/browse/FLINK-28326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingjie Cao resolved FLINK-28326.
-
Resolution: Fixed
> ResultPartitionTest.testIdleAndBackPressuredTime failed with AssertError
>
>
> Key: FLINK-28326
> URL: https://issues.apache.org/jira/browse/FLINK-28326
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Huang Xingbo
>Assignee: Weijie Guo
>Priority: Major
> Labels: pull-request-available, stale-assigned, test-stability
> Fix For: 1.17.0, 1.16.2
>
>
> {code:java}
> 2022-06-30T09:23:24.0469768Z Jun 30 09:23:24 [INFO]
> 2022-06-30T09:23:24.0470382Z Jun 30 09:23:24 [ERROR] Failures:
> 2022-06-30T09:23:24.0471581Z Jun 30 09:23:24 [ERROR]
> ResultPartitionTest.testIdleAndBackPressuredTime:414
> 2022-06-30T09:23:24.0472898Z Jun 30 09:23:24 Expected: a value greater than
> <0L>
> 2022-06-30T09:23:24.0474090Z Jun 30 09:23:24 but: <0L> was equal to <0L>
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37406=logs=0da23115-68bb-5dcd-192c-bd4c8adebde1=24c3384f-1bcb-57b3-224f-51bf973bbee8
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-28326) ResultPartitionTest.testIdleAndBackPressuredTime failed with AssertError
[
https://issues.apache.org/jira/browse/FLINK-28326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685678#comment-17685678
]
Yingjie Cao commented on FLINK-28326:
-
1.16: 096c4a5e29cdffc8b4ed72f7bc6fc7b42dab2e9b
> ResultPartitionTest.testIdleAndBackPressuredTime failed with AssertError
>
>
> Key: FLINK-28326
> URL: https://issues.apache.org/jira/browse/FLINK-28326
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Huang Xingbo
>Assignee: Weijie Guo
>Priority: Major
> Labels: pull-request-available, stale-assigned, test-stability
> Fix For: 1.17.0
>
>
> {code:java}
> 2022-06-30T09:23:24.0469768Z Jun 30 09:23:24 [INFO]
> 2022-06-30T09:23:24.0470382Z Jun 30 09:23:24 [ERROR] Failures:
> 2022-06-30T09:23:24.0471581Z Jun 30 09:23:24 [ERROR]
> ResultPartitionTest.testIdleAndBackPressuredTime:414
> 2022-06-30T09:23:24.0472898Z Jun 30 09:23:24 Expected: a value greater than
> <0L>
> 2022-06-30T09:23:24.0474090Z Jun 30 09:23:24 but: <0L> was equal to <0L>
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37406=logs=0da23115-68bb-5dcd-192c-bd4c8adebde1=24c3384f-1bcb-57b3-224f-51bf973bbee8
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] wsry merged pull request #21844: [BP-1.16][FLINK-28326][test] fix unstable test testIdleAndBackPressuredTime.
wsry merged PR #21844: URL: https://github.com/apache/flink/pull/21844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-27716) Add Flink ML Python API docs
[ https://issues.apache.org/jira/browse/FLINK-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685673#comment-17685673 ] Dong Lin commented on FLINK-27716: -- This is merged to the apache/flink-ml mater branch at 76d5b864dd7df95adebebe6f7e47d52ae0f689c7 > Add Flink ML Python API docs > > > Key: FLINK-27716 > URL: https://issues.apache.org/jira/browse/FLINK-27716 > Project: Flink > Issue Type: Improvement > Components: API / Python, Documentation, Library / Machine Learning >Reporter: Huang Xingbo >Assignee: Jiang Xin >Priority: Major > Labels: pull-request-available > Fix For: ml-2.2.0 > > > We can use sphinx same as pyflink or other tools to generate Python API docs > of ML -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-27716) Add Flink ML Python API docs
[ https://issues.apache.org/jira/browse/FLINK-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin closed FLINK-27716. Resolution: Fixed > Add Flink ML Python API docs > > > Key: FLINK-27716 > URL: https://issues.apache.org/jira/browse/FLINK-27716 > Project: Flink > Issue Type: Improvement > Components: API / Python, Documentation, Library / Machine Learning >Reporter: Huang Xingbo >Assignee: Jiang Xin >Priority: Major > Labels: pull-request-available > Fix For: ml-2.2.0 > > > We can use sphinx same as pyflink or other tools to generate Python API docs > of ML -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-27716) Add Flink ML Python API docs
[ https://issues.apache.org/jira/browse/FLINK-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated FLINK-27716: - Summary: Add Flink ML Python API docs (was: Add Python API docs in ML) > Add Flink ML Python API docs > > > Key: FLINK-27716 > URL: https://issues.apache.org/jira/browse/FLINK-27716 > Project: Flink > Issue Type: Improvement > Components: API / Python, Documentation, Library / Machine Learning >Reporter: Huang Xingbo >Assignee: Jiang Xin >Priority: Major > Labels: pull-request-available > Fix For: ml-2.2.0 > > > We can use sphinx same as pyflink or other tools to generate Python API docs > of ML -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-ml] Fanoid closed pull request #207: [FLINK-30939][draft] Add public APIs of GBTClassifier.
Fanoid closed pull request #207: [FLINK-30939][draft] Add public APIs of GBTClassifier. URL: https://github.com/apache/flink-ml/pull/207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] lindong28 merged pull request #201: [FLINK-27716] Add Python API docs in ML
lindong28 merged PR #201: URL: https://github.com/apache/flink-ml/pull/201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on pull request #201: [FLINK-27716] Add Python API docs in ML
lindong28 commented on PR #201: URL: https://github.com/apache/flink-ml/pull/201#issuecomment-1421958739 Thanks for the update. LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21887: [FLINK-30944] ExecutionGraphPartitionReleaseTest leaks threads
flinkbot commented on PR #21887: URL: https://github.com/apache/flink/pull/21887#issuecomment-1421956269 ## CI report: * 803239fa8de807f5d5bf6235dc0687a78d874d32 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-30944) ExecutionGraphPartitionReleaseTest leaks threads
[
https://issues.apache.org/jira/browse/FLINK-30944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-30944:
---
Labels: pull-request-available test-stability (was: test-stability)
> ExecutionGraphPartitionReleaseTest leaks threads
>
>
> Key: FLINK-30944
> URL: https://issues.apache.org/jira/browse/FLINK-30944
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Tests
>Affects Versions: 1.17.0, 1.15.3, 1.16.1, 1.18.0
>Reporter: Matthias Pohl
>Assignee: Weijie Guo
>Priority: Major
> Labels: pull-request-available, test-stability
>
> {{ExecutionGraphPartitionReleaseTest}} leaks threads through
> {{ExecutionGraphPartitionReleaseTest.scheduledExecutorService}}. The
> {{ScheduledExecutorService}} is instantiated but never shut down.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] reswqa opened a new pull request, #21887: [FLINK-30944] ExecutionGraphPartitionReleaseTest leaks threads
reswqa opened a new pull request, #21887: URL: https://github.com/apache/flink/pull/21887 ## What is the purpose of the change *`ExecutionGraphPartitionReleaseTest` leaks threads through `ExecutionGraphPartitionReleaseTest.scheduledExecutorService`. The `ScheduledExecutorService` is instantiated but never shut down.* ## Brief change log - *Fix the problem of thread leaking for `ExecutionGraphPartitionReleaseTest`* ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-30944) ExecutionGraphPartitionReleaseTest leaks threads
[
https://issues.apache.org/jira/browse/FLINK-30944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weijie Guo reassigned FLINK-30944:
--
Assignee: Weijie Guo
> ExecutionGraphPartitionReleaseTest leaks threads
>
>
> Key: FLINK-30944
> URL: https://issues.apache.org/jira/browse/FLINK-30944
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Tests
>Affects Versions: 1.17.0, 1.15.3, 1.16.1, 1.18.0
>Reporter: Matthias Pohl
>Assignee: Weijie Guo
>Priority: Major
> Labels: test-stability
>
> {{ExecutionGraphPartitionReleaseTest}} leaks threads through
> {{ExecutionGraphPartitionReleaseTest.scheduledExecutorService}}. The
> {{ScheduledExecutorService}} is instantiated but never shut down.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (FLINK-30707) Let speculative execution take input data amount into account when detecting slow tasks
[ https://issues.apache.org/jira/browse/FLINK-30707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-30707. --- Release Note: The slow task detecting is improved for speculative execution. Previously, it only considers the execution time of tasks when deciding which tasks are slow. It now takes the input data volume of tasks into account. Tasks which has a longer execution time but consumes more data may not be considered as slow. This improvement helps to eliminate the negative impacts of data skew on slow task detecting. Resolution: Done > Let speculative execution take input data amount into account when detecting > slow tasks > --- > > Key: FLINK-30707 > URL: https://issues.apache.org/jira/browse/FLINK-30707 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.17.0 >Reporter: xingbe >Assignee: xingbe >Priority: Major > Labels: pull-request-available > Fix For: 1.17.0 > > > The default slowTaskDetector in speculative execution is based on execution > time now, if the data amount allocated to each subtask is uneven, time-based > slow task detection may not reflect the actual performance of the subtask. > Therefore, it is necessary to take the input information into account when > detecting slow tasks. > The major changes are: > * Need to know the amount of data processed by each execution vertex. > * ExecutionTimeBasedSlowTaskDetector needs to introduce the amount of input > data to weight it with execution time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-30944) ExecutionGraphPartitionReleaseTest leaks threads
[
https://issues.apache.org/jira/browse/FLINK-30944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685670#comment-17685670
]
Weijie Guo commented on FLINK-30944:
Thanks [~mapohl] for reporting this, I'd like to do this work.
> ExecutionGraphPartitionReleaseTest leaks threads
>
>
> Key: FLINK-30944
> URL: https://issues.apache.org/jira/browse/FLINK-30944
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Tests
>Affects Versions: 1.17.0, 1.15.3, 1.16.1, 1.18.0
>Reporter: Matthias Pohl
>Priority: Major
> Labels: test-stability
>
> {{ExecutionGraphPartitionReleaseTest}} leaks threads through
> {{ExecutionGraphPartitionReleaseTest.scheduledExecutorService}}. The
> {{ScheduledExecutorService}} is instantiated but never shut down.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (FLINK-30707) Let speculative execution take input data amount into account when detecting slow tasks
[ https://issues.apache.org/jira/browse/FLINK-30707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reopened FLINK-30707: - > Let speculative execution take input data amount into account when detecting > slow tasks > --- > > Key: FLINK-30707 > URL: https://issues.apache.org/jira/browse/FLINK-30707 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.17.0 >Reporter: xingbe >Assignee: xingbe >Priority: Major > Labels: pull-request-available > Fix For: 1.17.0 > > > The default slowTaskDetector in speculative execution is based on execution > time now, if the data amount allocated to each subtask is uneven, time-based > slow task detection may not reflect the actual performance of the subtask. > Therefore, it is necessary to take the input information into account when > detecting slow tasks. > The major changes are: > * Need to know the amount of data processed by each execution vertex. > * ExecutionTimeBasedSlowTaskDetector needs to introduce the amount of input > data to weight it with execution time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-27219) CliClientITCase.testSqlStatements failed on azure with jdk11
[
https://issues.apache.org/jira/browse/FLINK-27219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shengkai Fang closed FLINK-27219.
-
Resolution: Fixed
> CliClientITCase.testSqlStatements failed on azure with jdk11
>
>
> Key: FLINK-27219
> URL: https://issues.apache.org/jira/browse/FLINK-27219
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.15.0, 1.16.0
>Reporter: Yun Gao
>Assignee: Shengkai Fang
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.1
>
>
> {code:java}
> Apr 13 04:56:44 [ERROR] Could not execute SQL statement. Reason:
> Apr 13 04:56:44 java.lang.ClassCastException: class
> jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
> java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
> java.net.URLClassLoader are in module java.base of loader 'bootstrap')
> Apr 13 04:56:44 !error
> Apr 13 04:56:44
> Apr 13 04:56:44 # test "ctas" only supported in Hive Dialect
> Apr 13 04:56:44 CREATE TABLE foo as select 1;
> Apr 13 04:56:44 [ERROR] Could not execute SQL statement. Reason:
> Apr 13 04:56:44 java.lang.ClassCastException: class
> jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
> java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
> java.net.URLClassLoader are in module java.base of loader 'bootstrap')
> Apr 13 04:56:44 !error
> Apr 13 04:56:44
> Apr 13 04:56:44 # list the configured configuration
> Apr 13 04:56:44 set;
> Apr 13 04:56:44 [ERROR] Could not execute SQL statement. Reason:
> Apr 13 04:56:44 java.lang.ClassCastException: class
> jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
> java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
> java.net.URLClassLoader are in module java.base of loader 'bootstrap')
> Apr 13 04:56:44 !error
> Apr 13 04:56:44
> Apr 13 04:56:44 # reset the configuration
> Apr 13 04:56:44 reset;
> Apr 13 04:56:44 [ERROR] Could not execute SQL statement. Reason:
> Apr 13 04:56:44 java.lang.ClassCastException: class
> jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
> java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
> java.net.URLClassLoader are in module java.base of loader 'bootstrap')
> Apr 13 04:56:44 !error
> Apr 13 04:56:44
> Apr 13 04:56:44 set;
> Apr 13 04:56:44 [ERROR] Could not execute SQL statement. Reason:
> Apr 13 04:56:44 java.lang.ClassCastException: class
> jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
> java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
> java.net.URLClassLoader are in module java.base of loader 'bootstrap')
> Apr 13 04:56:44 !error
> ...
> Apr 13 04:56:44 [ERROR] Could not execute SQL statement. Reason:
> Apr 13 04:56:44 org.apache.flink.sql.parser.impl.ParseException: Encountered
> "STRING" at line 10, column 27.
> Apr 13 04:56:44 Was expecting one of:
> Apr 13 04:56:44 ")" ...
> Apr 13 04:56:44 "," ...
> Apr 13 04:56:44
> Apr 13 04:56:44 !error
> ...
> Apr 13 04:56:44 SHOW JARS;
> Apr 13 04:56:44 Empty set
> Apr 13 04:56:44 !ok
> Apr 13 04:56:44 "
> Apr 13 04:56:44 at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> Apr 13 04:56:44 at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> Apr 13 04:56:44 at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> Apr 13 04:56:44 at
> org.apache.flink.table.client.cli.CliClientITCase.testSqlStatements(CliClientITCase.java:139)
> Apr 13 04:56:44 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Apr 13 04:56:44 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Apr 13 04:56:44 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Apr 13 04:56:44 at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> Apr 13 04:56:44 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> Apr 13 04:56:44 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Apr 13 04:56:44 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> Apr 13 04:56:44 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Apr 13 04:56:44 at
>
[GitHub] [flink-ml] jiangxin369 commented on a diff in pull request #201: [FLINK-27716] Add Python API docs in ML
jiangxin369 commented on code in PR #201: URL: https://github.com/apache/flink-ml/pull/201#discussion_r1099611012 ## flink-ml-python/docs/reference/pyflink.ml.core/api.rst: ## @@ -0,0 +1,92 @@ +.. + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and +limitations under the License. + + + +Core API + + +Stage +- + +Base class for a node in a :class:`Pipeline` or :class:`Graph`. The interface is only a concept, +and does not have any actual functionality. Its subclasses could be :class:`Estimator`, :class:`Model`, +:class:`Transformer` or :class:`AlgoOperator`. No other classes should inherit this interface directly. + +Each stage is with parameters, and requires a public empty constructor for restoration. + +.. currentmodule:: pyflink.ml.core.api Review Comment: [FLINK-30865](https://issues.apache.org/jira/browse/FLINK-30865) is fixed and the PR is updated based on the latest module structure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] jiangxin369 commented on a diff in pull request #201: [FLINK-27716] Add Python API docs in ML
jiangxin369 commented on code in PR #201: URL: https://github.com/apache/flink-ml/pull/201#discussion_r1099611012 ## flink-ml-python/docs/reference/pyflink.ml.core/api.rst: ## @@ -0,0 +1,92 @@ +.. + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and +limitations under the License. + + + +Core API + + +Stage +- + +Base class for a node in a :class:`Pipeline` or :class:`Graph`. The interface is only a concept, +and does not have any actual functionality. Its subclasses could be :class:`Estimator`, :class:`Model`, +:class:`Transformer` or :class:`AlgoOperator`. No other classes should inherit this interface directly. + +Each stage is with parameters, and requires a public empty constructor for restoration. + +.. currentmodule:: pyflink.ml.core.api Review Comment: https://issues.apache.org/jira/browse/FLINK-30865 is fixed and the PR is updated based on the latest module structure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] Myracle commented on pull request #21661: [FLINK-30629][Client/Job Submission] Fix the unstable test ClientHeartbeatTest.testJobRunningIfClientReportHeartbeat
Myracle commented on PR #21661: URL: https://github.com/apache/flink/pull/21661#issuecomment-1421917948 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-benchmarks] fredia commented on pull request #63: [FLINK-27571][benchmark] Recognize 'less is better' benchmarks in regression detection script
fredia commented on PR #63: URL: https://github.com/apache/flink-benchmarks/pull/63#issuecomment-1421914703 Thanks for your review, I dropped the comment and squashed the commits. The current notifications are as follows: > [mapRebalanceMapSink.F27_UNBOUNDED](http://codespeed.dak8s.net:8000/timeline/#/?exe=1=mapRebalanceMapSink.F27_UNBOUNDED=on=on=off=2=200) median=17231.7398765 recent_median=16483.040695 [~~calculateRegionToRestart.BATCH~~](http://codespeed.dak8s.net:8000/timeline/#/?exe=8=calculateRegionToRestart.BATCH=on=on=off=2=200) median=12.827356 recent_median=11.9279615 [~~partitionRelease.BATCH~~](http://codespeed.dak8s.net:8000/timeline/#/?exe=8=partitionRelease.BATCH=on=on=off=2=200) median=23.112702 recent_median=21.378349 [checkpointMultiInput](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=checkpointMultiInput=on=on=off=2=200) median=2.5996435 recent_median=2.490419 [checkpointSingleInput.UNALIGNED](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=checkpointSingleInput.UNALIGNED=on=on=off=2=200) median=338.060806 recent_median=308.487419 [checkpointSingleInput.UNALIGNED_1](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=checkpointSingleInput.UNALIGNED_1=on=on=off=2=200) median=215.0116175 recent_median=192.939931 [fireProcessingTimers](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=fireProcessingTimers=on=on=off=2=200) median=50.6221895 recent_median=47.733433 [globalWindow](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=globalWindow=on=on=off=2=200) median=5459.689767 recent_median=4600.3655335 [stateBackends.FS](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=stateBackends.FS=on=on=off=2=200) median=3269.6932005 recent_median=3084.736179 [stateBackends.FS_ASYNC](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=stateBackends.FS_ASYNC=on=on=off=2=200) median=3261.0470545 recent_median=2894.1319305 [stateBackends.MEMORY](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=stateBackends.MEMORY=on=on=off=2=200) median=3220.0443795 recent_median=2836.4578395 [tupleKeyBy](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=tupleKeyBy=on=on=off=2=200) median=4173.447853 recent_median=3483.1905105 After this PR, `calculateRegionToRestart.BATCH` and `partitionRelease.BATCH` will disappear, `buildTopology.STREAMING`, `createScheduler.STREAMING` and `deployDownstreamTasks.BATCH` will be added: > [mapRebalanceMapSink.F27_UNBOUNDED](http://codespeed.dak8s.net:8000/timeline/#/?exe=1=mapRebalanceMapSink.F27_UNBOUNDED=on=on=off=2=200) median=17146.759315 recent_median=16434.359172 [checkpointSingleInput.UNALIGNED](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=checkpointSingleInput.UNALIGNED=on=on=off=2=200) median=337.6920935 recent_median=308.487419 [checkpointSingleInput.UNALIGNED_1](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=checkpointSingleInput.UNALIGNED_1=on=on=off=2=200) median=214.956786 recent_median=193.5968605 [fireProcessingTimers](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=fireProcessingTimers=on=on=off=2=200) median=50.5016795 recent_median=48.207456 [globalWindow](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=globalWindow=on=on=off=2=200) median=5470.4972005 recent_median=4600.3655335 [stateBackends.FS](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=stateBackends.FS=on=on=off=2=200) median=3276.4542755 recent_median=2886.6052725 [stateBackends.FS_ASYNC](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=stateBackends.FS_ASYNC=on=on=off=2=200) median=3261.0470545 recent_median=2906.11805 [stateBackends.MEMORY](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=stateBackends.MEMORY=on=on=off=2=200) median=3224.299765 recent_median=2836.4578395 [tupleKeyBy](http://codespeed.dak8s.net:8000/timeline/#/?exe=6=tupleKeyBy=on=on=off=2=200) median=4173.447853 recent_median=3483.1905105 [**buildTopology.STREAMING**](http://codespeed.dak8s.net:8000/timeline/#/?exe=8=buildTopology.STREAMING=on=on=off=2=200) **median=116.1662695 recent_median=121.501681** [**createScheduler.STREAMING**](http://codespeed.dak8s.net:8000/timeline/#/?exe=8=createScheduler.STREAMING=on=on=off=2=200) **median=283.1718575 recent_median=295.282613** [**deployDownstreamTasks.BATCH**](http://codespeed.dak8s.net:8000/timeline/#/?exe=8=deployDownstreamTasks.BATCH=on=on=off=2=200) **median=72.3862015 recent_median=90.633917** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] yyx-bigdata commented on pull request #21886: Update filesystem.md
yyx-bigdata commented on PR #21886: URL: https://github.com/apache/flink/pull/21886#issuecomment-1421911977 > run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tsreaper commented on a diff in pull request #21759: [FLINK-29237][table] Remove RexSimplify from Flink code, SearchOperator code generation for RexUnknown.nullAs
tsreaper commented on code in PR #21759:
URL: https://github.com/apache/flink/pull/21759#discussion_r1099591819
##
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/SearchOperatorGen.scala:
##
@@ -112,7 +114,7 @@ object SearchOperatorGen {
var rangeChecks: Seq[GeneratedExpression] =
sarg.rangeSet.asRanges.asScala.toSeq
.map(RangeSets.map(_, rangeToExpression))
- if (sarg.containsNull) {
+ if (sarg.nullAs == RexUnknownAs.TRUE) {
rangeChecks =
Seq(generateIsNull(target, new
BooleanType(target.resultType.isNullable))) ++ rangeChecks
}
Review Comment:
You only deal with `sarg.nullAs == RexUnknownAs.TRUE` here, what about
`sarg.nullAs == RexUnknownAs.FALSE` and `sarg.nullAs == RexUnknownAs.UNKNOWN`?
We're facing a similar problem with the above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] wangyang0918 commented on a diff in pull request #21742: [FLINK-30765][runtime] Aligns the LeaderElectionService.stop() contract
wangyang0918 commented on code in PR #21742:
URL: https://github.com/apache/flink/pull/21742#discussion_r1099582674
##
flink-runtime/src/main/java/org/apache/flink/runtime/leaderelection/DefaultLeaderElectionService.java:
##
@@ -100,6 +100,7 @@ public final void stop() throws Exception {
if (!running) {
return;
}
+leaderContender.revokeLeadership();
Review Comment:
Sorry for the late response.
I believe that you are right we have some redundant codes in
`close/closeAsync()` and `revokeLeadership()` of all `LeaderContender`
implementations. And I get your point why you want to call the
`revokeLeadership` in the shutdown process.
However, I strongly have a mind that the leader information stored in the
ZNode and ConfigMap needs to be cleaned up if a leader contender is revoked
leadership. Before this change, this is true since the `revokeLeadership`
happens along with `clearConfirmedLeaderInformation`.
Moreover, from the point of view of leader elector, I think it does not lose
leadership when stopping since the timeout has not yet expired.
So I lean to not explicitly call the `leaderContender.revokeLeadership()`
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-30957) Support other missing features (see description)
[ https://issues.apache.org/jira/browse/FLINK-30957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30957: - Summary: Support other missing features (see description) (was: Support other missing features) > Support other missing features (see description) > > > Key: FLINK-30957 > URL: https://issues.apache.org/jira/browse/FLINK-30957 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > > # Support weights. > # Support leaf ID. > # Support feature importance. > # Support 1-order gradient. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30957) Support other missing features
Fan Hong created FLINK-30957: Summary: Support other missing features Key: FLINK-30957 URL: https://issues.apache.org/jira/browse/FLINK-30957 Project: Flink Issue Type: Sub-task Components: Library / Machine Learning Reporter: Fan Hong # Support weights. # Support leaf ID. # Support feature importance. # Support 1-order gradient. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30956) Add Python implementation and documents of GBTClassifier and GBTRegressor.
[ https://issues.apache.org/jira/browse/FLINK-30956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30956: - Summary: Add Python implementation and documents of GBTClassifier and GBTRegressor. (was: Add Python implementation of GBTClassifer and GBTRegressor.) > Add Python implementation and documents of GBTClassifier and GBTRegressor. > -- > > Key: FLINK-30956 > URL: https://issues.apache.org/jira/browse/FLINK-30956 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-30925) Add docs for the SQL Client gateway mode
[ https://issues.apache.org/jira/browse/FLINK-30925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shengkai Fang closed FLINK-30925. - Fix Version/s: 1.18.0 Resolution: Implemented Merged into master: 76a6edc7ff2f416920b6ff67d2e2b69d21ffcbab bd33e02547ec1faef9195fd6968803f76917c55e Merged into release-1.17: 78e2c9a13043d926bb1c4efb62f45cf4ef38561f ace89597edb5074a7455d6d69b0d087bdfb8704e > Add docs for the SQL Client gateway mode > > > Key: FLINK-30925 > URL: https://issues.apache.org/jira/browse/FLINK-30925 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Table SQL / Client >Affects Versions: 1.17.0 >Reporter: Shengkai Fang >Assignee: Shengkai Fang >Priority: Major > Labels: pull-request-available > Fix For: 1.17.0, 1.18.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30956) Add Python implementation of GBTClassifer and GBTRegressor.
Fan Hong created FLINK-30956: Summary: Add Python implementation of GBTClassifer and GBTRegressor. Key: FLINK-30956 URL: https://issues.apache.org/jira/browse/FLINK-30956 Project: Flink Issue Type: Sub-task Components: Library / Machine Learning Reporter: Fan Hong -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-ml] zhipeng93 merged pull request #208: [FLINK-30730][followup] Fix test failure in StringIndexerTest
zhipeng93 merged PR #208: URL: https://github.com/apache/flink-ml/pull/208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] zhipeng93 commented on pull request #208: [FLINK-30730][followup] Fix test failure in StringIndexerTest
zhipeng93 commented on PR #208: URL: https://github.com/apache/flink-ml/pull/208#issuecomment-1421896367 Thanks for the fix. LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fsk119 closed pull request #21877: [FLINK-30925][docs] Add docs about SQL Client remote mode
fsk119 closed pull request #21877: [FLINK-30925][docs] Add docs about SQL Client remote mode URL: https://github.com/apache/flink/pull/21877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-30955) Support early stopping with validation set.
Fan Hong created FLINK-30955: Summary: Support early stopping with validation set. Key: FLINK-30955 URL: https://issues.apache.org/jira/browse/FLINK-30955 Project: Flink Issue Type: Sub-task Components: Library / Machine Learning Reporter: Fan Hong -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30952) Add main training and transforming part
[ https://issues.apache.org/jira/browse/FLINK-30952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30952: - Summary: Add main training and transforming part (was: Add main training and transforming part.) > Add main training and transforming part > --- > > Key: FLINK-30952 > URL: https://issues.apache.org/jira/browse/FLINK-30952 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > > Implement main training and transforming part. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30954) Add estimator and transformer for GBTRegressor
[ https://issues.apache.org/jira/browse/FLINK-30954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30954: - Summary: Add estimator and transformer for GBTRegressor (was: Add estimator and transformer for GBTRegressor.) > Add estimator and transformer for GBTRegressor > -- > > Key: FLINK-30954 > URL: https://issues.apache.org/jira/browse/FLINK-30954 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30953) Support checkpoint machanism and model save/load
[ https://issues.apache.org/jira/browse/FLINK-30953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30953: - Summary: Support checkpoint machanism and model save/load (was: Support intermediate state management and model save/load.) > Support checkpoint machanism and model save/load > > > Key: FLINK-30953 > URL: https://issues.apache.org/jira/browse/FLINK-30953 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30954) Add estimator and transformer for GBTRegressor.
[ https://issues.apache.org/jira/browse/FLINK-30954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30954: - Component/s: Library / Machine Learning > Add estimator and transformer for GBTRegressor. > --- > > Key: FLINK-30954 > URL: https://issues.apache.org/jira/browse/FLINK-30954 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30953) Support intermediate state management and model save/load.
[ https://issues.apache.org/jira/browse/FLINK-30953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30953: - Component/s: Library / Machine Learning > Support intermediate state management and model save/load. > -- > > Key: FLINK-30953 > URL: https://issues.apache.org/jira/browse/FLINK-30953 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30952) Add main training and transforming part.
[ https://issues.apache.org/jira/browse/FLINK-30952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30952: - Component/s: Library / Machine Learning > Add main training and transforming part. > > > Key: FLINK-30952 > URL: https://issues.apache.org/jira/browse/FLINK-30952 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > > Implement main training and transforming part. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-ml] Fanoid commented on pull request #208: [hotfix] Fix a failed test in StringIndexerTest
Fanoid commented on PR #208: URL: https://github.com/apache/flink-ml/pull/208#issuecomment-1421881372 @zhipeng93 @jiangxin369 Could you help review this hotfix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-30951) Release Testing: Verify FLINK-29635 Hive sink should support merge files in batch mode
[ https://issues.apache.org/jira/browse/FLINK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] luoyuxia updated FLINK-30951: - Description: The issue aims to verfiy FLINK-29635. Please verify in batch mode, the document is in [https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/hive/hive_read_write/#file-compaction]: 1: enable auto-compaction, write some data to a Hive table which results in the average size of files is less than compaction.small-files.avg-size(16MB by default), verfiy these files should be merged. 2: enable auto-compaction, set compaction.small-files.avg-size to a smaller values, then write some data to a Hive table which results in the average size of files is greater thant the compaction.small-files.avg-size, verfiy these files shouldn't be merged. 3. set sink.parallelism manually, check the parallelism of the compact operator is equal to sink.parallelism. 4. set compaction.parallelism manually, check the parallelism of the compact operator is equal to compaction.parallelism. 5. set compaction.file-size, check the size of the each target file merged is about the `compaction.file-size`. We shoud verify it with writing non-partitioned table, static partition table, dynamic partition table. We can find the example sql for how to create & write hive table in the codebase [HiveTableCompactSinkITCase]([https://github.com/apache/flink/search?q=HiveTableCompactSinkITCase]). > Release Testing: Verify FLINK-29635 Hive sink should support merge files in > batch mode > -- > > Key: FLINK-30951 > URL: https://issues.apache.org/jira/browse/FLINK-30951 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive >Reporter: luoyuxia >Priority: Blocker > Fix For: 1.17.0 > > > The issue aims to verfiy FLINK-29635. > Please verify in batch mode, the document is in > [https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/hive/hive_read_write/#file-compaction]: > > 1: enable auto-compaction, write some data to a Hive table which results in > the average size of files is less than compaction.small-files.avg-size(16MB > by default), verfiy these files should be merged. > 2: enable auto-compaction, set compaction.small-files.avg-size to a smaller > values, then write some data to a Hive table which results in the average > size of files is greater thant the compaction.small-files.avg-size, verfiy > these files shouldn't be merged. > 3. set sink.parallelism manually, check the parallelism of the compact > operator is equal to sink.parallelism. > 4. set compaction.parallelism manually, check the parallelism of the compact > operator is equal to compaction.parallelism. > 5. set compaction.file-size, check the size of the each target file merged is > about the `compaction.file-size`. > > We shoud verify it with writing non-partitioned table, static partition > table, dynamic partition table. > We can find the example sql for how to create & write hive table in the > codebase > [HiveTableCompactSinkITCase]([https://github.com/apache/flink/search?q=HiveTableCompactSinkITCase]). > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] yyx-bigdata closed pull request #21886: Update filesystem.md
yyx-bigdata closed pull request #21886: Update filesystem.md URL: https://github.com/apache/flink/pull/21886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21886: Update filesystem.md
flinkbot commented on PR #21886: URL: https://github.com/apache/flink/pull/21886#issuecomment-1421846449 ## CI report: * e221ec486d3e90f2fb65fe8c511cd6e192ff27ba UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-30954) Add estimator and transformer for GBTRegressor.
Fan Hong created FLINK-30954: Summary: Add estimator and transformer for GBTRegressor. Key: FLINK-30954 URL: https://issues.apache.org/jira/browse/FLINK-30954 Project: Flink Issue Type: Sub-task Reporter: Fan Hong -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30953) Support intermediate state management and model save/load.
Fan Hong created FLINK-30953: Summary: Support intermediate state management and model save/load. Key: FLINK-30953 URL: https://issues.apache.org/jira/browse/FLINK-30953 Project: Flink Issue Type: Sub-task Reporter: Fan Hong -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30952) Add main training and transforming part.
Fan Hong created FLINK-30952: Summary: Add main training and transforming part. Key: FLINK-30952 URL: https://issues.apache.org/jira/browse/FLINK-30952 Project: Flink Issue Type: Sub-task Reporter: Fan Hong Implement main training and transforming part. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30939) Add public APIs for GBTClassifer
[ https://issues.apache.org/jira/browse/FLINK-30939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Hong updated FLINK-30939: - Summary: Add public APIs for GBTClassifer (was: Add public APIs and topmost framework for GBTClassifer) > Add public APIs for GBTClassifer > > > Key: FLINK-30939 > URL: https://issues.apache.org/jira/browse/FLINK-30939 > Project: Flink > Issue Type: Sub-task > Components: Library / Machine Learning >Reporter: Fan Hong >Priority: Major > Labels: pull-request-available > > Add Estimator and Transformer for GBTClassifer with no actual training and > transforming code being provided. > The public parameters are expected to cover all reasonable ones from > SparkML's org.apache.spark.ml.classification.GBTClassifier. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] yyx-bigdata opened a new pull request, #21886: Update filesystem.md
yyx-bigdata opened a new pull request, #21886: URL: https://github.com/apache/flink/pull/21886 修改文档代码MemorySize.ofMebiBytes(1024)，否则策略说明中文件大小应为1MB ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change Please make sure both new and modified tests in this PR follows the conventions defined in our code quality guide: https://flink.apache.org/contributing/code-style-and-quality-common.html#testing *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluster with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-30951) Release Testing: Verify FLINK-29635 Hive sink should support merge files in batch mode
luoyuxia created FLINK-30951: Summary: Release Testing: Verify FLINK-29635 Hive sink should support merge files in batch mode Key: FLINK-30951 URL: https://issues.apache.org/jira/browse/FLINK-30951 Project: Flink Issue Type: Sub-task Components: Connectors / Hive Reporter: luoyuxia Fix For: 1.17.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-29825) Improve benchmark stability
[ https://issues.apache.org/jira/browse/FLINK-29825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685641#comment-17685641 ] Dong Lin commented on FLINK-29825: -- Thanks [~Yanfei Lei] for implementing and evaluating the algorithm! [~pnowojski] Cool, I think we have agreed to make incremental improvements and used the algorithm proposed in the above doc to detect regression for Flink benchmarks. We probably still have different understandings regarding the pros/cons of these alternative choices. It will be great if you or someone else can help implement an alternative choice and show that it can do better than the one we are going to use. I probably won't have time to try the Hunter algorithm myself in the near future. > Improve benchmark stability > --- > > Key: FLINK-29825 > URL: https://issues.apache.org/jira/browse/FLINK-29825 > Project: Flink > Issue Type: Improvement > Components: Benchmarks >Affects Versions: 1.17.0 >Reporter: Yanfei Lei >Assignee: Yanfei Lei >Priority: Minor > > Currently, regressions are detected by a simple script which may have false > positives and false negatives, especially for benchmarks with small absolute > values, small value changes would cause large percentage changes. see > [here|https://github.com/apache/flink-benchmarks/blob/master/regression_report.py#L132-L136] > for details. > And all benchmarks are executed on one physical machine, it might happen that > hardware issues affect performance, like "[FLINK-18614] Performance > regression 2020.07.13". > > This ticket aims to improve the precision and recall of the regression-check > script. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-ml] Fanoid opened a new pull request, #208: [hotfix] Fix a failed test in StringIndexerTest
Fanoid opened a new pull request, #208: URL: https://github.com/apache/flink-ml/pull/208 ## What is the purpose of the change Fix a failed test in StringIndexerTest, which is encountered with certain probability. ## Brief change log - Fixes the assertion in the failed test of StringIndexerTest. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] snuyanzin commented on a diff in pull request #21759: [FLINK-29237][table] Remove RexSimplify from Flink code, SearchOperator code generation for RexUnknown.nullAs
snuyanzin commented on code in PR #21759:
URL: https://github.com/apache/flink/pull/21759#discussion_r1099444836
##
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/SearchOperatorGen.scala:
##
@@ -112,7 +114,7 @@ object SearchOperatorGen {
var rangeChecks: Seq[GeneratedExpression] =
sarg.rangeSet.asRanges.asScala.toSeq
.map(RangeSets.map(_, rangeToExpression))
- if (sarg.containsNull) {
+ if (sarg.nullAs == RexUnknownAs.TRUE) {
rangeChecks =
Seq(generateIsNull(target, new
BooleanType(target.resultType.isNullable))) ++ rangeChecks
}
Review Comment:
This i didn't get...
The question is why do we need such check here if this check implicitly
happens in
`org.apache.flink.table.planner.codegen.calls.ScalarOperatorGens#generateOr` ...
Or did I miss something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] snuyanzin commented on a diff in pull request #21759: [FLINK-29237][table] Remove RexSimplify from Flink code, SearchOperator code generation for RexUnknown.nullAs
snuyanzin commented on code in PR #21759:
URL: https://github.com/apache/flink/pull/21759#discussion_r1099441783
##
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/SearchOperatorGen.scala:
##
@@ -96,7 +96,9 @@ object SearchOperatorGen {
|boolean $nullTerm = true;
|if (!${needle.nullTerm}) {
| $resultTerm = $negation$setTerm.contains(${needle.resultTerm});
- | $nullTerm = !$resultTerm && $setTerm.containsNull();
+ | $nullTerm = false;
+ |} else {
+ | $resultTerm = $setTerm.containsNull();
Review Comment:
thanks for bearing with me
I updated the code in a way similar to what you've mentioned above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (FLINK-30950) Remove flink-connector-aws-base from Flink main repo
Hong Liang Teoh created FLINK-30950: --- Summary: Remove flink-connector-aws-base from Flink main repo Key: FLINK-30950 URL: https://issues.apache.org/jira/browse/FLINK-30950 Project: Flink Issue Type: Sub-task Reporter: Hong Liang Teoh Remove flink-connector-aws-base from Flink main repo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30950) Remove flink-connector-aws-base from Flink main repo
[ https://issues.apache.org/jira/browse/FLINK-30950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh updated FLINK-30950: Component/s: Connectors / Common > Remove flink-connector-aws-base from Flink main repo > > > Key: FLINK-30950 > URL: https://issues.apache.org/jira/browse/FLINK-30950 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Common >Reporter: Hong Liang Teoh >Priority: Major > > Remove flink-connector-aws-base from Flink main repo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30949) Sync flink-connector-aws-base from Flink main to flink-connector-aws
Hong Liang Teoh created FLINK-30949: --- Summary: Sync flink-connector-aws-base from Flink main to flink-connector-aws Key: FLINK-30949 URL: https://issues.apache.org/jira/browse/FLINK-30949 Project: Flink Issue Type: Sub-task Components: Connectors / Common Reporter: Hong Liang Teoh Sync flink-connector-aws-base from Flink main to flink-connector-aws -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30948) Remove flink-avro-glue-schema-registry and flink-json-glue-schema-registry from Flink main repo
[ https://issues.apache.org/jira/browse/FLINK-30948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh updated FLINK-30948: Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile) > Remove flink-avro-glue-schema-registry and flink-json-glue-schema-registry > from Flink main repo > --- > > Key: FLINK-30948 > URL: https://issues.apache.org/jira/browse/FLINK-30948 > Project: Flink > Issue Type: Sub-task > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) > Environment: Remove flink-avro-glue-schema-registry and > flink-json-glue-schema-registry from Flink main repo, along with associated > end-to-end tests >Reporter: Hong Liang Teoh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30948) Remove flink-avro-glue-schema-registry and flink-json-glue-schema-registry from Flink main repo
Hong Liang Teoh created FLINK-30948: --- Summary: Remove flink-avro-glue-schema-registry and flink-json-glue-schema-registry from Flink main repo Key: FLINK-30948 URL: https://issues.apache.org/jira/browse/FLINK-30948 Project: Flink Issue Type: Sub-task Environment: Remove flink-avro-glue-schema-registry and flink-json-glue-schema-registry from Flink main repo, along with associated end-to-end tests Reporter: Hong Liang Teoh -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30947) Sync flink-avro-glue-schema-registry and flink-json-glue-schema-registry to flink-connector-aws
[ https://issues.apache.org/jira/browse/FLINK-30947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh updated FLINK-30947: Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile) > Sync flink-avro-glue-schema-registry and flink-json-glue-schema-registry to > flink-connector-aws > --- > > Key: FLINK-30947 > URL: https://issues.apache.org/jira/browse/FLINK-30947 > Project: Flink > Issue Type: Sub-task > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) >Reporter: Hong Liang Teoh >Priority: Major > > Copy flink-avro-glue-schema-registry and flink-json-glue-schema-registry, > along with associated end-to-end tests to flink-connector-aws -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30947) Sync flink-avro-glue-schema-registry and flink-json-glue-schema-registry to flink-connector-aws
Hong Liang Teoh created FLINK-30947: --- Summary: Sync flink-avro-glue-schema-registry and flink-json-glue-schema-registry to flink-connector-aws Key: FLINK-30947 URL: https://issues.apache.org/jira/browse/FLINK-30947 Project: Flink Issue Type: Sub-task Reporter: Hong Liang Teoh Copy flink-avro-glue-schema-registry and flink-json-glue-schema-registry, along with associated end-to-end tests to flink-connector-aws -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30946) Migrate AWS connector formats to flink-connector-aws
[ https://issues.apache.org/jira/browse/FLINK-30946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh updated FLINK-30946: Description: Migrate flink-avro-glue-schema-registry and flink-json-glue-schema-registry to flink-connector-aws. h4. was: Update flink-connector-aws to take the flink-avro-glue-schema-registry from the Flink codebase. h4. > Migrate AWS connector formats to flink-connector-aws > > > Key: FLINK-30946 > URL: https://issues.apache.org/jira/browse/FLINK-30946 > Project: Flink > Issue Type: Technical Debt > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) >Reporter: Hong Liang Teoh >Priority: Major > > Migrate flink-avro-glue-schema-registry and flink-json-glue-schema-registry > to flink-connector-aws. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30946) Migrate AWS connector formats to flink-connector-aws
[ https://issues.apache.org/jira/browse/FLINK-30946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh updated FLINK-30946: Summary: Migrate AWS connector formats to flink-connector-aws (was: Sync master Avro GlueSchemaRegistry formats flink-connector-aws) > Migrate AWS connector formats to flink-connector-aws > > > Key: FLINK-30946 > URL: https://issues.apache.org/jira/browse/FLINK-30946 > Project: Flink > Issue Type: Technical Debt > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) >Reporter: Hong Liang Teoh >Priority: Major > > Update flink-connector-aws to take the flink-avro-glue-schema-registry from > the Flink codebase. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30946) Sync master Avro GlueSchemaRegistry formats flink-connector-aws
Hong Liang Teoh created FLINK-30946: --- Summary: Sync master Avro GlueSchemaRegistry formats flink-connector-aws Key: FLINK-30946 URL: https://issues.apache.org/jira/browse/FLINK-30946 Project: Flink Issue Type: Technical Debt Reporter: Hong Liang Teoh Update flink-connector-aws to take the flink-avro-glue-schema-registry from the Flink codebase. h4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-30946) Sync master Avro GlueSchemaRegistry formats flink-connector-aws
[ https://issues.apache.org/jira/browse/FLINK-30946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh updated FLINK-30946: Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile) > Sync master Avro GlueSchemaRegistry formats flink-connector-aws > --- > > Key: FLINK-30946 > URL: https://issues.apache.org/jira/browse/FLINK-30946 > Project: Flink > Issue Type: Technical Debt > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) >Reporter: Hong Liang Teoh >Priority: Major > > Update flink-connector-aws to take the flink-avro-glue-schema-registry from > the Flink codebase. > h4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] hlteoh37 commented on a diff in pull request #21798: [FLINK-30837] Remove use of MutableByteArrayInputStream
hlteoh37 commented on code in PR #21798: URL: https://github.com/apache/flink/pull/21798#discussion_r1099391455 ## flink-formats/flink-avro-glue-schema-registry/src/main/java/org/apache/flink/formats/avro/glue/schema/registry/GlueSchemaRegistryInputStreamDeserializer.java: ## @@ -73,12 +72,10 @@ public Schema getSchemaAndDeserializedStream(InputStream in) throws IOException in.read(inputBytes); in.reset(); -MutableByteArrayInputStream mutableByteArrayInputStream = (MutableByteArrayInputStream) in; String schemaDefinition = glueSchemaRegistryDeserializationFacade.getSchemaDefinition(inputBytes); byte[] deserializedBytes = glueSchemaRegistryDeserializationFacade.getActualData(inputBytes); -mutableByteArrayInputStream.setBuffer(deserializedBytes); Review Comment: Yes, as far as I can tell as well... it is not doing anything, since the mutableByteArrayInputStream is not read further down the callchain... Compared with the implementations of `SchemaCoder.readSchema()` in `ConfluentSchemaRegistryCoder` as well. The buffer doesn't seem to be reset there, nor is it used after. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
