[jira] [Created] (FLINK-32941) Table API Bridge `toDataStream(targetDataType)` function not working correctly for Java List

2023-08-22 Thread Tan Kim (Jira)
Tan Kim created FLINK-32941: --- Summary: Table API Bridge `toDataStream(targetDataType)` function not working correctly for Java List Key: FLINK-32941 URL: https://issues.apache.org/jira/browse/FLINK-32941

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-08-22 Thread Becket Qin
Hi Weihua, Thanks for the explanation. From the doc, it looks like the current behaviors of "execution.attached=true" between Yarn and K8S session cluster are exactly the opposite. For YARN it basically means the cluster will shutdown if the client disconnects. For K8S, it means the cluster will

[jira] [Created] (FLINK-32940) Support projection pushdown to table source for column projections through UDTF

2023-08-22 Thread Venkata krishnan Sowrirajan (Jira)
Venkata krishnan Sowrirajan created FLINK-32940: --- Summary: Support projection pushdown to table source for column projections through UDTF Key: FLINK-32940 URL:

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-22 Thread Becket Qin
Hi Jark, Regarding the migration path, it would be useful to scrutinize the use case of FiledReferenceExpression and ResolvedExpressions. There are two kinds of use cases: 1. A ResolvedExpression is constructed by the user or connector / plugin developers. 2. A ResolvedExpression is constructed

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-22 Thread Jing Ge
Hi Timo, Your last suggestion sounds good. Best regards, Jing On Mon, Aug 21, 2023 at 4:21 AM Benchao Li wrote: > It sounds good to me too, that we avoid introducing the concept of "system > columns" for now. > > Timo Walther 于2023年8月18日周五 22:38写道: > > > Great, I also like my last suggestion

Re: [Discuss] FLIP-355: Add parent dir of files to classpath using yarn.provided.lib.dirs

2023-08-22 Thread Venkatakrishnan Sowrirajan
Thanks for the FLIP, Archit. This is definitely quite a useful addition to the *yarn.provided.lib.dirs* . +1. IMO, except for the fact that *yarn.provided.lib.dirs* (platform specific jars can be cached) takes only directories whereas *yarn.ship-files* (user files) takes both files and dirs, the

[jira] [Created] (FLINK-32939) pyflink 1.17.0 has missing transitive dependency for pyopenssl

2023-08-22 Thread Nathanael England (Jira)
Nathanael England created FLINK-32939: - Summary: pyflink 1.17.0 has missing transitive dependency for pyopenssl Key: FLINK-32939 URL: https://issues.apache.org/jira/browse/FLINK-32939 Project:

[Discuss] FLIP-355: Add parent dir of files to classpath using yarn.provided.lib.dirs

2023-08-22 Thread Archit Goyal
Hi all, Gentle ping if I can get a review on the FLIP. Thanks, Archit Goyal From: Archit Goyal Date: Thursday, August 17, 2023 at 5:51 PM To: dev@flink.apache.org Subject: [Discuss] FLIP-355: Add parent dir of files to classpath using yarn.provided.lib.dirs Hi All, I am opening this thread

[jira] [Created] (FLINK-32938) flink-connector-pulsar should remove all `PulsarAdmin` calls

2023-08-22 Thread Neng Lu (Jira)
Neng Lu created FLINK-32938: --- Summary: flink-connector-pulsar should remove all `PulsarAdmin` calls Key: FLINK-32938 URL: https://issues.apache.org/jira/browse/FLINK-32938 Project: Flink Issue

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-22 Thread Jark Wu
Hi Becket, I totally agree we should try to have a consistent API for a final state. The only concern I have mentioned is the "smooth" migration path. The FiledReferenceExpression is widely used in many public APIs, not only in the SupportsFilterPushDown. Yes, we can change every methods in

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-08-22 Thread Weihua Hu
Hi, Jiangjie 'execution.attached' can be used to attach an existing cluster and stop it [1][2], which is not related to job submission. So does YARN session mode[3]. IMO, this behavior should not be controlled by the new option 'client.attached.after.submission'. [1]

[jira] [Created] (FLINK-32937) Create a new version in JIRA

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32937: --- Summary: Create a new version in JIRA Key: FLINK-32937 URL: https://issues.apache.org/jira/browse/FLINK-32937 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32935) CLONE - Propose a pull request for website updates

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32935: --- Summary: CLONE - Propose a pull request for website updates Key: FLINK-32935 URL: https://issues.apache.org/jira/browse/FLINK-32935 Project: Flink

[jira] [Created] (FLINK-32933) CLONE - Build and stage Java and Python artifacts

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32933: --- Summary: CLONE - Build and stage Java and Python artifacts Key: FLINK-32933 URL: https://issues.apache.org/jira/browse/FLINK-32933 Project: Flink

[jira] [Created] (FLINK-32934) CLONE - Stage source and binary releases on dist.apache.org

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32934: --- Summary: CLONE - Stage source and binary releases on dist.apache.org Key: FLINK-32934 URL: https://issues.apache.org/jira/browse/FLINK-32934 Project: Flink

[jira] [Created] (FLINK-32936) CLONE - Vote on the release candidate

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32936: --- Summary: CLONE - Vote on the release candidate Key: FLINK-32936 URL: https://issues.apache.org/jira/browse/FLINK-32936 Project: Flink Issue Type:

[jira] [Created] (FLINK-32932) Build Release Candidate: 1.18.0-rc1

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32932: --- Summary: Build Release Candidate: 1.18.0-rc1 Key: FLINK-32932 URL: https://issues.apache.org/jira/browse/FLINK-32932 Project: Flink Issue Type: New

[jira] [Created] (FLINK-32931) Publish the Dockerfiles for the new release

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32931: --- Summary: Publish the Dockerfiles for the new release Key: FLINK-32931 URL: https://issues.apache.org/jira/browse/FLINK-32931 Project: Flink Issue

[jira] [Created] (FLINK-32929) Deploy artifacts to Maven Central Repository

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32929: --- Summary: Deploy artifacts to Maven Central Repository Key: FLINK-32929 URL: https://issues.apache.org/jira/browse/FLINK-32929 Project: Flink Issue

[jira] [Created] (FLINK-32930) Create Git tag and mark version as released in Jira

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32930: --- Summary: Create Git tag and mark version as released in Jira Key: FLINK-32930 URL: https://issues.apache.org/jira/browse/FLINK-32930 Project: Flink

[jira] [Created] (FLINK-32928) Deploy Python artifacts to PyPI

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32928: --- Summary: Deploy Python artifacts to PyPI Key: FLINK-32928 URL: https://issues.apache.org/jira/browse/FLINK-32928 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32927) Verify that no exclusions were erroneously added to the japicmp plugin

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32927: --- Summary: Verify that no exclusions were erroneously added to the japicmp plugin Key: FLINK-32927 URL: https://issues.apache.org/jira/browse/FLINK-32927

[jira] [Created] (FLINK-32926) Create a release branch

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32926: --- Summary: Create a release branch Key: FLINK-32926 URL: https://issues.apache.org/jira/browse/FLINK-32926 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32925) Select executing Release Manager

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32925: --- Summary: Select executing Release Manager Key: FLINK-32925 URL: https://issues.apache.org/jira/browse/FLINK-32925 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32924) Review Release Notes in JIRA

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32924: --- Summary: Review Release Notes in JIRA Key: FLINK-32924 URL: https://issues.apache.org/jira/browse/FLINK-32924 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32923) Triage release-blocking issues in JIRA

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32923: --- Summary: Triage release-blocking issues in JIRA Key: FLINK-32923 URL: https://issues.apache.org/jira/browse/FLINK-32923 Project: Flink Issue Type:

[jira] [Created] (FLINK-32922) Review and update documentation

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32922: --- Summary: Review and update documentation Key: FLINK-32922 URL: https://issues.apache.org/jira/browse/FLINK-32922 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32921) Prepare Flink 1.18 Release

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32921: --- Summary: Prepare Flink 1.18 Release Key: FLINK-32921 URL: https://issues.apache.org/jira/browse/FLINK-32921 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-32920) Finalize release 1.18.0

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32920: --- Summary: Finalize release 1.18.0 Key: FLINK-32920 URL: https://issues.apache.org/jira/browse/FLINK-32920 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-32918) CLONE - Update reference data for Migration Tests

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32918: --- Summary: CLONE - Update reference data for Migration Tests Key: FLINK-32918 URL: https://issues.apache.org/jira/browse/FLINK-32918 Project: Flink

[jira] [Created] (FLINK-32919) CLONE - Updates the docs stable version

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32919: --- Summary: CLONE - Updates the docs stable version Key: FLINK-32919 URL: https://issues.apache.org/jira/browse/FLINK-32919 Project: Flink Issue Type:

[jira] [Created] (FLINK-32916) CLONE - Apache mailing lists announcements

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32916: --- Summary: CLONE - Apache mailing lists announcements Key: FLINK-32916 URL: https://issues.apache.org/jira/browse/FLINK-32916 Project: Flink Issue Type:

[jira] [Created] (FLINK-32917) CLONE - Other announcements

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32917: --- Summary: CLONE - Other announcements Key: FLINK-32917 URL: https://issues.apache.org/jira/browse/FLINK-32917 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32913) CLONE - Update japicmp configuration

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32913: --- Summary: CLONE - Update japicmp configuration Key: FLINK-32913 URL: https://issues.apache.org/jira/browse/FLINK-32913 Project: Flink Issue Type:

[jira] [Created] (FLINK-32915) CLONE - Remove outdated versions

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32915: --- Summary: CLONE - Remove outdated versions Key: FLINK-32915 URL: https://issues.apache.org/jira/browse/FLINK-32915 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32914) CLONE - Merge website pull request

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32914: --- Summary: CLONE - Merge website pull request Key: FLINK-32914 URL: https://issues.apache.org/jira/browse/FLINK-32914 Project: Flink Issue Type:

[jira] [Created] (FLINK-32912) Promote release 1.18

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32912: --- Summary: Promote release 1.18 Key: FLINK-32912 URL: https://issues.apache.org/jira/browse/FLINK-32912 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-32911) Release Testing: Verify FLINK-28386: Trigger an immediate checkpoint after all sinks finished

2023-08-22 Thread Jiang Xin (Jira)
Jiang Xin created FLINK-32911: - Summary: Release Testing: Verify FLINK-28386: Trigger an immediate checkpoint after all sinks finished Key: FLINK-32911 URL: https://issues.apache.org/jira/browse/FLINK-32911

[jira] [Created] (FLINK-32910) Silence curls in test code

2023-08-22 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32910: - Summary: Silence curls in test code Key: FLINK-32910 URL: https://issues.apache.org/jira/browse/FLINK-32910 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-32909) The jobmanager.sh pass arguments failed

2023-08-22 Thread Alex Wu (Jira)
Alex Wu created FLINK-32909: --- Summary: The jobmanager.sh pass arguments failed Key: FLINK-32909 URL: https://issues.apache.org/jira/browse/FLINK-32909 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-08-22 Thread Becket Qin
Hi Weihua, Just want to clarify a little bit, what is the impact of `execution.attached` on a cluster startup before a client submits a job to that cluster? Does this config only become effective after a job submission? Currently, the cluster behavior has an independent config of

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-22 Thread Becket Qin
Thanks for the reply, Jark. I think it will be helpful to understand the final state we want to eventually achieve first, then we can discuss the steps towards that final state. It looks like there are two proposed end states now: 1. Have a separate NestedFieldReferenceExpression class; keep

[jira] [Created] (FLINK-32908) Fix wrong materialization id setting when switching from non-log-based checkpoint to log-based checkpoint

2023-08-22 Thread Feifan Wang (Jira)
Feifan Wang created FLINK-32908: --- Summary: Fix wrong materialization id setting when switching from non-log-based checkpoint to log-based checkpoint Key: FLINK-32908 URL:

[jira] [Created] (FLINK-32907) CheckpointAfterAllTasksFinishedITCase hangs on AZP

2023-08-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32907: --- Summary: CheckpointAfterAllTasksFinishedITCase hangs on AZP Key: FLINK-32907 URL: https://issues.apache.org/jira/browse/FLINK-32907 Project: Flink

[RELEASE][TEST] Flink 1.18 Release Tests need more contributors

2023-08-22 Thread Jing Ge
Hi devs, The community is focusing on the 1.18 release. There is an umbrella ticket[1] which contains all testing tasks that should be done before we can create new release candidates. Currently, there are still some open tasks. We would like to ask the community for more volunteers to join us

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-22 Thread Jark Wu
Hi Venkat, Thanks for the proposal. I have some minor comments about the FLIP. 1. I think we don't need to add SupportsFilterPushDown#supportsNestedFilters() method, because connectors can skip nested filters by putting them in Result#remainingFilters(). And this is backward-compatible because

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-08-22 Thread Weihua Hu
Hi Jiangjie Sorry for the late reply, I fully agree with the three user sensible behaviors you described. I would like to bring up a point. Currently, 'execution.attached' is not only used for submitting jobs, But also for starting a new cluster (YARN and Kubernetes). If it's true, the starting