Re: Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread godfrey he
+1, thanks for driving this Best, Godfrey Mang Zhang 于2022年7月5日周二 11:56写道: > > Hi everyone, > I'm sorry to bother you all, but since FLIP-218[1] has been updated, I'm > going to relaunch VOTE. > The main contents of the modification are: > 1. remove rtas from option name > 2. no longer

Re: Python Job Support for the Kubernetes Operator

2022-07-06 Thread Yang Wang
I think maybe we make the SQL or Python jobs submission more convenient in the future by introducing the *"**pyScriptURL"* or *"SQLScriptURI"* similar to *"jarURI"*. Best, Yang Thomas Weise 于2022年7月7日周四 01:06写道: > Since SQL or Python are essentially just examples of how to use the > operator

Re: [DISCUSS] FLIP-250: Support Customized Kubernetes Schedulers Proposal

2022-07-06 Thread Yang Wang
Thanks zhaobo for starting the discussion and preparing the FLIP. The customized Kubernetes Schedulers support will be very helpful for the users who still hesitates to migrate the Flink workloads from YARN to Kubernetes. Now leveraging the ability of customized K8s scheduler, many advanced

[jira] [Created] (FLINK-28434) Extend "NestedProjectionUtil" to unify support for projection pushdown for physical fields or metadata only

2022-07-06 Thread lincoln lee (Jira)
lincoln lee created FLINK-28434: --- Summary: Extend "NestedProjectionUtil" to unify support for projection pushdown for physical fields or metadata only Key: FLINK-28434 URL:

[jira] [Created] (FLINK-28433) Allow connection to mysql through mariadb driver

2022-07-06 Thread PengLei (Jira)
PengLei created FLINK-28433: --- Summary: Allow connection to mysql through mariadb driver Key: FLINK-28433 URL: https://issues.apache.org/jira/browse/FLINK-28433 Project: Flink Issue Type:

[DISCUSS] FLIP-250: Support Customized Kubernetes Schedulers Proposal

2022-07-06 Thread bo zhaobo
Hi, all. I would like to raise a discussion in Flink dev ML about Support Customized Kubernetes Schedulers. Currentlly, Kubernetes becomes more and more polular for Flink Cluster deployment, and its ability is better, especially, it supports customized scheduling. Essentially, in

Re: [DISCUSS] FLIP-238: Introduce FLIP-27-based Data Generator Source

2022-07-06 Thread Alexander Fedulov
Hi Becket, interesting points about the discrepancies in the *RuntimeContext* "wrapping" throughout the framework, but I agree - this is something that needs to be tackled separately. For now, I adjusted the FLIP and the PoC implementation to only expose the parallelism. Best, Alexander Fedulov

[jira] [Created] (FLINK-28432) HiveCatalogTable only support using the latest columns as partition keys

2022-07-06 Thread Jing Ge (Jira)
Jing Ge created FLINK-28432: --- Summary: HiveCatalogTable only support using the latest columns as partition keys Key: FLINK-28432 URL: https://issues.apache.org/jira/browse/FLINK-28432 Project: Flink

Re: [DISCUSS] Publish flink-end-to-end-tests-common to Maven

2022-07-06 Thread Alexander Fedulov
I decided that the classes listed above are generic enough to be placed into *flink-test-utils*. Here is the PR for the discussed changes that needs a review: https://github.com/apache/flink/pull/20188 Best, Alex On Wed, Jul 6, 2022 at 1:35 PM Alexander Fedulov wrote: > Hi Chesnay, > > the

Re: Python Job Support for the Kubernetes Operator

2022-07-06 Thread Thomas Weise
Since SQL or Python are essentially just examples of how to use the operator vs. features of the operator itself, they should not affect the release schedule and can be added anytime, as examples to the operator or elsewhere. Thanks, Thomas On Wed, Jul 6, 2022 at 8:33 AM Gyula Fóra wrote: > >

[jira] [Created] (FLINK-28431) CompletedCheckPoints stored on ZooKeeper is not up-to-date, when JobManager is restarted it fails to recover the job due to "checkpoint FileNotFound exception"

2022-07-06 Thread aresyhzhang (Jira)
aresyhzhang created FLINK-28431: --- Summary: CompletedCheckPoints stored on ZooKeeper is not up-to-date, when JobManager is restarted it fails to recover the job due to "checkpoint FileNotFound exception" Key: FLINK-28431

[jira] [Created] (FLINK-28430) HiveFunctionWrapper should use UserClassLoader to load class

2022-07-06 Thread luoyuxia (Jira)
luoyuxia created FLINK-28430: Summary: HiveFunctionWrapper should use UserClassLoader to load class Key: FLINK-28430 URL: https://issues.apache.org/jira/browse/FLINK-28430 Project: Flink Issue

[jira] [Created] (FLINK-28429) Reduce pyflink tests time

2022-07-06 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-28429: Summary: Reduce pyflink tests time Key: FLINK-28429 URL: https://issues.apache.org/jira/browse/FLINK-28429 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Support partition pruning for streaming reading

2022-07-06 Thread cao zou
Hi, godfrey and Jark, thanks for joining the discussion. The implications for FileSource Sorry about that, I have missed an important feature that the FileSource supports continuous reading. I think we can do the same thing for the FileSource, which is using the partition pruning function to

Re: Python Job Support for the Kubernetes Operator

2022-07-06 Thread Gyula Fóra
Hi All! One thing we could do already now is to add a simple example on how to execute Python jobs like java jobs (with the right main class, args etc). It would be similar to https://github.com/apache/flink-kubernetes-operator/tree/main/examples/flink-sql-runner-example but slightly simpler as

[jira] [Created] (FLINK-28428) Example in the Elasticsearch doc of fault tolerance section is missing

2022-07-06 Thread LuNing Wang (Jira)
LuNing Wang created FLINK-28428: --- Summary: Example in the Elasticsearch doc of fault tolerance section is missing Key: FLINK-28428 URL: https://issues.apache.org/jira/browse/FLINK-28428 Project: Flink

Re: [DISCUSS] Publish flink-end-to-end-tests-common to Maven

2022-07-06 Thread Alexander Fedulov
Hi Chesnay, the following classes also need to be migrated along with the *FlinkContainerTestEnvironment*: *JobSubmission* *SQLJobSubmission* *ParameterProperty* *FileUtils* We can either put them directly into* flink-connector-test-utils *or into *flink-test-utils *and add a dependency

[jira] [Created] (FLINK-28427) MiniClusterResource should not override rest port settings

2022-07-06 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-28427: Summary: MiniClusterResource should not override rest port settings Key: FLINK-28427 URL: https://issues.apache.org/jira/browse/FLINK-28427 Project: Flink

Re: [VOTE] Apache Flink ML Release 2.1.0, release candidate #2

2022-07-06 Thread Becket Qin
+1 (binding) Verified the following: - verified the checksum and signature of the source and binary distribution - built from source code without exception. (Some unit tests failed because rocksdb doesn't work well with the Apple silicon). - checked the source jar and it does not contain any

Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread yuxia
Thanks for driving this FLIP. +1 (non-binding) Best regards, Yuxia - 原始邮件 - 发件人: "zhangmang1" 收件人: "dev" 发送时间: 星期二, 2022年 7 月 05日 下午 2:18:02 主题: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS) Hi everyone, Thanks for all the feedback so far. Based on the discussion

[jira] [Created] (FLINK-28426) PyFlink provides M1 wheel package

2022-07-06 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-28426: Summary: PyFlink provides M1 wheel package Key: FLINK-28426 URL: https://issues.apache.org/jira/browse/FLINK-28426 Project: Flink Issue Type: New Feature

Re: Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread Jing Ge
+1 Thanks for your effort! Best regards, Jing On Wed, Jul 6, 2022 at 10:31 AM Mang Zhang wrote: > Hi Jing, > Since I updated the FLIP scenario after the last vote, I have relaunch a > vote. > Because the email title is the same, so I am sorry for the trouble. > Please vote in this thread.

Re:Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread Mang Zhang
Hi Jing, Since I updated the FLIP scenario after the last vote, I have relaunch a vote. Because the email title is the same, so I am sorry for the trouble. Please vote in this thread. thank you! -- Best regards, Mang Zhang At 2022-07-06 16:16:58, "Jing Ge" wrote: >should we go back

RE: Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread Rui Fan
Thanks Mang for this FLIP. I think it will be useful for users. +1(non-binding) Best wishes Rui Fan On 2022/07/06 08:13:11 Jiangang Liu wrote: > +1 for the design > > Jark Wu 于2022年7月5日周二 16:04写道: > > > +1 (binding) > > > > Best, > > Jark > > > > > > On Tue, 5 Jul 2022 at 14:18, Mang Zhang

Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread Jing Ge
should we go back to the voting thread to vote or just vote in this thread? Best regards, Jing On Wed, Jul 6, 2022 at 10:13 AM Jiangang Liu wrote: > +1 for the design > > Jark Wu 于2022年7月5日周二 16:04写道: > > > +1 (binding) > > > > Best, > > Jark > > > > > > On Tue, 5 Jul 2022 at 14:18, Mang

Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread Jiangang Liu
+1 for the design Jark Wu 于2022年7月5日周二 16:04写道: > +1 (binding) > > Best, > Jark > > > On Tue, 5 Jul 2022 at 14:18, Mang Zhang wrote: > > > Hi everyone, > > > > > > > > > > Thanks for all the feedback so far. Based on the discussion [1], we seem > > to have consensus. So, I would like to

[jira] [Created] (FLINK-28425) E2E tests run out of disk space on Azure

2022-07-06 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-28425: -- Summary: E2E tests run out of disk space on Azure Key: FLINK-28425 URL: https://issues.apache.org/jira/browse/FLINK-28425 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-28424) JdbcExactlyOnceSinkE2eTest hangs on Azure

2022-07-06 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-28424: -- Summary: JdbcExactlyOnceSinkE2eTest hangs on Azure Key: FLINK-28424 URL: https://issues.apache.org/jira/browse/FLINK-28424 Project: Flink Issue Type:

[jira] [Created] (FLINK-28423) ExecutionVertexCancelTest crashed with exit code 239

2022-07-06 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-28423: -- Summary: ExecutionVertexCancelTest crashed with exit code 239 Key: FLINK-28423 URL: https://issues.apache.org/jira/browse/FLINK-28423 Project: Flink

[jira] [Created] (FLINK-28422) Fall back to lastest supported HiveShim when user use a higher hive version

2022-07-06 Thread luoyuxia (Jira)
luoyuxia created FLINK-28422: Summary: Fall back to lastest supported HiveShim when user use a higher hive version Key: FLINK-28422 URL: https://issues.apache.org/jira/browse/FLINK-28422 Project: Flink

[jira] [Created] (FLINK-28421) Support full caching in lookup table source

2022-07-06 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-28421: - Summary: Support full caching in lookup table source Key: FLINK-28421 URL: https://issues.apache.org/jira/browse/FLINK-28421 Project: Flink Issue Type:

[jira] [Created] (FLINK-28420) Support partial lookup caching in lookup join runners

2022-07-06 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-28420: - Summary: Support partial lookup caching in lookup join runners Key: FLINK-28420 URL: https://issues.apache.org/jira/browse/FLINK-28420 Project: Flink

[jira] [Created] (FLINK-28419) Add runtime provider interface for full caching lookup

2022-07-06 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-28419: - Summary: Add runtime provider interface for full caching lookup Key: FLINK-28419 URL: https://issues.apache.org/jira/browse/FLINK-28419 Project: Flink

[jira] [Created] (FLINK-28418) Add runtime provider interface for partial caching lookup

2022-07-06 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-28418: - Summary: Add runtime provider interface for partial caching lookup Key: FLINK-28418 URL: https://issues.apache.org/jira/browse/FLINK-28418 Project: Flink

[jira] [Created] (FLINK-28417) Add interface and default implementation for cache in lookup table

2022-07-06 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-28417: - Summary: Add interface and default implementation for cache in lookup table Key: FLINK-28417 URL: https://issues.apache.org/jira/browse/FLINK-28417 Project: Flink

[jira] [Created] (FLINK-28416) Add the new LookupFunction interface for Lookup table source

2022-07-06 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-28416: - Summary: Add the new LookupFunction interface for Lookup table source Key: FLINK-28416 URL: https://issues.apache.org/jira/browse/FLINK-28416 Project: Flink

[jira] [Created] (FLINK-28415) FLIP-221: Support Partial and Full Caching in Lookup Table Source

2022-07-06 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-28415: - Summary: FLIP-221: Support Partial and Full Caching in Lookup Table Source Key: FLINK-28415 URL: https://issues.apache.org/jira/browse/FLINK-28415 Project: Flink