[RESULT][VOTE] Creating benchmark channel in Apache Flink slack

2022-07-15 Thread Anton Kalashnikov
Hi everyone, I’m happy to announce that 'Creating new benchmark Slack channel' has been accepted, with 7 approving votes, 5 of which are binding: - Yun Gao (binding) - Yuan Mei (binding) - Godfrey He - Marton Balassi (binding) - Alexander Fedulov - Yun Tang (binding) - Jing Zhang (binding)

[DISCUSS] FLIP-252: Amazon DynamoDB Sink Connector

2022-07-15 Thread Danny Cranmer
Hello all, We would like to start a discussion thread on FLIP-252: Amazon DynamoDB Sink Connector [1] where we propose to provide a sink connector for Amazon DynamoDB [2] based on the Async Sink [3]. Looking forward to comments and feedback. Thank you. [1]

[jira] [Created] (FLINK-28573) Nested type will lose nullability when converting from TableSchema

2022-07-15 Thread Jane Chan (Jira)
Jane Chan created FLINK-28573: - Summary: Nested type will lose nullability when converting from TableSchema Key: FLINK-28573 URL: https://issues.apache.org/jira/browse/FLINK-28573 Project: Flink

Re: [DISCUSS] Add new JobStatus fields to Flink Kubernetes Operator CRD

2022-07-15 Thread Gyula Fóra
Hi Matyas! So to clarify your suggestion, we would have the following JobStatus fields: jobId : String state : String savepointInfo : SavepointInfo jobDetailsInfo : String (optional) - output of Flink Rest API job details And the user could configure with a flag whether to include

Re: [DISCUSS] Add new JobStatus fields to Flink Kubernetes Operator CRD

2022-07-15 Thread Őrhidi Mátyás
Hi Gyula, since the jobDetailsInfo could evolve, another option would be to dump it as yaml/json into the metadata. Best, Matyas On Fri, Jul 15, 2022 at 2:58 PM Gyula Fóra wrote: > Based on some further though, a reasonable middleground would be to add an > optional metadata/jobDetailsInfo

[jira] [Created] (FLINK-28572) FlinkSQL executes Table.execute multiple times on the same Table, and changing the Table.execute code position will produce different phenomena

2022-07-15 Thread Andrew Chan (Jira)
Andrew Chan created FLINK-28572: --- Summary: FlinkSQL executes Table.execute multiple times on the same Table, and changing the Table.execute code position will produce different phenomena Key: FLINK-28572 URL:

Re: [DISCUSS] Add new JobStatus fields to Flink Kubernetes Operator CRD

2022-07-15 Thread Gyula Fóra
Based on some further though, a reasonable middleground would be to add an optional metadata/jobDetailsInfo field to the JobStatus. We would also add an accompanying config option (default false) whether to populate this field for jobs. This way operator users could decide if they want to expose

Re: [DISCUSS] Add new JobStatus fields to Flink Kubernetes Operator CRD

2022-07-15 Thread Gyula Fóra
Hi All! I fully acknowledge the general need to access more info about the running deployments. This need however is very specific to the use-cases / platforms built on the operator. I think we need a good way to tackle this without growing the status arbitrarily. Currently the JobStatus in the

[jira] [Created] (FLINK-28571) Add Chi-squared test as Transformer to ml.feature

2022-07-15 Thread Simon Tao (Jira)
Simon Tao created FLINK-28571: - Summary: Add Chi-squared test as Transformer to ml.feature Key: FLINK-28571 URL: https://issues.apache.org/jira/browse/FLINK-28571 Project: Flink Issue Type: New

Re: [DISCUSS] Add new JobStatus fields to Flink Kubernetes Operator CRD

2022-07-15 Thread WONG, DAREN
Hi Martin, Yes, that's understandable. I think adding job endTime, duration, jobPlan is useful to other Flink users too as they now have info to track: 1. endTime: If the job has ended, the user can know when it has ended. If the job is still streaming, then the user can know as it defaults to

Re: [DISCUSS] FLIP-247 Bulk fetch of table and column statistics for given partitions

2022-07-15 Thread godfrey he
Hi Jing, Thanks for the driving this, LGTM. Best, Godfrey Jingsong Li 于2022年7月15日周五 11:38写道: > > Thanks for starting this discussion. > > Have we considered introducing a listPartitionWithStats() in Catalog? > > Best, > Jingsong > > On Fri, Jul 15, 2022 at 10:08 AM Jark Wu wrote: > > > > Hi

[jira] [Created] (FLINK-28570) Introduces a StreamPhysicalPlanChecker to validate if there's any non-deterministic updates which may cause wrong result

2022-07-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-28570: --- Summary: Introduces a StreamPhysicalPlanChecker to validate if there's any non-deterministic updates which may cause wrong result Key: FLINK-28570 URL:

[jira] [Created] (FLINK-28569) SinkUpsertMaterializer should be aware of the input upsertKey if it is not empty

2022-07-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-28569: --- Summary: SinkUpsertMaterializer should be aware of the input upsertKey if it is not empty Key: FLINK-28569 URL: https://issues.apache.org/jira/browse/FLINK-28569

[jira] [Created] (FLINK-28568) Implements a new lookup join operator (sync mode only) with state to eliminate the non determinism

2022-07-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-28568: --- Summary: Implements a new lookup join operator (sync mode only) with state to eliminate the non determinism Key: FLINK-28568 URL: https://issues.apache.org/jira/browse/FLINK-28568

[jira] [Created] (FLINK-28567) Introduce predicate inference from one side of join to the other

2022-07-15 Thread Alexander Trushev (Jira)
Alexander Trushev created FLINK-28567: - Summary: Introduce predicate inference from one side of join to the other Key: FLINK-28567 URL: https://issues.apache.org/jira/browse/FLINK-28567 Project:

[jira] [Created] (FLINK-28566) Adds materialization support to eliminate the non determinism generated by lookup join node

2022-07-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-28566: --- Summary: Adds materialization support to eliminate the non determinism generated by lookup join node Key: FLINK-28566 URL: https://issues.apache.org/jira/browse/FLINK-28566

[RESULT][VOTE] FLIP-249: Flink Web UI Enhancement for Speculative Execution

2022-07-15 Thread Gen Luo
Hi everyone, I’m happy to announce that FLIP-249[1] has been accepted, with 4 approving votes, 3 of which are binding[2]: - Zhu Zhu (binding) - Lijie Wang - Jing Zhang (binding) - Yun Gao (binding) There is no disapproving vote. [1]

[jira] [Created] (FLINK-28565) Create NOTICE file for flink-table-store-hive-catalog

2022-07-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-28565: Summary: Create NOTICE file for flink-table-store-hive-catalog Key: FLINK-28565 URL: https://issues.apache.org/jira/browse/FLINK-28565 Project: Flink Issue

[jira] [Created] (FLINK-28564) Update NOTICE/LICENCE files for 1.1.0 release

2022-07-15 Thread Matyas Orhidi (Jira)
Matyas Orhidi created FLINK-28564: - Summary: Update NOTICE/LICENCE files for 1.1.0 release Key: FLINK-28564 URL: https://issues.apache.org/jira/browse/FLINK-28564 Project: Flink Issue Type:

[jira] [Created] (FLINK-28563) Add Transformer for VectorSlicer

2022-07-15 Thread weibo zhao (Jira)
weibo zhao created FLINK-28563: -- Summary: Add Transformer for VectorSlicer Key: FLINK-28563 URL: https://issues.apache.org/jira/browse/FLINK-28563 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-28562) Rocksdb state backend is too slow when using AggregateFunction

2022-07-15 Thread Yue Shang (Jira)
Yue Shang created FLINK-28562: - Summary: Rocksdb state backend is too slow when using AggregateFunction Key: FLINK-28562 URL: https://issues.apache.org/jira/browse/FLINK-28562 Project: Flink

[jira] [Created] (FLINK-28561) Merge subpartition shuffle data read request for better sequential IO

2022-07-15 Thread Yingjie Cao (Jira)
Yingjie Cao created FLINK-28561: --- Summary: Merge subpartition shuffle data read request for better sequential IO Key: FLINK-28561 URL: https://issues.apache.org/jira/browse/FLINK-28561 Project: Flink