[jira] [Created] (FLINK-29954) Improve DynamoDB Connector Sample Code

2022-11-08 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-29954: - Summary: Improve DynamoDB Connector Sample Code Key: FLINK-29954 URL: https://issues.apache.org/jira/browse/FLINK-29954 Project: Flink Issue Type:

[jira] [Created] (FLINK-29953) Get rid of flink-connector-hive dependency in flink-table-store-hive

2022-11-08 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-29953: Summary: Get rid of flink-connector-hive dependency in flink-table-store-hive Key: FLINK-29953 URL: https://issues.apache.org/jira/browse/FLINK-29953 Project: Flink

[jira] [Created] (FLINK-29952) Using DROP TABLE XXX to drop a temporary table (or view) in sql-client can get an exception with incomplete explaination.

2022-11-08 Thread Hang HOU (Jira)
Hang HOU created FLINK-29952: Summary: Using DROP TABLE XXX to drop a temporary table (or view) in sql-client can get an exception with incomplete explaination. Key: FLINK-29952 URL:

[jira] [Created] (FLINK-29951) Kafka SQL connector supports setting end offset/timestamp

2022-11-08 Thread Xuannan Su (Jira)
Xuannan Su created FLINK-29951: -- Summary: Kafka SQL connector supports setting end offset/timestamp Key: FLINK-29951 URL: https://issues.apache.org/jira/browse/FLINK-29951 Project: Flink Issue

[jira] [Created] (FLINK-29950) Add 'contentType' and 'FieldFormat' to 'ResultSet' for improvement

2022-11-08 Thread yuzelin (Jira)
yuzelin created FLINK-29950: --- Summary: Add 'contentType' and 'FieldFormat' to 'ResultSet' for improvement Key: FLINK-29950 URL: https://issues.apache.org/jira/browse/FLINK-29950 Project: Flink

[jira] [Created] (FLINK-29949) Implement 'RemoteExecutor' to perform the communication with Gateway

2022-11-08 Thread yuzelin (Jira)
yuzelin created FLINK-29949: --- Summary: Implement 'RemoteExecutor' to perform the communication with Gateway Key: FLINK-29949 URL: https://issues.apache.org/jira/browse/FLINK-29949 Project: Flink

[jira] [Created] (FLINK-29948) Introduce ErrorResponse in Gateway RestEndpoint.

2022-11-08 Thread yuzelin (Jira)
yuzelin created FLINK-29948: --- Summary: Introduce ErrorResponse in Gateway RestEndpoint. Key: FLINK-29948 URL: https://issues.apache.org/jira/browse/FLINK-29948 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-29947) Support configuring session API in Gateway RestEndpoint

2022-11-08 Thread yuzelin (Jira)
yuzelin created FLINK-29947: --- Summary: Support configuring session API in Gateway RestEndpoint Key: FLINK-29947 URL: https://issues.apache.org/jira/browse/FLINK-29947 Project: Flink Issue Type:

[jira] [Created] (FLINK-29946) Introduce ‘ErrorDetail’ in SQL Gateway Rest Endpoint for error handling

2022-11-08 Thread yuzelin (Jira)
yuzelin created FLINK-29946: --- Summary: Introduce ‘ErrorDetail’ in SQL Gateway Rest Endpoint for error handling Key: FLINK-29946 URL: https://issues.apache.org/jira/browse/FLINK-29946 Project: Flink

[jira] [Created] (FLINK-29945) Refactor ‘Executor’ interface to work with Gateway API

2022-11-08 Thread yuzelin (Jira)
yuzelin created FLINK-29945: --- Summary: Refactor ‘Executor’ interface to work with Gateway API Key: FLINK-29945 URL: https://issues.apache.org/jira/browse/FLINK-29945 Project: Flink Issue Type:

[jira] [Created] (FLINK-29944) Support accumaltor in source reader

2022-11-08 Thread hk__lrzy (Jira)
hk__lrzy created FLINK-29944: Summary: Support accumaltor in source reader Key: FLINK-29944 URL: https://issues.apache.org/jira/browse/FLINK-29944 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-29943) Bump Table Store 0.2 Flink version to 1.15.2

2022-11-08 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-29943: --- Summary: Bump Table Store 0.2 Flink version to 1.15.2 Key: FLINK-29943 URL: https://issues.apache.org/jira/browse/FLINK-29943 Project: Flink Issue Type:

[jira] [Created] (FLINK-29942) Improve average performance for TPC-DS

2022-11-08 Thread Jark Wu (Jira)
Jark Wu created FLINK-29942: --- Summary: Improve average performance for TPC-DS Key: FLINK-29942 URL: https://issues.apache.org/jira/browse/FLINK-29942 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-29941) Support SQL Client connects to SQL Gateway

2022-11-08 Thread Jark Wu (Jira)
Jark Wu created FLINK-29941: --- Summary: Support SQL Client connects to SQL Gateway Key: FLINK-29941 URL: https://issues.apache.org/jira/browse/FLINK-29941 Project: Flink Issue Type: New Feature

Re: [DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-08 Thread Yanfei Lei
Hi Roman, Thanks for the proposal, this allows State Backend to make better use of memory. After reading the ticket, I'm curious about some points: 1. Is shared-memory only for the state backend? If both "taskmanager.memory.managed.shared-fraction: >0" and "state.backend.rocksdb.memory.managed:

[jira] [Created] (FLINK-29940) ExecutionGraph logs job state change at ERROR level when job fails

2022-11-08 Thread Mingliang Liu (Jira)
Mingliang Liu created FLINK-29940: - Summary: ExecutionGraph logs job state change at ERROR level when job fails Key: FLINK-29940 URL: https://issues.apache.org/jira/browse/FLINK-29940 Project: Flink

[jira] [Created] (FLINK-29939) Add metrics for Kubernetes Client Response 5xx count and rate

2022-11-08 Thread Zhou Jiang (Jira)
Zhou Jiang created FLINK-29939: -- Summary: Add metrics for Kubernetes Client Response 5xx count and rate Key: FLINK-29939 URL: https://issues.apache.org/jira/browse/FLINK-29939 Project: Flink

[jira] [Created] (FLINK-29938) Add open() Method to AsyncSink ElementConverter

2022-11-08 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-29938: - Summary: Add open() Method to AsyncSink ElementConverter Key: FLINK-29938 URL: https://issues.apache.org/jira/browse/FLINK-29938 Project: Flink Issue

[jira] [Created] (FLINK-29937) Enhanced DynamoDB Element Converter

2022-11-08 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-29937: - Summary: Enhanced DynamoDB Element Converter Key: FLINK-29937 URL: https://issues.apache.org/jira/browse/FLINK-29937 Project: Flink Issue Type:

[DISCUSS] Allow sharing (RocksDB) memory between slots

2022-11-08 Thread Roman Khachatryan
Hi everyone, I'd like to discuss sharing RocksDB memory across slots as proposed in FLINK-29928 [1]. Since 1.10 / FLINK-7289 [2], it is possible to: - share these objects among RocksDB instances of the same slot - bound the total memory usage by all RocksDB instances of a TM However, the memory

[jira] [Created] (FLINK-29936) Make operator more robust to namespaces access errors

2022-11-08 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-29936: -- Summary: Make operator more robust to namespaces access errors Key: FLINK-29936 URL: https://issues.apache.org/jira/browse/FLINK-29936 Project: Flink Issue

Re: Questions about Flink Table Store

2022-11-08 Thread Leonard Xu
Hi, Alexander Happy to hear that you’re willing to contribute to Flink, the answer of these specific tech design/functionality questions may need to dig the code. I’m not familiar with FTS code base, but I’v CC my colleagues who are the core maintainers of FTS and maybe they can give your some

Re: [Discuss]- Donate Iceberg Flink Connector

2022-11-08 Thread OpenInx
Hi Sorry for the late reply. I'm one of the core flink iceberg connector maintainers at the early stage (flink 1.12, flink 1.13, flink 1.14). For the later flink releases, I've had some adjustments in my work and had less interactions with apache flink+iceberg, thanks Ryan, Steven, Kyle,

Re: support escaping `#` in flink job spec in Flink-operator

2022-11-08 Thread Maximilian Michels
Taking a step back here: I think this needs to be handled in the application mode in any case. Even if we had a better parser, it would still treat # as a comment char. The application mode needs to be fixed to come up with an escape scheme. YAML supports this via \# but that won't work with our

[jira] [Created] (FLINK-29935) Setup CI runs against 1.16/1.17-SNAPSHOT

2022-11-08 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-29935: Summary: Setup CI runs against 1.16/1.17-SNAPSHOT Key: FLINK-29935 URL: https://issues.apache.org/jira/browse/FLINK-29935 Project: Flink Issue Type:

[jira] [Created] (FLINK-29934) maven-assembly-plugin 2.4 make assemble plugin could find xml in flink clients module

2022-11-08 Thread jackylau (Jira)
jackylau created FLINK-29934: Summary: maven-assembly-plugin 2.4 make assemble plugin could find xml in flink clients module Key: FLINK-29934 URL: https://issues.apache.org/jira/browse/FLINK-29934

[jira] [Created] (FLINK-29933) Bump Flink version to 1.16.0

2022-11-08 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-29933: Summary: Bump Flink version to 1.16.0 Key: FLINK-29933 URL: https://issues.apache.org/jira/browse/FLINK-29933 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-29932) Upgrade Calcite version to 1.29.0

2022-11-08 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-29932: --- Summary: Upgrade Calcite version to 1.29.0 Key: FLINK-29932 URL: https://issues.apache.org/jira/browse/FLINK-29932 Project: Flink Issue Type:

[jira] [Created] (FLINK-29931) Pin Flink binary URL and cache binary

2022-11-08 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-29931: Summary: Pin Flink binary URL and cache binary Key: FLINK-29931 URL: https://issues.apache.org/jira/browse/FLINK-29931 Project: Flink Issue Type:

[jira] [Created] (FLINK-29930) ParquetFileStatsExtractorTest is unstable

2022-11-08 Thread Shammon (Jira)
Shammon created FLINK-29930: --- Summary: ParquetFileStatsExtractorTest is unstable Key: FLINK-29930 URL: https://issues.apache.org/jira/browse/FLINK-29930 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-29929) Correct conversions from ResolvedSchema to TableSchema

2022-11-08 Thread Yubin Li (Jira)
Yubin Li created FLINK-29929: Summary: Correct conversions from ResolvedSchema to TableSchema Key: FLINK-29929 URL: https://issues.apache.org/jira/browse/FLINK-29929 Project: Flink Issue Type:

[GitHub] [flink-connector-shared-utils] zentol commented on a diff in pull request #1: [FLINK-29472] Add first version of release scripts

2022-11-08 Thread GitBox
zentol commented on code in PR #1: URL: https://github.com/apache/flink-connector-shared-utils/pull/1#discussion_r1016492375 ## README.md: ## @@ -1 +1,79 @@ -This repository contains utilities for [Apache Flink](https://flink.apache.org/) connectors. \ No newline at end of

Re: [DISCUSS] FLIP-271: Autoscaling

2022-11-08 Thread Maximilian Michels
>> # Horizontal scaling V.S. Vertical scaling > >True. We left out vertical scaling intentionally. For now we assume CPU / memory is set up by the user. While definitely useful, vertical scaling >adds another dimension to the scaling problem which we wanted to tackle later. I'll update the FLIP to

[jira] [Created] (FLINK-29928) Allow sharing (RocksDB) memory between slots

2022-11-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-29928: - Summary: Allow sharing (RocksDB) memory between slots Key: FLINK-29928 URL: https://issues.apache.org/jira/browse/FLINK-29928 Project: Flink Issue

[VOTE] Release flink-shaded 16.1, release candidate #1

2022-11-08 Thread Chesnay Schepler
Hi everyone, Please review and vote on the release candidate #1 for the version 16.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1],

Re: [DISCUSS] FLIP-271: Autoscaling

2022-11-08 Thread Gyula Fóra
I had 2 extra comments to Max's reply: 1. About pre-allocating resources: This could be done through the operator when the standalone deployment mode is used relatively easily as there we have better control of pods/resources. 2. Session jobs: There is a FLIP (

Re: [DISCUSS] FLIP-271: Autoscaling

2022-11-08 Thread Maximilian Michels
@Yang >Since the current auto-scaling needs to fully redeploy the application, it may fail to start due to lack of resources. Great suggestions. I agree that we will have to have to preallocate / reserve resources to ensure the rescaling doesn't take longer as expected. This is not only a

Re: [VOTE] Release Apache Flink Elasticsearch connector 3.0.0, rc2

2022-11-08 Thread Martijn Visser
+1 (binding) - Downloaded artifacts - Checked hash and signature - No binaries in source archive found - Verified NOTICE files - Build from source code - Spinned up a Flink 1.16 cluster with Kafka and Elasticsearch7 and used flink-sql-connector-elasticsearch7-3.0.0.jar to successfully sent data

Re: ASF Slack

2022-11-08 Thread Xintong Song
Hi Max, Thanks for bringing this up. I'm open to a re-evaluation of the Slack usages. In the initial discussion for creating the Slack workspace [1], leveraging the ASF Slack was indeed brought up as an alternative by many folks. I think we have chosen a dedicated Flink Slack over the ASF Slack

[jira] [Created] (FLINK-29927) AkkaUtils#getAddress may cause memory leak

2022-11-08 Thread Gen Luo (Jira)
Gen Luo created FLINK-29927: --- Summary: AkkaUtils#getAddress may cause memory leak Key: FLINK-29927 URL: https://issues.apache.org/jira/browse/FLINK-29927 Project: Flink Issue Type: Bug

Re: ASF Slack

2022-11-08 Thread Martijn Visser
If you click on the link from Beam via an incognito window/logged out of Slack, you will be prompted to provide the workspace URL of the ASF. If you do that, you're prompted for a login screen or you can create an account. Creating an account prompts you to have an @apache.org email address. See

Re: [DISCUSS]Introduce a time-segment based restart strategy

2022-11-08 Thread Weihua Hu
HI, @Paul Lam Thanks for the reply. I think it makes a lot of sense to distinguish exceptions, but it might add complexity to the restart policy maintenance, and some exceptions might be wrapped in the FlinkRuntimeException or something else. Maybe we can implement the first version based on the

[jira] [Created] (FLINK-29926) File source continuous monitoring mode ignoring files during savepoint upgrade mode

2022-11-08 Thread Avinash (Jira)
Avinash created FLINK-29926: --- Summary: File source continuous monitoring mode ignoring files during savepoint upgrade mode Key: FLINK-29926 URL: https://issues.apache.org/jira/browse/FLINK-29926 Project: