[jira] [Created] (FLINK-19146) createMiniBatchTrigger() use OR ，table.exec.mini-batch.size and table.exec.mini-batch.allow-latency

badqiu created FLINK-19146: -- Summary: createMiniBatchTrigger() use OR ，table.exec.mini-batch.size and table.exec.mini-batch.allow-latency Key: FLINK-19146 URL: https://issues.apache.org/jira/browse/FLINK-19146

[jira] [Created] (FLINK-19161) Port File Sources to FLIP-27 API

Stephan Ewen created FLINK-19161: Summary: Port File Sources to FLIP-27 API Key: FLINK-19161 URL: https://issues.apache.org/jira/browse/FLINK-19161 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

Yes, I think we can address the problem of indeterminacy in a separate FLIP because we're already in it. Aljoscha On 07.09.20 17:00, Dawid Wysakowicz wrote: @Seth That's a very good point. I agree that RocksDB has the same problem. I think we can use the same approach for the sorted shuffles

Re: [VOTE] FLIP-139: General Python User-Defined Aggregate Function on Table API

+1(binding) Best, Jincheng Xingbo Huang 于2020年9月7日周一 下午5:45写道： > Hi, > > +1 (non-binding) > > Best, > Xingbo > > Wei Zhong 于2020年9月7日周一 下午2:37写道： > > > Hi all, > > > > I would like to start the vote for FLIP-139[1] which is discussed and > > reached consensus in the discussion thread[2]. > >

Re: [VOTE] FLIP-137: Support Pandas UDAF in PyFlink

+1(binding) Best, Jincheng Shuiqiang Chen 于2020年9月7日周一 下午3:50写道： > +1 (non-binding) > > > 在 2020年9月7日，下午2:38，Wei Zhong 写道： > > > > +1 (non-binding) > > > >> 在 2020年9月7日，10:00，Dian Fu 写道： > >> > >> +1 > >> > >>> 在 2020年9月4日，上午11:12，Xingbo Huang 写道： > >>> > >>> Hi all, > >>> I would like to

Re: [VOTE] FLIP-137: Support Pandas UDAF in PyFlink

+1 (binding) On Tue, Sep 8, 2020 at 7:43 AM jincheng sun wrote: > +1(binding) > > Best, > Jincheng > > > Shuiqiang Chen 于2020年9月7日周一 下午3:50写道： > > > +1 (non-binding) > > > > > 在 2020年9月7日，下午2:38，Wei Zhong 写道： > > > > > > +1 (non-binding) > > > > > >> 在 2020年9月7日，10:00，Dian Fu 写道： > > >> > >

[jira] [Created] (FLINK-19162) Allow Split Reader based sources to reuse record batches

Stephan Ewen created FLINK-19162: Summary: Allow Split Reader based sources to reuse record batches Key: FLINK-19162 URL: https://issues.apache.org/jira/browse/FLINK-19162 Project: Flink

[ANNOUNCE] Weekly Community Update 2020/36

Dear community, happy to share another community update for the past week. This time with the upcoming release of Flink 1.11.2, a proposal for more efficient aggregation for batch processing with the DataStream API, and the comeback of two FLIPs that have been abandoned for a bit. Flink

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

Hi Dawid, thanks for bringing this up, it's really exciting to see that batch execution is introduced in DataStream. From the flip, it seems we are sticking with sort based execution mode (at least for now), which will sort the whole input data before any *keyed* operation is executed. I have two

Re: Merge SupportsComputedColumnPushDown and SupportsWatermarkPushDown

Thanks to Shengkai for summarizing the problems on the FLIP-95 interfaces and solutions. I think the new proposal, i.e. only pushing the "WatermarkStrategy" is much cleaner and easier to develop than before. So I'm +1 to the proposal. Best, Jark On Sat, 5 Sep 2020 at 13:44, Shengkai Fang

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Hi, Timo Thanks for you explanation and update, I have only one question for the latest FLIP. About the MAP DataType of key 'debezium-json.source', if user want to use the table name metadata, they need to write: tableName STRING AS CAST(SYSTEM_METADATA('debeuim-json.source') AS

Re: [VOTE] FLIP-141: Intra-Slot Managed Memory Sharing

+1 for the proposal! Regarding the name of `BATCH_OP/ROCKSDB`, we can reserve the configuration names for batch and rocksdb respectively, ` batch_ OP` for batch, "ROCKSDB" for roockdb. and the default value as follows: { BATCH_OP: 70, ROCKSDB : 70, PYTHON : 30 } Only one of `BATCH_

Re: [VOTE] FLIP-137: Support Pandas UDAF in PyFlink

+1 Thanks, Zhu Hequn Cheng 于2020年9月8日周二 上午8:57写道： > +1 (binding) > > > On Tue, Sep 8, 2020 at 7:43 AM jincheng sun > wrote: > > > +1(binding) > > > > Best, > > Jincheng > > > > > > Shuiqiang Chen 于2020年9月7日周一 下午3:50写道： > > > > > +1 (non-binding) > > > > > > > 在 2020年9月7日，下午2:38，Wei Zhong

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Thanks Timo ~ The FLIP was already in pretty good shape, I have only 2 questions here: 1. “`CAST(SYSTEM_METADATA("offset") AS INT)` would be a valid read-only computed column for Kafka and can be extracted by the planner.” What is the pros we follow the SQL-SERVER syntax here ? Usually an

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Hi Timo, 1. "`Map listReadableMetadata()` only allows one possible DataType for a metadata key." I think the main purpose of the metadata feature is to access the Kafka timestamp and use it as a rowtime attribute. If we force users to use the specific type, then this feature might be tricky to

Re: [VOTE] FLIP-137: Support Pandas UDAF in PyFlink

+1 (non-binding) > 在 2020年9月7日，10:00，Dian Fu 写道： > > +1 > >> 在 2020年9月4日，上午11:12，Xingbo Huang 写道： >> >> Hi all, >> I would like to start the vote for FLIP-137[1], which is discussed and >> reached a consensus in the discussion thread[2]. >> >> The vote will be open for at least 72h, unless

Re: [VOTE] FLIP-137: Support Pandas UDAF in PyFlink

+1 (non-binding) > 在 2020年9月7日，下午2:38，Wei Zhong 写道： > > +1 (non-binding) > >> 在 2020年9月7日，10:00，Dian Fu 写道： >> >> +1 >> >>> 在 2020年9月4日，上午11:12，Xingbo Huang 写道： >>> >>> Hi all, >>> I would like to start the vote for FLIP-137[1], which is discussed and >>> reached a consensus in the

[jira] [Created] (FLINK-19152) Remove Kafka 0.10.x and 0.11.x connectors

Aljoscha Krettek created FLINK-19152: Summary: Remove Kafka 0.10.x and 0.11.x connectors Key: FLINK-19152 URL: https://issues.apache.org/jira/browse/FLINK-19152 Project: Flink Issue

[jira] [Created] (FLINK-19147) Support AutoClose in CliClient

darion yaphet created FLINK-19147: - Summary: Support AutoClose in CliClient Key: FLINK-19147 URL: https://issues.apache.org/jira/browse/FLINK-19147 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-19148) Table crashed in Flink Table API & SQL Docs

Leonard Xu created FLINK-19148: -- Summary: Table crashed in Flink Table API & SQL Docs Key: FLINK-19148 URL: https://issues.apache.org/jira/browse/FLINK-19148 Project: Flink Issue Type: Bug

Re: [DISCUSS][Release 1.12] Stale blockers and build instabilities

Hi all, another two weeks have passed. We now have 5 blockers (Up 3 from 2 weeks ago), but they are all making progress. We currently have 79 test-instabilities

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Ignore my question(4), I’ve found the answer in the doc : 'value.fields-include' = ‘EXCEPT_KEY' (all fields of the schema minus fields of the key) > 在 2020年9月7日，16:33，Leonard Xu 写道： > > (4) About Reading and writing from key and value section, we bind that the > fields of key part must

[VOTE] FLIP-139: General Python User-Defined Aggregate Function on Table API

Hi all, I would like to start the vote for FLIP-139[1] which is discussed and reached consensus in the discussion thread[2]. The vote will be open for at least 72 hours. I'll try to close it by 2020-09-10 07:00 UTC, unless there is an objection or not enough votes. Best, Wei [1]

[jira] [Created] (FLINK-19149) Compacted Kafka Topic can be interpreted as Changelog Stream

Konstantin Knauf created FLINK-19149: Summary: Compacted Kafka Topic can be interpreted as Changelog Stream Key: FLINK-19149 URL: https://issues.apache.org/jira/browse/FLINK-19149 Project: Flink

Flink stateful functions : compensating callback to invoked functions on a timeout

Hi all, I am implementing a use case in Flink stateful functions. My specification highlights that starting from a *stateful function f* a business workflow (in other words a group of stateful functions f1, f2, … fn are called either sequentially or in parallel or both ). Stateful function f

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Thanks Timo for the update ! I like the proposal in general, I have some question about the doc. (1) About the DDL `CAST(SYSTEM_METADATA("offset") AS INT)`, It looks like we use `CAST` to resolve the nullability of meta column type, Could we use explicit type (i.e. SYSTEM_METADATA("offset”)

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Hi Timo, Thanks a lot for picking up this FLIP. I believe it's a very important one for almost everyone who uses Flink SQL with Kafka. Also +1 to leave out partitioning for now. Best, Konstantin On Fri, Sep 4, 2020 at 1:37 PM Aljoscha Krettek wrote: > I like the proposal! I didn't check the

[RESULT][VOTE] FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API)

Hi all, The voting time for FLIP-131 [1] has passed. I'm closing the vote now. Including my implicit vote, there were 7 + 1 votes, 5 of which are binding: - Dawid Wysakowicz (binding) - Piotr Nowojski (binding) - David Anderson (binding) - Zhu Zhu (binding) - Aljoscha Krettek There were no -1

[jira] [Created] (FLINK-19150) Behaviour change after migration from 1.9 to 1.11

Jiayi Liao created FLINK-19150: -- Summary: Behaviour change after migration from 1.9 to 1.11 Key: FLINK-19150 URL: https://issues.apache.org/jira/browse/FLINK-19150 Project: Flink Issue Type:

[jira] [Created] (FLINK-19151) Flink does not normalize container resource with correct configurations when Yarn FairScheduler is used

Xintong Song created FLINK-19151: Summary: Flink does not normalize container resource with correct configurations when Yarn FairScheduler is used Key: FLINK-19151 URL:

[jira] [Created] (FLINK-19155) ResultPartitionTest is unstable

Aljoscha Krettek created FLINK-19155: Summary: ResultPartitionTest is unstable Key: FLINK-19155 URL: https://issues.apache.org/jira/browse/FLINK-19155 Project: Flink Issue Type: New

Re: [VOTE] FLIP-139: General Python User-Defined Aggregate Function on Table API

Hi, +1 (non-binding) Best, Xingbo Wei Zhong 于2020年9月7日周一 下午2:37写道： > Hi all, > > I would like to start the vote for FLIP-139[1] which is discussed and > reached consensus in the discussion thread[2]. > > The vote will be open for at least 72 hours. I'll try to close it by > 2020-09-10 07:00

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Hi Leonard, thanks for your feedback. (1) Actually, I discuss this already in the FLIP. But let me summarize our options again if it was not clear enough in the FLIP: a) CREATE TABLE t (a AS CAST(SYSTEM_METADATA("offset") AS INT)) pro: readable, complex arithmetic possible, more SQL

h

f 568793005 邮箱：568793...@qq.com 签名由 网易邮箱大师 定制

Re: [jira] [Created] (FLINK-16595) Support extra hadoop filesystem URLs for which to request delegation tokens

Hi all , I hava solved this problem in my production environment, and I would like to do something for the community, could you assign this issue to me？ -- Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

[jira] [Created] (FLINK-19156) Migration of transactionIdHint in Kafka is never applied

Dawid Wysakowicz created FLINK-19156: Summary: Migration of transactionIdHint in Kafka is never applied Key: FLINK-19156 URL: https://issues.apache.org/jira/browse/FLINK-19156 Project: Flink

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Thanks Timo, I think this FLIP is already in great shape! I have following questions: 1. `Map listReadableMetadata()` only allows one possible DataType for a metadata key. However, users may expect to use different types, e.g. for "timestamp" metadata, users may use it as BIGINT, or

[jira] [Created] (FLINK-19154) Always clean up HA data when application completion

Husky Zeng created FLINK-19154: -- Summary: Always clean up HA data when application completion Key: FLINK-19154 URL: https://issues.apache.org/jira/browse/FLINK-19154 Project: Flink Issue Type:

[jira] [Created] (FLINK-19158) Revisit java e2e download timeouts

Robert Metzger created FLINK-19158: -- Summary: Revisit java e2e download timeouts Key: FLINK-19158 URL: https://issues.apache.org/jira/browse/FLINK-19158 Project: Flink Issue Type:

[jira] [Created] (FLINK-19153) FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API)

Aljoscha Krettek created FLINK-19153: Summary: FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API) Key: FLINK-19153 URL: https://issues.apache.org/jira/browse/FLINK-19153

[jira] [Created] (FLINK-19157) Jdbc PhoenixDialect

Zhou Zach created FLINK-19157: - Summary: Jdbc PhoenixDialect Key: FLINK-19157 URL: https://issues.apache.org/jira/browse/FLINK-19157 Project: Flink Issue Type: Task Components:

Re: Flink stateful functions : compensating callback to invoked functions on a timeout

Hi, I am not an expert on Stateful functions, but I doubt there is something in particular that would solve you use case. I think you need to implement that in the user space. You will need some way of keeping track of which state changes come from which originating event and then have some

Re: [DISCUSS] Releasing Flink 1.11.2

Hi All, Since there are still two 1.11.2 blockers items in progress, RC1 creation will be postponed to tomorrow. Thanks, Zhu Zhu Zhu 于2020年9月4日周五 下午3:50写道： > @Dawid > Thanks for the information and preparing a fix for FLINK-19133! > I have made it a blocker for 1.11.2 and will keep tracking

[jira] [Created] (FLINK-19160) When backpressured AsyncWaitOperator/ContinousFileReaderOperator are not idle

Konstantin Knauf created FLINK-19160: Summary: When backpressured AsyncWaitOperator/ContinousFileReaderOperator are not idle Key: FLINK-19160 URL: https://issues.apache.org/jira/browse/FLINK-19160

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Sorry, I forgot to ask one more question. 4. Do we allow to use the SYSTEM_METADATA as a sub-expression? For example, checksum AS CAST(CAST(SYSTEM_METADATA("headers") AS MAP)['checksum'] AS STRING), myvalue AS CAST(CAST(SYSTEM_METADATA("headers") AS MAP)['mykey'] AS BIGINT) And we will push

[jira] [Created] (FLINK-19159) Using Scalafmt to format scala source code

darion yaphet created FLINK-19159: - Summary: Using Scalafmt to format scala source code Key: FLINK-19159 URL: https://issues.apache.org/jira/browse/FLINK-19159 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

Hi Jark, 1. "`Map listReadableMetadata()` only allows one possible DataType for a metadata key." I was thinking about this topic a lot today. My conclusion is: yes, we should force users to specify the type as documented. Users can further cast or compute using expressions to more specific

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

@Seth That's a very good point. I agree that RocksDB has the same problem. I think we can use the same approach for the sorted shuffles then. @Aljoscha I agree we should think about making it more resilient, as I guess users might have problems already if they use keys with non-deterministic