Why documentation always say checkpoint does not support Flink specific features like rescaling.

2018-08-27 Thread Yun Tang
Hi, all >From the latest 1.6 (or all the previous verions) documentation >https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/state/checkpoints.html#difference-to-savepoints, > it said "Checkpoints have a few differences from savepoints, they do not >support Flink specific features

Re: How to use RocksDBStateBackend predefined options

2018-11-13 Thread Yun Tang
not included in our pre-wrapped configuration. We might combine these two kind of operations together. I'd like to create a JIRA about this improvement, what's your opinions? Best Yun Tang From: Thomas Weise Sent: Tuesday, November 13, 2018 1:53 To: dev

Re: [DISCUSS] Contributing Chinese website and docs to Apache Flink

2019-01-28 Thread Yun Tang
/cn> alluxio/docs/cn at master · Alluxio/alluxio · GitHub<https://github.com/Alluxio/alluxio/tree/master/docs/cn> You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. github.com B

Re: [DISCUSS] Remove forceAvro() and forceKryo() from the ExecutionConfig

2019-03-26 Thread Yun Tang
ExecutionConfig. Best Yun Tang [1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/custom_serializers.html From: Konstantin Knauf Sent: Tuesday, March 26, 2019 17:37 To: Stephan Ewen Cc: Yun Tang; dev; user Subject: Re: [DISCUSS] Remove forceAvro

Re: [DISCUSS] Remove forceAvro() and forceKryo() from the ExecutionConfig

2019-03-26 Thread Yun Tang
for enableForceAvro() and enableForceKryo()? I think if https://issues.apache.org/jira/browse/FLINK-11917 merged, we could support to migrate state which was POJO but serialized using Kryo. Best Yun Tang From: Stephan Ewen Sent: Tuesday, March 26, 2019 2:31

Re: Where does RocksDB's mem allocation occur

2019-02-22 Thread Yun Tang
-in-RocksDB#indexes-and-filter-blocks> A library that provides an embeddable, persistent key-value store for fast storage. - facebook/rocksdb github.com Best Yun Tang From: Roshan Naik Sent: Saturday, February 23, 2019 10:09 To: dev@flink.apache.org Subject: Wh

Re: JobManager scale limitation - Slow S3 checkpoint deletes

2019-03-06 Thread Yun Tang
`notifyCheckpointComplete` come. * Let state backend on task manager side could did something on its side, which would be really helpful for specific state backend disaggregating computation and storage. Best Yun Tang From: Thomas Weise Sent: Thursday, March 7, 2019

Re: [Discuss][FLINK-8297]A solution for FLINK-8297 Timebased RocksDBListState

2019-04-15 Thread Yun Tang
state as expected and not recommend users to directly use this feature if they're not sure the list state really large. Best Yun Tang From: Andrey Zagrebin Sent: Monday, April 15, 2019 17:09 To: dev Subject: Re: [Discuss][FLINK-8297]A solution for FLINK-8297

Re: Something wrong with travis?

2019-06-24 Thread Yun Tang
 Best Yun Tang From: Chesnay Schepler Sent: Wednesday, June 19, 2019 16:59 To: dev@flink.apache.org; Kurt Young Subject: Re: Something wrong with travis? Recent builds are passing again. On 18/06/2019 08:34, Kurt Young wrote: > Hi dev, > > I noticed

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

2019-06-24 Thread Yun Tang
Congratulations Jincheng! Best Yun Tang From: Yu Li Sent: Tuesday, June 25, 2019 0:07 To: dev Subject: Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC Congratulations Jincheng! Well deserved! Best Regards, Yu On Tue, 25 Jun 2019 at 00:01, Zili Chen

Re: [VOTE] Migrate to sponsored Travis account

2019-07-04 Thread Yun Tang
I noticed that switching to a separate Travis account to run CI is actually impossible from what https://issues.apache.org/jira/browse/INFRA-18703 said. Hope another option from Chesnay to speed up the CI progress would work soon. Best Yun Tang From: Jark Wu

Re: A question about RocksDBListState and RocksDBMapState.

2019-07-05 Thread Yun Tang
Hi kaka You're correct, these comments are not correct for RocksDBMapState now, will correct it with a hotfix. Best Yun Tang From: kaka chen Sent: Friday, July 5, 2019 15:05 To: dev@flink.apache.org Subject: A question about RocksDBListState and RocksDBMapState

Re: [DISCUSS] ARM support for Flink

2019-07-10 Thread Yun Tang
better support to release RocksDB-java with ARM officially. [1] https://github.com/facebook/rocksdb/blob/master/java/RELEASE.md [2] https://github.com/dataArtisans/frocksdb Best Yun Tang From: Xiyuan Wang Sent: Tuesday, July 9, 2019 10:52 To: dev@flink.apache.org

Re: Flink 1.6.4 Issue on RocksDB incremental checkpoints and fs.default-scheme

2019-07-03 Thread Yun Tang
Hi Andrea This should be a bug already fixed by https://issues.apache.org/jira/browse/FLINK-12042 , you could upgrade to at least 1.7.3 version to avoid this bug. Best Yun Tang From: Andrea Spina Sent: Wednesday, July 3, 2019 15:46 To: dev@flink.apache.org

Re: Flink 1.6.4 Issue on RocksDB incremental checkpoints and fs.default-scheme

2019-07-01 Thread Yun Tang
welcome. [1] https://github.com/apache/flink/blob/6f4148180ba372a2c12c1d54bea8579350af6c98/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L2568 Best Yun Tang From: Andrea Spina

Re: Flink 1.6.4 Issue on RocksDB incremental checkpoints and fs.default-scheme

2019-07-01 Thread Yun Tang
please provide your job manager's log. Best Yun Tang From: Andrea Spina Sent: Monday, July 1, 2019 23:14 To: dev@flink.apache.org Subject: Re: Flink 1.6.4 Issue on RocksDB incremental checkpoints and fs.default-scheme Hi Yun, rocksDB configuration is set as follows

Re: [VOTE] Flink Project Bylaws

2019-08-13 Thread Yun Tang
tributing/contribute-code.html#code-contribution-process Best Yun Tang From: JingsongLee Sent: Tuesday, August 13, 2019 23:56 To: dev Subject: Re: [VOTE] Flink Project Bylaws +1 (non-binding) Thanks Becket. I've learned a lot from current bylaws.

Re: Something wrong with travis?

2019-07-30 Thread Yun Tang
I met this problem again at https://api.travis-ci.com/v3/job/220732163/log.txt . Is there any place we could ask for help to contact tarvis or any clues we could use to figure out this? Best Yun Tang From: Yun Tang Sent: Monday, June 24, 2019 14:22 To: dev

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-15 Thread Yun Tang
make me feel fear about the OOM errors. Look forward for more powerful enrichment to state-backend, and help Flink to achieve better performance together. Best Yun Tang From: Stephan Ewen Sent: Thursday, August 15, 2019 23:07 To: dev Subject: Re: [DISCUSS] FLIP-50

Re: [DISCUSS] Contribute Pulsar Flink connector back to Flink

2019-09-03 Thread Yun Tang
source [1] and sink [2] connector? Will the contribution contains part of those PRs or totally different implementation? [1] https://issues.apache.org/jira/browse/FLINK-9641 [2] https://issues.apache.org/jira/browse/FLINK-9168 Best Yun Tang From: Yijie Shen Sent

Re: CiBot Update

2019-09-01 Thread Yun Tang
template and come here for the command line. Best Yun Tang From: Congxian Qiu Sent: Monday, August 26, 2019 20:08 To: dev@flink.apache.org Subject: Re: CiBot Update Thanks Chesnay for the nice work, it's very helpful Best, Congxian Terry Wang 于2019年8月26日周一 下午6:59写道:

Re: instable checkpointing after migration to flink 1.8

2019-09-05 Thread Yun Tang
main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java#L453 [2] https://github.com/apache/flink/blob/ba1cd43b6ed42070fd9013159a5b65adb18b3975/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/snapshot/RocksIncrementalS

Re: instable checkpointing after migration to flink 1.8

2019-09-11 Thread Yun Tang
/projects/flink/flink-docs-stable/dev/stream/operators/process_function.html#timer-coalescing Best Yun Tang From: Bekir Oguz Sent: Wednesday, September 11, 2019 19:39 To: Yun Tang Cc: dev@flink.apache.org ; Stephan Ewen ; Niels Alebregtse ; Vladislav Bakayev Subject

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

2019-09-11 Thread Yun Tang
Congratulations Zili Best Yun Tang From: Yun Gao Sent: Thursday, September 12, 2019 10:12 To: dev Subject: Re: [ANNOUNCE] Zili Chen becomes a Flink committer Congratulations Zili! Best, Yun

Re: [DISCUSS] Support customize state in customized KeyedStateBackend

2019-09-08 Thread Yun Tang
/flink/runtime/state/heap/HeapKeyedStateBackend.java#L79 [2] https://github.com/apache/flink/blob/576228651382db040aaa006cf9142f6568930cb1/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L114 Best Yun Tang

Re: [ANNOUNCE] Hequn becomes a Flink committer

2019-08-07 Thread Yun Tang
Congratulations Hequn. Best Yun Tang From: Rong Rong Sent: Thursday, August 8, 2019 0:41 Cc: dev ; user Subject: Re: [ANNOUNCE] Hequn becomes a Flink committer Congratulations Hequn, well deserved! -- Rong On Wed, Aug 7, 2019 at 8:30 AM mailto:xingc

Re: Custom type serializer inside Flink state

2019-08-07 Thread Yun Tang
[1] for more information. CC @Tzu-Li (Gordon) Tai<mailto:tzuli...@apache.org> as he might provide more information about this. [1] https://issues.apache.org/jira/browse/FLINK-11333 Best Yun Tang From: Ying Xu Sent: Thursday, August 8, 2019 1:51 T

Re: [ANNOUNCE] Kete Young is now part of the Flink PMC

2019-07-23 Thread Yun Tang
Congratulations Kurt, Well deserved. Best From: Xu Forward Sent: Tuesday, July 23, 2019 17:53 To: dev Subject: Re: [ANNOUNCE] Kete Young is now part of the Flink PMC Congratulations Kurt! best forward Jark Wu 于2019年7月23日周二 下午5:49写道: > Congratulations

Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project

2019-07-22 Thread Yun Tang
Congratulations Zhijiang, well deserved. Best From: Yingjie Cao Sent: Tuesday, July 23, 2019 10:23 To: dev@flink.apache.org Subject: Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project Congratulations Zhijiang! yangtao.yt

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

2019-07-18 Thread Yun Tang
Congratulations Becket! Well deserved. Best Yun Tang From: Robert Metzger Sent: Thursday, July 18, 2019 15:41 To: dev Subject: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project Hi all, I'm excited to announce that Jiangjie

Re: [DISCUSS] Setup a bui...@flink.apache.org mailing list for travis builds

2019-07-21 Thread Yun Tang
+1 sounds good, more people could be encouraged to involve in paying attention to failure commit. Best Yun Tang From: Becket Qin Sent: Monday, July 22, 2019 9:44 To: dev Subject: Re: [DISCUSS] Setup a bui...@flink.apache.org mailing list for travis builds +1

Re: Checkpoint in FlinkSQL

2019-11-04 Thread Yun Tang
://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#checkpointing Best Yun Tang From: Simon Su Date: Tuesday, November 5, 2019 at 10:38 AM To: dev , user Subject: Checkpoint in FlinkSQL Hi All Does current Flink support to set checkpoint properties while using Flink SQL ? For example

Re: RocksDB state on HDFS seems not being cleanned up

2019-11-05 Thread Yun Tang
/jira/browse/FLINK-13390 [4] https://github.com/facebook/rocksdb/blob/834feaff05a4bf7ae49c736305d5eb180aed4011/include/rocksdb/compaction_filter.h#L61 Best Yun Tang From: shuwen zhou Date: Wednesday, November 6, 2019 at 12:02 PM To: dev , Yun Tang , Till Rohrmann Subject: Re: RocksDB state

Re: RocksDB state on HDFS seems not being cleanned up

2019-11-05 Thread Yun Tang
state ttl configure as `cleanupIncrementally()`, moreover, what is the actual meaning of "removed automatically"? [1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/state/state.html#cleanup-in-background Best Yun Tang On 11/5/19, 11:24 PM, "Till Rohrman

Re: flink checkpoint state data corruption

2019-11-14 Thread Yun Tang
Hi Jeffrey Thanks for your willingness to try this un-merged PR. Would you please share your exception stack and your proto file? If you could just summed up a job which could be able to reproduce this bug and share this to me, that would be better to fix this problem. Best Yun Tang From

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

2019-11-09 Thread Yun Tang
Congratulations, Jark Best Yun Tang On 11/10/19, 10:40 AM, "vino yang" wrote: Congratulations, Jark. Best, Vino Leonard Xu 于2019年11月8日周五 下午8:40写道: > Congratulations, Jark. > Thanks for your contribution and help. > >

Re: [DISCUSSION] Kafka Metrics Reporter

2019-11-18 Thread Yun Tang
these metrics data. [1] https://www.alibabacloud.com/help/doc-detail/29003.htm Best Yun Tang On 11/18/19, 8:19 AM, "Becket Qin" wrote: Hi Gyula, Thanks for bringing this up. It is a useful addition to have a Kafka metrics reporter. I understand that we already have

Re: Cron style for checkpoint

2019-11-21 Thread Yun Tang
Hi Shuwen Conceptually, checkpoints in Flink behaves more like a system mechanism to achieve fault tolerance and transparent for users. On the other hand, savepoint in Flink behaves more like a user control behavior, can savepoint not satisfy your demands for crontab? Best Yun Tang From

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

2019-12-05 Thread Yun Tang
found we cannot view the logs of some IT cases which would be uploaded by traivs_watchdog to transfer.sh previously. I think this feature is also easy to implement in AZP, right? Best Yun Tang On 12/6/19, 12:19 AM, "Robert Metzger" wrote: I've created a first draft of my plans i

Re: [DISCUSS] Drop Heap Backend Synchronous snapshots

2019-12-05 Thread Yun Tang
+1 from my side for I did not see any real benefits if using synchronous snapshots. Moreover, I think we should also remove the support of synchronous snapshots in DefaultOpeatorStateBackend and deprecate the config state.backend.async Best Yun Tang On 12/5/19, 8:06 PM, "Stephan Ewen&qu

Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer

2019-12-15 Thread Yun Tang
Congratulations ZZ Best Yun Tang From: Guowei Ma Sent: Monday, December 16, 2019 11:15 To: dev Subject: Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer Congrats Zhuzhu! Best, Guowei Zhenghua Gao 于2019年12月16日周一 上午10:47写道: > Congrats! > > *Bes

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

2019-10-28 Thread Yun Tang
Congratulations, Becket Best Yun Tang From: Rong Rong Sent: Tuesday, October 29, 2019 0:19 To: dev Cc: Becket Qin Subject: Re: [ANNOUNCE] Becket Qin joins the Flink PMC Congratulations Becket!! -- Rong On Mon, Oct 28, 2019, 7:53 AM Jark Wu wrote

Re: set job level TTL

2019-11-24 Thread Yun Tang
Hi Chen Currently, state TTL setting is per-state scoped, and there is no job-level TTL setting so far. Do you want similar settings in flink-conf to enable Flink SQL could also benefit from state TTL? Best Yun Tang On 11/25/19, 12:16 AM, "Chen Qin" wrote: Hi there,

Re: [DISCUSS] Remove old WebUI

2019-11-24 Thread Yun Tang
+1, I like the new UI On 11/24/19, 11:13 PM, "Hequn Cheng" wrote: +1 to drop the old UI. On Sun, Nov 24, 2019 at 9:43 PM Jeff Zhang wrote: > +1 > > > Jark Wu 于2019年11月24日周日 下午8:46写道: > > > +1 to drop the old UI. > > > > On Fri, 22 Nov 2019 at

Re: set job level TTL

2019-11-25 Thread Yun Tang
could also achieve #1 but leave window state not limited to TTL. I think this deserves an issue to track this request. Best Yun Tang On 11/25/19, 8:04 AM, "Chen Qin" wrote: I am a bit confused. Job level TTL is different from state TTL despite share same name. Wh

Re: [DISCUSS] FLIP-83: Flink End-to-end Performance Testing Framework

2019-11-01 Thread Yun Tang
+1, I like the idea of this improvement which acts as a watchdog for developers' code change. By the way, do you think it's worthy to add a checkpoint mode which just disable checkpoint to run end-to-end jobs? And when will stage2 and stage3 be discussed in more details? Best Yun Tang On 11

Re: Best coding practises guide while programming using flink apis

2019-10-10 Thread Yun Tang
an important role. [1] https://flink.apache.org/contributing/code-style-and-quality-preamble.html [2] https://ci.apache.org/projects/flink/flink-docs-stable/concepts/programming-model.html#levels-of-abstraction Best Yun Tang From: Deepak Sharma Sent: Monday

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-10 Thread Yun Tang
Hi Arvid +1 for this future which has been hoped for a long time. End-to-end exactly once job could benefit from quicker checkpoint completion. Best Yun Tang From: Yun Gao Sent: Thursday, October 10, 2019 18:39 To: dev Subject: Re: [DISCUSS] FLIP-76

Re: [VOTE] Release 1.9.1, release candidate #1

2019-10-13 Thread Yun Tang
I think FLINK-14385<https://issues.apache.org/jira/browse/FLINK-14385> is not introduced from release-1.9 but since when we decide to release hadoop-free Flink form 1.8 [1] [1] https://issues.apache.org/jira/browse/FLINK-11266 Best Yun Tang From: Hequn

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-26 Thread Yun Tang
Congratulations and well deserved! Best Yun Tang From: Canbin Zheng Sent: Monday, February 24, 2020 16:07 To: dev Subject: Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer Congratulations !! Dawid Wysakowicz 于2020年2月24日周一 下午3:55写道: > Congratulati

Re: [VOTE] Integrate Flink Docker image publication into Flink release process

2020-01-28 Thread Yun Tang
+1 (non-binding) From: Stephan Ewen Sent: Tuesday, January 28, 2020 21:36 To: dev ; patr...@ververica.com Subject: Re: [VOTE] Integrate Flink Docker image publication into Flink release process +1 On Tue, Jan 28, 2020 at 2:20 PM Patrick Lucas wrote: > Thanks

Re: [DISCUSS] Make AppendingState#add refuse to add null element

2020-01-15 Thread Yun Tang
le/dev/stream/operators/windows.html#reducefunction Best Yun Tang From: Yu Li Sent: Thursday, January 9, 2020 14:09 To: dev Subject: Re: [DISCUSS] Make AppendingState#add refuse to add null element +1 for unifying the behavior to refusing adding null element. Nice

Re: Flink 1.6, increment Checkpoint, the shared dir stored the last year checkpoint state

2020-01-19 Thread Yun Tang
be removed by checkpoint coordinator but takes too long to complete before job shut down. 3. This file is still useful. This is possible in theory because some specific rocksDB sst file might not be selected during compactions for a long time. Best Yun Tang From

Re: [DISCUSS] Add N-Ary Stream Operator

2020-01-13 Thread Yun Tang
Hi I noticed that previous design doc [1] also talked about the topic of introducing new KeyedStreamOperatorNG, I wonder is that a must-do to introduce N-ary stream operator? [1] https://docs.google.com/document/d/1ZFzL_0xGuUEnBsFyEiHwWcmCcjhd9ArWsmhrgnt05RI Best Yun Tang

Re: [ANNOUNCE] Dian Fu becomes a Flink committer

2020-01-16 Thread Yun Tang
Congratulations, Dian! Best Yun Tang From: Benchao Li Sent: Thursday, January 16, 2020 22:27 To: Congxian Qiu Cc: dev@flink.apache.org ; Jingsong Li ; jincheng sun ; Shuo Cheng ; Xingbo Huang ; Wei Zhong ; Hequn Cheng ; Leonard Xu ; Jeff Zhang ; user ; user

Re: [DISCUSS] Change default for RocksDB timers: Java Heap => in RocksDB

2020-01-16 Thread Yun Tang
Hi Stephan, I am +1 for the change which stores timers in RocksDB by default. Some users hope the checkpoint could be completed as fast as possible, which also need the timer stored in RocksDB to not affect the sync part of checkpoint. Best Yun Tang From

Re: [DISCUSS] Make AppendingState#add refuse to add null element

2020-01-16 Thread Yun Tang
@Aljoscha Krettek<mailto:aljos...@apache.org> , got it and thanks for your clarification. Best Yun Tang 获取 Outlook for Android<https://aka.ms/ghei36> From: Aljoscha Krettek Sent: Thursday, January 16, 2020 8:40:57 PM To: dev@flink.apache.org

Re: Why aren't Range Scans Supported in MapState?

2020-01-06 Thread Yun Tang
* <https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/typeutils/ordered/OrderedBytes.java> [2] https://github.com/facebook/rocksdb/pull/3334 Best Yun Tang * From: Aaron Langford Sent: Tuesday, Jan

Re: [ANNOUNCE] Apache Flink Stateful Functions 2.0.0 released

2020-04-08 Thread Yun Tang
Excited to see the stateful functions release! Thanks for the great work of manager Gordon and everyone who ever contributed to this. Best Yun Tang From: Till Rohrmann Sent: Wednesday, April 8, 2020 14:30 To: dev Cc: Oytun Tez ; user Subject: Re: [ANNOUNCE

[DISCUSS] Creating a new repo to host Flink benchmarks

2020-04-09 Thread Yun Tang
[4], which is under GPLv2 license. What do you think? Best, Yun Tang [1] https://github.com/dataArtisans/flink-benchmarks [2] https://issues.apache.org/jira/browse/FLINK-16850 [3] https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release [4] https://openjdk.java.net/projects/code

Re: Configuring autolinks to Flink JIRA ticket in github repos

2020-04-09 Thread Yun Tang
in the pull requests. e.g FLINK-16850 could point to the link instead of [FLINK-16850](https://issues.apache.org/jira/browse/FLINK-16850) [1] https://github.com/apache/flink/commits/master Best Yun Tang From: Till Rohrmann Sent: Thursday, April 2, 2020 23:11

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

2020-04-14 Thread Yun Tang
Apache project. Best Yun Tang From: Yu Li Sent: Friday, April 10, 2020 21:03 To: dev Subject: Re: [DISCUSS] Creating a new repo to host Flink benchmarks +1 for the proposal, thanks for driving this Yun. Best Regards, Yu On Fri, 10 Apr 2020 at 11:25, Dian Fu

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

2020-04-20 Thread Yun Tang
Thanks for Gordon's help! We will move on and create FLINK-17280<https://issues.apache.org/jira/browse/FLINK-17280> [1] to track the progress. [1] https://issues.apache.org/jira/browse/FLINK-17280 Best Yun Tang From: Tzu-Li (Gordon) Tai Sent: Monday,

Re: [PROPOSAL] Contribute training materials to Apache Flink

2020-04-15 Thread Yun Tang
could be run well as expected. Best Yun Tang From: Till Rohrmann Sent: Wednesday, April 15, 2020 16:08 To: dev Cc: Eduardo Winpenny Tejedor ; Seth Wiesman ; Niels Basjes ; user Subject: Re: [PROPOSAL] Contribute training materials to Apache Flink Hi David

Re: [ANNOUNCE] New Apache Flink PMC Member - Hequn Chen

2020-04-17 Thread Yun Tang
Congratulations , Hequn Best Yun Tang From: Guowei Ma Sent: Friday, April 17, 2020 18:27 To: dev Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Hequn Chen Congratulations! Best, Guowei Xintong Song 于2020年4月17日周五 下午5:21写道: > Congratulations, He

Re: [ANNOUNCE] New Committers and PMC member

2020-04-01 Thread Yun Tang
Congratulations to all of you! Best Yun Tang From: Yang Wang Sent: Wednesday, April 1, 2020 22:28 To: dev Subject: Re: [ANNOUNCE] New Committers and PMC member Congratulations all. Best, Yang Leonard Xu 于2020年4月1日周三 下午10:15写道: > Congratulations Konstan

Configuring autolinks to Flink JIRA ticket in github repos

2020-04-01 Thread Yun Tang
github repos. Hope it could be resolved soon  [1] https://help.github.com/en/github/administering-a-repository/configuring-autolinks-to-reference-external-resources [2] https://issues.apache.org/jira/browse/INFRA-20055 Best Yun Tang

Re: [GitHub] [flink-web] klion26 commented on pull request #247: [FLINK-13683] Translate "Code Style - Component Guide" page into Chinese

2020-04-30 Thread Yun Tang
Has anyone known why this thread could occur in development related discussions at Flink? From: GitBox Sent: Thursday, April 30, 2020 15:55 To: dev@flink.apache.org Subject: [GitHub] [flink-web] klion26 commented on pull request #247: [FLINK-13683] Translate

Re: [DISCUSS] Align the behavior of internal return result of MapState#entries, keys, values and iterator.

2020-05-11 Thread Yun Tang
Thanks for everyone's positive response. FLINK-17610 [1] have been created to track this problem. [1] https://issues.apache.org/jira/browse/FLINK-17610 Best Yun Tang From: Congxian Qiu Sent: Monday, May 11, 2020 13:42 To: dev@flink.apache.org Subject: Re

Re: What is the RocksDB local directory in flink checkpointing?

2020-05-08 Thread Yun Tang
strapTools.java#L478-L489 Best Yun Tang From: Till Rohrmann Sent: Wednesday, May 6, 2020 17:35 To: LakeShen Cc: dev ; user ; user-zh Subject: Re: What is the RocksDB local directory in flink checkpointing? Hi LakeShen, `state.backend.rocksdb.localdir` defines th

Re: [DISCUSS] increase "state.backend.fs.memory-threshold" from 1K to 100K

2020-05-15 Thread Yun Tang
much more confident to increase this configuration as the memory footprint expands at that time by a lot of serialized TDDs. [1] https://github.com/apache/flink/blob/32bd0944d0519093c0a4d5d809c6636eb3a7fc31/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java#L752 Bes

Re: [DISCUSS] Exact feature freeze date

2020-05-15 Thread Yun Tang
+1 for Monday morning in Europe. Best Yun Tang From: Jingsong Li Sent: Friday, May 15, 2020 21:17 To: dev Subject: Re: [DISCUSS] Exact feature freeze date +1 for Monday morning. Best, Jingsong Lee On Fri, May 15, 2020 at 8:45 PM Till Rohrmann wrote: >

Re: [DISCUSS] increase "state.backend.fs.memory-threshold" from 1K to 100K

2020-05-16 Thread Yun Tang
Exception { while (running) { synchronized (ctx.getCheckpointLock()) { ctx.collect(ThreadLocalRandom.current().nextInt()); } Thread.sleep(100); } } @Override public void cancel() { running =

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Yun Tang
+1 (non-binding) Yun Tang From: Benchao Li Sent: Saturday, May 16, 2020 22:00 To: dev Subject: Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v) +1 (non-binding) Piotr Nowojski 于2020年5月

Re: [DISCUSS] increase "state.backend.fs.memory-threshold" from 1K to 100K

2020-05-14 Thread Yun Tang
/main/java/org/apache/flink/runtime/checkpoint/RoundRobinOperatorStateRepartitioner.java#L64-L87 Best Yun Tang From: Yu Li Sent: Thursday, May 14, 2020 23:51 To: Till Rohrmann Cc: dev ; Piotr Nowojski Subject: Re: [DISCUSS] increase "state.backend.fs.m

Re: [DISCUSS] Stability guarantees for @PublicEvolving classes

2020-05-14 Thread Yun Tang
to collect enough feedbacks, and upgrading API stable level every 3 major versions should be better. [1] https://flink.apache.org/downloads.html#flink Best Yun Tang From: Xintong Song Sent: Friday, May 15, 2020 11:04 To: dev Subject: Re: [DISCUSS] Stability

Re: Question about RocksDBStateBackend Compaction Filter state cleanup

2020-03-17 Thread Yun Tang
Yun Tang From: LakeShen Sent: Tuesday, March 17, 2020 15:30 To: dev ; user-zh ; user Subject: Question about RocksDBStateBackend Compaction Filter state cleanup Hi community , I see the flink RocksDBStateBackend state cleanup,now the code like

[DISCUSS] Align the behavior of internal return result of MapState#entries, keys, values and iterator.

2020-05-09 Thread Yun Tang
for users. I suggest to align these internal behavior to return empty iterator instead of null. What do you think? [1] https://issues.apache.org/jira/browse/FLINK-17015 Best Yun Tang

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

2020-09-09 Thread Yun Tang
live without checkpointCoordinator on JM side while 'StreamTask' always initialize the 'checkpointStorage' on task side. I think JM knows the inner relationship between state backend and checkpoint while TM seems mix them together. Best Yun Tang From: Konstantin

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-09 Thread Yun Tang
Congratulations, Zhu! Best Yun Tang From: Danny Chan Sent: Friday, October 9, 2020 13:51 To: dev@flink.apache.org Subject: Re: [ANNOUNCE] New PMC member: Zhu Zhu Congrats, Zhu Zhu ~ Best, Danny Chan 在 2020年10月9日 +0800 PM1:05,dev@flink.apache.org,写道

[DISCUSS][docker] Adopt Jemalloc as default memory allocator for debian based Flink docker image

2020-10-13 Thread Yun Tang
.io/manual/v/1.0/installation/docker#why-there-is-no-fluent-bit-docker-image-based-on-alpine-linux [6] https://github.com/home-assistant/core/pull/33237 Best Yun Tang

Re: [DISCUSS][docker] Adopt Jemalloc as default memory allocator for debian based Flink docker image

2020-10-16 Thread Yun Tang
fragmentation problem. Does anyone else have other ideas? Best Yun Tang From: Yang Wang Sent: Thursday, October 15, 2020 14:59 To: dev Subject: Re: [DISCUSS][docker] Adopt Jemalloc as default memory allocator for debian based Flink docker image Thanks Yun Tang

Re: [ANNOUNCE] Apache Flink 1.10.2 released

2020-08-25 Thread Yun Tang
Thanks for Zhu's work to manage this release and everyone who contributed to this! Best, Yun Tang From: Yangze Guo Sent: Tuesday, August 25, 2020 14:47 To: Dian Fu Cc: Zhu Zhu ; dev ; user ; user-zh Subject: Re: [ANNOUNCE] Apache Flink 1.10.2 released Thanks

Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-28 Thread Yun Tang
://issues.apache.org/jira/browse/FLINK-19035 Best, Yun Tang From: Aljoscha Krettek Sent: Thursday, August 27, 2020 16:45 To: dev@flink.apache.org Subject: Re: [DISCUSS] Removing deprecated methods from DataStream API Did you consider DataStream.project() yet

Re: [ANNOUNCE] New PMC member: Dian Fu

2020-08-27 Thread Yun Tang
Congratulations , Dian! Best Yun Tang From: Yang Wang Sent: Friday, August 28, 2020 10:28 To: Arvid Heise Cc: Benchao Li ; dev ; user-zh ; Dian Fu ; user Subject: Re: [ANNOUNCE] New PMC member: Dian Fu Congratulations Dian ! Best, Yang Arvid Heise

Re: [VOTE] Release 1.10.2, release candidate #2

2020-08-20 Thread Yun Tang
I checked to build from source code and run example on YARN and they all behave as expected. However, I think we should tell user explicitly for the interface changes to fix FLINK-18242 and already leave comments for the flink-web PR. Best Yun Tang From

Re: [VOTE] Release 1.10.2, release candidate #2

2020-08-20 Thread Yun Tang
Thanks for Zhu Zhu's quick response and LGTM now, +1 (non-binding) Best Yun Tang From: Zhu Zhu Sent: Thursday, August 20, 2020 19:39 To: dev ; Yun Tang Subject: Re: [VOTE] Release 1.10.2, release candidate #2 Thanks for pointing it out! @Yun Tang<mailto:m

Re: [ANNOUNCE] New Apache Flink Committer - Yun Tang

2020-09-17 Thread Yun Tang
Thanks for all your kind welcome and very glad to be one of the committers of Flink community. Best Yun Tang From: Congxian Qiu Sent: Wednesday, September 16, 2020 13:10 To: dev@flink.apache.org Cc: Zhijiang ; tangyun ; Yun Tang Subject: Re: [ANNOUNCE] New

Re: CheckpointedFunction initialization during checkpoint

2020-10-03 Thread Yun Tang
.java#L496 Best Yun Tang From: Aljoscha Krettek Sent: Tuesday, September 29, 2020 20:51 To: dev@flink.apache.org Subject: Re: CheckpointedFunction initialization during checkpoint Hi Teng, I think if the system is slowed down enough it can happen that some parts of

Re: [DISCUSS] Semantics of our JIRA fields

2020-05-25 Thread Yun Tang
ommunities. If so, I think we should provide this notice in the official web-site, JIRA template or any other place instead of just dev mail list to reach developer/users. [1] https://www.atlassian.com/agile/tutorials/versions Best Yun Tang From: Chesnay Schepler Se

Re: [DISCUSS] Switch to Azure Pipelines as the primary CI tool / switch off Travis

2020-05-25 Thread Yun Tang
I noticed that there still existed travis related bot commands in the github PR page, and I think we should remove the command hint now. From: Robert Metzger Sent: Thursday, April 23, 2020 15:44 To: dev Subject: Re: [DISCUSS] Switch to Azure Pipelines as the

Re: [DISCUSS] Switch to Azure Pipelines as the primary CI tool / switch off Travis

2020-05-25 Thread Yun Tang
If we still need to accept PRs for Flink-1.9/1.10, that could explain why we still need that command hint. Chesnay, thanks for your explanation. From: Chesnay Schepler Sent: Monday, May 25, 2020 18:17 To: dev@flink.apache.org ; Yun Tang Subject: Re: [DISCUSS

Re: Questions around MemoryStateBackend size limits

2020-06-02 Thread Yun Tang
/apache/flink/runtime/state/memory/MemCheckpointStreamFactory.java#L62-L69 [2] https://github.com/apache/flink/blob/1c78ab397de524836fd69c6218b1122aa387c251/flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskStateManagerImpl.java#L117-L122 Best Yun Tang

Re: [DISCUSS] (Document) Backwards Compatibility of Savepoints

2020-06-02 Thread Yun Tang
this in documentation to say explicitly Flink does not guarantee such kind of backwards compatibility. Best Yun Tang From: Ufuk Celebi Sent: Wednesday, May 27, 2020 16:42 To: dev@flink.apache.org Subject: Re: [DISCUSS] (Document) Backwards Compatibility of Savepoints I

Re: Re: [ANNOUNCE] New Apache Flink Committer - Godfrey He

2020-09-16 Thread Yun Tang
Congratulations, Godfrey ~ Best Yun Tang From: Kurt Young Sent: Wednesday, September 16, 2020 17:02 To: dev Subject: Re: Re: [ANNOUNCE] New Apache Flink Committer - Godfrey He Congrats and welcome, Godfrey! Best, Kurt On Wed, Sep 16, 2020 at 4:59 PM Xintong

Re: [ANNOUNCE] New Apache Flink Committer - Arvid Heise

2020-09-16 Thread Yun Tang
Congratulations , Arvid  Best Yun Tang From: Yuan Mei Sent: Wednesday, September 16, 2020 14:27 To: dev Subject: Re: [ANNOUNCE] New Apache Flink Committer - Arvid Heise Congrats, Arvid! ☺ On Wed, Sep 16, 2020 at 12:37 PM Guowei Ma wrote: > Congratulati

Re: [DISCUSS] Disable "Squash and merge" button for Flink repository on GitHub

2020-10-26 Thread Yun Tang
-GitHub/Authorship-of-merge-commits-made-by-Github-Apps-changed/td-p/48797/highlight/false/page/3 Best Yun Tang From: Jark Wu Sent: Monday, March 9, 2020 16:02 To: dev Subject: Re: [DISCUSS] Disable "Squash and merge" button for Flink repository on GitHub Hi

Re: Flink Redis connectivity

2020-07-21 Thread Yun Tang
apache/bahir-flink Best Yun Tang From: Yangze Guo Sent: Tuesday, July 21, 2020 18:50 To: dev Subject: Re: Flink Redis connectivity Hi, I think you could implement `RichMapFunction` and create `redisClient` in the `open` method. Best, Yangze Guo On Tue, Jul 21

Re: [DISCUSS] Planning Flink 1.12

2020-08-09 Thread Yun Tang
+1 for extending the feature freeze due date. From: Zhijiang Sent: Thursday, August 6, 2020 17:05 To: dev Subject: Re: [DISCUSS] Planning Flink 1.12 +1 on my side for feature freeze date by the end of Oct.

Re: Check pointing for simple pipeline

2020-07-07 Thread Yun Tang
Hi Prasanna Using incremental checkpoint is always better than not as this is faster and less memory consumed. However, incremental checkpoint is only supported by RocksDB state-backend. Best Yun Tang From: Prasanna kumar Sent: Tuesday, July 7, 2020 20:43

  1   2   3   4   5   6   >