Re: [DISCUSS] TravisCI status on GitHub Page

2017-03-20 Thread Bowen Li
I would argue for benefits of having build status. Instead of letting people go through all docs and wikis to find how Flink build system works, it guides people directly to where builds actually happen and ramps up new contributors faster. When my local tests fail during development, the

Re: [POLL] Who still uses Java 7 with Flink ?

2017-03-16 Thread Bowen Li
There's always a tradeoff we need to make. I'm in favor of upgrading to Java 8 to bring in all new Java features. The common way I've seen (and I agree) other software upgrading major things like this is 1) upgrade for next big release without backward compatibility and notify everyone 2)

Re: [DISCUSS] TravisCI status on GitHub Page

2017-03-21 Thread Bowen Li
ok, fair enough On Tue, Mar 21, 2017 at 11:00 AM, Stephan Ewen <se...@apache.org> wrote: > @Bowen Li > > There are various discussions currently about reworking repository > structure, tests, even switching CI services. > I would be up for revisiting this questions o

Re: How to register and send custom metrics from Flink?

2017-03-13 Thread Bowen Li
nce you create a metric for the flatMap > operation the metric will only > be exposed as long as the flatMap operation is active. Thus, if this > operation takes less than 10 seconds > (the default interval for all reporters) then it may never report it. > > Regards, > Chesnay > &

How to register and send custom metrics from Flink?

2017-03-13 Thread Bowen Li
Hi guys, I'm retrying to send some app related custom metrics from Flink to Datadog via StatsD. I followed https://ci.apache.org/projects/flink/flink-docs- release-1.2/monitoring/metrics.html to set up flink-conf.yaml and test code like this // flink-conf.yaml metrics.reporters:

Which branch should I build for Flink 1.3.2

2017-08-11 Thread Bowen Li
Hi guys, I'm trying to build Flink 1.3.2 myself from source code, and which branch I should build from? 'release-1.3' or 'release-1.3.2-rc3'? I think it should be the former one, but want you confirm on it. Thanks, Bowen

Re: Which branch should I build for Flink 1.3.2

2017-08-11 Thread Bowen Li
nk.apache.org/downloads.html#source > > On Fri, Aug 11, 2017 at 2:53 PM, Bowen Li <bowen...@offerupnow.com> wrote: > > > Hi guys, > > I'm trying to build Flink 1.3.2 myself from source code, and which > > branch I should build from? 'release-1.3' or 'release-1.3.2-rc3'? I think > > it should be the former one, but want you confirm on it. > > > > Thanks, > > Bowen > > >

Re: [DISCUSS] Release 1.3.2 planning

2017-07-13 Thread Bowen Li
Hi Aljoscha, I'd like to see https://issues.apache.org/jira/browse/FLINK-6951 fixed in 1.3.2, if it makes sense. Thanks, Bowen On Wed, Jul 12, 2017 at 3:06 AM, Aljoscha Krettek wrote: > Short update, we resolved some blockers and discovered some new ones. > There’s

Re: [POLL] Who still uses Java 7 with Flink ?

2017-07-12 Thread Bowen Li
5274018=com.atlassian.jira. > >> > > plugin.system.issuetabpanels:comment-tabpanel#comment-15274018 > >> > > > > > >> > > > > > >> > > > > > >> > > > > On Thu, Mar 23, 2017 at 2:42 PM, Theodore Vasiloudis < &g

Re: [ANNOUNCE] New Flink PMC member: Tzu-Li (Gordon) Tai

2017-07-12 Thread Bowen Li
Congrats! On Wed, Jul 12, 2017 at 3:30 AM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Congrats Gordon! > > On Wed, Jul 12, 2017 at 5:29 AM, Evans Ye wrote: > > > Congrats Gordon! > > > > Tzu-Li (Gordon) Tai 於 2017年7月12日 週三,上午2:28寫道: > > > >

Re: Akka dissassociated

2017-07-19 Thread Bowen Li
The amount of resource assigned to JobManager looks fine. How much resource (CPU and memory) did you allocate for TaskManager? On Wed, Jul 19, 2017 at 3:07 AM, Stephan Ewen wrote: > Hi Greg! > > Akka disassociation means that the network connection between the > JobManager and

Re: Switch to Scala 2.11 as a default build profile

2017-06-28 Thread Bowen Li
+1. AWS EMR eco system is using Scala 2.11, and breaks with Scala 2.10. We had to build several Flink components (e.g. flink-kinesis-connector) ourselves in order to run on EMR. Defaulting to Scala 2.11 will greatly reduce adoption cost for Flink on EMR On Wed, Jun 28, 2017 at 9:34 AM, Till

Re: Re: Switch to Scala 2.11 as a default build profile

2017-06-29 Thread Bowen Li
EMR's builtin Flink is always 1 or 2 versions behind Flink latest release. We choose to install Flink on EMR ourselves. On Thu, Jun 29, 2017 at 2:37 AM, 郭健 wrote: > +1 > Most of the time I need to build scala-2.11 version, although I can use > the script

Re: [DISCUSS] Feature Freeze

2017-04-27 Thread Bowen Li
Hi Ufuk, I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter) into release 1.3. It's in the final state of code review in https://github.com/apache/flink/pull/3736 Thanks, Bowen On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) < wangzhijiang...@aliyun.com> wrote: >

Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Bowen Li
Hi everyone, I read the doc, and found that we are not even testing Flink against Kinesis with *flink-connector-kinesis*. According to our experience, *flink-connector-kinesis *releases tend to have more issues w.r.t. dependency version conflicts, performance, capability, and stability. We've

Re: [RESULT][VOTE] Release 1.3.2, release candidate #3

2017-08-05 Thread Bowen Li
Hi Aljoscha, That'll be great. Look forward to a better Kinesis connector from Flink! Thanks, Bowen On Fri, Aug 4, 2017 at 11:07 PM, Aljoscha Krettek wrote: > The voting time has passed and I'm happy to announce that we've collected > enough votes to release this RC

Re: [ANNOUNCE] New Flink PMC member: Chesnay Schepler

2017-07-28 Thread Bowen Li
Congrats, Chesnay! On Fri, Jul 28, 2017 at 11:48 AM, N Sa wrote: > Congratulations Chesnay! > > On Jul 28, 2017 11:44 AM, "Aljoscha Krettek" wrote: > > Well deserved! Congrats. :-) > > > On 28. Jul 2017, at 17:45, Tzu-Li (Gordon) Tai

Which window function to use to start a window at anytime

2017-08-16 Thread Bowen Li
Hi guys, I sent this email to Flink user email alias but didn't get any feedbacks. I'm sending it here. We are trying use Flink to count millions of keyed items of an hour window hourly as `time(SlidingEventTimeWindows.of(1hour, 1hour))`. According to the sliding window doc

Re: Examples of Flink windowing unit tests, and option to bring in flink-spector to Flink

2017-05-03 Thread Bowen Li
chronous job API to listen on > Accumulator updates and wait until all required conditions have been met. > Then it would stop the job and the test would succeed. > > Best, > Aljoscha > > > On 2. May 2017, at 22:01, Bowen Li <bowen...@offerupnow.com> wrote: > > &g

Re: Incompatible Apache Http lib in Flink kinesis connector

2017-06-19 Thread Bowen Li
tpcore:jar:4.3.3:compile > > Checking dependency tree of flink, the highest version is 4.2.x > > You can try building flink with dependency on 4.3.y of httpclient / > httpcore > > FYI > > > > On Mon, Jun 19, 2017 at 4:52 PM, Bowen Li <bowen...@offerupnow.com> wrote:

Re: Incompatible Apache Http lib in Flink kinesis connector

2017-06-20 Thread Bowen Li
Guys, This is the PR https://github.com/apache/flink/pull/4150 On Tue, Jun 20, 2017 at 1:37 PM, Bowen Li <bowen...@offerupnow.com> wrote: > Hi Ted and Gordon, > I found the root cause and a solution. Basically > https://ci.apache.org/projects/flink/flink-docs- > release-

Re: Incompatible Apache Http lib in Flink kinesis connector

2017-06-20 Thread Bowen Li
httpclient5/src/main/java/org/apache/hc/client5/http/impl/ > DefaultConnectionKeepAliveStrategy.java > does contain INSTANCE. > > FYI > > On Mon, Jun 19, 2017 at 11:22 PM, Bowen Li <bowen...@offerupnow.com> > wrote: > > > Hi Gordon, > > I double c

Incompatible Apache Http lib in Flink kinesis connector

2017-06-19 Thread Bowen Li
Hi guys, I'm trying to enable Flink's checkpoint on our Flink app. I got the following Apache http jar compatibility error, and cannot figure out how to resolve it. Here's the stacktrace: ``` 2017-06-19 15:07:39,828 INFO org.apache.flink.runtime.taskmanager.Task

Examples of Flink windowing unit tests, and option to bring in flink-spector to Flink

2017-05-02 Thread Bowen Li
Hi Flink folks! Are there any concrete good examples of unit testing Flink windowing? I've been searching around, and all those available online are quite ambiguous. Another thing is, I found the flink-spector library is pretty awesome,

Re: Examples of Flink windowing unit tests, and option to bring in flink-spector to Flink

2017-05-03 Thread Bowen Li
egative condition is violated) and signal success by updating a unique > > Accumulator value. The test would use the asynchronous job API to listen > on > > Accumulator updates and wait until all required conditions have been met. > > Then it would stop the job and the test would succeed. &g

Re: [DISCUSS] Dropping Scala 2.10

2017-09-20 Thread Bowen Li
+1 for dropping support for Scala 2.10 On Tue, Sep 19, 2017 at 3:29 AM, Sean Owen wrote: > For the curious, here's the overall task in Spark: > > https://issues.apache.org/jira/browse/SPARK-14220 > > and most of the code-related changes: > >

Re: System resource logger

2017-10-05 Thread Bowen Li
System and processor info, marked as 'logged once' in gist shared by Piotr, should still be logged instead of registered as metrics, right? On Thu, Oct 5, 2017 at 2:38 AM, Till Rohrmann wrote: > Thanks for the proposal Piotr. I like it a lot since it will help people to >

Re: 回复:[DISCUSS] Releasing Flink 1.4

2017-10-13 Thread Bowen Li
+1. To give users a heads up, might be good to start to change the ‘target release’ of some Jira tickets from 1.4 to 1.5. Sent from my iPhone > On Oct 13, 2017, at 11:02 PM, Zhijiang(wangzhijiang999) > wrote: > > totally agree with the >

Re: [jira] [Created] (FLINK-7597) broken flink-connectors-kinesis setup in Intellij that potentially results from improper pom.xml

2017-09-07 Thread Bowen Li
t has a license that does not > allow us to build it by default :-( > (The Amazon Software License). > > Can you go to the maven profiles in IntelliJ and activate the profile to > include maven? That should help! > > Best, > Stephan > > > On Thu, Sep 7, 2017 at

Logistic Regression in Flink ML

2017-10-01 Thread Bowen Li
Hi guys, I checked that FlinkML still doesn't have a logistic regression model, and the following two PRs are waiting to be merged. - https://github.com/apache/flink/pull/3428 for https://issues.apache.org/jira/browse/FLINK-1743 - https://github.com/apache/flink/pull/3756 Can any committer

[Discuss] adding a new command line argument -sd/--savepointdirectory

2017-09-01 Thread Bowen Li
Hi guys, I want to start on discussion on FLINK-7566 if there's only one checkpointing metadata file in , `flink run -s ` should resume from that metadata file. We need a way to programmatically start a Flink job from a checkpoint without having

Re: Dependency convergence

2017-10-04 Thread Bowen Li
+1. This is great, Piotrek! BTW, can you clarify what you mean by 'project wide'? Is it the whole `flink` project or just `flink-connector-kafka`? I think it's useful to be applied to the whole flink project. I've seen dependencies conflict problem like this in flink-connector-kinesis. Enabling

Re: Unable to write snapshots to S3 on EMR

2017-10-06 Thread Bowen Li
Hi Andy, I believe it's because you didn't set your s3 impl correctly. Try to set your core-site.xml by following https://ci.apache.org/ projects/flink/flink-docs-release-1.4/ops/deployment/aws.html#s3afilesystem- recommended Bowen On Fri, Oct 6, 2017 at 7:59 AM, Andy M.

Clean up flink-contrib: Remove flink-streaming-contrib and migrate its class to other modules

2017-12-02 Thread Bowen Li
Hello Flink devs, I'd like to propose removinh of flink-contrib/flink-streaming-contrib and migrate its classes to flink-streaming-java/scala, for the following reasons: 1. flink-streaming-contrib is so small that has only 4 classes (3 java and 1 scala), which isn't worth a dedicated jar for

Re: [ANNOUNCE] Flink 1.4 feature-freeze date and release burn-down

2017-11-02 Thread Bowen Li
Hi Aljoscha, Thank you for taking on the releasing branch work! I'm wondering what's the status of it? And when will we open master branch for PRs of 1.5 features? Thanks! Bowen On Thu, Oct 19, 2017 at 2:17 AM, Aljoscha Krettek wrote: > Hi Everyone, > > It seems

Re: [ANNOUNCE] New committer: Haohui Mai

2017-11-01 Thread Bowen Li
Congratulations! On Wed, Nov 1, 2017 at 8:39 AM, Xingcan Cui wrote: > Congratulations! > > On Wed, Nov 1, 2017 at 9:37 PM, Kurt Young wrote: > > > Congrats and welcome on board! > > > > Best, > > Kurt > > > > On Wed, Nov 1, 2017 at 8:15 PM, Hai Zhou

Re: [VOTE] Release 1.4.0, release candidate #3

2017-12-07 Thread Bowen Li
lease branch, i.e. the 1.4 doc > on the website will update as soon as the doc on the release-1.4 branch is > updated. > > > On 6. Dec 2017, at 20:47, Bowen Li <bowen...@offerupnow.com> wrote: > > > > Hi Aljoscha, > > > > I found Flink's State doc a

Re: [VOTE] Release 1.4.0, release candidate #3

2017-12-06 Thread Bowen Li
Hi Aljoscha, I found Flink's State doc and javaDoc are very ambiguous on what the replacement of FoldingState is, which will confuse a lot of users. We need to fix it in 1.4 release. I have submitted a PR at https://github.com/apache/flink/pull/5129 Thanks, Bowen On Wed, Dec 6, 2017 at 5:56

Re: [Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

2018-05-14 Thread Bowen Li
w is that we have to distinguish > between user and TTL timers. AFAIK, the timer service does not support > timer tags (or another method) to distinguish timers. > > I've given you the permissions to create and edit wiki pages. > > Best, Fabian > > 2018-04-30 7:47 GMT+02:00 Bowen

Re: [VOTE] Release 1.5.0, release candidate #2

2018-05-10 Thread Bowen Li
Hi Till, I found that only file and kafka connectors are tested in the plan. @Gordon, shall we test the Kinesis connector? AFAIK, there're some major changes and AWS library upgrades in Flink 1.5. I would have tested it myself but I don't use Kinesis anymore. Thanks, Bowen On Thu, May 10,

Re: [ANNOUNCE] Two new committers: Xingcan Cui and Nico Kruber

2018-05-09 Thread Bowen Li
Congratulations! On Tue, May 8, 2018 at 11:06 PM, Zhijiang(wangzhijiang999) < wangzhijiang...@aliyun.com> wrote: > Congratulations, Xingcan and Nico ! > Nico is a good PR reviewer and I gained a lot from him. > :)--发件人:Fabian >

Re: KPL in current stable 1.4.2 and below, upcoming problem

2018-05-11 Thread Bowen Li
Thanks, this is a great heads-up! Flink 1.4 is using KPL 0.12.5, *so Flink version 1.4 or below will be effected.* Kinesis sink is in flink-kinesis-connector. Good news is that whoever is using flink-kinesis-connector right now is building it themself, because Flink doesn't publish that jar into

Re: CloudWatch Metrics Reporter

2018-05-16 Thread Bowen Li
To @Chesnay 's question on AWS library: yes, it would require using aws-java-sdk-core and aws-java-sdk-cloudwatch . Both are of Apache License 2.0. This is

Re: [PROPOSAL] Improving Flink’s timer management for large state

2018-05-27 Thread Bowen Li
+1 LGTM. RocksDB timer service is one of the most highly anticipated features from Flink users, and it's finally coming, officially. I also would love to see bringing timer more closely to state backend, for the sake of easier development and maintenance of code. On Fri, May 25, 2018 at 7:13 AM,

Re: [Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

2018-05-27 Thread Bowen Li
for exact TTL, I wonder if we can support a relax > TTL that could provides a better performance. Because to me, the reason > that I need TTL is just to prevent the state size growing infinitly(I > believe I'm not the only one like this), so a relax version is enough, if > there is a relax

[DISCUSS] Flink 1.4 and below STOPS writing to Kinesis after June 12th.

2018-06-06 Thread Bowen Li
1.5. Given that the public APIs don't change, flink-connector-kinesis in Flink 1.5 should work with Flink 1.4 and below, but that needs verification. What do you think? Thanks, Bowen -- Forwarded message -- From: Bowen Li Date: Fri, May 11, 2018 at 10:28 AM Subject: Re: KPL in curr

Re: [Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

2018-04-29 Thread Bowen Li
oc. >>> >>> In principle, I'm not sure if this functionality should be added to the >>> state backends. >>> We could also use the existing timer service which would have a few nice >>> benefits (see my comments in the docs). >>> >>&g

Re: Flink consuming more memory than expected

2018-05-03 Thread Bowen Li
Hi Gabriel, Yes, using RocksDB state backend can relieve your RAM usage. I see a few issues with your job: 1) it's keeping track of 672 windows (28x24), that's lots of data, so try to reduce number of windows 2) use reduce functions to incrementally aggregate state, rather than buffering data

Re: Improving Javadocs

2017-10-19 Thread Bowen Li
Hi Kent, A “[hotfix]” should be enough Sent from my iPhone > On Oct 20, 2017, at 1:58 AM, Kent Murra wrote: > > Does improving Javadocs require a JIRA or should I submit as a hotfix? > > In particular, I find the Javadocs for RichFunction.open and > RichFunction.close a bit

[SEATTLE MEETUP] Announcing First Seattle Apache Flink Meetup

2018-01-04 Thread Bowen Li
drinks will be provided. Plenty of onsite parking spots. DATE: Jan 17th, 2018, Wednesday TALKS: - Haitao Wang, Senior Staff Engineer at Alibaba, will give a presentation on large-scale streaming processing with Flink and Flink SQL at Alibaba and several internal use cases. - Bowen Li will talk

[Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

2018-02-05 Thread Bowen Li
Hi guys, I want to propose a new FLIP -- FLIP-25 - Support User State TTL Natively in Flink. This has been one of most handy and most frequently asked features in Flink community. The jira ticket is FLINK-3089 . I've written a rough design

Re: [DISCUSS] Move Flink Docker Files from Flink to 'docker-flink'

2018-02-21 Thread Bowen Li
+1. flink-contrib is an experiment land, docker-flink should graduate from it. This also helps to clean up flink-contrib/ On Wed, Feb 21, 2018 at 1:41 PM, Stephan Ewen wrote: > Hi all! > > I want to start the discussion to whether to drop the " >

Re: [ANNOUNCE] Apache Flink 1.4.1 released

2018-02-15 Thread Bowen Li
Congratulations everyone! On Thu, Feb 15, 2018 at 10:04 AM, Tzu-Li (Gordon) Tai wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.4.1, which is the first bugfix release for the Apache Flink 1.4 > series. > > > Apache Flink® is an

Re: [DISCUSS] Releasing Flink 1.5.0

2018-02-26 Thread Bowen Li
Hi guys, Can we get FLINK-8667 into 1.5.0? I've been waiting for it for quite a few days. The reason I really want to get it into 1.5.0 is because KeyedBroadcastProce ssFunction will be released in 1.5.0 and there's no compatibility issues. If we don't

Re: [ANNOUNCE] Apache Flink 1.5.2 released

2018-07-31 Thread Bowen Li
Congratulations, community! On Tue, Jul 31, 2018 at 1:44 AM Chesnay Schepler wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.5.2, which is the second bugfix release for the Apache Flink 1.5 > series. > > Apache Flink® is an open-source stream

Re: [DISCUSS] Add end-to-end tests using Docker Compose

2018-03-12 Thread Bowen Li
t; and "nightly-tests". The idea being > the the former are executed for each pull request, as they are now. The > latter are executed nightly or manually if you want to verify a release. > This way, the nightly tests can get quite involved without blowing up build > time. What

Re: [DISCUSS] Releasing Flink 1.3.3 with only critical fixes

2018-03-08 Thread Bowen Li
+1 On Thu, Mar 8, 2018 at 7:36 AM, Aljoscha Krettek wrote: > Hi, > > We fixed a critical bug in the ZooKeeper checkpoint store but never > released a 1.3.x version of Flink with the fix. I'm proposing to release > 1.3.3 with only fixes for this issue so that users can

Re: [DISCUSS] Add end-to-end tests using Docker Compose

2018-03-10 Thread Bowen Li
This would be nice. BTW, the [3] link is missing I have a few questions, given concerns that 1) the full build time will take longer and longer time and 2) build queue on Travis will be more congested - How much time will be added to full build time according to estimation? - Do we have

Re: [VOTE] Release 1.5.0, release candidate #1

2018-04-05 Thread Bowen Li
Hi Till, FLINK-8837 is marked as a blocker for release 1.5.0. I've opened a PR for it, can you please take a look at it? Bowen On Tue, Apr 3, 2018 at 8:45 AM, simone wrote: > > On 03/04/2018 17:40, simone wrote: >

Re: Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

2018-03-04 Thread Bowen Li
If ListShards() gives all the info that Flink needs, +1 on switching. DescribeStreams() has a limitation of 5 requests/sec, which is pretty bad But, I believe the goal of switching APIs should be *making Flink jobs that read from Kinesis more stable*, rather than having faster shard discovery

Re: Proposal - Change shard discovery in Flink Kinesis Connector to use ListShards

2018-03-05 Thread Bowen Li
Good for AWS has increased the limit from 5txn/s last year to 10txn/s On Mon, Mar 5, 2018 at 8:32 AM, Thomas Weise <t...@apache.org> wrote: > On Sun, Mar 4, 2018 at 11:38 PM, Bowen Li <bowenl...@gmail.com> wrote: > > > If ListShards() gives all the info that Flin

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-11-01 Thread Bowen Li
After taking a look at how other discussion threads work, I think it's actually fine just keep our discussion here. It's up to you, Xuefu. The google doc LGTM. I left some minor comments. On Thu, Nov 1, 2018 at 10:17 AM Bowen Li wrote: > Hi all, > > As Xuefu has published the d

Re: [DISCUSS] Flink SQL DDL Design

2018-11-02 Thread Bowen Li
Thanks Shuyi! I left some comments there. I think the design of SQL DDL and Flink-Hive integration/External catalog enhancements will work closely with each other. Hope we are well aligned on the directions of the two designs, and I look forward to working with you guys on both! Bowen On Thu,

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-11-01 Thread Bowen Li
Hi all, As Xuefu has published the design doc on google, I agree with Shuyi's suggestion that we probably should start a new email thread like "[DISCUSS] ... Hive integration design ..." on only dev mailing list for community devs to review. The current thread sends to both dev and user list.

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-11-13 Thread Bowen Li
to be rather general and > >> not specifically related to the Hive integration. > >> > > >> IMO, there are some aspects, especially those at the boundary of Hive and > >> Flink, that need a bit more discussion. For example > >> > >> * What do

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-11-15 Thread Bowen Li
elated to the Hive integration. > >> > > >> IMO, there are some aspects, especially those at the boundary of Hive and > >> Flink, that need a bit more discussion. For example > >> > >> * What does it take to make Flink schema compatible with Hive

Re: [DISCUSS] Flink SQL DDL Design

2018-11-28 Thread Bowen Li
; > > not > > > > > > > >> necessary, the framework determine the table referred is a > > > source > > > > > or a > > > > > > > sink > > > > > > > >> according to the context of the query using the table. it > wi

Re: [ANNOUNCE] Apache Flink 1.7.0 released

2018-12-04 Thread Bowen Li
Congratulations everyone! On Mon, Dec 3, 2018 at 11:55 PM Shuyi Chen wrote: > Thanks a lot for the hard work, Till! > > Shuyi > > On Sat, Dec 1, 2018 at 4:07 AM Dominik Wosiński wrote: > > > Thanks Till for being the release manager! > > Thanks Everyone and Great Job. > > > > Best Regards, > >

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2019-01-08 Thread Bowen Li
hing so we can > >> better coordinate. > >> > >> Thanks again for your interest and contributions. > >> > >> Thanks, > >> Xuefu > >> > >> > >> > >> ---

Re: [PROGRESS UPDATE] [DISCUSS] Flink-Hive Integration and Catalogs

2019-03-20 Thread Bowen Li
feedbacks on Flink-Hive integration." * > > Regards, > Shaoxuan > > On Wed, Mar 20, 2019 at 7:16 AM Bowen Li wrote: > >> Hi Flink users and devs, >> >> We want to get your feedbacks on integrating Flink with Hive. >> >> Background: In Flink Forwar

Re: [DISCUSS] Drop Elasticssearch 1 connector

2019-04-05 Thread Bowen Li
+1 for dropping elasticsearch 1 connector. On Wed, Apr 3, 2019 at 5:10 AM Chesnay Schepler wrote: > Hello everyone, > > I'm proposing to remove the connector for elasticsearch 1. > > The connector is used significantly less than more recent versions (2&5 > are downloaded 4-5x more), and hasn't

Re: [REMINDER] Please add entries for newly added dependencies to NOTICE file

2019-03-24 Thread Bowen Li
Hi, I agree with Ufuk that we can start with something simple, achievable, yet effective, like using flink-bot. The wiki that explains licensing of Flink is very good but hard to be found and noticed by new contributors, do we have plan to move it to a more discoverable place like

Re: [DISCUSS] Create a Flink ecosystem website

2019-03-05 Thread Bowen Li
Thanks for bring it up, Becket. That sounds very good to me. Spark also has such a page for ecosystem project https://spark.apache.org/third-party-projects.html and a hosted website https://spark-packages.org/ with metadata, categories/tags and stats mentioned in the doc. Bowen On Tue, Mar 5,

Re: [DISCUSS] Create a Flink ecosystem website

2019-03-08 Thread Bowen Li
Confluent hub for Kafka is another good example of this kind. I personally like it over the spark site. May worth checking it out with Kafka folks On Thu, Mar 7, 2019 at 6:06 AM Becket Qin wrote: > Absolutely! Thanks for the pointer. I'll submit a PR to update

[PROGRESS UPDATE] [DISCUSS] Flink-Hive Integration and Catalogs

2019-03-19 Thread Bowen Li
nts/258723322/>, We presented Integrating Flink with Hive <https://www.slideshare.net/BowenLi9/integrating-flink-with-hive-xuefu-zhang-and-bowen-li-seattle-flink-meetup-feb-2019> with a live demo to local community and got great response. As of mid March now, we have internally finished buil

Re: [DISCUSS] Improve the flinkbot

2019-03-22 Thread Bowen Li
+1, sounds good, Jark. On Fri, Mar 22, 2019 at 1:55 AM Fabian Hueske wrote: > Hi Jark, > > Thanks for driving the effort to integrate the Chinese website! > > We have the policy that new features / improvements should be documented in > the same PR for a long time. > So far, this was checked by

Re: [DISCUSS] Towards a leaner flink-dist

2019-01-24 Thread Bowen Li
+1 for leaner distribution and a better 'download' webpage. +1 for a full distribution if we can automate it besides supporting the leaner one. If we support both, I'd image release managers should be able to package two distributions with a single change of parameter instead of manually package

Re: [ANNOUNCE] Contributing Alibaba's Blink

2019-01-24 Thread Bowen Li
Exciting to see this happening! Wrt doc, have we done a diff which can show us how much differences are between Flink's and Blink's documentation (flink/docs)? For example, how many pages and how much percentage of each page is different? How many new pages (for new features) does Blink have?If

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Bowen Li
+1, a great starting point! It'll be nice if we can later come up with some way to bridge the knowledge between two email channels. On Thu, Jan 24, 2019 at 5:32 AM Fabian Hueske wrote: > Thanks Robert! > I think this is a very good idea. > +1 > > Fabian > > Am Do., 24. Jan. 2019 um 14:09 Uhr

Re: [DISCUSS] [FLINK SQL] External catalog for Confluent Kafka

2019-04-17 Thread Bowen Li
e missing FLIPs in the FLIP main page. I added > all the missing FLIP (incl. FLIP-14, FLIP-22, FLIP-29, FLIP-30, FLIP-31) to > the page. > > I also include @xuef...@alibaba-inc.com and @Bowen > Li into the thread who are familiar with the > latest catalog design. > >

Re: [ANNOUNCE] Apache Flink-shaded 7.0 released

2019-05-31 Thread Bowen Li
Thanks Jincheng for driving this release! On Thu, May 30, 2019 at 11:40 PM Terry Wang wrote: > Wow~ Glad to see this! > Thanks Jincheng and Chesnay for your effort! > > > 在 2019年5月31日,下午1:53,jincheng sun 写道: > > > > Hi all, > > > > The Apache Flink community is very happy to announce the

Re: [DISCUSS] Features for Apache Flink 1.9.0

2019-06-06 Thread Bowen Li
> > >>>> > > > >>>> Concerning the ORC format for StreamingFileSink, is there already > a > > > >> JIRA > > > >>>> ticket tracking that? If not, I suggest to first open one and see > if > > > >>> there >

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-06-14 Thread Bowen Li
t have much experience how to improve this. IMO, Baidu is not > > good at searching for technology-related information. > > > > > > > > Regards, > > Jark > > > > > > > > > > [1]. > > > https://mail-archives.apache.org/mod_mbo

[DISCUSS] solve unstable build capacity problem on TravisCI

2019-06-24 Thread Bowen Li
Hi devs, I've been experiencing the pain resulting from lack of stable build capacity on Travis for Flink PRs [1]. Specifically, I noticed often that no build in the queue is making any progress for hours, and suddenly 5 or 6 builds kick off all together after the long pause. I'm at PST (UTC-08)

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

2019-06-24 Thread Bowen Li
https://travis-ci.org/apache/flink/builds/549681530 This build request has been sitting at **HEAD of the queue** since I first saw it at PST 10:30am (not sure how long it's been there before 10:30am). It's PST 4:12pm now and it hasn't started yet. On Mon, Jun 24, 2019 at 2:48 PM Bowen Li wrote

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

2019-06-24 Thread Bowen Li
es to the PR activities. > https://travis-ci.org/apache/flink/jobs/54519 > > travis-ci for my personal repo often failed with exceeding time limit after > 4+ hours. > The job exceeded the maximum time limit for jobs, and has been terminated. > > On Mon, Jun 24, 2019 at 4:1

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

2019-06-25 Thread Bowen Li
Schepler wrote: > > On 24/06/2019 23:48, Bowen Li wrote: > > - Has anyone else experienced the same problem or have similar > observation > > on TravisCI? (I suspect it has things to do with time zone) > In Europe we have the same problem. > > > > - What pricing

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-06-13 Thread Bowen Li
I resonate with your concerns Robert. It's because most questions are asked in the DingTalk group which has close to 10,000 people now. Here're my observations and thoughts. Group chat is great for marketing, announce news and updates, broadcast live streams/events, and is just TERRIBLE for

Re: Updating Kinesis Connector to latest Apache licensed libs

2019-06-13 Thread Bowen Li
Hi Dyana, Thanks for bringing this up! You are right that ASL is the blocker for us to officially include flink-connecotr-kinesis as a connector module to build and publish to Maven central. I've been thru the mess of building, publishing, and maintaining flink-connector-kinesis via Jfrog, and

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

2019-06-24 Thread Bowen Li
Congratulations! On Mon, Jun 24, 2019 at 9:53 AM Yun Tang wrote: > Congratulations Jincheng! > > Best > Yun Tang > > From: Yu Li > Sent: Tuesday, June 25, 2019 0:07 > To: dev > Subject: Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC > >

Re: [ANNOUNCE] Feature freeze for Apache Flink 1.9.0 release

2019-06-24 Thread Bowen Li
Hi Gordon, Thanks for driving this effort. Xuefu responded to the discussion thread [1] and I want to bring that to our attention here: Hive integration depends on a few features that are actively developed. If the completion of those features don't leave enough time for us to integrate, then

Re: [DISCUSS] Adopting a Code Style and Quality Guide

2019-06-12 Thread Bowen Li
Hi Stephan, Definitely a good guide in principle IMO! I personally already find it useful in practice to learn from it myself, and also promote and cultivate a better coding habit around by referencing it. So big +1 to adopt it. Also want to highlight that we need to make real use of it, and

[ANNOUNCE] Seattle Flink Meetup at AWS on May 30

2019-05-20 Thread Bowen Li
Hi Greater Seattle folks! We are hosting our next meetup with AWS Kinesis Analytics team on May 30 next Thursday in downtown Seattle. We feature two talks this time: 1. *"AWS Kinesis Analytics: running Flink serverless in multi-tenant environment"* by Kinesis Analytics team on: -

Re: [DISCUSS] Releasing Flink-shaded 7.0

2019-04-30 Thread Bowen Li
+1 for releasing flink-shaded 7.0 On Tue, Apr 30, 2019 at 3:13 AM Chesnay Schepler wrote: > I'm in favor creating a flink-shaded release soon. I'm not aware of any > other changes being planned for flink-shaded, so let's get this out the > door rather sooner than later. > > Note that this

Re: [DISCUSS] Update our Roadmap

2019-05-01 Thread Bowen Li
+1 On Mon, Apr 29, 2019 at 11:41 PM jincheng sun wrote: > Hi Jeff, > > I have open the PR about add Python Table API section to the roadmap. I > appreciate if you have time to look at it. :) > > https://github.com/apache/flink-web/pull/204 > > Regards, > Jincheng > > jincheng sun 于2019年4月29日周一

Re: [DISCUSS] [FLINK SQL] External catalog for Confluent Kafka

2019-04-18 Thread Bowen Li
from > >> contributing new catalog implementations. So you could already start > >> designing an implementation. The implementation could be unit tested for > >> now until it can also be registered in a table environment for > >> integration tests/end-to-end tests. >

Re: [DISCUSS] Features for Apache Flink 1.9.0

2019-05-04 Thread Bowen Li
+1, exciting and ambitious goals, the rough timeline looks reasonable. Let's make it happen! On Sat, May 4, 2019 at 2:47 AM Jark Wu wrote: > +1 for the 1.9.0 feature list. Excited to see it is happening. > > Regards, > Jark > > On Thu, 2 May 2019 at 17:07, himansh1...@gmail.com > wrote: > > >

Re: [VOTE] Migrate to sponsored Travis account

2019-07-04 Thread Bowen Li
nt on resources provided by >> >> external companies I would like to vote on this. >> >> >> >> Please vote on this proposal, as follows: >> >> [ ] +1, Approve the migration to a Ververica-sponsored Travis >> >> account, provided that I

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

2019-06-26 Thread Bowen Li
cussed about > >>> this, and I > >>> > > think Till and George have > >>> > > already spend sometime investigating it. I have cced both of > >>> them, and > >>> > > maybe they can

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

2019-07-11 Thread Bowen Li
Congrats, Rong! On Thu, Jul 11, 2019 at 10:48 AM Oytun Tez wrote: > Congratulations Rong! > > --- > Oytun Tez > > *M O T A W O R D* > The World's Fastest Human Translation Platform. > oy...@motaword.com — www.motaword.com > > > On Thu, Jul 11, 2019 at 1:44 PM Peter Huang > wrote: > >>

  1   2   3   4   >