Registration open for Community Over Code North America

Hello! Registration is still open for the upcoming Community Over Code NA event in Halifax, NS! We invite you to register for the event https://communityovercode.org/registration/ Apache Committers, note that you have a special discounted rate for the conference at US$250. To take advantage of

Final reminder: ApacheCon North America call for presentations closing soon

[Note: You're receiving this because you are subscribed to one or more Apache Software Foundation project mailing lists.] This is your final reminder that the Call for Presetations for ApacheCon North America 2022 will close at 00:01 GMT on Monday, May 23rd, 2022. Please don't wait! Get your talk

Call for Presentations now open, ApacheCon North America 2022

to: Community Big Data Search IoT Cloud Fintech Pulsar Tomcat You can submit your session proposals starting today at https://cfp.apachecon.com/ Rich Bowen, on behalf of the ApacheCon Planners apachecon.com @apachecon

ApacheCon starts tomorrow!

on Twitter - https://twitter.com/apachecon - for tips and announcements during the event. See you tomorrow! -- Rich Bowen, VP Conferences The Apache Software Foundation https://apachecon.com/ @apachecon

ApacheCon is just 3 weeks away!

vability; Search; Tomcat. We will also feature keynotes from Ashley Wolf, Mark Cox, Alison Parker and Michael Weinberg. Details on the schedule, and these keynotes, can be found at https://www.apachecon.com/acah2021/tracks/ We look forward to seeing you at this year’s ApacheCon! – Ri

flink hive batch作业报FileNotFoundException

Hi,大家好 我执行的是batch table写入hive时,会出现FileNotFound的错误,找不到.staging文件 版本是 1.12.1 搭建方式是 standalone 报错信息如下： 11:28 Caused by: java.lang.Exception: Failed to finalize execution on master ... 33 more Caused by: org.apache.flink.table.api.TableException: Exception in finalizeGlobal at

flink集群提交任务挂掉

Hi,大家好： 现在我们遇到的场景是这样的,提交任务的时候会报错。我们使用的版本是1.12.1,搭建模式是standalone的。下面是报错信息。 java.lang.OutOfMemoryError: Direct buffer memory. The direct out-of-memory error has occurred. This can mean two things: either job(s) require(s) a larger size of JVM direct memory or there is a direct memory

Call for Presentations for ApacheCon 2021 now open

forward to reviewing your contribution to one of the most popular open source software events in the world! Rich, for the ApacheCon Planners -- Rich Bowen, VP Conferences The Apache Software Foundation https://apachecon.com/ @apachecon

Announcing ApacheCon @Home 2020

. Submit your talks today at https://acna2020.jamhosted.net/ We hope to see you at the event! Rich Bowen, VP Conferences, The Apache Software Foundation

Re: 使用Flink1.10.0读取hive时source并行度问题

@JingsongLee 把当前的hive sink并发度配置策略加到文档里吧 https://issues.apache.org/jira/browse/FLINK-16448 On Tue, Mar 3, 2020 at 9:31 PM Jun Zhang <825875...@qq.com> wrote: > > 嗯嗯，其实我觉得我写的这个示例sql应该是一个使用很广泛的sql，我新建了hive表，并且导入了数据之后，一般都会使用类似的sql来验证一下表建的对不对，数据是否正确。 > > > > > > > 在2020年03月4日 13:25，JingsongLee

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

Congrats, Jingsong! On Fri, Feb 21, 2020 at 7:28 AM Till Rohrmann wrote: > Congratulations Jingsong! > > Cheers, > Till > > On Fri, Feb 21, 2020 at 4:03 PM Yun Gao wrote: > >> Congratulations Jingsong! >> >>Best, >>Yun >> >>

Re: Flink connect hive with hadoop HA

Hi sunfulin, Sounds like you didn't config the hadoop HA correctly on the client side according to [1]. Let us know if it helps resolve the issue. [1] https://stackoverflow.com/questions/25062788/namenode-ha-unknownhostexception-nameservice1 On Mon, Feb 10, 2020 at 7:11 AM Khachatryan Roman

Re: Flink connect hive with hadoop HA

Hi sunfulin, Sounds like you didn't config the hadoop HA correctly on the client side according to [1]. Let us know if it helps resolve the issue. [1] https://stackoverflow.com/questions/25062788/namenode-ha-unknownhostexception-nameservice1 On Mon, Feb 10, 2020 at 7:11 AM Khachatryan Roman

Re: Read data from Oracle using Flink SQL API

Hi Flavio, +1 for adding Oracle (potentially more dbms like SqlServer, etc) to flink-jdbc. Would you mind open a parent ticket and some subtasks, each one for one to-be-added dbms you've thought of? On Sun, Feb 2, 2020 at 10:11 PM Jingsong Li wrote: > Yes, And I think we should add

Re: batch job OOM

Hi Fanbin， You can install your own Flink build in AWS EMR, and it frees you from Emr’s release cycles On Thu, Jan 23, 2020 at 03:36 Jingsong Li wrote: > Fanbin, > > I have no idea now, can you created a JIRA to track it? You can describe > complete SQL and some data informations. > > Best, >

Call for presentations for ApacheCon North America 2020 now open

Dear Apache enthusiast, (You’re receiving this message because you are subscribed to one or more project mailing lists at the Apache Software Foundation.) The call for presentations for ApacheCon North America 2020 is now open at https://apachecon.com/acna2020/cfp ApacheCon will be held at

Re: [ANNOUNCE] Dian Fu becomes a Flink committer

Congrats! On Thu, Jan 16, 2020 at 13:45 Peter Huang wrote: > Congratulations, Dian! > > > Best Regards > Peter Huang > > On Thu, Jan 16, 2020 at 11:04 AM Yun Tang wrote: > >> Congratulations, Dian! >> >> Best >> Yun Tang >> -- >> *From:* Benchao Li >> *Sent:*

Re: [ANNOUNCE] Dian Fu becomes a Flink committer

Congrats! On Thu, Jan 16, 2020 at 13:45 Peter Huang wrote: > Congratulations, Dian! > > > Best Regards > Peter Huang > > On Thu, Jan 16, 2020 at 11:04 AM Yun Tang wrote: > >> Congratulations, Dian! >> >> Best >> Yun Tang >> -- >> *From:* Benchao Li >> *Sent:*

Re: flink sql confluent schema avro topic注册成表

Hi 陈帅, 这是一个非常合理的需求。我们需要开发一个 Flink ConfluentSchemaRegistryCatalog 完成元数据的获取。社区希望的用户体验是用户只需要给出confluent schema registry的链接，Flink SQL可以通过 ConfluentSchemaRegistryCatalog自动获取读写所需的信息，不再需要用户手动写DDL和format。 社区内部已经开始讨论了，我们应该会在1.11中完成，请关注 https://issues.apache.org/jira/browse/FLINK-12256 On Wed, Dec 18,

Re: [DISCUSS] What parts of the Python API should we focus on next ?

- integrate PyFlink with Jupyter notebook - Description: users should be able to run PyFlink seamlessly in Jupyter - Benefits: Jupyter is the industrial standard notebook for data scientists. I’ve talked to a few companies in North America, they think Jupyter is the #1 way to empower

Re: [DISCUSS] What parts of the Python API should we focus on next ?

- integrate PyFlink with Jupyter notebook - Description: users should be able to run PyFlink seamlessly in Jupyter - Benefits: Jupyter is the industrial standard notebook for data scientists. I’ve talked to a few companies in North America, they think Jupyter is the #1 way to empower

Re: [DISCUSS] have separate Flink distributions with built-in Hive dependencies

cc user ML in case anyone want to chime in On Fri, Dec 13, 2019 at 00:44 Bowen Li wrote: > Hi all, > > I want to propose to have a couple separate Flink distributions with Hive > dependencies on specific Hive versions (2.3.4 and 1.2.1). The distributions > will be provided to

Re: [ANNOUNCE] Launch of flink-packages.org: A website to foster the Flink Ecosystem

Great work, glad to see this finally happening! On Tue, Nov 19, 2019 at 6:26 AM Robert Metzger wrote: > Thanks. > > I added a ticket for this nice idea: > https://github.com/ververica/flink-ecosystem/issues/84 > > On Tue, Nov 19, 2019 at 11:29 AM orips wrote: > >> This is great. >> >> Can we

Re: [ANNOUNCE] Weekly Community Update 2019/45

by *Sreekanth Krishnavajjala & Vinod Kataria (AWS)* > includes a hands-on introduction to Apache Flink on AWS EMR. [7] > * Upcoming Meetups > * At the next Athens Big Data Group on the 14th of November *Chaoran > Yu *of Lightbend will talk about Flink and Spark on Kubernete

Re: Streaming write to Hive

Hi, I'm not sure if there's one yet. Feel free to create one if not. On Wed, Sep 4, 2019 at 11:28 PM Qi Luo wrote: > Hi Bowen, > > Thank you for the information! Streaming write to Hive is a very common > use case for our users. Is there any open issue for this to which

Re: Streaming write to Hive

. It's on our list for 1.10, not high priority though. Bowen On Wed, Sep 4, 2019 at 2:23 AM Qi Luo wrote: > Hi guys, > > In Flink 1.9 HiveTableSink is added to support writing to Hive, but it > only supports batch mode. StreamingFileSink can write to HDFS in streaming > mode, but

Re: kinesis table connector support

@Fanbin, I don't think there's one yet. Feel free to create a ticket and submit a PR for it On Mon, Sep 2, 2019 at 8:13 AM Biao Liu wrote: > Hi Fanbin, > > I'm not familiar with table module. Maybe someone else could help. > > @jincheng sun > Do you know there is any plan for kinesis table

[ANNOUNCE] Kinesis connector becomes part of Flink releases

drive this initiative forward. Cheers, Bowen

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

Congratulations Andrey! On Wed, Aug 14, 2019 at 10:18 PM Rong Rong wrote: > Congratulations Andrey! > > On Wed, Aug 14, 2019 at 10:14 PM chaojianok wrote: > > > Congratulations Andrey! > > At 2019-08-14 21:26:37, "Till Rohrmann" wrote: > > >Hi everyone, > > > > > >I'm very happy to announce

[ANNOUNCE] Seattle Flink Meetup at Uber on 8/22

Flink Checkout event details and RSVP at https://www.meetup.com/seattle-flink/events/263782233/ . See you soon! Bowen

Re: Status of the Integration of Flink with Hive

features seems to be in development. > Some really cool features have been described here: > https://fr.slideshare.net/BowenLi9/integrating-flink-with-hive-xuefu-zhang-and-bowen-li-seattle-flink-meetup-feb-2019 > My first need is to read and update Hive metadata. > Concerning the Hive data I can store th

Re: Cannot access the data from Hive-Tables in Blink

Hi Yebgenya, This is caused by Hive version mismatch, you are either not using the right Hive version (double check your Hive version is supported by Blink), or not specifying the right version in yaml config (e.g. you use 2.3.4 but specify it as 1.2.1). Bowen On Tue, Jul 16, 2019 at 11:22 AM

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

Congrats, Rong! On Thu, Jul 11, 2019 at 10:48 AM Oytun Tez wrote: > Congratulations Rong! > > --- > Oytun Tez > > *M O T A W O R D* > The World's Fastest Human Translation Platform. > oy...@motaword.com — www.motaword.com > > > On Thu, Jul 11, 2019 at 1:44 PM Peter Huang > wrote: > >>

Re: Hive in sql-client

Hi Yebgenya, To use Blink's integration with Hive in SQL CLI, you can reference Blink's documentation at [1], [2], and [3] Note that Hive integration is actually available in **Flink master branch** now and will be released soon as part of Flink 1.9.0. The end-to-end integration should be

Re:

is well written, I don't think it's acceptable to not have a well written subject. Note that this is brought up not specific to you as a person, but specific to a common practice everyone in the community should follow. Bowen On Sun, Jul 7, 2019 at 1:19 PM Konstantin Knauf wrote: > Hi Wang, >

Re: Source Kafka and Sink Hive managed tables via Flink Job

that Hive ACID is not the problem, please share some high level code of your job, so we can take another look. Bowen On Thu, Jul 4, 2019 at 6:50 AM Youssef Achbany wrote: > Thank you Li for your answer and sorry for the dev mistake :). > > *To be more clear:* > > We write multiple e

Re: Source Kafka and Sink Hive managed tables via Flink Job

BTW, I'm adding user@ mailing list since this is a user question and should be asked there. dev@ mailing list is only for discussions of Flink development. Please see https://flink.apache.org/community.html#mailing-lists On Wed, Jul 3, 2019 at 12:34 PM Bowen Li wrote: > Hi Youssef, >

ApacheCon North America 2019 Schedule Now Live!

also be running ApacheCon Europe in Berlin in October. Details of that event are at https://aceu19.apachecon.com/ Follow us on Twitter - @ApacheCon - for all the latest updates. See you in Las Vegas! Rich Bowen, VP Conferences, The ASF rbo...@apache.org http://apachecon.com/

Re: [External] Flink 1.7.1 on EMR metrics

To answer your question on your debugging code, your reporter has a bug: log.info("STATSD SENDING: ", name, value); should be -> log.info("STATSD SENDING: {} {}", name, value); - On Sat, Jun 1, 2019 at 7:30 PM Padarn Wilson wrote: > Thanks both: Using the the inbuilt Slf4j reporter is a

[ANNOUNCE] Seattle Flink Meetup at AWS on May 30

t developers onboard given the considerably rising interest of contributing to Flink from local tech community Please find details and RSVP at https://www.meetup.com/seattle-flink/events/260865206/ See you next week! Cheers, Bowen

Re: Flink 与 Hive 集成问题

Hi, 我们正在做 Flink-Hive 平台级的元数据和数据的集成，你可以关注下： flink-connector-hive module， Hive元数据 FLINK-11479 ，Hive数据 FLINK-10729

Re: [DISCUSS] Drop Elasticssearch 1 connector

+1 for dropping elasticsearch 1 connector. On Wed, Apr 3, 2019 at 5:10 AM Chesnay Schepler wrote: > Hello everyone, > > I'm proposing to remove the connector for elasticsearch 1. > > The connector is used significantly less than more recent versions (2&5 > are downloaded 4-5x more), and hasn't

Re: Re:[进度更新] [讨论] Flink 对 Hive 的兼容 和 Catalogs

尝试flink > > on hive > > > - > > > *各位使用Flink-Hive的动机是什么？只维护一套数据处理系统？使用Flink获取更好的性能？*//技术迭代，当然理想的状况是批流统一，只维护一套数据处理系统。spark的性能已经很棒了，所以追求更好的性能这个对我们不需要。 > > > - *各位如何使用Hive？数据量有多大？主要是读，还是读写都有？*//大的表 数据量不小，主要是读 > > > - *有多少Hive UDF？都是什么类型？*//挺多 >

Re: [PROGRESS UPDATE] [DISCUSS] Flink-Hive Integration and Catalogs

ang wrote: > Hi Bowen, > Thanks for driving this. I am CCing this email/survey to user-zh@ > flink.apache.org as well. > I heard there are lots of interests on Flink-Hive from the field. One of > the biggest requests the hive users are raised is "the support of > out-of-d

Re: [PROGRESS UPDATE] [DISCUSS] Flink-Hive Integration and Catalogs

ang wrote: > Hi Bowen, > Thanks for driving this. I am CCing this email/survey to user-zh@ > flink.apache.org as well. > I heard there are lots of interests on Flink-Hive from the field. One of > the biggest requests the hive users are raised is "the support of > out-of-d

[进度更新] [讨论] Flink 对 Hive 的兼容 和 Catalogs

处理系统？使用Flink获取更好的性能？* - *各位如何使用Hive？数据量有多大？主要是读，还是读写都有？* - *有多少Hive UDF？都是什么类型？* - *对项目有什么问题或者建议？* 大家的建议对我们很重要。我们希望这些工作能真正的尽快惠及社区用户。我们争取这周做个调查问卷，更全面的收集各位的反馈和建议。 Bowen

[PROGRESS UPDATE] [DISCUSS] Flink-Hive Integration and Catalogs

nts/258723322/>, We presented Integrating Flink with Hive <https://www.slideshare.net/BowenLi9/integrating-flink-with-hive-xuefu-zhang-and-bowen-li-seattle-flink-meetup-feb-2019> with a live demo to local community and got great response. As of mid March now, we have internally finished buil

Re: [DISCUSS] Create a Flink ecosystem website

>>> > > Forgot to provide the link... > >>> > > > >>> > > [1] https://www.apache.org/dev/services.html#blogs (Apache infra > >>> services) > >>> > > [2] https://www.apache.org/dev/freebsd-jails (FreeBSD Jail > provid

4 Apache Events in 2019: DC Roadshow soon; next up Chicago, Las Vegas, and Berlin!

Dear Apache Enthusiast, (You’re receiving this because you are subscribed to one or more user mailing lists for an Apache Software Foundation project.) TL;DR: * Apache Roadshow DC is in 3 weeks. Register now at https://apachecon.com/usroadshowdc19/ * Registration for Apache Roadshow Chicago is

Re: [DISCUSS] Create a Flink ecosystem website

Thanks for bring it up, Becket. That sounds very good to me. Spark also has such a page for ecosystem project https://spark.apache.org/third-party-projects.html and a hosted website https://spark-packages.org/ with metadata, categories/tags and stats mentioned in the doc. Bowen On Tue, Mar 5

Re: TimeZone shift problem in Flink SQL

Hi, Did you consider timezone in conversion in your UDF? On Tue, Jan 22, 2019 at 5:29 AM 徐涛 wrote: > Hi Experts, > I have the following two UDFs, > unix_timestamp: transform from string to Timestamp, with the > arguments (value:String, format:String), return Timestamp >

Re: [DISCUSS] Towards a leaner flink-dist

+1 for leaner distribution and a better 'download' webpage. +1 for a full distribution if we can automate it besides supporting the leaner one. If we support both, I'd image release managers should be able to package two distributions with a single change of parameter instead of manually package

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

detailed discussions with? It's gonna be a great and influential project , and I'd love to participate into it to move FlinkSQL's adoption and ecosystem even further. Thanks, Bowen > 在 2018年10月12日，下午3:37，Jörn Franke 写道： > > Thank you very nice , I fully agree with that. > >> Am 1

Speakers needed for Apache DC Roadshow

We need your help to make the Apache Washington DC Roadshow on Dec 4th a success. What do we need most? Speakers! We're bringing a unique DC flavor to this event by mixing Open Source Software with talks about Apache projects as well as OSS CyberSecurity, OSS in Government and and OSS Career

Re: [ANNOUNCE] Apache Flink 1.5.2 released

Congratulations, community! On Tue, Jul 31, 2018 at 1:44 AM Chesnay Schepler wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.5.2, which is the second bugfix release for the Apache Flink 1.5 > series. > > Apache Flink® is an open-source stream

Re: Is KeyedProcessFunction available in Flink 1.4?

g more access to timers' key, thus you can refer to examples of ProcessFunction in that document. Bowen On Thu, Jul 19, 2018 at 3:26 PM anna stax wrote: > Hello all, > I am using Flink 1.4 because thats the version provided by the latest AWS > EMR. > Is KeyedProcessFunction avai

Re: Is Flink using even-odd versioning system

Hi Alexander, AFAIK, Flink releases don't do that. The community has done its best to ensure every release is at its best state. Thanks, Bowen On Tue, Jul 10, 2018 at 4:54 AM Alexander Smirnov < alexander.smirn...@gmail.com> wrote: > to denote development and stable releases? >

Register now for ApacheCon and save $250

to be there. Register today at http://apachecon.com/acna18 and we’ll see you in Montreal! -- Rich Bowen VP, Conferences, The Apache Software Foundation h...@apachecon.com @ApacheCon

Re: Flink and AWS S3 integration: java.lang.NullPointerException: null uri host

Did you run Flink on AWS EMR or somewhere else? Have you read and followed instructions on https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/deployment/aws.html#amazon-web-services-aws ? On Wed, May 30, 2018 at 7:08 AM, Fabian Wollert wrote: > Hi, I'm trying to set up

Re: [ANNOUNCE] Apache Flink 1.5.0 release

Congratulations, everyone! On Mon, May 28, 2018 at 1:15 AM, Fabian Hueske wrote: > Thank you Till for serving as a release manager for Flink 1.5! > > 2018-05-25 19:46 GMT+02:00 Till Rohrmann : > > > Quick update: I had to update the date of the release blog post which > also > > changed the

Re: Clarification in TumblingProcessing TimeWindow Documentation

with better comment. Thanks, Bowen On Sun, May 27, 2018 at 12:43 PM, Dhruv Kumar <gargdhru...@gmail.com> wrote: > Hi > > I was looking at TumblingProcessingTimeWindows.java > <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/st

Re: Missing MapState when Timer fires after restored state

trying to get is null or not. I've also created https://issues.apache.org/jira/browse/FLINK-9362 to document this. Thanks Bowen On Mon, May 14, 2018 at 4:00 AM, Juho Autio <juho.au...@rovio.com> wrote: > We have a Flink streaming job (1.5-SNAPSHOT) that uses timers to clear old >

Re: Recommended books

I'd recommend this book, *Stream Processing with Apache Flink: Fundamentals, Implementation, and Operation of Streaming Applications.* It's probably the most authentic book about Flink on the market. You can buy and read the early release on OReilly,

ApacheCon North America 2018 schedule is now live.

. And if your company wants to sponsor this event, get in touch at h...@apachecon.com for opportunities that are still available. See you in Montreal! Rich Bowen VP Conferences, The Apache Software Foundation h...@apachecon.com @ApacheCon [1] http://cloudstackcollab.org/ [2] http

Re: 1.4.3 release/roadmap

​to find bug fixes that are going into​ 1.4.x, say 1.4.3, you can filter jira tickets with 'Fix Versions' as '1.4.3' On Thu, Apr 19, 2018 at 1:36 AM, Daniel Harper wrote: > Hi there, > > There are some bug fixes that are in the 1.4 branch that we would like to > be made

Save the date: ApacheCon North America, September 24-27 in Montréal

@ApacheCon on Twitter to be the first to hear announcements about keynotes, the schedule, evening events, and everything you can expect to see at the event. See you in Montréal! Sincerely, Rich Bowen, V.P. Events, on behalf of the entire ApacheCon team [1] http://www.apachecon.com/acna18 [2

Re: [ANNOUNCE] Apache Flink 1.4.1 released

Congratulations everyone! On Thu, Feb 15, 2018 at 10:04 AM, Tzu-Li (Gordon) Tai wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.4.1, which is the first bugfix release for the Apache Flink 1.4 > series. > > > Apache Flink® is an

[SEATTLE MEETUP] Announcing First Seattle Apache Flink Meetup

drinks will be provided. Plenty of onsite parking spots. DATE: Jan 17th, 2018, Wednesday TALKS: - Haitao Wang, Senior Staff Engineer at Alibaba, will give a presentation on large-scale streaming processing with Flink and Flink SQL at Alibaba and several internal use cases. - Bowen Li will talk

Re: Could not flush and close the file system output stream to s3a, is this fixed?

14, 2017 at 2:05 AM, Fabian Hueske <fhue...@gmail.com> wrote: > Bowen Li (in CC) closed the issue but there is no fix (or at least it is > not linked in the JIRA). > Maybe it was resolved in another issue or can be differently resolved. > > @Bowen, can you comment on ho

Re: [DISCUSS] Dropping Scala 2.10

+1 for dropping support for Scala 2.10 On Tue, Sep 19, 2017 at 3:29 AM, Sean Owen wrote: > For the curious, here's the overall task in Spark: > > https://issues.apache.org/jira/browse/SPARK-14220 > > and most of the code-related changes: > >

Re: Does RocksDB need a dedicated CPU?

t; SSDs to achieve the best performance. > > Regards, > > Kien > > > > On 9/5/2017 1:15 PM, Bowen Li wrote: > >> Hi guys, >> >> Does RocksDB need a dedicated CPU? Do we need to allocate one CPU for >> each RocksDB while deploying Flink cluster wi

Does RocksDB need a dedicated CPU?

Hi guys, Does RocksDB need a dedicated CPU? Do we need to allocate one CPU for each RocksDB while deploying Flink cluster with RocksDB state backend? I think there's probably no need since RocksDB is a native 'library', but I want to confirm it with Flink community. Thanks, Bowen

Re: Even out the number of generated windows

That's exactly what I found yesterday! Thank you Aljoscha for confirming it! On Mon, Aug 28, 2017 at 2:57 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi Bowen, > > There is not built-in TTL but you can use a ProcessFunction to set a timer > that clears state. > &

Re: Even out the number of generated windows

to expire keyed state by time? Thanks, Bowen On Sun, Aug 27, 2017 at 5:41 AM, Robert Metzger <rmetz...@apache.org> wrote: > Hi Bowen, > > I don't know what kind of relationship your company has to AWS, maybe they > are willing to look into the issue from their side. > >

Re: Even out the number of generated windows

sink"? How to add the throttling into Flink's fluent API? Thanks, Bowen On Fri, Aug 25, 2017 at 2:31 PM, Robert Metzger <rmetz...@apache.org> wrote: > Hi Bowen, > > (very nice graphics :) ) > > I don't think you can do anything about the windows itself (unless you

Re: Which window function to use to start a window at anytime

Hi Aljoscha, Thank you very much! We imagined it's going to be very expensive to achieve that, and your answer verified our understanding of how Flink works. Regards, Bowen On Fri, Aug 25, 2017 at 8:18 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi, >

Re: Flink doesn't free YARN slots after restarting

to downstream Kinesis sink, as my ASCII charts demonstrated. Does my example make sense? Thanks, Bowen On Fri, Aug 25, 2017 at 12:01 AM, Till Rohrmann <trohrm...@apache.org> wrote: > Hi Bowen, > > having a sliding window of one day with a slide of one hour basically > mean

Even out the number of generated windows

| | | |___ time Thanks, Bowen

Re: Flink doesn't free YARN slots after restarting

this? load | | | |___ time Thanks, Bowen On Tue, Aug 22, 2017 at 2:56 AM, Till Rohrmann <trohrm...@apache.org> wrote: > Hi Bowen, > > sorry for my late answer. I dug through some of the logs and it seems

Re: akka timeout

Hi Steven, Yes, GC is a big overhead, it may cause your CPU utilization to reach 100%, and every process stopped working. We ran into this a while too. How much memory did you assign to TaskManager? How much the your CPU utilization when your taskmanager is considered 'killed'? Bowen

Re: [Survey] How many people use Flink with AWS Kinesis sink

to learn 1) what KPL configuration values (rate limit, record_ttl, etc) are the best for Flink 2) what the deployment strategy of KPL (parallelism, any dedicated cores or memory?) works best with Flink. Thanks! Bowen On Mon, Aug 21, 2017 at 10:55 AM, Stephan Ewen <se...@apache.org> wrote:

[Survey] How many people use Flink with AWS Kinesis sink

it further requires to make Flink works with Kinesis. If not many Flink users had good experience with Kinesis, we'll probably need to look for some alternatives. I really appreciate your time and your insight! Thank you very much! Bowen

Which window function to use to start a window at anytime

would be 2:12:24.123 - 3:12:24.122, and so on - for item B, the first event arrives at 1:10:20:321, so the window would be 1:10:20:321 - 2:10:20:320, and the next window would be 2:10:20:321 - 3:10:20:320, and so on. Do you have any insights on how to achieve it? Thanks! Bowen

Re: Flink doesn't free YARN slots after restarting

as a result. Why the slots are not released? Or are they actually crushed and thus no longer available? I'm trying to deploy the first Flink cluster within out company. And this issue is slowing us down from proving that Flink actually works for us. We'd appreciate your help on it! Thanks, Bowen

Re: Flink doesn't free YARN slots after restarting

Hi Till, Thanks for taking this issue. We are not comfortable sending logs to a email list which is this open. I'll send logs to you. Thanks, Bowen On Wed, Aug 9, 2017 at 2:46 AM, Till Rohrmann <trohrm...@apache.org> wrote: > Hi Bowen, > > if I'm not mistaken, then

Flink doesn't free YARN slots after restarting

-ytm 4 xxx.jar ``` Does anyone know what's going wrong? Thanks, Bowen

Re: [POLL] Who still uses Java 7 with Flink ?

5274018=com.atlassian.jira. > >> > > plugin.system.issuetabpanels:comment-tabpanel#comment-15274018 > >> > > > > > >> > > > > > >> > > > > > >> > > > > On Thu, Mar 23, 2017 at 2:42 PM, Theodore Vasiloudis < &g

Re: confusing RocksDBStateBackend parameters

Hope that helps. > > Best > Ziyad > > Best Regards > *Ziyad Muhammed Mohiyudheen * > 407, Internationales Studienzentrum Berlin > Theodor-Heuss-Platz 5 > 14052 Berlin > *Ph: +49 176 6587 3343 <%2B49%20176%206587%203343>* > *Mail to*: *mmzi...@gmail.com <mmzi

confusing RocksDBStateBackend parameters

tml> explained the first three well. But what's the 4th one for? What's its difference from others? I'd appreciate your clarification. Thanks very much! Bowen

Re: Clarification on state backend parameters

:20 PM, bowen.li <bowen...@offerupnow.com> wrote: > Hi guys, > This is great clarification! > > An extended question from me is, what's the difference between > `state.checkpoints.dir` and the param you pass in to RocksDBStateBackend > constructor in`public

Last chance: ApacheCon is just three weeks away

://lists.apache.org/list.html?apachecon-disc...@apache.org * Podcasts and speaker interviews: http://feathercast.apache.org/ * IRC: #apachecon on the https://freenode.net/ We look forward to seeing you in Miami! -- Rich Bowen - VP Conferences, The Apache Software Foundation http://apachecon.com

Re: [POLL] Who still uses Java 7 with Flink ?

There's always a tradeoff we need to make. I'm in favor of upgrading to Java 8 to bring in all new Java features. The common way I've seen (and I agree) other software upgrading major things like this is 1) upgrade for next big release without backward compatibility and notify everyone 2)

FINAL REMINDER: CFP for ApacheCon closes February 11th

in Miami! -- Rich Bowen VP Conferences, Apache Software Foundation rbo...@apache.org Twitter: @apachecon (You are receiving this email because you are subscribed to a dev@ or users@ list of some Apache Software Foundation project. If you do not wish to receive email from these lists any more, you

ApacheCon CFP closing soon (11 February)

. Or contact me - rbo...@apache.org - with any questions or concerns. Thanks! Rich Bowen, VP Conferences, Apache Software Foundation -- (You've received this email because you're on a dev@ or users@ mailing list of an Apache Software Foundation project. For subscription and unsubscription information

Save the date: ApacheCon Miami, May 15-19, 2017

today at http://apachecon.com/ and submit your talks. ApacheCon Miami is going to be our best ApacheCon yet, and you, and your project, can’t afford to miss it. -- Rich Bowen - rbo...@apache.org VP, Conferences http://apachecon.com @apachecon

ApacheCon is now less than a month away!

Bowen: VP, Conferences rbo...@apache.org http://apachecon.com/ @apachecon

[ANNOUNCE] CFP open for ApacheCon North America 2016

Community growth starts by talking with those interested in your project. ApacheCon North America is coming, are you? We are delighted to announce that the Call For Presentations (CFP) is now open for ApacheCon North America. You can submit your proposed sessions at