Re: [DISCUSS] Best practice to run flink on kubernetes

Yang Wang wrote: Through mixed-run with online services, they could get better resource utilization and reduce the cost. Flink, as an important case, the dynamical resource allocation is the basic requirement. That's why we want to move the progress more faster. vote +1. If flink

Re: [DISCUSS] Best practice to run flink on kubernetes

Hi dev and users, I just want to revive this discussion because we have some meaningful progress about kubernetes native integration. I have made a draft implementation to complete the poc. Cli and submission are both working as expected. The design doc[1] has been updated, including the detailed

Re: 怎么执行flink代码里边的测试用例

首先你要进入测试所在module的目录，在你这个例子中是 flink-connnectors\flink-connector-kafka-base\ 然后执行 mvn -Dtest=KafkaProducerTestBase#testExactlyOnceCustomOperator test -Dtest=后面可以跟<类名>#<方法名>执行某个测试用例，也可以跟<类名>执行某个类的所有测试用例 Thank you~ Xintong Song On Sun, Sep 29, 2019 at 4:32 PM gaofeilong198...@163.com <

[ANNOUNCE] Weekly Community Update 2019/39

Dear community, happy to share this week's community update with news about Flink 1.10 & 1.9.1, two FLIPs for better programmatic job and cluster control, improvements to the web user interface and a bit more. Flink Development == * [releases] Jark has started a discussion on

Re:Re: map can't return null

hi， ok，thanks.I'll read it. Then I have another problem, which was that I had caught the exception ,but it still came out. At 2019-09-29 17:05:20, "Biao Liu" wrote: Hi allan, It's not a bug. Flink does not support null value, see discussion [1]. In you example, you have to return

Re: map can't return null

Hi allan, It's not a bug. Flink does not support null value, see discussion [1]. In you example, you have to return something with MapFunction even there is nothing to return. Maybe you could use flatmap instead of map to handle this null value scenario. It's allowed to collect nothing (skip

Re:Re: map不能返回null值吗

ok，我知道了。确定一下，之前没发现，跟了一下代码，所以问一下。多谢！ 在 2019-09-29 16:44:53，"Qi Luo" 写道： >Hi Allan， > >map只能返回非null，你可以考虑使用flatMap。 > >Qi > >On Sun, Sep 29, 2019 at 4:31 PM allan <18612537...@163.com> wrote: > >> Hi, >> 发现了一个问题，就是在调用map函数后，返回的null值，map的返回值为tuple2，在下一步filter过滤null值是，作业报异常。 >> >>

Re: map can't return null

Hi Allan, Map does support null but tuple serializer does not. You might want to use pojo or row types if you need to deal with null values. Read more here . - Abhishek On Sun, 29

Re: map不能返回null值吗

Hi Allan， map只能返回非null，你可以考虑使用flatMap。 Qi On Sun, Sep 29, 2019 at 4:31 PM allan <18612537...@163.com> wrote: > Hi, > 发现了一个问题，就是在调用map函数后，返回的null值，map的返回值为tuple2，在下一步filter过滤null值是，作业报异常。 > > java.lang.NullPointerException > at >

怎么执行flink代码里边的测试用例

hi, 我的linux环境可以正常执行 mvn clean install -DskipTests -Dfast 和 mvn clean install -DskipTests 对flink源码进行编译， 那么，我想执行flink里边的测试用例，比如 flink-connectors\flink-connector-kafka-base\src\test\java\org\apache\flink\streaming\connectors\kafka\KafkaProducerTestBase.java#testExactlyOnceCustomOperator, 应该怎么执行？

map can't return null

Hi guys， When I use like the code, .map(new MapFunction>() { @Override public Tuple2 map(String value) throws Exception { if (properties != null) { return new Tuple2<>(cv_no, json.toJSONString()); } return null;

Could not forward element to next operator

Hi， 最近发现作业一直在报错，我的窗口是一分钟的窗口。这是什么原因，谁能帮助一下？错误如下： TimerException{org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException: Could not forward element to next operator} at

map不能返回null值吗

Hi, 发现了一个问题，就是在调用map函数后，返回的null值，map的返回值为tuple2，在下一步filter过滤null值是，作业报异常。 java.lang.NullPointerException at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.copy(TupleSerializer.java:104) at

Re: Could not forward element to next operator

问题可能出在被省略的部分，检查一下最底层的 caused by Thanks, Biao /'bɪ.aʊ/ On Sun, 29 Sep 2019 at 13:17, <18612537...@163.com> wrote: > 我看过这个我不是这个问题，作业没有设置水印，作业可以正常运行，最近可能是运行一天多会报这个异常 > > 发自我的 iPhone > > > 在 2019年9月29日，上午11:49，Wesley Peng 写道： > > > > Hello, > > > > May this article match your issue? > >

Re: task-manager中taskslot的classloader隔离问题

同一个 TM 中，相同 job 的 subtask 会共享一个 classloader Thanks, Biao /'bɪ.aʊ/ On Sat, 28 Sep 2019 at 09:30, Ever <439674...@qq.com> wrote: > 有一个job有2个task，每个task分别有3个subtask(并行度为3), 如下图所示。 > > 每个subtask会占用一个taskslot， 但是同一个job的不同task的subtask可以共享同一个taskslot， > 所以这里应该是一个taskslot会有2个subtask。 >