抱歉，没完全看懂你的问题，尝试回答下，更新拓扑目前需要重启
杨胜松(鼓翅) 于2019年6月24日周一 下午4:26写道：
>
> 你好！
>
> 请教下，假设我一个拓扑现在有三个sql已经在跑了，现在我想加第四个sql进来，那么我一定要重发这个拓扑么？有什么办法可以在不影响这三个sql计算的情况下，把第四个sql也加进来么？
> 刚接触flink，还处在摸索阶段，有什么资料能分享下更好啦~
>
>
>
>
>
>
>
抱歉，没看懂你的问题
1. group id 和 offset 有什么关系？
2. "在kafka 里看不到group id" 指什么？
雷水鱼 于2019年6月24日周一 下午5:05写道：
> 现象
> 使用这个pom ,在kafka 里看不到group id
>
> com.alibaba.blink
> flink-streaming-scala_2.11
> blink-3.2.2
>
>
> org.slf4j
> slf4j-api
>
>
>
>
> com.alibaba.blink
>
Thanks for opening this ticket and I would watch it.
Flink does not handle OOM issue specially. I remembered we ever discussed the
similar issue before but forgot the conclusion then or have other concerns for
it.
I am not sure whether it is worth to fix atm, maybe Till or Chesnay could give
a
Hi Zhijiang
Thank you for your analysis. I agree with it. The solution may be to let tm
exit like you mentioned when any type of oom occurs, because the flink has
no control on a tm when a oom occurs.
I fired a jira before, https://issues.apache.org/jira/browse/FLINK-12889.
Don't know it is
Hi Syed
You could use 'mvn clean package -pl :flink-streaming-java_2.11 -DskipTests
-am' to build flink-streaming-java and flink-runtime modules. If the 'already
built binary' means the flink-dist-*.jar package, the former mvn command would
not update the dist jar package. As far as I know, a
You need to specify flink-dist in -pl. Module flink-dist will build the
flink binary distribution.
syed 于2019年6月25日周二 上午9:14写道：
> Hi;
> I am trying to modify some core functionalities of flink for my through
> understanding about flink. I already build the flink from source, now I am
> looking
Hi,
I am using flink version 1.7.2 , I am trying to use S3 like object
storage EMC ECS(
https://www.emc.com/techpubs/ecs/ecs_s3_supported_features-1.htm). Not
all S3 apis are supported by EMC ESC according to this document. Here
is my config
s3.endpoint: SU73ECSG1P1d.***.COM
s3.access-key:
Hi;
I am trying to modify some core functionalities of flink for my through
understanding about flink. I already build the flink from source, now I am
looking to build only a few modules which I have modified. Is this possible,
or every time I have to build the flink in full (all modules). As it
Hi Ken,
Thanks for reaching out, I created a compliant bucket with name
aip-featuretoolkit. I now get the exception "Unable to execute HTTP
request: aip-featuretoolkit.SU73ECSG1P1d.***.COM: Name or service not
known" from
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.s3a.Invoker.class
Dear Community,
I am using Flink (processing-time) timers along with a Process Function.
What I would like to do is to "postpone" eventually registered timers for
the given key: I would like to do it since I might process plenty of events
in a row (think about it as a session) so that I will able
Hi Shu Su,
the first point exactly pinpointed the issue I bumped into. I forgot to put
that dependency to "provided". Thank you!
Il giorno lun 24 giu 2019 alle ore 05:35 Shu Su ha
scritto:
> Hi Andrea
>
> Actually It’s caused by Flink’s ClassLoader. It’s because flink use
> parent
I posted my related observation here in a separated thread.
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Does-making-synchronize-call-might-choke-the-whole-pipeline-tc28383.html
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
private static void doWork(long tid) throws InterruptedException
{
if (!sortedTid.contains(tid)) {
sortedTid.add(tid);
}
// simulate a straggler, make the thread with the lowest tid a
slow
processor
if
Fabian,
Thank you for replying.
If I understand your previous comment correctly, I setup up a consumer with
parallelism 1 and connect a worker task with parallelism 2.
If worker thread one is making a block call and stuck for 60s, the consumer
thread should continue fetching from the partition
Ah, that's great!
Thanks for letting us know :-)
Am Mo., 24. Juni 2019 um 11:33 Uhr schrieb Wouter Zorgdrager <
w.d.zorgdra...@tudelft.nl>:
> Hi Fabian,
>
> Thanks for your reply. I managed to resolve this issue. Actually this
> behavior was not so unexpected, I messed up using xStream as a
Hi Fabian,
Thanks for your reply. I managed to resolve this issue. Actually this
behavior was not so unexpected, I messed up using xStream as a 'base' while
I needed to use yStream as a 'base'. I.e. yStream.element - 60 min <=
xStream.element <= yStream.element + 30 min. Interchanging both
Hi Wouter,
Not sure what is going wrong there, but something that you could try is to
use a custom watemark assigner and always return a watermark of 0.
When the source finished serving the watermarks, it emits a final
Long.MAX_VALUE watermark.
Hence the join should consume all events and store
Hi Ben,
Flink's Kafka consumers track their progress independent of any worker.
They keep track of the reading offset for themselves (committing progress
to Kafka is optional and only necessary to have progress monitoring in
Kafka's metrics).
As soon as a consumer reads and forwards an event, it
现象
使用这个pom ,在kafka 里看不到group id
com.alibaba.blink
flink-streaming-scala_2.11
blink-3.2.2
org.slf4j
slf4j-api
com.alibaba.blink
flink-connector-kafka-0.11_2.11
blink-3.2.0
org.slf4j
slf4j-api
使用开源版本，可以看到在kafka 看到group id
org.apache.flink
flink-streaming-java_2.12
你好！
请教下，假设我一个拓扑现在有三个sql已经在跑了，现在我想加第四个sql进来，那么我一定要重发这个拓扑么？有什么办法可以在不影响这三个sql计算的情况下，把第四个sql也加进来么？
刚接触flink，还处在摸索阶段，有什么资料能分享下更好啦~
--
风险能力中台-产品技术部-风控平台-计算平台
感谢大佬的解决，1.8确实已经修改
> On Jun 24, 2019, at 3:49 PM, Biao Liu wrote:
>
> 你好，看了下代码，1.7.2 确实有这问题，最新的代码已经 fix，见[1]
> 如果可以的话，升级到1.8.0就包含了该 fixing
>
> 1.
> https://github.com/apache/flink/commit/24c2e17c8d52ae2f0f897a5806a3a44fdf62b0a5
>
> 巫旭阳 于2019年6月24日周一 下午2:40写道：
>
>> 源码在 BucketingSink 615行
>>
你好，"Connection reset by peer" 是网络 TCP 层的异常，见[1]
建议查一下该 task 的下游消费者 task
1.
https://stackoverflow.com/questions/1434451/what-does-connection-reset-by-peer-mean
罗学焕/予之 于2019年6月18日周二 下午5:08写道：
> 大家好：
> Flink应用，如以100笔/s的交易量向kafka写入数据（数据量不大），Flink程序接受并处理数据，涉及到 20个左右的 流表 Join
> 。和大量的异步操作读取hbase 维表。
你好，建议自己先排查一下，把问题范围缩小，别人才好帮忙
可以看看 back pressure 具体慢在哪里，再具体分析
haibin <191560...@qq.com> 于2019年6月19日周三 上午5:50写道：
> hello，各位大佬：
>
>
> 在做实时etl的时候，source（kafka）->map->filter->flatmap->map->sink(kafka)这样流程etl的时候，发现处理速度很慢，有什么好的方法提高处理速度。
>有25个作业同时消费同一个topic（32个分区），会不会有性能问题？
>
>
>
> --
> Sent from:
你好，看了下代码，1.7.2 确实有这问题，最新的代码已经 fix，见[1]
如果可以的话，升级到1.8.0就包含了该 fixing
1.
https://github.com/apache/flink/commit/24c2e17c8d52ae2f0f897a5806a3a44fdf62b0a5
巫旭阳 于2019年6月24日周一 下午2:40写道：
> 源码在 BucketingSink 615行
> Path testPath = new Path(basePath, UUID.randomUUID().toString());
> try
你好，我觉得问题有点太抽象，建议先看看官网文档[1]，还有 Flink 自带的例子[2]
1.
https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/operators/windows.html
2.
https://github.com/apache/flink/tree/release-1.8/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/windowing
源码在 BucketingSink 615行
Path testPath = new Path(basePath, UUID.randomUUID().toString());
try (FSDataOutputStream outputStream = fs.create(testPath)) {
outputStream.writeUTF("hello");
} catch (IOException e) {
LOG.error("Could not create file for checking if truncate works.", e);
throw new
27 matches
Mail list logo
