Hi Filip,
Could you share the version of the ZK server you are connecting to?
Best,
Zhanghao Chen
From: Filip Karnicki
Sent: Friday, March 4, 2022 23:12
To: user
Subject: Shaded zookeeper - curator mismatch?
Hi, I believe there's a mismatch in shaded
Hi Shane,
Flink provides a generic counter interface with a few implementations. The
default implementation SimpleCounter, which is not thread-safe, is used when
you calling counter(String name) on a MetricGroup. Therefore, you'll need to
use your own thread-safe implementation, check out the
kafka实时流关联hive的最新分区表数据时，关于缓存刷新的问题
'streaming-source.monitor-interval'='12 h'
这个参数我理解是：按照启动开始时间算起，每12小时读取一下最新分区的数据是吧？
还有个问题是读取最新分区的时间间隔之间，实时流里面进入了预关联新分区的数据，那么是不是就相当于关联的还是上一次的最新分区数据吧？
Hi,
I think you are looking for this answer from David:
https://stackoverflow.com/questions/69799181/flink-streaming-do-the-events-get-distributed-to-each-task-slots-separately-acc
I think then you could technically create your partitioner - though
little bit cubersome - by mapping your
Hi all,
I need to be able to control which slot a keyBy group goes to, in order to
compensate for a badly skewed dataset.
Any recommended approach to use here?
Previously (with a DataSet) I used groupBy followed by a withPartitioner, and
provided my own custom partitioner.
I posted this same
Hi all,
For Flink counters [1], are increment operations guaranteed to be atomic across
all parallel tasks? I.e., is there a guarantee that the counter values will not
be higher than expected?
Thanks,
Shane
---
[1]
Hi,
If I understand the code correctly, the only option is to implement a
custom SplitFetcherManager. There, you can either:
1) override maybeShutdownFinishedFetchers(), or
2) override createSplitFetcher() to return a custom fetcher; that
fetcher would override isIdle() and return true after some
I am working on developing a custom source with the new Source api.
What I'm noticing is that during periods of low incoming data it repeatedly
will shutdown and restart the fetchers when the split assignments are empty
and periodically added.
I get log message such as
Hi, I believe there's a mismatch in shaded zookeeper/curator dependencies.
I see that curator 4.2.0 needs zookeeper 3.5.4-beta, but it's still used in
flink-shaded-zookeeper-34, which as far as I can tell is used by flink
runtime 1.14.3
Hi, Puneet~
AFAIK, that should be expected behavior that jobs on crashed TaskManager
restarts. HA means there is no single point risk but Flink job still need
to through failover to ensure state and data consistency. You may refer
Hi Vidya,
> Why is the incremental checkpointing taking more time for the snapshot at the
> end of the window duration?
I guess that this is because the job is under back pressure on end of window.
You can expand the checkpoint details to see whether that the async duration of
each task is
Hi, Martijn,
Thanks for your information. It seems that the situation is similar to
what I know, I will follow FLIP-190.
Also congratulations on becoming a Flink committer!
Best regards,
Chen Zihao
Martijn Visser 于2022年3月4日周五 16:18写道：
> Hi,
>
> Per the documentation [1] stateful
Hi All!
We're running a statefun uber jar on a shared cloudera flink cluster,
the latter of which launches with some ancient protobuf dependencies
because of reasons[1].
Setting the following flink-config settings on the entire cluster
classloader.parent-first-patterns.additional:
Hi,
Per the documentation [1] stateful upgrades for SQL are currently not
supported when upgrading from one minor version to another. There's ongoing
work to improve this (via FLIP-190 [2]) but that's currently not yet
available.
Best regards,
Martijn Visser
https://twitter.com/MartijnVisser82
14 matches
Mail list logo
