Could it be same as FLINK-21028[1] (titled as “Streaming application didn’t
stop properly”, fixed in 1.11.4, 1.12.2, 1.13.0) ?
[1]: https://issues.apache.org/jira/browse/FLINK-21028
Best,
Kezhu Wang
On June 8, 2021 at 22:54:10, Yun Gao (yungao...@aliyun.com) wrote:
Hi Thomas,
I tried but do
们都会过滤掉非期望的回调（返回 TriggerResult.CONTINUS）。
Best,
Kezhu Wang
On March 8, 2021 at 15:37:52, smq (374060...@qq.com) wrote:
Trigger 抽象类其中有两个方法，onProcessingTime （）和OnEventTime （），在源码描述中，当trigger
context设置的定时器触发时会调用上面两个方法。但是在实际测试中，发现，就算没有设置定时器，上面的方法也能在水印超过窗口最大时间之后触发。请问有人知道了解过这个吗。
你需要发邮件到
Best,
Kezhu Wang
On March 25, 2021 at 10:15:56, drewfranklin (drewfrank...@163.com) wrote:
退订
To + 1) %
numberOfChannels;
return nextChannelToSendTo;
}
一个随机，一个严格 round-robin。
Best,
Kezhu Wang
On March 15, 2021 at 22:02:33, 赢峰 (si_ji_f...@163.com) wrote:
Flink 中 shuffle Partitioner 和 rebalance partitoner 有什么区别？
nyu...@outlook.com) wrote:
你好，我参考StatefulSinkWriterOperator重写了迁移方法，进行测试，还是遇到了上面的错误。 这个错误似乎发生在source的
initializeState之前。有没有可能flink在读取savepoint文件时，就做了反序列化，在这个过程中失败了。请问有什么办法能帮忙定位吗？
感谢~
2021年3月11日 上午11:36，Kezhu Wang 写道：
新的集群使用的是更新之后的 pulsar connector ？我看了下 pulsar-flink 的代码，这个更新对 state 是破
新的集群使用的是更新之后的 pulsar connector ？我看了下 pulsar-flink 的代码，这个更新对 state 是破坏性的。
+unionOffsetStates = stateStore.getUnionListState(
+new ListStateDescriptor<>(
+OFFSETS_STATE_NAME,
+TypeInformation.of(new TypeHint>() {
+
Hi all,
My BAD!!!
Sorry for apparent mess up in that moment.
I will write a separate test for stream iterations.
The stateful function part should be a separated issue.
Best,
Kezhu Wang
On March 4, 2021 at 22:13:48, Piotr Nowojski (piotr.nowoj...@gmail.com)
wrote:
Hi Meissner,
Can you
ep memory from freeing.
Besides this, I saw an attachment “task-manager-thrad-print.txt” in initial
mail, when and where did you capture ? Task Manager ? Is there any job
still running ?
Best,
Kezhu Wang
On March 1, 2021 at 18:38:55, Tamir Sagi (tamir.s...@niceactimize.com)
wrote:
lot of ProducerFencedExceptions and
all jobs go into a restart cycle.
FLINK-11654: https://issues.apache.org/jira/browse/FLINK-11654
Best,
Kezhu Wang
On February 28, 2021 at 22:35:02, Morgan Geldenhuys (
morgan.geldenh...@tu-berlin.de) wrote:
Greetings all,
I am having an issue instantiating multi
Hi,
You could also try `cancel —withSavepoint [savepointDir]` even it is in
deprecation. Comparing to take-savepoints and then cancel approach, there
will be no checkpoints in between. This may be important if there are two
phase commit operators in your job.
Best,
Kezhu Wang
On February 28
#pipeline-max-parallelism
Best,
Kezhu Wang
On February 28, 2021 at 16:45:03, Padarn Wilson (pad...@gmail.com) wrote:
Hi all,
Sorry for the basic question, but is it possible to set max
parallelism using the flink conf file, rather than explicitly in code:
https://ci.apache.org/projects/flink
Hi,
Glad to hear.
Normally, you would not encounter this if there are massive data.
`WatermarkStrategy.withIdleness` could be more appropriate in production.
Best,
Kezhu Wang
On February 24, 2021 at 22:35:11, sagar (sagarban...@gmail.com) wrote:
Thanks Kezhu, It worked!!!
On Wed, Feb 24
Hi,
Thanks for reporting. I think it is a Flink bug and have created
FLINK-21522 for it. You could track progress there.
FLINK-21522: https://issues.apache.org/jira/browse/FLINK-21522
Best,
Kezhu Wang
On February 28, 2021 at 00:59:04, Meissner, Dylan (
dylan.t.meiss...@nordstrom.com) wrote
.
Or you could try `WatermarkStrategy.withIdleness`.
Best,
Kezhu Wang
On February 24, 2021 at 15:43:47, sagar (sagarban...@gmail.com) wrote:
It is fairly simple requirement, if I changed it to PRocessing time it
works fine , but not working with event time..help appreciated!
On Wed, Feb 24, 2021 at 10
I saw one potential issue. Your timestamp assigner returns timestamp in
second resolution while Flink requires millisecond resolution.
Best,
Kezhu Wang
On February 24, 2021 at 11:49:59, sagar (sagarban...@gmail.com) wrote:
I have simple flink stream program, where I am using socket as my
-queryable-state/flink-queryable-state-runtime/src/test/java/org/apache/flink/queryablestate/itcases/AbstractQueryableStateTestBase.java#L912
Best,
Kezhu Wang
On February 19, 2021 at 20:27:41, Sandeep khanzode (sand...@shiftright.ai)
wrote:
Hello,
Is there an example setup of Queryable State
Piotr is right. So just ignore my words. It is the price of going deep down
the rabbit hole:-).
Best,
Kezhu Wang
On February 17, 2021 at 23:48:30, Piotr Nowojski (pnowoj...@apache.org)
wrote:
Note^2: InputSelectable is `@PublicEvolving` API, so it can be used.
However as Timo pointed out
not be bounded before FLIP-147 delivered.
[1] FLINK-21392: https://issues.apache.org/jira/browse/FLINK-21392
Best,
Kezhu Wang
On February 17, 2021 at 22:58:23, Arvid Heise (ar...@apache.org) wrote:
Note that the question is also posted on SO [1].
[1]
https://stackoverflow.com/questions/66236004
it is canonical or not.
Best,
Kezhu Wang
On February 17, 2021 at 13:03:42, Salva Alcántara (salcantara...@gmail.com)
wrote:
What is the canonical way to accomplish this:
>Given a ConnectedStreams, e.g., a CoFlatMap UDF, how to prevent any
processing of the data stream until >the control
With an initial `y`, I think you could compute new `y` on new stream value.
Upon recovering from checkpoint, may be `KeyedStateBackend.applyToAllKeys`
could help you to rebuild an initial `y`.
Best,
Kezhu Wang
On February 17, 2021 at 13:09:39, Salva Alcántara (salcantara...@gmail.com)
wrote:
I
Hi Sandeep,
I must mislead you by inaccurate words. I did not mean using
CoGroupedStreams, but only CoGroupedStreams.apply as reference for how to
union streams together and keyBy them. This way you can have all three
streams’ states in downstream without duplication.
Best,
Kezhu Wang
chieve this. Thanks.
On 10-Feb-2021, at 12:44 PM, Kezhu Wang wrote:
Flink has broadcast state to broadcast one stream to other in case you are
not aware of it. It actually duplicates state.
1. Broadcast state:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_stat
Flink has broadcast state to broadcast one stream to other in case you are
not aware of it. It actually duplicates state.
1. Broadcast state:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html
Best,
Kezhu Wang
On February 10, 2021 at 13:03:36, Sandeep
(a) It is by design. For keyed state, you can only access state for that
key, not others. If you want one value per key, ValueState fits more
appropriate that MapState.
(b) state-processor-api aims to access/create/modify/upgrade offline
savepoint but not running state. Queryable state may meet
e helpful.
Best,
Kezhu Wang
On February 4, 2021 at 23:17:07, Antonis Papaioannou (papai...@ics.forth.gr)
wrote:
Hi,
reading through the documentation regarding waterrmarks, it is not clear to
me if watermarks are also used by map/flatmpat operators or just by window
operators.
My application r
自定义 state 的 type-serializer，可以尝试下 SimpleVersionedSerializer/avro/protobuf,
etc.
复杂状态应该尽量避免使用依赖反射的 serializer，java Serializable 同理，这些都不好做 state upgrade。
On January 31, 2021 at 11:29:25, 赵一旦 (hinobl...@gmail.com) wrote:
这个问题有人知道吗？
reduceFunction 是和 ReducingStateDescriptor 配合的， 并不是
“window的”。“WindowOperator” 的 function 是 “InternalWindowFunction”，这个可以是
“RichFunction”。
Flink 中的 state 除了可以绑定 key，也可以绑定 namespace，只是 namespace
并没有直接暴露给用户。如果需要的话，可以自己写个 WindowOperator，暴露个 WindowFunction。
Interface WindowFunction {
// You could
https://github.com/apache/iceberg/blob/master/flink/src/main/java/org/apache/iceberg/flink/data/FlinkOrcWriters.java#L259
On January 23, 2021 at 13:49:23, 赵一旦 (hinobl...@gmail.com) wrote:
目前通过自定义OrcBulkWriterFactory方式，拿到一个一个的ColumnVector，然后设置值。
28 matches
Mail list logo
