Hi Team,
I am facing some issue with Custom Partitioner in flink Streaming . I am
using watcher to read file from folder and then I have to partition records
and send to sink .
- This is happening if parallelism > 1 .
- Checkpoint is enabled .
- If I don't use partitioner , then everything
Thanks Hequn!. Is it also thread safe when the same UDF is called multiple
times in the same record.
Is the UDF called sequentially for each fields a single record, I have a
query like -
select GetName(data.id, 'city'), GetName(data.id, 'zone') from ..
--
Sent from:
Thanks Qiu,
this is an useful information indeed, but this strategy will only reduce the
chance of re-execution whole graph. I think it won't help if TM crash, which
anyhow the whole cluster need to restart to redistribute states, am I right?
From: Congxian Qiu
Hi, Yow
I think there is another restart strategy in flink: region failover[1], but
I could not find the documentation, maybe someone else may help here, For
region failover, please take a look at this issue[2] before you use it. And
you can take a look at this FLIP[3].
[1]
Hi Qiu thanks again!
Based on my experience on Flink 1.3, when one of the TM crash the whole cluster
need to be restarted so i guess this is the recovery you mentioned. But it
sounds defeat the purpose of cluster as one TM crash should not crash the whole
cluster. May i know is this still the
Hi James,
Usually Flink ML is highly integrated with Scala. I did poke around to and
try to make the example work in Java and it does require a significant
amount of effort, but you can try:
First the implicit type parameters needs to be passed over to the execution
environment to generate the
Hi, Siew Wai Yow
Yes, David is correct, the TM must be recovered, the number of TMs before
and after the crash must be the same.
In my last reply, I want to say that the states may not on the same TM
after the crash. Sorry for the unclear description.
Siew Wai Yow 于2019年1月12日周六 下午6:44写道：
>
Thanks Qiu but David has different view from stackoverflow. He mentioned the
Crashed TM must be recovered.
https://stackoverflow.com/questions/54149134/what-happen-to-state-in-flink-task-manager-when-crash/54153686?noredirect=1#comment95144500_54153686
"The crashed TM must be recovered, and
Hi Alieh,
Which kind of API do you use? TableApi or SQL or DataStream or DataSet.
Would be great if you can show us some information about your pipeline or
provide a way to reproduce the problem.
Best, Hequn
On Sat, Jan 12, 2019 at 1:58 AM Alieh
wrote:
> Hello all,
>
> I have a very very long
Hi Anil,
It is thread-safe.
Each udf instance will only run in one task. And for each udf, it processes
data synchronously, i.e, the next record will not be processed until the
current record is processed.
Best, Hequn
On Sat, Jan 12, 2019 at 3:12 AM Anil wrote:
> Is the eval method invoked in
Hi,
I want to use Stochastic Outlier Selection in ML Library. But after I read the
document [1] , I find there is not Java example. Sorry I am not familiar with
Scala
So I appreciate that someone can share a Java example.
[1]
11 matches
Mail list logo
