; *From: *Kostas Kloudas
> *Date: *Monday, June 8, 2020 at 2:35 AM
> *To: *Arvid Heise
> *Cc: *M Singh , User-Flink
> *Subject: *Re: Stopping a job
>
>
>
> What Arvid said is correct.
>
> The only thing I have to add is that "stop" allows also exactly-once sink
: Kostas Kloudas
Date: Monday, June 8, 2020 at 2:35 AM
To: Arvid Heise
Cc: M Singh , User-Flink
Subject: Re: Stopping a job
What Arvid said is correct.
The only thing I have to add is that "stop" allows also exactly-once sinks to
push out their buffered data to their final destin
What Arvid said is correct.
The only thing I have to add is that "stop" allows also exactly-once sinks
to push out their buffered data to their final destination (e.g.
Filesystem). In other words, it takes into account side-effects, so it
guarantees exactly-once end-to-end, assuming that you are
It was before I joined the dev team, so the following are kind of
speculative:
The concept of stoppable functions never really took off as it was a bit of
a clumsy approach. There is no fundamental difference between stopping and
cancelling on (sub)task level. Indeed if you look in the twitter
Hi Arvid:
Thanks for the links.
A few questions:
1. Is there any particular interface in 1.9+ that identifies the source as
stoppable ?2. Is there any distinction b/w stop and cancel in 1.9+ ?3. Is
there any list of sources which are documented as stoppable besides the one
listed in
Yes, it seems as if FlinkKinesisConsumer does not implement it.
Here are the links to the respective javadoc [1] and code [2]. Note that in
later releases (1.9+) this interface has been removed. Stop is now
implemented through a cancel() on source level.
In general, I don't think that in a
Hi Arvid:
I check the link and it indicates that only Storm SpoutSource, TwitterSource
and NifiSource support stop.
Does this mean that FlinkKinesisConsumer is not stoppable ?
Also, can you please point me to the Stoppable interface mentioned in the link
? I found the following but am not
Hi,
could you check if this SO thread [1] helps you already?
[1]
https://stackoverflow.com/questions/53735318/flink-how-to-solve-error-this-job-is-not-stoppable
On Thu, Jun 4, 2020 at 7:43 PM M Singh wrote:
> Hi:
>
> I am running a job which consumes data from Kinesis and send data to
>
Hi:
I am running a job which consumes data from Kinesis and send data to another
Kinesis queue. I am using an older version of Flink (1.6), and when I try to
stop the job I get an exception
Caused by: java.util.concurrent.ExecutionException:
agrebin
>> wrote:
>>
>>> Could you share the source code of your RichAsyncFunction?
>>> Looks like netty threads of vertx are still being shutdown after user
>>> code class loader has been shutdown.
>>> It probably means that RichAsyncFunction
etty threads of vertx are still being shutdown after user code
> class loader has been shutdown.
> It probably means that RichAsyncFunction was not closed properly or not all
> resources have been fully freed there (logging your RichAsyncFunction.close
> could help).
> Do you
gt; It probably means that RichAsyncFunction was not closed properly or not
>> all resources have been fully freed there (logging your
>> RichAsyncFunction.close could help).
>> Do you mean cancelation by stopping the job?
>>
>> On Wed, May 15, 2019 at 10:02 PM
shutdown.
> It probably means that RichAsyncFunction was not closed properly or not
> all resources have been fully freed there (logging your
> RichAsyncFunction.close could help).
> Do you mean cancelation by stopping the job?
>
> On Wed, May 15, 2019 at 10:02 PM John Smith
> wr
RichAsyncFunction.close
could help).
Do you mean cancelation by stopping the job?
On Wed, May 15, 2019 at 10:02 PM John Smith wrote:
> So these are the two exceptions I see in the logs...
>
> Exception in thread "vert.x-worker-thread-0" Exception in thread
> "
So these are the two exceptions I see in the logs...
Exception in thread "vert.x-worker-thread-0" Exception in thread
"vert.x-internal-blocking-0" java.lang.NoClassDefFoundError:
io/netty/util/concurrent/FastThreadLocal
at
Hi John,
could you share the full stack trace or better logs?
It looks like something is trying to be executed in vertx.io code after the
local task has been stopped and the class loader for the user code has been
unloaded. Maybe from some daemon thread pool.
Best,
Andrey
On Wed, May 15, 2019
Hi,
I'm using vertx.io as an async JDBC client for a RichAsyncFunction it works
fine but when I stop the job I get...
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
io/vertx/core/impl/VertxImpl$SharedWorkerPool
Is there a way to avoid/fix this?
:6123 to connect to JobManager.
>
>
> The program finished with the following exception:
>
> java.lang.Exception: Stopping the job with ID
> 7f5a5f95353a2b486572f4cdefa813b8 failed.
> at org.apache
I can't stop the job, every time the exception like follows.
Retrieving JobManager.
Using address /10.6.192.141:6123 to connect to JobManager.
The program finished with the following exception:
java.lang.Exception: Stopping the job
Hi everyone,
I have an integration test for which a use a LocalStreamEnvironment.
Currently, the Flink Job is started in a separated thread, which I
interrupt after some time and then do some assertions.
In this situation is there a better way to stop/cancel a running job in
20 matches
Mail list logo
