Hi Jose
In some scenarios I think it would make sense to optionally allow the job
going on even if there are some exceptions.
But IMHO the scenario might be more likely to debug something but the
production. And in my own limited experience most of user actually could do
this themself i.e. Dealin
Ok, so I loaded the dump into Eclipse Mat and followed:
https://cwiki.apache.org/confluence/display/FLINK/Debugging+ClassLoader+leaks
- On the Histogram, I got over 30 entries for: ChildFirstClassLoader
- Then I clicked on one of them "Merge Shortest Path..." and picked
"Exclude all phantom/weak/s
Hi Harshit,
I guess you are using the reference code of the master instead of Flink
1.14 which is the version you are using.
TumblingEventTimeWindows is introduced in Flink 1.16 which is still not
released. However, it could be seen as an utility class and so I think you
could just copy it into y
I can look into RocksDB metrics, I need to configure Prometheus at some point
anyway. However, going back to the original question, is there no way to gain
more insight into this with the state processor API? You've mentioned potential
issues (too many states, missing compaction) but, with my ad
Hi Martin,
Thanks for your answer. Regarding my contribution, I will for sure check the
contributing guide and get familiar with Flink source code. I hope it will end
up well and I will be able to write that functionality.
Best regards,
Jasmin
On 19.04.2022., at 09:39, Martijn Visser
mailto:m
I am deploying as a docker on our servers, due to some restrictions I can
only pass Keystore URLs.
one option is yarn.ship-files !. can you help me with pointing to the
sample code, and how job manager can ship this file?
download as part of job's main function and send to all task managers..
wi
Hello Community,
We are trying to adopt flink-stop instead of orchestrating the stop
manually. While using the command, it failed with error "Operation not
found under key:
org.apache.flink.runtime.rest.handler.job.AsynchronousJobOperationKey@9fbfc15a".
the job eventually stopped but I wanted to s
Also https://shopify.engineering/optimizing-apache-flink-applications-tips
might be helpful (has a section on profiling, as well as classloading).
On Tue, Apr 19, 2022 at 4:35 AM Chesnay Schepler wrote:
> We have a very rough "guide" in the wiki (it's just the specific steps I
> took to debug an
> I assume that when you say "new states", that is related to new descriptors
> with different names? Because, in the case of windowing for example, each
> window "instance" has its own scoped (non-global and keyed) state, but that's
> not regarded as a separate column family, is it?
Yes, that's
Dear Team,
I am new to pyflink and request for your support in issue I am facing with
Pyflink. I am using Pyflink version 1.14.4 & using reference code from
pyflink getting started pages.
I am getting following error .
ImportError: cannot import name 'TumblingEventTimeWindows' from
'pyfl
Hi Flink Users,
Does anyone know what happened to the /status endpoint of a job?
Calling /jobs/0c39e6ce662379449e7f7f965ff1eee0/status gives me a 404.
Thanks & best, Peter
Hi,
I have a question regarding flink checkpointing configuration.
I am obtaining my knowledge from the official docs here:
https://nightlies.apache.org/flink/flink-docs-stable/docs/deployment/config/
and running Flink 1.14.4
I would like to be able to do a checkpoint every 10 minutes which at l
Hello,
We are actually working on a similar problem against S3. The checkpointing
thing got me thinking if the checkpoint would indeed succeed with a large
backlog of files. I always imagined that SplitEnumerator lists all
available files and SourceReader is responsible for reading those files
aft
Hi,
After the previous JobManager fails, K8S start the new JobManager, but the
Leader saved in HA is still the old JobManager address. After the Dispatcher
gets the old JobManager leader, it will try to connect to it.
This error can be ignored, and it will return to normal after waiting for a
p
Hi Roman,
I assume that when you say "new states", that is related to new descriptors
with different names? Because, in the case of windowing for example, each
window "instance" has its own scoped (non-global and keyed) state, but that's
not regarded as a separate column family, is it?
For the
We have a very rough "guide" in the wiki (it's just the specific steps I
took to debug another leak):
https://cwiki.apache.org/confluence/display/FLINK/Debugging+ClassLoader+leaks
On 19/04/2022 12:01, huweihua wrote:
Hi, John
Sorry for the late reply. You can use MAT[1] to analyze the dump fil
Hi, John
Sorry for the late reply. You can use MAT[1] to analyze the dump file. Check
whether have too many loaded classes.
[1] https://www.eclipse.org/mat/
> 2022年4月18日 下午9:55，John Smith 写道：
>
> Hi, can anyone help with this? I never looked at a dump file before.
>
> On Thu, Apr 14, 2022 at
Hi Alexis,
Thanks a lot for the information,
MANIFEST files list RocksDB column families (among other info); ever
growing size of these files might indicate that some new states are
constantly being created.
Could you please confirm that the number of state names is constant?
> Could you confirm
Hi Jasmin,
Vertica is not implemented as a JDBC dialect, which is a requirement for
Flink to support this. You can find an overview of the currently supported
JDBC dialects in the documentation [1]. It would be interesting if you
could contribute support for Vertica towards Flink.
Best regards,
Hi,
I'm using Flink-1.14.4 and failed to load in WindowReaderFunction the state
of a stateful trigger attached to a session window.
I found that the following data become available in WindowReaderFunction:
- the state defined in the ProcessWindowFunction
- the registered timers of the stateful tri
20 matches
Mail list logo
