Hi,
We put a lot of logging in our code the flow of events as they flow
between operators,
this is done at the DEBUG level.
Normally this logging would be turn off by setting the logging level to
INFO or above,
because logging everything is expensive and a cluttered log is harder to
analyze.
When we need to switch the logging to DEBUG level,
we currently change the configuration files, then restart the job,
so that Flink would pick up and use the updated files during redeployment.
The other option is to manually track down the location of each YARN
container,
and update the configuration files directly, but that's a lot of work,
and not possible if you don't have superuser right on that machine, due
to our security setup.
That's why we want an easier way to change the logging level.
Best regards,
Kien
On 4/20/2018 12:15 AM, Qian Ye wrote:
Thanks for your kind reply.
But I still have some question. What does the logging level mean in your
system? Why do you need to re-deploy the cluster to change the logging level?
As far as I know, the debugging information can be divided into level like
info, warn, error, etc. Is these information totally logged during the system
running? And then they can further be analyzed by category them by level?
Best,
Stephen
On Apr 19, 2018, at 7:19 AM, Kien Truong <duckientru...@gmail.com> wrote:
Hi,
Our most useful tool when debugging Flink is actually the simple log files,
because debugger just slow things down too much for us.
However, having to re-deploy the entire cluster to change the logging level is
a pain (we use YARN),
so we would really like an easier method to change the logging level at runtime.
Regards,
Kien
On 4/19/2018 5:53 AM, Qian Ye wrote:
Hi
I’m wondering if new debugging methods/tools are urgent for Flink development.
I know there already exists some debug methods for Flink, e.g., remote
debugging of flink
clusters(https://cwiki.apache.org/confluence/display/FLINK/Remote+Debugging+of+Flink+Clusters).
But are they are convenient enough?
Best regards.
