[
https://issues.apache.org/jira/browse/OOZIE-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655423#comment-17655423
]
Prabhu Joseph edited comment on OOZIE-3712 at 1/6/23 12:30 PM:
---------------------------------------------------------------
Thanks [~dionusos] for reviewing this.
{quote} May I suggest to have this property fine tuned for system level, for
workflow level and for action level too?
{quote}
Yes this gives more control to user. Will use this way.
{quote}Also what do you suggest as a "verbose" option, what debug level should
be turned on? Verbose class loading? Debug/Trace level logging for the given
action? Should we allow the user to somehow configure in an easy way (switches)
what verbose options they want?
{quote}
I had only Java verbose class loading option (-verbose:class) in mind, as that
is useful to find jar conflicts leading to NoSuchMethodErrors. But we could use
it to configure any common JVM flags which is needed across all JVMs part of a
specific Action or all Actions.
Examples:
A. Enable Java Verbose Class Loading
{code:java}
oozie.workflow.action.common.jvmflags=-verbose:class
{code}
B. Enable log4j debug option
{code:java}
oozie.workflow.action.common.jvmflags=-Dlog4j.debug
{code}
C. Print GC Details
{code:java}
oozie.workflow.action.common.jvmflags=-XX:+PrintGCDetails and
-XX:+PrintGCTimeStamps
{code}
bq. Anyway, I like the idea. Do you plan working on it?
Yes, my teammate has started working on it. Will assign it to her once JIRA
Account is created.
was (Author: prabhu joseph):
Thanks [~dionusos] for reviewing this.
{quote} May I suggest to have this property fine tuned for system level, for
workflow level and for action level too?
{quote}
Yes this gives more control to user. Will use this way.
{quote}Also what do you suggest as a "verbose" option, what debug level should
be turned on? Verbose class loading? Debug/Trace level logging for the given
action? Should we allow the user to somehow configure in an easy way (switches)
what verbose options they want?
{quote}
I had only Java verbose class loading option (-verbose:class) in mind, as that
is useful to find jar conflicts leading to NoSuchMethodErrors. But we could use
it configure any common JVM flags with default maybe -verbose:class and which
will be set across all JVMs part of specific Action or all Actions.
Examples:
A. Enable Java Verbose Class Loading
{code:java}
oozie.workflow.action.common.jvmflags=-verbose:class
{code}
B. Enable log4j debug option
{code:java}
oozie.workflow.action.common.jvmflags=-Dlog4j.debug
{code}
C. Print GC Details
{code:java}
oozie.workflow.action.common.jvmflags=-XX:+PrintGCDetails and
-XX:+PrintGCTimeStamps
{code}
bq. Anyway, I like the idea. Do you plan working on it?
Yes, my teammate has started working on it. Will assign it to her once JIRA
Account is created.
> Oozie Verbose Option at Workflow Level which sets Java Verbose to all JVMs
> involved in the actions
> --------------------------------------------------------------------------------------------------
>
> Key: OOZIE-3712
> URL: https://issues.apache.org/jira/browse/OOZIE-3712
> Project: Oozie
> Issue Type: Improvement
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
>
> The most common issue in Oozie is NoSuhMethodError. And the best way to debug
> is to set the JVM option -verbose to the right JVM which has failed. The
> challenge is to identify which JVM failed and what is the OPTS option through
> which -verbose can be set. In MapReduce Action, MRAppMaster, MapTask and
> ReduceTask each have different way of passing OPTS. In Spark Action, Spark
> ApplicationMaster, Driver and Executors have different way of passing OPTS.
> This Jira intends to expose one config say oozie.workflow.enable.verbose=true
> in job.properties to user and internally Oozie sets -verbose to all the JVMs
> involved in the Actions (SparkAction, SqoopAction or MapReduceAction). This
> will be very useful for debugging the most common issue NoSuhMethodError.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
