[
https://issues.apache.org/jira/browse/OOZIE-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807637#comment-16807637
]
Hadoop QA commented on OOZIE-3249:
----------------------------------
Testing JIRA OOZIE-3249
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any star imports
. {color:green}+1{color} the patch does not introduce any line longer than
132
. {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} Javadoc generation succeeded with the patch
. {color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:red}-1{color} There are [1] new bugs found below threshold in
[sharelib/oozie] that must be fixed.
. You can find the SpotBugs diff here (look for the red and orange ones):
sharelib/oozie/findbugs-new.html
. The most important SpotBugs errors are:
. At ShellMain.java:[line 92]: This usage of
java/lang/ProcessBuilder.<init>(Ljava/util/List;)V can be vulnerable to
Command Injection
. At ShellMain.java:[line 90]: At ShellMain.java:[line 89]
. At ShellMain.java:[line 91]
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 3169
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1070/
> [tools] Instrumentation log parser
> ----------------------------------
>
> Key: OOZIE-3249
> URL: https://issues.apache.org/jira/browse/OOZIE-3249
> Project: Oozie
> Issue Type: Improvement
> Components: tools
> Affects Versions: 5.0.0
> Reporter: Andras Piros
> Assignee: Andras Piros
> Priority: Major
> Attachments: OOZIE-3249.001.patch, OOZIE-3249.002.patch,
> oozie-instrumentation-localhost.log.2018-05-09,
> oozie-instrumentation-localhost.log.2018-05-09.out
>
>
> Oozie instrumentation logs contain a lot of information, but are difficult to
> parse, because per instrumentation log entry there is always one header line
> in plain text format (containing timestamp), and multiple other lines in JSON
> format (not containing timestamp). Those lines of course belong together.
> {noformat}
> 2018-05-02 02:48:13,426 INFO oozieinstrumentation:520 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-]
> {
> ...
> "counters" : {
> ...
> "callablequeue.executed" : {
> "count" : 5954144
> },
> ...
> "callablequeue.queued" : {
> "count" : 10596129
> },
> ...
> },
> ...
> }
> {noformat}
> There should be a simple script in {{tools/bin}} that takes as parameters:
> * input file name ({{-i}}), e.g. {{-i /path/to/oozie-instrumentation.log}}
> * output file name ({{-o}}), e.g. {{-o
> /path/to/oozie-instrumentation.log.out}}
> * parameters to extract ({{-p}}) in the format of
> {{path/to/json/value1,path/to/json/value2}}, in this case {{-p
> counters/callablequeue.executed/count,counters/callablequeue.queued/count}}
> The output file should contain in CSV format:
> * a header line containing column names for
> * one line per parsed input header / JSON lines, containing:
> ** first cell is the minutes part of the timestamp
> ** consecutive cells are parsed JSON values given each parameter to extract
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
