[jira] [Commented] (OOZIE-3298) OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED
[
https://issues.apache.org/jira/browse/OOZIE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590773#comment-16590773
]
Hadoop QA commented on OOZIE-3298:
--
Testing JIRA OOZIE-3298
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:red}-1{color} There are [1] new bugs found below threshold in [core]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
core/findbugs-new.html
. The most important FindBugs errors are:
. At MapReduceActionExecutor.java:[lines 566-588]:
org.apache.oozie.action.hadoop.MapReduceActionExecutor$YarnApplicationIdComparator
implements Comparator but not Serializable
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2938
.{color:orange}Tests failed at first run:{color}
TestCoordActionInputCheckXCommand#testNone
TestCoordActionsKillXCommand#testActionKillCommandDate
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/780/
> OYA: external ID is not filled properly and failing MR job is treated as
> SUCCEEDED
> --
>
> Key: OOZIE-3298
> URL: https://issues.apache.org/jira/browse/OOZIE-3298
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-3298.001.patch, OOZIE-3298.002.patch,
> OOZIE-3298.003.patch, OOZIE-3298.004.patch, OOZIE-3298.005.patch,
> OOZIE-3298.007.patch
>
>
> When a MapReduce action is launched from Oozie (OYA), we don't properly fill
> the external ID field. It gets populated with the YARN id of the LauncherAM,
> not with the id of the actual MR job. If the MR job is succesfully submitted
> then fails, it will be treated as a successfully executed action, which is
> very misleading and can potentially confuse Oozie users.
--
Failed: OOZIE-3298 PreCommit Build #780
Jira: https://issues.apache.org/jira/browse/OOZIE-3298 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/780/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 1.77 MB...] [DEBUG] There are no new bugs found in [sharelib/distcp]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [sharelib/spark]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [client]. [ERROR] There are [1] new bugs found below threshold in total that must be fixed. [TRACE] FindBugs diffs checked and reports created [TRACE] Summary file size is 2838 bytes [TRACE] Full summary file size is 1783 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS grep: /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/TESTS.out: No such file or directory Running test-patch task DISTRO Testing JIRA OOZIE-3298 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces -1 the patch contains 1 line(s) longer than 132 characters +1 the patch adds/modifies 6 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warning(s) WARNING: the current HEAD has 100 Javadoc warning(s) +1 the patch does not seem to introduce new Javadoc error(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [1] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [webapp]. -1 There are [1] new bugs found below threshold in [core] that must be fixed. You can find the FindBugs diff here (look for the red and orange ones): core/findbugs-new.html The most important FindBugs errors are: At MapReduceActionExecutor.java:[lines 566-588]: org.apache.oozie.action.hadoop.MapReduceActionExecutor$YarnApplicationIdComparator implements Comparator but not Serializable +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [fluent-job/fluent-job-api]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [client]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2938 Tests failed at first run: TestCoordActionInputCheckXCommand#testNone TestCoordActionsKillXCommand#testActionKillCommandDate For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/780/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
[jira] [Commented] (OOZIE-3298) OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED
[ https://issues.apache.org/jira/browse/OOZIE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590675#comment-16590675 ] Hadoop QA commented on OOZIE-3298: -- PreCommit-OOZIE-Build started > OYA: external ID is not filled properly and failing MR job is treated as > SUCCEEDED > -- > > Key: OOZIE-3298 > URL: https://issues.apache.org/jira/browse/OOZIE-3298 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.0.0 >Reporter: Peter Bacsko >Assignee: Andras Piros >Priority: Blocker > Fix For: 5.1.0 > > Attachments: OOZIE-3298.001.patch, OOZIE-3298.002.patch, > OOZIE-3298.003.patch, OOZIE-3298.004.patch, OOZIE-3298.005.patch, > OOZIE-3298.007.patch > > > When a MapReduce action is launched from Oozie (OYA), we don't properly fill > the external ID field. It gets populated with the YARN id of the LauncherAM, > not with the id of the actual MR job. If the MR job is succesfully submitted > then fails, it will be treated as a successfully executed action, which is > very misleading and can potentially confuse Oozie users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3298) OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED
[ https://issues.apache.org/jira/browse/OOZIE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3298: Attachment: OOZIE-3298.007.patch > OYA: external ID is not filled properly and failing MR job is treated as > SUCCEEDED > -- > > Key: OOZIE-3298 > URL: https://issues.apache.org/jira/browse/OOZIE-3298 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.0.0 >Reporter: Peter Bacsko >Assignee: Andras Piros >Priority: Blocker > Fix For: 5.1.0 > > Attachments: OOZIE-3298.001.patch, OOZIE-3298.002.patch, > OOZIE-3298.003.patch, OOZIE-3298.004.patch, OOZIE-3298.005.patch, > OOZIE-3298.007.patch > > > When a MapReduce action is launched from Oozie (OYA), we don't properly fill > the external ID field. It gets populated with the YARN id of the LauncherAM, > not with the id of the actual MR job. If the MR job is succesfully submitted > then fails, it will be treated as a successfully executed action, which is > very misleading and can potentially confuse Oozie users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 68237: OOZIE-3298 OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68237/ --- (Updated Aug. 23, 2018, 6:35 p.m.) Review request for oozie, Attila Sasvari and Peter Bacsko. Repository: oozie-git Description --- OOZIE-3298 OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED Diffs (updated) - core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 8f0f244013932476d8ae454d224f235948529b34 core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 83a23f5220aa72ba15edc8b98ef80a74213fcee8 core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java f39bba2c691435354dac6da7794e5142b511d937 core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java a31079a41d30677d35a253a4a69505c21aa585f6 core/src/test/java/org/apache/oozie/action/hadoop/TestYarnApplicationIdComparator.java PRE-CREATION core/src/test/java/org/apache/oozie/action/hadoop/TestYarnApplicationIdFinder.java PRE-CREATION core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java 893405e38ad71d22460768b76ed03ac4d9a0b95d sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java f460b6bd11f60dfb397c6bba82be1427c2d1b570 Diff: https://reviews.apache.org/r/68237/diff/4/ Changes: https://reviews.apache.org/r/68237/diff/3-4/ Testing --- Tested on a real cluster, plus added test cases to `TestMapReduceActionExecutor` and new test classes. Please note that I could find no proper way of actually getting MapReduce to start a new job while using `OozieClient#submit()` - apparently no YARN child application is created. Please advise what's the best way to advance, maybe call `JobClient#submitJob()` directly from `ActionExecutorTestCase#startWorkflowAndFailChildMRJob()`. Thanks, András Piros
[jira] [Commented] (OOZIE-3229) Improved filtering options in V2SLAServlet
[
https://issues.apache.org/jira/browse/OOZIE-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590555#comment-16590555
]
Hadoop QA commented on OOZIE-3229:
--
Testing JIRA OOZIE-3229
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2963
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/779/
> Improved filtering options in V2SLAServlet
> --
>
> Key: OOZIE-3229
> URL: https://issues.apache.org/jira/browse/OOZIE-3229
> Project: Oozie
> Issue Type: New Feature
> Components: client
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Andras Piros
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3229-1.patch, OOZIE-3229-2.patch,
> OOZIE-3229-3.patch, OOZIE-3229-4.patch, OOZIE-3229-5.patch, OOZIE-3229-6.patch
>
>
> Currently we can apply a range of filters on top of {{V2SLAServlet}} that can
> be used in a rich but undocumented set of ways:
> * {{id}}
> * {{parent_id}}
> * {{event_status}}
> * {{app_name}}
> * {{nominal_start}}
> * {{nominal_end}}
> Need to refactor {{V2SLAServlet}} to feature:
> * a richer set of {{SLAEvent}}, {{SLARegistration}}, and {{SLASummary}}
> filtering based on their attributes
> * filter options will always be {{AND}}-ed, never {{OR}}-ed to each other
> * maintain compatibility with the parameter names and behavior used thus far
> * remove {{SLASummaryFilter}} and refactor
> {{SLASummaryGetForFilterJPAExecutor}} as just another possibility for
> confusion
> * document new functionality with rich use case / example library so that
> users can leverage
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3229 PreCommit Build #779
Jira: https://issues.apache.org/jira/browse/OOZIE-3229
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/779/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.76 MB...]
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [webapp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [fluent-job/fluent-job-api].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2448 bytes
[TRACE] Full summary file size is 1399 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
grep:
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/TESTS.out:
No such file or directory
Running test-patch task DISTRO
Testing JIRA OOZIE-3229
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 1 line(s) longer than 132 characters
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [fluent-job/fluent-job-api].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2963
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/779/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0100 31020 0 100 3102 0 4865 --:--:-- --:--:-- --:--:--
4862{"self":"https://issues.apache.org/jira/rest/api/2/issue/13155493/comment/16590555","id":"16590555","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small=hadoopqa=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall=hadoopqa=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium=hadoopqa=10393"},"displayName":"Hadoop
[jira] [Commented] (OOZIE-2684) Bad database schema error for WF_ACTIONS table
[
https://issues.apache.org/jira/browse/OOZIE-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590386#comment-16590386
]
Andras Piros commented on OOZIE-2684:
-
Thanks [~kmarton]! Can you please use
[*{{\@SuppressFBWarnings}}*|http://findbugs.sourceforge.net/api/edu/umd/cs/findbugs/annotations/SuppressFBWarnings.html]
on method level in case FindBugs errors are not applicable? Thanks!
> Bad database schema error for WF_ACTIONS table
> --
>
> Key: OOZIE-2684
> URL: https://issues.apache.org/jira/browse/OOZIE-2684
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Julia Kinga Marton
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-2684-00.patch, OOZIE-2684-001.patch,
> OOZIE-2684-002.patch
>
>
> In {{SchemaCheckerService}}, Oozie compares expected and found indexed
> columns and generates the below error message which could be confusing to the
> users.
> {noformat}
> 2016-09-16 12:39:26,564 INFO SchemaCheckXCommand:520 -
> SERVER[c6402.ambari.apache.org] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] About to check database schema
> 2016-09-16 12:39:26,703 ERROR SchemaCheckXCommand:517 -
> SERVER[c6402.ambari.apache.org] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] Found [1] extra indexes for columns in table [WF_ACTIONS]: [wf_id]
> 2016-09-16 12:39:26,723 ERROR SchemaCheckXCommand:517 -
> SERVER[c6402.ambari.apache.org] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] Database schema is BAD! Check previous error log messages for
> details
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3229) Improved filtering options in V2SLAServlet
[
https://issues.apache.org/jira/browse/OOZIE-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590383#comment-16590383
]
Hadoop QA commented on OOZIE-3229:
--
PreCommit-OOZIE-Build started
> Improved filtering options in V2SLAServlet
> --
>
> Key: OOZIE-3229
> URL: https://issues.apache.org/jira/browse/OOZIE-3229
> Project: Oozie
> Issue Type: New Feature
> Components: client
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Andras Piros
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3229-1.patch, OOZIE-3229-2.patch,
> OOZIE-3229-3.patch, OOZIE-3229-4.patch, OOZIE-3229-5.patch, OOZIE-3229-6.patch
>
>
> Currently we can apply a range of filters on top of {{V2SLAServlet}} that can
> be used in a rich but undocumented set of ways:
> * {{id}}
> * {{parent_id}}
> * {{event_status}}
> * {{app_name}}
> * {{nominal_start}}
> * {{nominal_end}}
> Need to refactor {{V2SLAServlet}} to feature:
> * a richer set of {{SLAEvent}}, {{SLARegistration}}, and {{SLASummary}}
> filtering based on their attributes
> * filter options will always be {{AND}}-ed, never {{OR}}-ed to each other
> * maintain compatibility with the parameter names and behavior used thus far
> * remove {{SLASummaryFilter}} and refactor
> {{SLASummaryGetForFilterJPAExecutor}} as just another possibility for
> confusion
> * document new functionality with rich use case / example library so that
> users can leverage
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 68140: OOZIE-3229 Improved filtering options in V2SLAServlet
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68140/ --- (Updated Aug. 23, 2018, 3:27 p.m.) Review request for oozie, András Piros and Kinga Marton. Changes --- Removing my SelectQuery (+builder) class, using JPA CriteriaAPI instead. Repository: oozie-git Description --- Added lots of new filter fields, also refactored SLASummaryGetForFilterJPAExecutor class to eliminate FindBugs errors. I'm not sure about the filter field names. Diffs (updated) - client/src/main/java/org/apache/oozie/client/OozieClient.java 949b4532 core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java b54161e9 core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java 3982d1e0 core/src/test/java/org/apache/oozie/executor/jpa/sla/TestSLASummaryGetForFilterJPAExecutor.java PRE-CREATION core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java aa633225 docs/src/site/twiki/DG_SLAMonitoring.twiki c91c227d webapp/src/main/webapp/console/sla/css/oozie-sla.css d2f2deee webapp/src/main/webapp/console/sla/js/oozie-sla.js 2ecad228 webapp/src/main/webapp/console/sla/oozie-sla.html e5bf6275 Diff: https://reviews.apache.org/r/68140/diff/4/ Changes: https://reviews.apache.org/r/68140/diff/3-4/ Testing --- Thanks, Andras Salamon
[jira] [Updated] (OOZIE-3229) Improved filtering options in V2SLAServlet
[
https://issues.apache.org/jira/browse/OOZIE-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated OOZIE-3229:
--
Attachment: OOZIE-3229-6.patch
> Improved filtering options in V2SLAServlet
> --
>
> Key: OOZIE-3229
> URL: https://issues.apache.org/jira/browse/OOZIE-3229
> Project: Oozie
> Issue Type: New Feature
> Components: client
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Andras Piros
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3229-1.patch, OOZIE-3229-2.patch,
> OOZIE-3229-3.patch, OOZIE-3229-4.patch, OOZIE-3229-5.patch, OOZIE-3229-6.patch
>
>
> Currently we can apply a range of filters on top of {{V2SLAServlet}} that can
> be used in a rich but undocumented set of ways:
> * {{id}}
> * {{parent_id}}
> * {{event_status}}
> * {{app_name}}
> * {{nominal_start}}
> * {{nominal_end}}
> Need to refactor {{V2SLAServlet}} to feature:
> * a richer set of {{SLAEvent}}, {{SLARegistration}}, and {{SLASummary}}
> filtering based on their attributes
> * filter options will always be {{AND}}-ed, never {{OR}}-ed to each other
> * maintain compatibility with the parameter names and behavior used thus far
> * remove {{SLASummaryFilter}} and refactor
> {{SLASummaryGetForFilterJPAExecutor}} as just another possibility for
> confusion
> * document new functionality with rich use case / example library so that
> users can leverage
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3318) [build] Fix Javadoc check in the pre-commit
[ https://issues.apache.org/jira/browse/OOZIE-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590377#comment-16590377 ] Andras Salamon commented on OOZIE-3318: --- [~andras.piros] Based on my suggestion [~kmarton] already simplified the grep + awk combo and eliminated one of the awk calls. The other awk is necessary to avoid counting non-javadoc related warnings. > [build] Fix Javadoc check in the pre-commit > --- > > Key: OOZIE-3318 > URL: https://issues.apache.org/jira/browse/OOZIE-3318 > Project: Oozie > Issue Type: Sub-task > Components: scripts >Affects Versions: 5.0.0 >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.1.0 > > Attachments: OOZIE-3318-003.patch, OOZIE-3318-004.patch, > OOZIE-3318.001.patch, OOZIE-3318.002.patch > > > Javadoc check builds are broken in the pre-commit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3318) [build] Fix Javadoc check in the pre-commit
[
https://issues.apache.org/jira/browse/OOZIE-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590368#comment-16590368
]
Andras Piros commented on OOZIE-3318:
-
Thanks for the contribution [~kmarton]!
Can you please:
* substitute the {{grep | awk}} combo w/ {{grep -c}} as per [~asalamon74]'s
suggestion
* post sample {{JAVADOC}} outputs for all paths while injecting Javadoc faults
on demand into Oozie source
> [build] Fix Javadoc check in the pre-commit
> ---
>
> Key: OOZIE-3318
> URL: https://issues.apache.org/jira/browse/OOZIE-3318
> Project: Oozie
> Issue Type: Sub-task
> Components: scripts
>Affects Versions: 5.0.0
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-3318-003.patch, OOZIE-3318-004.patch,
> OOZIE-3318.001.patch, OOZIE-3318.002.patch
>
>
> Javadoc check builds are broken in the pre-commit
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3318) [build] Fix Javadoc check in the pre-commit
[ https://issues.apache.org/jira/browse/OOZIE-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3318: Summary: [build] Fix Javadoc check in the pre-commit (was: Fix Javadoc check in the pre-commit) > [build] Fix Javadoc check in the pre-commit > --- > > Key: OOZIE-3318 > URL: https://issues.apache.org/jira/browse/OOZIE-3318 > Project: Oozie > Issue Type: Sub-task > Components: scripts >Affects Versions: 5.0.0 >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.1.0 > > Attachments: OOZIE-3318-003.patch, OOZIE-3318-004.patch, > OOZIE-3318.001.patch, OOZIE-3318.002.patch > > > Javadoc check builds are broken in the pre-commit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3317) [build] Fix false positive precommit reports
[ https://issues.apache.org/jira/browse/OOZIE-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590348#comment-16590348 ] Andras Piros commented on OOZIE-3317: - Thanks for the contribution [~kmarton]! +1 > [build] Fix false positive precommit reports > > > Key: OOZIE-3317 > URL: https://issues.apache.org/jira/browse/OOZIE-3317 > Project: Oozie > Issue Type: Sub-task > Components: scripts >Affects Versions: 5.0.0 >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3317-001.patch, OOZIE-3317-DO_NOT_MERGE.patch, > OOZIE-3317-DO_NOT_MERGE2.patch, OOZIE-3317-TESTING.patch, consoleFull.txt > > > Some steps of the current precommit are failing but we get a +1 anyways ( see > OOZIE-3304 for a recent example) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3317) [build] Fix false positive precommit reports
[ https://issues.apache.org/jira/browse/OOZIE-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3317: Summary: [build] Fix false positive precommit reports (was: Fix false positive precommit reports) > [build] Fix false positive precommit reports > > > Key: OOZIE-3317 > URL: https://issues.apache.org/jira/browse/OOZIE-3317 > Project: Oozie > Issue Type: Sub-task > Components: scripts >Affects Versions: 5.0.0 >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3317-001.patch, OOZIE-3317-DO_NOT_MERGE.patch, > OOZIE-3317-DO_NOT_MERGE2.patch, OOZIE-3317-TESTING.patch, consoleFull.txt > > > Some steps of the current precommit are failing but we get a +1 anyways ( see > OOZIE-3304 for a recent example) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3264) Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
[
https://issues.apache.org/jira/browse/OOZIE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590266#comment-16590266
]
Andras Piros commented on OOZIE-3264:
-
Committed to {{master}}.
> Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
> -
>
> Key: OOZIE-3264
> URL: https://issues.apache.org/jira/browse/OOZIE-3264
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3264-1.patch, OOZIE-3264-2.patch
>
>
> The test {{testLastOnlyMaterialization}} failed.
> {noformat}
> junit.framework.AssertionFailedError: expected: but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:253)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.checkCoordActionsStatus(TestCoordMaterializeTransitionXCommand.java:1415)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testLastOnlyMaterialization(TestCoordMaterializeTransitionXCommand.java:1114)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3264) Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
[
https://issues.apache.org/jira/browse/OOZIE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590257#comment-16590257
]
Andras Piros commented on OOZIE-3264:
-
Thanks for the contribution [~asalamon74]! +1
> Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
> -
>
> Key: OOZIE-3264
> URL: https://issues.apache.org/jira/browse/OOZIE-3264
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3264-1.patch, OOZIE-3264-2.patch
>
>
> The test {{testLastOnlyMaterialization}} failed.
> {noformat}
> junit.framework.AssertionFailedError: expected: but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:253)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.checkCoordActionsStatus(TestCoordMaterializeTransitionXCommand.java:1415)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testLastOnlyMaterialization(TestCoordMaterializeTransitionXCommand.java:1114)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590246#comment-16590246
]
Andras Piros commented on OOZIE-3320:
-
[~roeyshemtov24] thanks for the updated patch!
Some propositions:
* test {{ShellActionExecutor#resolveExecutable()}} separately
* check in {{ShellActionExecutor#resolveExecutable()}} whether the {{exec}} is
really present and is a file
* let's not modify the existing license header of {{ShellActionExecutor}} -
pre-commit hook takes a predefined template to check against
* document the existing and new functionality in
{{DG_ShellActionExtension.twiki}} describing both ways
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3264) Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
[
https://issues.apache.org/jira/browse/OOZIE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590241#comment-16590241
]
Hadoop QA commented on OOZIE-3264:
--
Testing JIRA OOZIE-3264
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2922
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/778/
> Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
> -
>
> Key: OOZIE-3264
> URL: https://issues.apache.org/jira/browse/OOZIE-3264
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3264-1.patch, OOZIE-3264-2.patch
>
>
> The test {{testLastOnlyMaterialization}} failed.
> {noformat}
> junit.framework.AssertionFailedError: expected: but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:253)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.checkCoordActionsStatus(TestCoordMaterializeTransitionXCommand.java:1415)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testLastOnlyMaterialization(TestCoordMaterializeTransitionXCommand.java:1114)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3264 PreCommit Build #778
Jira: https://issues.apache.org/jira/browse/OOZIE-3264
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/778/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.76 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2456 bytes
[TRACE] Full summary file size is 1399 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3264
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 4 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [fluent-job/fluent-job-api].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2922
+1 DISTRO
+1 distro tarball builds with the patch
+1 Overall result, good!, no -1s
There is at least one warning, please check
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/778/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13162422/comment/16590241","id":"16590241","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small=hadoopqa=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall=hadoopqa=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium=hadoopqa=10393"},"displayName":"Hadoop
[jira] [Updated] (OOZIE-3331) [spark-action] Inconsistency while parsing quoted Spark options
[
https://issues.apache.org/jira/browse/OOZIE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3331:
Summary: [spark-action] Inconsistency while parsing quoted Spark options
(was: SparkOptionsSplitter inconsistency)
> [spark-action] Inconsistency while parsing quoted Spark options
> ---
>
> Key: OOZIE-3331
> URL: https://issues.apache.org/jira/browse/OOZIE-3331
> Project: Oozie
> Issue Type: Test
> Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
> Fix For: 5.1.0
>
> Attachments: OOZIE-3331-1.patch, OOZIE-3331-wip.patch
>
>
> According to the [javadoc
> comment|https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkOptionsSplitter.java#L125-L138]
> of SparkOptionsSplitter we should unquote the option value if and only if
> it has quotes at both ends, and doesn't have any quotes in between. One of
> the listed examples shows that the following should remain quoted:
> {noformat}
> key="value1 value2" "value3 value4"{noformat}
> The current code unquotes this example which contradicts the javadoc comment.
> We should either change the javadoc or the code (after defining the correct
> behaviour) and we should also add a unit test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3331) SparkOptionsSplitter inconsistency
[
https://issues.apache.org/jira/browse/OOZIE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590165#comment-16590165
]
Andras Piros commented on OOZIE-3331:
-
Thanks for the contribution [~asalamon74]! +1
> SparkOptionsSplitter inconsistency
> --
>
> Key: OOZIE-3331
> URL: https://issues.apache.org/jira/browse/OOZIE-3331
> Project: Oozie
> Issue Type: Test
> Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
> Fix For: 5.1.0
>
> Attachments: OOZIE-3331-1.patch, OOZIE-3331-wip.patch
>
>
> According to the [javadoc
> comment|https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkOptionsSplitter.java#L125-L138]
> of SparkOptionsSplitter we should unquote the option value if and only if
> it has quotes at both ends, and doesn't have any quotes in between. One of
> the listed examples shows that the following should remain quoted:
> {noformat}
> key="value1 value2" "value3 value4"{noformat}
> The current code unquotes this example which contradicts the javadoc comment.
> We should either change the javadoc or the code (after defining the correct
> behaviour) and we should also add a unit test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 68482: OOZIE 3331 - SparkOptionsSplitter inconsistency
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68482/#review207813 --- Ship it! Ship It! - András Piros On Aug. 23, 2018, 7:04 a.m., Andras Salamon wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68482/ > --- > > (Updated Aug. 23, 2018, 7:04 a.m.) > > > Review request for oozie, András Piros and Kinga Marton. > > > Repository: oozie-git > > > Description > --- > > OOZIE 3331 - SparkOptionsSplitter inconsistency > > > Diffs > - > > > sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkOptionsSplitter.java > ac6ee81a > > sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkOptionsSplitter.java > 61ded5ee > > > Diff: https://reviews.apache.org/r/68482/diff/1/ > > > Testing > --- > > TestSparkOptionsSplitter, TestSparkArgsExtractor unit tests > > > Thanks, > > Andras Salamon > >
[jira] [Commented] (OOZIE-3264) Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
[
https://issues.apache.org/jira/browse/OOZIE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590136#comment-16590136
]
Hadoop QA commented on OOZIE-3264:
--
PreCommit-OOZIE-Build started
> Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
> -
>
> Key: OOZIE-3264
> URL: https://issues.apache.org/jira/browse/OOZIE-3264
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3264-1.patch, OOZIE-3264-2.patch
>
>
> The test {{testLastOnlyMaterialization}} failed.
> {noformat}
> junit.framework.AssertionFailedError: expected: but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:253)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.checkCoordActionsStatus(TestCoordMaterializeTransitionXCommand.java:1415)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testLastOnlyMaterialization(TestCoordMaterializeTransitionXCommand.java:1114)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3264) Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
[
https://issues.apache.org/jira/browse/OOZIE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590137#comment-16590137
]
Andras Piros commented on OOZIE-3264:
-
Retriggered.
> Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
> -
>
> Key: OOZIE-3264
> URL: https://issues.apache.org/jira/browse/OOZIE-3264
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3264-1.patch, OOZIE-3264-2.patch
>
>
> The test {{testLastOnlyMaterialization}} failed.
> {noformat}
> junit.framework.AssertionFailedError: expected: but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:253)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.checkCoordActionsStatus(TestCoordMaterializeTransitionXCommand.java:1415)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testLastOnlyMaterialization(TestCoordMaterializeTransitionXCommand.java:1114)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3331) SparkOptionsSplitter inconsistency
[
https://issues.apache.org/jira/browse/OOZIE-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589930#comment-16589930
]
Hadoop QA commented on OOZIE-3331:
--
Testing JIRA OOZIE-3331
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2926
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/777/
> SparkOptionsSplitter inconsistency
> --
>
> Key: OOZIE-3331
> URL: https://issues.apache.org/jira/browse/OOZIE-3331
> Project: Oozie
> Issue Type: Test
> Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
> Fix For: 5.1.0
>
> Attachments: OOZIE-3331-1.patch, OOZIE-3331-wip.patch
>
>
> According to the [javadoc
> comment|https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkOptionsSplitter.java#L125-L138]
> of SparkOptionsSplitter we should unquote the option value if and only if
> it has quotes at both ends, and doesn't have any quotes in between. One of
> the listed examples shows that the following should remain quoted:
> {noformat}
> key="value1 value2" "value3 value4"{noformat}
> The current code unquotes this example which contradicts the javadoc comment.
> We should either change the javadoc or the code (after defining the correct
> behaviour) and we should also add a unit test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Review Request 68482: OOZIE 3331 - SparkOptionsSplitter inconsistency
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68482/ --- Review request for oozie, András Piros and Kinga Marton. Repository: oozie-git Description --- OOZIE 3331 - SparkOptionsSplitter inconsistency Diffs - sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkOptionsSplitter.java ac6ee81a sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkOptionsSplitter.java 61ded5ee Diff: https://reviews.apache.org/r/68482/diff/1/ Testing --- TestSparkOptionsSplitter, TestSparkArgsExtractor unit tests Thanks, Andras Salamon
[jira] [Commented] (OOZIE-3264) Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
[
https://issues.apache.org/jira/browse/OOZIE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589737#comment-16589737
]
Andras Salamon commented on OOZIE-3264:
---
Here is the failed Jenkins build:
[https://builds.apache.org/job/PreCommit-OOZIE-Build/776/]
I think the failure has nothing to do with this patch.
[~andras.piros] can you please retrigger the Jenkins build?
> Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization
> -
>
> Key: OOZIE-3264
> URL: https://issues.apache.org/jira/browse/OOZIE-3264
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3264-1.patch, OOZIE-3264-2.patch
>
>
> The test {{testLastOnlyMaterialization}} failed.
> {noformat}
> junit.framework.AssertionFailedError: expected: but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:253)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.checkCoordActionsStatus(TestCoordMaterializeTransitionXCommand.java:1415)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testLastOnlyMaterialization(TestCoordMaterializeTransitionXCommand.java:1114)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (98 issues) Subscriber: ooziedaily Key Summary OOZIE-3332 Update the spark action examples to make it easier to run them on yarn in client and cluster mode https://issues.apache.org/jira/browse/OOZIE-3332 OOZIE-3326 Sqoop Action should support tez delegation tokens for hive-import https://issues.apache.org/jira/browse/OOZIE-3326 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3318 Fix Javadoc check in the pre-commit https://issues.apache.org/jira/browse/OOZIE-3318 OOZIE-3317 Fix false positive precommit reports https://issues.apache.org/jira/browse/OOZIE-3317 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3298 OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED https://issues.apache.org/jira/browse/OOZIE-3298 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3264 Flaky test TestCoordMaterializeTransitionXCommand#testLastOnlyMaterialization https://issues.apache.org/jira/browse/OOZIE-3264 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3249 [tools] Instrumentation log parser https://issues.apache.org/jira/browse/OOZIE-3249 OOZIE-3229 Improved filtering options in V2SLAServlet https://issues.apache.org/jira/browse/OOZIE-3229 OOZIE-3218 Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space https://issues.apache.org/jira/browse/OOZIE-3218 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3194 Oozie should set proper permissions to sharelib after upload https://issues.apache.org/jira/browse/OOZIE-3194 OOZIE-3186 Oozie is unable to use configuration linked using jceks://file/... https://issues.apache.org/jira/browse/OOZIE-3186 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3160 PriorityDelayQueue put()/take() can cause significant CPU load due to busy waiting https://issues.apache.org/jira/browse/OOZIE-3160 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3061 Kill only those child jobs which are not already killed https://issues.apache.org/jira/browse/OOZIE-3061 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging
