[jira] [Commented] (OOZIE-3077) SLA documentation fixes
[
https://issues.apache.org/jira/browse/OOZIE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197889#comment-16197889
]
Hadoop QA commented on OOZIE-3077:
--
Testing JIRA OOZIE-3077
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1835
.Tests rerun: 82
.Tests failed at first run:
org.apache.oozie.servlet.TestV1JobServlet,org.apache.oozie.servlet.TestJsonRestServlet,org.apache.oozie.servlet.TestV2SLAServlet,org.apache.oozie.jms.TestDefaultConnectionContext,org.apache.oozie.TestDagEngine,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.email.TestEmailActionExecutor,org.apache.oozie.compression.TestCodecFactory,org.apache.oozie.servlet.TestV2ValidateServlet,org.apache.oozie.TestBundleJobBean,org.apache.oozie.jms.TestJMSSLAEventListener,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/94/
> SLA documentation fixes
> ---
>
> Key: OOZIE-3077
> URL: https://issues.apache.org/jira/browse/OOZIE-3077
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Minor
> Attachments: OOZIE-3077-1.patch, OOZIE-3077-2.patch
>
>
> Following properties can be specified in job.xml -> Following properties can
> be specified in job.xml, job.properties
> Default is 24 hours -> Default is 48 hours
> oozie.sla.disable.alerts -> oozie.sla.enable.alerts for property name in the
> example configuration having 2009-01-01T01:00Z::2009-05-31T23:59Z as value
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3077) SLA documentation fixes
[ https://issues.apache.org/jira/browse/OOZIE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197771#comment-16197771 ] Rohini Palaniswamy commented on OOZIE-3077: --- +1 > SLA documentation fixes > --- > > Key: OOZIE-3077 > URL: https://issues.apache.org/jira/browse/OOZIE-3077 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley >Priority: Minor > Attachments: OOZIE-3077-1.patch, OOZIE-3077-2.patch > > > Following properties can be specified in job.xml -> Following properties can > be specified in job.xml, job.properties > Default is 24 hours -> Default is 48 hours > oozie.sla.disable.alerts -> oozie.sla.enable.alerts for property name in the > example configuration having 2009-01-01T01:00Z::2009-05-31T23:59Z as value -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3031) Coord job with only unresolved dependencies doesn't timeout
[
https://issues.apache.org/jira/browse/OOZIE-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197762#comment-16197762
]
Satish Subhashrao Saley commented on OOZIE-3031:
Thank you Puru. Committed to master.
> Coord job with only unresolved dependencies doesn't timeout
> ---
>
> Key: OOZIE-3031
> URL: https://issues.apache.org/jira/browse/OOZIE-3031
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Critical
> Attachments: OOZIE-3031-V1.patch, OOZIE-3031-V2.patch
>
>
> {code:title=CoordActionInputCheckXCommand.java}
> else if (!isTimeout(currentTime)) {
> if (!status) {
> long addtionalDelay = isChangeInDependency ? 0
> :
> ConfigurationService.getInt(CONF_COORD_INPUT_CHECK_REQUEUE_INTERVAL_ADDITIONAL_DELAY)
> * 1000L;
> queue(new
> CoordActionInputCheckXCommand(coordAction.getId(), coordAction.getJobId()),
> addtionalDelay +
> getCoordInputCheckRequeueInterval());
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> else {
> if (!nonExistListStr.isEmpty() && isPushDependenciesMet) {
> queue(new CoordActionTimeOutXCommand(coordAction,
> coordJob.getUser(), coordJob.getAppName()));
> }
> else {
> // Let CoordPushDependencyCheckXCommand queue the timeout
> queue(new
> CoordPushDependencyCheckXCommand(coordAction.getId()));
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> }
> {code}
> {{!nonExistList Str.isEmpty()}} check is unnecessary, once the action has
> passed the timeout, it should be marked as timedout.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3031) Coord job with only unresolved dependencies doesn't timeout
[
https://issues.apache.org/jira/browse/OOZIE-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197756#comment-16197756
]
Satish Subhashrao Saley commented on OOZIE-3031:
+1
> Coord job with only unresolved dependencies doesn't timeout
> ---
>
> Key: OOZIE-3031
> URL: https://issues.apache.org/jira/browse/OOZIE-3031
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Critical
> Attachments: OOZIE-3031-V1.patch, OOZIE-3031-V2.patch
>
>
> {code:title=CoordActionInputCheckXCommand.java}
> else if (!isTimeout(currentTime)) {
> if (!status) {
> long addtionalDelay = isChangeInDependency ? 0
> :
> ConfigurationService.getInt(CONF_COORD_INPUT_CHECK_REQUEUE_INTERVAL_ADDITIONAL_DELAY)
> * 1000L;
> queue(new
> CoordActionInputCheckXCommand(coordAction.getId(), coordAction.getJobId()),
> addtionalDelay +
> getCoordInputCheckRequeueInterval());
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> else {
> if (!nonExistListStr.isEmpty() && isPushDependenciesMet) {
> queue(new CoordActionTimeOutXCommand(coordAction,
> coordJob.getUser(), coordJob.getAppName()));
> }
> else {
> // Let CoordPushDependencyCheckXCommand queue the timeout
> queue(new
> CoordPushDependencyCheckXCommand(coordAction.getId()));
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> }
> {code}
> {{!nonExistList Str.isEmpty()}} check is unnecessary, once the action has
> passed the timeout, it should be marked as timedout.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3077) SLA documentation fixes
[ https://issues.apache.org/jira/browse/OOZIE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-3077: --- Attachment: OOZIE-3077-2.patch > SLA documentation fixes > --- > > Key: OOZIE-3077 > URL: https://issues.apache.org/jira/browse/OOZIE-3077 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley >Priority: Minor > Attachments: OOZIE-3077-1.patch, OOZIE-3077-2.patch > > > Following properties can be specified in job.xml -> Following properties can > be specified in job.xml, job.properties > Default is 24 hours -> Default is 48 hours > oozie.sla.disable.alerts -> oozie.sla.enable.alerts for property name in the > example configuration having 2009-01-01T01:00Z::2009-05-31T23:59Z as value -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3077) SLA documentation fixes
[ https://issues.apache.org/jira/browse/OOZIE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197741#comment-16197741 ] Satish Subhashrao Saley commented on OOZIE-3077: Updated patch > SLA documentation fixes > --- > > Key: OOZIE-3077 > URL: https://issues.apache.org/jira/browse/OOZIE-3077 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley >Priority: Minor > Attachments: OOZIE-3077-1.patch, OOZIE-3077-2.patch > > > Following properties can be specified in job.xml -> Following properties can > be specified in job.xml, job.properties > Default is 24 hours -> Default is 48 hours > oozie.sla.disable.alerts -> oozie.sla.enable.alerts for property name in the > example configuration having 2009-01-01T01:00Z::2009-05-31T23:59Z as value -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3077) SLA documentation fixes
[ https://issues.apache.org/jira/browse/OOZIE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197677#comment-16197677 ] Rohini Palaniswamy commented on OOZIE-3077: --- Couple of minor rewordings. 1) Following are ways to enable/disable SLA alerts. -> Following are ways to enable/disable SLA alerts for coordinator actions. 2) 1. Following properties can be specified in bundle xml as properties for coordinator. -> 1. Specify in Bundle XML during submission \n Following properties can be specified in bundle xml as properties for coordinator. 3) Change using job command -> Specify during coordinator job submission or update > SLA documentation fixes > --- > > Key: OOZIE-3077 > URL: https://issues.apache.org/jira/browse/OOZIE-3077 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley >Priority: Minor > Attachments: OOZIE-3077-1.patch > > > Following properties can be specified in job.xml -> Following properties can > be specified in job.xml, job.properties > Default is 24 hours -> Default is 48 hours > oozie.sla.disable.alerts -> oozie.sla.enable.alerts for property name in the > example configuration having 2009-01-01T01:00Z::2009-05-31T23:59Z as value -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3031) Coord job with only unresolved dependencies doesn't timeout
[
https://issues.apache.org/jira/browse/OOZIE-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197623#comment-16197623
]
Hadoop QA commented on OOZIE-3031:
--
Testing JIRA OOZIE-3031
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2039
.Tests rerun: 54
.Tests failed at first run:
org.apache.oozie.action.TestActionFailover,org.apache.oozie.action.hadoop.TestJavaActionExecutor,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/93/
> Coord job with only unresolved dependencies doesn't timeout
> ---
>
> Key: OOZIE-3031
> URL: https://issues.apache.org/jira/browse/OOZIE-3031
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Critical
> Attachments: OOZIE-3031-V1.patch, OOZIE-3031-V2.patch
>
>
> {code:title=CoordActionInputCheckXCommand.java}
> else if (!isTimeout(currentTime)) {
> if (!status) {
> long addtionalDelay = isChangeInDependency ? 0
> :
> ConfigurationService.getInt(CONF_COORD_INPUT_CHECK_REQUEUE_INTERVAL_ADDITIONAL_DELAY)
> * 1000L;
> queue(new
> CoordActionInputCheckXCommand(coordAction.getId(), coordAction.getJobId()),
> addtionalDelay +
> getCoordInputCheckRequeueInterval());
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> else {
> if (!nonExistListStr.isEmpty() && isPushDependenciesMet) {
> queue(new CoordActionTimeOutXCommand(coordAction,
> coordJob.getUser(), coordJob.getAppName()));
> }
> else {
> // Let CoordPushDependencyCheckXCommand queue the timeout
> queue(new
> CoordPushDependencyCheckXCommand(coordAction.getId()));
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> }
> {code}
> {{!nonExistList Str.isEmpty()}} check is unnecessary, once the action has
> passed the timeout, it
Success: OOZIE-3031 PreCommit Build #93
Jira: https://issues.apache.org/jira/browse/OOZIE-3031 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/93/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.59 MB...] +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warnings WARNING: the current HEAD has 77 Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/spark]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2039 Tests rerun: 54 Tests failed at first run: org.apache.oozie.action.TestActionFailover,org.apache.oozie.action.hadoop.TestJavaActionExecutor, +1 DISTRO +1 distro tarball builds with the patch +1 Overall result, good!, no -1s There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/93/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 3706k 100 3706k0 0 4866k 0 --:--:-- --:--:-- --:--:-- 11.6M Adding comment to JIRA Comment added. test-patch exit code: 0 [description-setter] Description set: OOZIE-3031 Archiving artifacts [Fast Archiver] Compressed 1.80 MB of artifacts by 64.4% relative to #91 Recording test results Email was triggered for: Success Sending email for trigger: Success ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (OOZIE-2004) Improve Oozie version info output
[
https://issues.apache.org/jira/browse/OOZIE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197617#comment-16197617
]
Artem Ervits commented on OOZIE-2004:
-
[~asasvari] shouldn't it then be like so:
{code}
{"buildVersion":"5.0.0-SNAPSHOT", "buildInfo":"{build.time=${build.time},
vc.url=${vc.url}, build.user=asasvari, vc.revision=${vc.revision}"}
{code}
> Improve Oozie version info output
> -
>
> Key: OOZIE-2004
> URL: https://issues.apache.org/jira/browse/OOZIE-2004
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: 4.0.1
>Reporter: Chunjun Xiao
>Assignee: Artem Ervits
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: HBASE-2004-2.patch, OOZIE-2004-0.patch,
> OOZIE-2004-1.patch, OOZIE-2004-2.patch
>
>
> Current oozie version info output is quite simple:
> {code}
> [root@svm5 ~] # oozie version
> Oozie client build version: 4.0.1
> [root@svm5 ~] #
> {code}
> Can we improve it to follow the convention of Hadoop/HBase, as:
> {code}
> [root@svm5 ~] # oozie version
> Oozie 4.0.1
> Source code repository: ssh://git@XXX/oozie.git -r
> 3171cb656a4e4c1b2c1e9c6a47abed7df5bbcc8d
> Compiled by root on Sun Jul 6 04:38:08 PDT 2014
> From source with checksum XXX
> [root@svm5 ~] #
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3062) Set HADOOP_CONF_DIR for spark action
[
https://issues.apache.org/jira/browse/OOZIE-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197536#comment-16197536
]
Hadoop QA commented on OOZIE-3062:
--
Testing JIRA OOZIE-3062
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 2 line(s) longer than 132
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/92/
> Set HADOOP_CONF_DIR for spark action
>
>
> Key: OOZIE-3062
> URL: https://issues.apache.org/jira/browse/OOZIE-3062
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-3062-1.patch, OOZIE-3062-2.patch,
> OOZIE-3062-3.patch, OOZIE-3062-4.patch, OOZIE-3062-5.patch
>
>
> OOZIE-2569 created core-site.xml, hdfs-site.xml, mapred-site.xml,
> yarn-site.xml for spark action. But for spark to consider these files ([spark
> code
> ref|https://github.com/apache/spark/blob/83fe3b5e10f8dc62245ea37143abb96be0f39805/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala#L284]),
> we need to set HADOOP_CONF_DIR environmental variable, otherwise it will
> point to default HADOOP_CONF_DIR configured on the node. Therefore, Oozie
> should set HADOOP_CONF_DIR.
> Due to a bug YARN-4727, yarn was not considering user specified value for
> whiltelisted environment variables including HADOOP_CONF_DIR. So, this will
> work on specified hadoop fix versions in YARN-4727.
> This fix in Oozie will be no-op on Hadoop without YARN-4727 fix and Oozie
> will behave as it is.
> To verify this on one node machine.
> 1. In hadoop mapred-site.xml, set
> {code}
>
>mapreduce.fileoutputcommitter.marksuccessfuljobs
>false
>
> {code}
> 2. Run any spark file copy example from Oozie. Check output directory,
> _SUCCESS flag is missing. It should be there because we explicitly set
> conf.set("mapreduce.fileoutputcommitter.marksuccessfuljobs", "true"); in
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L293
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Failed: OOZIE-3062 PreCommit Build #92
Jira: https://issues.apache.org/jira/browse/OOZIE-3062 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/92/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.53 MB...] -1 the patch contains 2 line(s) longer than 132 characters +1 the patch does adds/modifies 1 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warnings WARNING: the current HEAD has 77 Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files -1 TESTS - patch does not compile, cannot run testcases +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/92/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 3706k0 159150 0 26721 0 0:02:22 --:--:-- 0:02:22 26721100 3706k 100 3706k0 0 4686k 0 --:--:-- --:--:-- --:--:-- 18.4M Adding comment to JIRA Comment added. test-patch exit code: 1 Build step 'Execute shell' marked build as failure [description-setter] Description set: OOZIE-3062 Archiving artifacts [Fast Archiver] Compressed 1.79 MB of artifacts by 68.0% relative to #91 Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (OOZIE-2004) Improve Oozie version info output
[
https://issues.apache.org/jira/browse/OOZIE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197499#comment-16197499
]
Peter Cseh commented on OOZIE-2004:
---
Ohh, I've missed this during the review...
I've assumed that JsonTags.BUILD_VERSION and BuildInfo.BUILD_VERSION are the
same.
I'd prefer the second solution, it should be compatible enough for most use
cases. What do you think [~asasvari]?
> Improve Oozie version info output
> -
>
> Key: OOZIE-2004
> URL: https://issues.apache.org/jira/browse/OOZIE-2004
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: 4.0.1
>Reporter: Chunjun Xiao
>Assignee: Artem Ervits
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: HBASE-2004-2.patch, OOZIE-2004-0.patch,
> OOZIE-2004-1.patch, OOZIE-2004-2.patch
>
>
> Current oozie version info output is quite simple:
> {code}
> [root@svm5 ~] # oozie version
> Oozie client build version: 4.0.1
> [root@svm5 ~] #
> {code}
> Can we improve it to follow the convention of Hadoop/HBase, as:
> {code}
> [root@svm5 ~] # oozie version
> Oozie 4.0.1
> Source code repository: ssh://git@XXX/oozie.git -r
> 3171cb656a4e4c1b2c1e9c6a47abed7df5bbcc8d
> Compiled by root on Sun Jul 6 04:38:08 PDT 2014
> From source with checksum XXX
> [root@svm5 ~] #
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3031) Coord job with only unresolved dependencies doesn't timeout
[
https://issues.apache.org/jira/browse/OOZIE-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purshotam Shah updated OOZIE-3031:
--
Attachment: OOZIE-3031-V2.patch
> Coord job with only unresolved dependencies doesn't timeout
> ---
>
> Key: OOZIE-3031
> URL: https://issues.apache.org/jira/browse/OOZIE-3031
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Critical
> Attachments: OOZIE-3031-V1.patch, OOZIE-3031-V2.patch
>
>
> {code:title=CoordActionInputCheckXCommand.java}
> else if (!isTimeout(currentTime)) {
> if (!status) {
> long addtionalDelay = isChangeInDependency ? 0
> :
> ConfigurationService.getInt(CONF_COORD_INPUT_CHECK_REQUEUE_INTERVAL_ADDITIONAL_DELAY)
> * 1000L;
> queue(new
> CoordActionInputCheckXCommand(coordAction.getId(), coordAction.getJobId()),
> addtionalDelay +
> getCoordInputCheckRequeueInterval());
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> else {
> if (!nonExistListStr.isEmpty() && isPushDependenciesMet) {
> queue(new CoordActionTimeOutXCommand(coordAction,
> coordJob.getUser(), coordJob.getAppName()));
> }
> else {
> // Let CoordPushDependencyCheckXCommand queue the timeout
> queue(new
> CoordPushDependencyCheckXCommand(coordAction.getId()));
> }
> updateCoordAction(coordAction, isChangeInDependency);
> }
> }
> {code}
> {{!nonExistList Str.isEmpty()}} check is unnecessary, once the action has
> passed the timeout, it should be marked as timedout.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
Attila Sasvari created OOZIE-3081:
-
Summary: testSubmitWithLauncherQueue fails when FairScheduler is
used
Key: OOZIE-3081
URL: https://issues.apache.org/jira/browse/OOZIE-3081
Project: Oozie
Issue Type: Bug
Components: tests
Affects Versions: 5.0.0
Reporter: Attila Sasvari
Running TestJavaActionExecutor fails with fair scheduler
{code}
Failed tests:
testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
expected:<[]test> but was:<[root.]test>
{code}
Reason: FairScheduler adds a "root." prefix to the queue name in
[org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
A potential way to fix the test case:
- we can add a new function
{code}
private boolean isFairSchedulerUsed(Configuration conf) {
return
conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
}
{code}
that can be called from {{testSubmitWithLauncherQueue()}}:
{code}
...
if (isFairSchedulerUsed(conf)) {
assertEquals("root.test", queue);
}
else {
assertEquals("test", queue);
}
...
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null
[ https://issues.apache.org/jira/browse/OOZIE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197333#comment-16197333 ] Satish Subhashrao Saley commented on OOZIE-3079: Thank you Rohini and Andras for review. Committed to master. > Filtering coordinators returns bundle id as null > > > Key: OOZIE-3079 > URL: https://issues.apache.org/jira/browse/OOZIE-3079 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Attachments: OOZIE-3079-1.patch, OOZIE-3079-2.patch, > OOZIE-3079-3.patch > > > If we filter coordinator jobs, the json reply shows (parent) bundle id as > null even though coordinator was started from a bundle. > Example query - > http://localhost:11000/oozie/v1/jobs?filter=user=saley=coordinator -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null
[ https://issues.apache.org/jira/browse/OOZIE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197281#comment-16197281 ] Rohini Palaniswamy commented on OOZIE-3079: --- +1 > Filtering coordinators returns bundle id as null > > > Key: OOZIE-3079 > URL: https://issues.apache.org/jira/browse/OOZIE-3079 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Attachments: OOZIE-3079-1.patch, OOZIE-3079-2.patch, > OOZIE-3079-3.patch > > > If we filter coordinator jobs, the json reply shows (parent) bundle id as > null even though coordinator was started from a bundle. > Example query - > http://localhost:11000/oozie/v1/jobs?filter=user=saley=coordinator -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2004) Improve Oozie version info output
[
https://issues.apache.org/jira/browse/OOZIE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197222#comment-16197222
]
Attila Sasvari commented on OOZIE-2004:
---
This change breaks API compatibility. Existing clients of the Oozie server that
depend on {{/v1/admin/build-version}} will break without a client-side update.
For example, a monitoring client might depend on the {{buildVersion}} element
in the returned JSON. Right now the server returns:
{code}
{"buildInfo":"{build.time=${build.time}, vc.url=${vc.url}, build.user=asasvari,
vc.revision=${vc.revision}, build.version=5.0.0-SNAPSHOT}"}
{code}
It would be better to have either a new version {{/v3/admin/build-version}} or
only add content like:
{code}
{"buildVersion":"5.0.0-SNAPSHOT", "buildInfo":"{build.time=${build.time},
vc.url=${vc.url}, build.user=asasvari, vc.revision=${vc.revision},
build.version=5.0.0-SNAPSHOT}"}
{code}
> Improve Oozie version info output
> -
>
> Key: OOZIE-2004
> URL: https://issues.apache.org/jira/browse/OOZIE-2004
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: 4.0.1
>Reporter: Chunjun Xiao
>Assignee: Artem Ervits
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: HBASE-2004-2.patch, OOZIE-2004-0.patch,
> OOZIE-2004-1.patch, OOZIE-2004-2.patch
>
>
> Current oozie version info output is quite simple:
> {code}
> [root@svm5 ~] # oozie version
> Oozie client build version: 4.0.1
> [root@svm5 ~] #
> {code}
> Can we improve it to follow the convention of Hadoop/HBase, as:
> {code}
> [root@svm5 ~] # oozie version
> Oozie 4.0.1
> Source code repository: ssh://git@XXX/oozie.git -r
> 3171cb656a4e4c1b2c1e9c6a47abed7df5bbcc8d
> Compiled by root on Sun Jul 6 04:38:08 PDT 2014
> From source with checksum XXX
> [root@svm5 ~] #
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments
[
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197115#comment-16197115
]
Attila Sasvari commented on OOZIE-3078:
---
Thanks for the review, committed to master!
> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch, OOZIE-3078-02.patch
>
>
> If
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
> is called with a {{null}} argument, a NPE will be thrown. It should return
> false.
> Note: [SparkMain |
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
> uses this method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments
[
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196935#comment-16196935
]
Peter Cseh commented on OOZIE-3078:
---
+1
Thanks for the fix!
> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch, OOZIE-3078-02.patch
>
>
> If
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
> is called with a {{null}} argument, a NPE will be thrown. It should return
> false.
> Note: [SparkMain |
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
> uses this method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments
[
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196899#comment-16196899
]
Hadoop QA commented on OOZIE-3078:
--
Testing JIRA OOZIE-3078
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2036
.Tests rerun: 81
.Tests failed at first run:
org.apache.oozie.command.coord.TestCoordActionsKillXCommand,org.apache.oozie.servlet.TestBulkMonitorWebServiceAPI,org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.action.hadoop.TestJavaActionExecutor,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/91/
> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch, OOZIE-3078-02.patch
>
>
> If
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
> is called with a {{null}} argument, a NPE will be thrown. It should return
> false.
> Note: [SparkMain |
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
> uses this method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Success: OOZIE-3078 PreCommit Build #91
Jira: https://issues.apache.org/jira/browse/OOZIE-3078 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/91/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.59 MB...] +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warnings WARNING: the current HEAD has 77 Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2036 Tests rerun: 81 Tests failed at first run: org.apache.oozie.command.coord.TestCoordActionsKillXCommand,org.apache.oozie.servlet.TestBulkMonitorWebServiceAPI,org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.action.hadoop.TestJavaActionExecutor, +1 DISTRO +1 distro tarball builds with the patch +1 Overall result, good!, no -1s There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/91/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 3706k 100 3706k0 0 4694k 0 --:--:-- --:--:-- --:--:-- 7852k Adding comment to JIRA Comment added. test-patch exit code: 0 [description-setter] Description set: OOZIE-3078 Archiving artifacts [Fast Archiver] Compressed 1.79 MB of artifacts by 74.9% relative to #87 Recording test results Email was triggered for: Success Sending email for trigger: Success ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Comment Edited] (OOZIE-2871) when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate via:[TOKEN, KERBEROS]”
[ https://issues.apache.org/jira/browse/OOZIE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196807#comment-16196807 ] Denes Bodo edited comment on OOZIE-2871 at 10/9/17 11:00 AM: - [~gezapeti] I think yes, based on "Cannot do cross cluster distcp" between two secure clusters. Oozie does not pass any HDFS delegation token to the distcp job. was (Author: dionusos): [~gezapeti] I think yes, based on "Cannot do cross cluster distcp" between two secure clusters. > when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate > via:[TOKEN, KERBEROS]” > --- > > Key: OOZIE-2871 > URL: https://issues.apache.org/jira/browse/OOZIE-2871 > Project: Oozie > Issue Type: Bug > Components: security >Affects Versions: 4.2.0 > Environment: Oozie version :4.2.0 > Hadoop version:2.7.2 > Both Oozie and Hadoop are enabled kerberos. >Reporter: yangfang >Priority: Critical > > When Oozie and Hadoop both enabled kerberos， I submitted a mapreduce job > to oozie，then I got the error as below： > 2017-04-27 13:37:12,677 WARN MapReduceActionExecutor: 523 - SERVER[zdh143] > USER[mr] GROUP[-] TOKEN[] APP[map-reduce-wf] > JOB[008-170427133546167-oozie-mr-W] > ACTION[008-170427133546167-oozie-mr-W@mr-node] Launcher exception: Failed > on local exception: java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; > destination host is: "zdh143":9000; > java.io.IOException: Failed on local exception: java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; > destination host is: "zdh143":9000; > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:773) > at org.apache.hadoop.ipc.Client.call(Client.java:1479) > at org.apache.hadoop.ipc.Client.call(Client.java:1412) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) > at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426) > at > org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:130) > at > org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:268) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1299) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1296) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561) > at > org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:102) > at >
[jira] [Commented] (OOZIE-2871) when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate via:[TOKEN, KERBEROS]”
[ https://issues.apache.org/jira/browse/OOZIE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196807#comment-16196807 ] Denes Bodo commented on OOZIE-2871: --- [~gezapeti] I think yes, based on "Cannot do cross cluster distcp" between two secure clusters. > when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate > via:[TOKEN, KERBEROS]” > --- > > Key: OOZIE-2871 > URL: https://issues.apache.org/jira/browse/OOZIE-2871 > Project: Oozie > Issue Type: Bug > Components: security >Affects Versions: 4.2.0 > Environment: Oozie version :4.2.0 > Hadoop version:2.7.2 > Both Oozie and Hadoop are enabled kerberos. >Reporter: yangfang >Priority: Critical > > When Oozie and Hadoop both enabled kerberos， I submitted a mapreduce job > to oozie，then I got the error as below： > 2017-04-27 13:37:12,677 WARN MapReduceActionExecutor: 523 - SERVER[zdh143] > USER[mr] GROUP[-] TOKEN[] APP[map-reduce-wf] > JOB[008-170427133546167-oozie-mr-W] > ACTION[008-170427133546167-oozie-mr-W@mr-node] Launcher exception: Failed > on local exception: java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; > destination host is: "zdh143":9000; > java.io.IOException: Failed on local exception: java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; > destination host is: "zdh143":9000; > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:773) > at org.apache.hadoop.ipc.Client.call(Client.java:1479) > at org.apache.hadoop.ipc.Client.call(Client.java:1412) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) > at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426) > at > org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:130) > at > org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:268) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1299) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1296) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561) > at > org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:102) > at > org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:64) > at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47) > at > org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:38) > at
[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments
[
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196798#comment-16196798
]
Attila Sasvari commented on OOZIE-3078:
---
[~gezapeti] thanks for the review! Attached new patch to address latest
comments.
> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch, OOZIE-3078-02.patch
>
>
> If
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
> is called with a {{null}} argument, a NPE will be thrown. It should return
> false.
> Note: [SparkMain |
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
> uses this method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3078) PasswordMasker throws NPE with null arguments
[
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-3078:
--
Attachment: OOZIE-3078-02.patch
> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch, OOZIE-3078-02.patch
>
>
> If
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
> is called with a {{null}} argument, a NPE will be thrown. It should return
> false.
> Note: [SparkMain |
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
> uses this method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Failed: OOZIE-2406 PreCommit Build #90
Jira: https://issues.apache.org/jira/browse/OOZIE-2406 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/90/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.60 MB...] +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [tools]. 0 There are [1] new bugs found in [core] that would be nice to have fixed. You can find the FindBugs diff here: core/findbugs-new.html +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files -1 TESTS Tests run: 2036 Tests failed: 1 Tests errors: 1 The patch failed the following testcases: testSimpleGraphPng(org.apache.oozie.util.graph.TestGraphGenerator) Tests failing with errors: testGraphWithManyNodes(org.apache.oozie.util.graph.TestGraphGenerator) +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/90/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 4 3706k4 172k0 0 232k 0 0:00:15 --:--:-- 0:00:15 232k100 3706k 100 3706k0 0 4154k 0 --:--:-- --:--:-- --:--:-- 23.1M Adding comment to JIRA Comment added. test-patch exit code: 1 Build step 'Execute shell' marked build as failure [description-setter] Description set: OOZIE-2406 Archiving artifacts [Fast Archiver] Compressed 1.80 MB of artifacts by 69.5% relative to #87 Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## 2 tests failed. FAILED: org.apache.oozie.util.graph.TestGraphGenerator.testGraphWithManyNodes Error Message: resource [graph-workflow-many-actions.xml] not found Stack Trace: java.lang.IllegalArgumentException: resource [graph-workflow-many-actions.xml] not found at org.apache.oozie.util.IOUtils.getResourceAsString(IOUtils.java:134) at org.apache.oozie.util.graph.TestGraphGenerator.readXmlFromClasspath(TestGraphGenerator.java:162) at org.apache.oozie.util.graph.TestGraphGenerator.testGraphWithManyNodes(TestGraphGenerator.java:154) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84) at org.junit.runners.Suite.runChild(Suite.java:127) at org.junit.runners.Suite.runChild(Suite.java:26) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED:
[jira] [Commented] (OOZIE-2406) Completely rewrite GraphGenerator code
[
https://issues.apache.org/jira/browse/OOZIE-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196728#comment-16196728
]
Hadoop QA commented on OOZIE-2406:
--
Testing JIRA OOZIE-2406
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2036
.Tests failed: 1
.Tests errors: 1
.The patch failed the following testcases:
. testSimpleGraphPng(org.apache.oozie.util.graph.TestGraphGenerator)
.Tests failing with errors:
. testGraphWithManyNodes(org.apache.oozie.util.graph.TestGraphGenerator)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/90/
> Completely rewrite GraphGenerator code
> --
>
> Key: OOZIE-2406
> URL: https://issues.apache.org/jira/browse/OOZIE-2406
> Project: Oozie
> Issue Type: New Feature
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-2406.004.patch, OOZIE-2406.004.patch,
> OOZIE-2406.005.patch, OOZIE-2406.006.patch, OOZIE-2406.007.patch,
> OOZIE-2406.008.patch, OOZIE-2406.009.patch, OOZIE-2406.010.patch,
> OOZIE-2406.011.patch, OOZIE-2406.012.patch, OOZIE-2406.012.patch,
> OOZIE-2406.012.patch, OOZIE-2406.012.patch, OOZIE-2406.012.patch, Screen Shot
> 2017-10-02 at 1.55.58 PM.png, Screen Shot 2017-10-02 at 1.56.33 PM.png,
> graphviz-10-action-nodes.png, graphviz-25-action-nodes.png,
> graphviz-50-action-nodes.png, graphviz-many-nodes.png, jung-many-nodes.png
>
>
> The Web UI currently generates a graph of the workflow DAG as a png image
> that it can show to the user ({{GraphGenerator}} class). Unfortunately,
> there are a number of downsides to the current implementation:
> # The image is generated server-side, which doesn't scale well and eats up
> lots of memory
> #- To help combat this issue, we only generate graphs for workflows with less
> than 25 nodes and we disabled the refresh button in the UI
> # It's slow
> # It uses a library ({{net.sf.jung}} from http://jung.sourceforge.net), which
> hasn't
[jira] [Resolved] (OOZIE-3080) testOldFormatRead fails with classnotfound
[
https://issues.apache.org/jira/browse/OOZIE-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna resolved OOZIE-3080.
---
Resolution: Not A Problem
> testOldFormatRead fails with classnotfound
> --
>
> Key: OOZIE-3080
> URL: https://issues.apache.org/jira/browse/OOZIE-3080
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Reporter: Yesha Vora
>
> testOldFormatRead UT fails with classnotfound
> {code}
> Error Message
> java.lang.ClassNotFoundException: null
> Stacktrace
> java.io.IOException: java.lang.ClassNotFoundException: null
> at
> org.apache.oozie.workflow.lite.NodeDef.readVersionZero(NodeDef.java:140)
> at org.apache.oozie.workflow.lite.NodeDef.readFields(NodeDef.java:190)
> at
> org.apache.oozie.workflow.lite.LiteWorkflowApp.readFields(LiteWorkflowApp.java:147)
> at
> org.apache.oozie.workflow.lite.TestLiteWorkflowApp.testOldFormatRead(TestLiteWorkflowApp.java:75)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: null
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.oozie.workflow.lite.NodeDef.readVersionZero(NodeDef.java:137)
> ... 24 more{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-2871) when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate via:[TOKEN, KERBEROS]”
[ https://issues.apache.org/jira/browse/OOZIE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196615#comment-16196615 ] Peter Cseh commented on OOZIE-2871: --- Can OOZIE-2900 be related here? > when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate > via:[TOKEN, KERBEROS]” > --- > > Key: OOZIE-2871 > URL: https://issues.apache.org/jira/browse/OOZIE-2871 > Project: Oozie > Issue Type: Bug > Components: security >Affects Versions: 4.2.0 > Environment: Oozie version :4.2.0 > Hadoop version:2.7.2 > Both Oozie and Hadoop are enabled kerberos. >Reporter: yangfang >Priority: Critical > > When Oozie and Hadoop both enabled kerberos， I submitted a mapreduce job > to oozie，then I got the error as below： > 2017-04-27 13:37:12,677 WARN MapReduceActionExecutor: 523 - SERVER[zdh143] > USER[mr] GROUP[-] TOKEN[] APP[map-reduce-wf] > JOB[008-170427133546167-oozie-mr-W] > ACTION[008-170427133546167-oozie-mr-W@mr-node] Launcher exception: Failed > on local exception: java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; > destination host is: "zdh143":9000; > java.io.IOException: Failed on local exception: java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; > destination host is: "zdh143":9000; > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:773) > at org.apache.hadoop.ipc.Client.call(Client.java:1479) > at org.apache.hadoop.ipc.Client.call(Client.java:1412) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) > at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426) > at > org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:130) > at > org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:268) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1299) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1296) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561) > at > org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:102) > at > org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:64) > at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47) > at > org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:38) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >
[jira] [Updated] (OOZIE-2406) Completely rewrite GraphGenerator code
[
https://issues.apache.org/jira/browse/OOZIE-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2406:
Attachment: OOZIE-2406.012.patch
> Completely rewrite GraphGenerator code
> --
>
> Key: OOZIE-2406
> URL: https://issues.apache.org/jira/browse/OOZIE-2406
> Project: Oozie
> Issue Type: New Feature
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-2406.004.patch, OOZIE-2406.004.patch,
> OOZIE-2406.005.patch, OOZIE-2406.006.patch, OOZIE-2406.007.patch,
> OOZIE-2406.008.patch, OOZIE-2406.009.patch, OOZIE-2406.010.patch,
> OOZIE-2406.011.patch, OOZIE-2406.012.patch, OOZIE-2406.012.patch,
> OOZIE-2406.012.patch, OOZIE-2406.012.patch, OOZIE-2406.012.patch, Screen Shot
> 2017-10-02 at 1.55.58 PM.png, Screen Shot 2017-10-02 at 1.56.33 PM.png,
> graphviz-10-action-nodes.png, graphviz-25-action-nodes.png,
> graphviz-50-action-nodes.png, graphviz-many-nodes.png, jung-many-nodes.png
>
>
> The Web UI currently generates a graph of the workflow DAG as a png image
> that it can show to the user ({{GraphGenerator}} class). Unfortunately,
> there are a number of downsides to the current implementation:
> # The image is generated server-side, which doesn't scale well and eats up
> lots of memory
> #- To help combat this issue, we only generate graphs for workflows with less
> than 25 nodes and we disabled the refresh button in the UI
> # It's slow
> # It uses a library ({{net.sf.jung}} from http://jung.sourceforge.net), which
> hasn't been updated since 2010.
> #- This library also has a dependency on a fork of Commons-Collections
> ({{net.sourceforge.collections}} from
> http://sourceforge.net/projects/collections/), which similarly hasn't been
> updated since 2010. This is a problem because we can't update
> Commons-Collections when there are security concerns such as COLLECTIONS-580
> (though Oozie is *not* susceptible to this attack).
> It would be good to do a complete rewrite, using a different library and
> remove Jung and the Commons-Collections fork. Whatever we choose, it should
> an svg which will draw the image on the browser.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (110 issues) Subscriber: ooziedaily Key Summary OOZIE-3079 Filtering coordinators returns bundle id as null https://issues.apache.org/jira/browse/OOZIE-3079 OOZIE-3078 PasswordMasker throws NPE with null arguments https://issues.apache.org/jira/browse/OOZIE-3078 OOZIE-3077 SLA documentation fixes https://issues.apache.org/jira/browse/OOZIE-3077 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3031 Coord job with only unresolved dependencies doesn't timeout https://issues.apache.org/jira/browse/OOZIE-3031 OOZIE-3022 fix for warning has no file and won't be listed in dependency files details https://issues.apache.org/jira/browse/OOZIE-3022 OOZIE-3002 address findbugs errors in client lib https://issues.apache.org/jira/browse/OOZIE-3002 OOZIE-3001 core library has many instances of warnings with trailing spaces and lines longer than 132 chars https://issues.apache.org/jira/browse/OOZIE-3001 OOZIE-2997 files contain trailing white spaces in client lib https://issues.apache.org/jira/browse/OOZIE-2997 OOZIE-2996 add option for -UseGCOverheadLimit to maven opts as sometimes local testing fails https://issues.apache.org/jira/browse/OOZIE-2996 OOZIE-2978 Remove code that handles Pig versions before 0.8 https://issues.apache.org/jira/browse/OOZIE-2978 OOZIE-2975 code clean up in pig sharelib, replace Exception with more explicit, add try with resources, StringBuilder instead of StringBuffer https://issues.apache.org/jira/browse/OOZIE-2975 OOZIE-2969 Drop support for Java 1.7 https://issues.apache.org/jira/browse/OOZIE-2969 OOZIE-2964 Add -Xdoclint:none to javadoc opts to avoid warnings https://issues.apache.org/jira/browse/OOZIE-2964 OOZIE-2962 bump maven-javadoc-plugin to 2.10.4 https://issues.apache.org/jira/browse/OOZIE-2962 OOZIE-2957 Documentation states that starting a coordinator is possible https://issues.apache.org/jira/browse/OOZIE-2957 OOZIE-2956 Fix Findbugs warnings related to reliance on default encoding in oozie-core https://issues.apache.org/jira/browse/OOZIE-2956 OOZIE-2955 Fix Findbugs warnings related to reliance on default encoding in oozie-client https://issues.apache.org/jira/browse/OOZIE-2955 OOZIE-2954 Fix Checkstyle issues in oozie-client https://issues.apache.org/jira/browse/OOZIE-2954 OOZIE-2953 Fix Checkstyle issues in oozie-tools https://issues.apache.org/jira/browse/OOZIE-2953 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2942 Fix Findbugs warnings in oozie-examples https://issues.apache.org/jira/browse/OOZIE-2942 OOZIE-2937 Remove redundant groupId from the child pom's https://issues.apache.org/jira/browse/OOZIE-2937 OOZIE-2934 Fix "Exceptional return value of java.io.File.mkdirs() ignored" Findbugs error in oozie-sharelib-spark https://issues.apache.org/jira/browse/OOZIE-2934 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2914 Consolidate Trim https://issues.apache.org/jira/browse/OOZIE-2914 OOZIE-2883 OOZIE throw the error "Missing [oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups] property" https://issues.apache.org/jira/browse/OOZIE-2883 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2867 Timezone handling for Coordinators: emphasize "Continent/City" format https://issues.apache.org/jira/browse/OOZIE-2867 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2829 Improve sharelib upload to accept multiple source folders https://issues.apache.org/jira/browse/OOZIE-2829 OOZIE-2826 Falcon feed fails to aws s3; Oozie joda time version does not meet required jar version 2.2 or later https://issues.apache.org/jira/browse/OOZIE-2826 OOZIE-2812
