[jira] [Commented] (OOZIE-3192) OozieDBCLI does not clean up temporary files
[
https://issues.apache.org/jira/browse/OOZIE-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433361#comment-16433361
]
Prabhu Joseph commented on OOZIE-3192:
--
Thanks [~gezapeti].
> OozieDBCLI does not clean up temporary files
>
>
> Key: OOZIE-3192
> URL: https://issues.apache.org/jira/browse/OOZIE-3192
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3192.1.patch, OOZIE-3192.2.patch
>
>
> Oozie runtime directories are getting accumulated under /tmp. OozieDBCLI does
> not destroy the Services and so the code to delete runtime directories are
> skipped.
>
> {code:java}
> [root@bigdata2 libtools]# ls /tmp/oozie-oozi*
> /tmp/oozie-oozi1000989288737845910.dir:
> /tmp/oozie-oozi1036845504637967814.dir:
> /tmp/oozie-oozi2051016416521814301.dir:
> /tmp/oozie-oozi3194893167383531504.dir:
> /tmp/oozie-oozi3750814910369499964.dir:
> /tmp/oozie-oozi3988391799981903880.dir:
> /tmp/oozie-oozi4308477407910112054.dir:
> /tmp/oozie-oozi4717881377452217148.dir:
> /tmp/oozie-oozi5894080803700698940.dir:
> /tmp/oozie-oozi6367849996578983710.dir:
> /tmp/oozie-oozi682733264186033679.dir:
> /tmp/oozie-oozi7006370502487976692.dir:
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3039) Add Sqoop2 action
[ https://issues.apache.org/jira/browse/OOZIE-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432994#comment-16432994 ] Satish Subhashrao Saley commented on OOZIE-3039: Hi [~huLiu] are you working on Sqoop2 action. If not, I will be happy to take this. > Add Sqoop2 action > - > > Key: OOZIE-3039 > URL: https://issues.apache.org/jira/browse/OOZIE-3039 > Project: Oozie > Issue Type: Improvement >Reporter: Hu Liu, >Assignee: Hu Liu, >Priority: Major > > Now Sqoop2 is released and we should add support for it. > I'm glad to work on it if anyone could assign it to me. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3216) Enable Authorization testing on REST endpoints
Peter Orova created OOZIE-3216: -- Summary: Enable Authorization testing on REST endpoints Key: OOZIE-3216 URL: https://issues.apache.org/jira/browse/OOZIE-3216 Project: Oozie Issue Type: Improvement Reporter: Peter Orova Currently there is no straightforward way of testing the Authorization service's functionality together with the Rest endpoints. The authorization decisions are made based on the contents of the authToke and the authToken in the request is filled by the AuthenticationFilter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3215) Improve testability for authorization
Peter Orova created OOZIE-3215: -- Summary: Improve testability for authorization Key: OOZIE-3215 URL: https://issues.apache.org/jira/browse/OOZIE-3215 Project: Oozie Issue Type: Improvement Reporter: Peter Orova The LocalOozie should be enhanced to be able to run integration tests on Authorisation. One way of achieving this could be to launch an embedded servlet container within. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3196) Authorization: restrict world readability by user
[ https://issues.apache.org/jira/browse/OOZIE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3196: Affects Version/s: 5.0.0 > Authorization: restrict world readability by user > - > > Key: OOZIE-3196 > URL: https://issues.apache.org/jira/browse/OOZIE-3196 > Project: Oozie > Issue Type: New Feature > Components: bundle, coordinator, workflow >Affects Versions: 5.0.0b1, 5.0.0 >Reporter: Andras Piros >Assignee: Peter Orova >Priority: Major > Fix For: 5.1.0 > > > The [*current authorization > model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the > enterprise requirements as everything is readable and writable by everyone by > default. > Write access can be restricted using authorization but restricting read > rights is only possible via Yarn ACLs and HDFS rights which still does not > prevent accessing the workflow, coordinator or bundle job’s configurations > for everyone. > Improve authorization so it’s possible to configure read/write access for > workflows, coordinators, and bundles in a more granular way. Could involve > Sentry during implementation or create and design a new system that fits the > needs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3196) Authorization: restrict world readability by user
[ https://issues.apache.org/jira/browse/OOZIE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3196: Fix Version/s: 5.1.0 > Authorization: restrict world readability by user > - > > Key: OOZIE-3196 > URL: https://issues.apache.org/jira/browse/OOZIE-3196 > Project: Oozie > Issue Type: New Feature > Components: bundle, coordinator, workflow >Affects Versions: 5.0.0b1, 5.0.0 >Reporter: Andras Piros >Assignee: Peter Orova >Priority: Major > Fix For: 5.1.0 > > > The [*current authorization > model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the > enterprise requirements as everything is readable and writable by everyone by > default. > Write access can be restricted using authorization but restricting read > rights is only possible via Yarn ACLs and HDFS rights which still does not > prevent accessing the workflow, coordinator or bundle job’s configurations > for everyone. > Improve authorization so it’s possible to configure read/write access for > workflows, coordinators, and bundles in a more granular way. Could involve > Sentry during implementation or create and design a new system that fits the > needs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2683) Rewrite the Oozie Web UI
[ https://issues.apache.org/jira/browse/OOZIE-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2683: Affects Version/s: 5.0.0 > Rewrite the Oozie Web UI > > > Key: OOZIE-2683 > URL: https://issues.apache.org/jira/browse/OOZIE-2683 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Major > Fix For: 5.2.0 > > > We're currently relying on a version of ExtJS that's so old, it's not linked > on their official website, and is often temporarily deleted. Looks like it's > now been deleted for good (OOZIE-2622). It also has a GPL license, so we > can't include it with Oozie, and users are forced to go and download it > themselves. And finally, it's a really outdated UI that's not very good. > We should invest in a new Web UI that's more modern and compatibly licensed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2683) Rewrite the Oozie Web UI
[ https://issues.apache.org/jira/browse/OOZIE-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2683: Fix Version/s: 5.2.0 > Rewrite the Oozie Web UI > > > Key: OOZIE-2683 > URL: https://issues.apache.org/jira/browse/OOZIE-2683 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Major > Fix For: 5.2.0 > > > We're currently relying on a version of ExtJS that's so old, it's not linked > on their official website, and is often temporarily deleted. Looks like it's > now been deleted for good (OOZIE-2622). It also has a GPL license, so we > can't include it with Oozie, and users are forced to go and download it > themselves. And finally, it's a really outdated UI that's not very good. > We should invest in a new Web UI that's more modern and compatibly licensed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3186) Oozie is unable to use configuration linked using jceks://file/...
[
https://issues.apache.org/jira/browse/OOZIE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432295#comment-16432295
]
Denes Bodo commented on OOZIE-3186:
---
Thank you, [~andras.piros].
Here is the review board ticket: https://reviews.apache.org/r/66530/
> Oozie is unable to use configuration linked using jceks://file/...
> --
>
> Key: OOZIE-3186
> URL: https://issues.apache.org/jira/browse/OOZIE-3186
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1, 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: usability
> Fix For: trunk
>
> Attachments: OOZIE-3186-001.patch, OOZIE-3186-002.patch
>
>
> When Oozie is used with Ambari, the next configuration makes Oozie fail to
> start:
> {noformat}
>
> hadoop.security.credential.provider.path
> jceks://file/.../oozie-site.jceks
>
> {noformat}
> Value should have *localjceks://* instead of *jceks://*. But Ambari does not
> let change this value. I propose change the url when Oozie loads it.
>
> Stacktrace, when the issue occurs:
> {code:java}
> org.apache.oozie.service.ServiceException: E0103: Could not load service
> classes, Could not load password for [oozie.service.JPAService.jdbc.password]
> at org.apache.oozie.service.Services.loadServices(Services.java:309) at
> org.apache.oozie.service.Services.init(Services.java:213) at
> org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
> at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
> at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780)
> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at
> org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at
> org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069) at
> org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061) at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at
> org.apache.catalina.core.StandardService.start(StandardService.java:525) at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:761) at
> org.apache.catalina.startup.Catalina.start(Catalina.java:595) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by:
> java.lang.IllegalArgumentException: Could not load password for
> [oozie.service.JPAService.jdbc.password] at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:615)
> at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:602)
> at org.apache.oozie.service.JPAService.init(JPAService.java:147) at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:386) at
> org.apache.oozie.service.Services.setService(Services.java:372) at
> org.apache.oozie.service.Services.loadServices(Services.java:305) ... 26 more
> Caused by: java.lang.reflect.InvocationTargetException at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:608)
> ... 31 more Caused by: java.lang.UnsupportedOperationException: Accessing
> local file system is not allowed at
> org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
> at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47)
> at
[jira] [Commented] (OOZIE-3194) Oozie should set proper permissions to sharelib after upload
[ https://issues.apache.org/jira/browse/OOZIE-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432240#comment-16432240 ] Denes Bodo commented on OOZIE-3194: --- [~andras.piros] Thanks for your time. Here you are: https://reviews.apache.org/r/66529/ > Oozie should set proper permissions to sharelib after upload > > > Key: OOZIE-3194 > URL: https://issues.apache.org/jira/browse/OOZIE-3194 > Project: Oozie > Issue Type: Bug > Components: tools >Affects Versions: 4.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Critical > Fix For: 4.3.1 > > Attachments: OOZIE-3194-v1.patch, OOZIE-3194-v2.patch > > > When user changes HDFS umask that does not support world readability to newly > created files, upgrading sharelib causes failing Oozie jobs. > I suggest set the required permissions to sharelib after upgrading it. Files > to 544 and directories to 755. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3214) Allow configurable timezone for coordinators
[
https://issues.apache.org/jira/browse/OOZIE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3214:
Description:
Right now in case of coordinators only daylight saving time corrections are
applied, the processing timezone for the coordinators is always the Oozie
processing timezone.
It would be more transparent to have an option for changing the {{timezone}}
attribute itself, such as an additional attribute in the {{coordinator.xml}}
(meaning a new version of {{coordinator.xsd}}). I would make this option
switched off by default(to have the actual behavior).
was:
Right now in case of coordinators only DST corrections are applied, the
processing timezone for the coordinators is always the Oozie processing
timezone.
It would be more transparent to have an option for changing the timezone
itself, such as an additional attribute in the coordinator.xml. I would make
this option switched off by default(to have the actual behaviour).
> Allow configurable timezone for coordinators
>
>
> Key: OOZIE-3214
> URL: https://issues.apache.org/jira/browse/OOZIE-3214
> Project: Oozie
> Issue Type: Improvement
> Components: coordinator
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
>
> Right now in case of coordinators only daylight saving time corrections are
> applied, the processing timezone for the coordinators is always the Oozie
> processing timezone.
> It would be more transparent to have an option for changing the {{timezone}}
> attribute itself, such as an additional attribute in the {{coordinator.xml}}
> (meaning a new version of {{coordinator.xsd}}). I would make this option
> switched off by default(to have the actual behavior).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3214) Allow configurable timezone for coordinators
Julia Kinga Marton created OOZIE-3214: - Summary: Allow configurable timezone for coordinators Key: OOZIE-3214 URL: https://issues.apache.org/jira/browse/OOZIE-3214 Project: Oozie Issue Type: Improvement Components: coordinator Affects Versions: trunk Reporter: Julia Kinga Marton Assignee: Julia Kinga Marton Right now in case of coordinators only DST corrections are applied, the processing timezone for the coordinators is always the Oozie processing timezone. It would be more transparent to have an option for changing the timezone itself, such as an additional attribute in the coordinator.xml. I would make this option switched off by default(to have the actual behaviour). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3186) Oozie is unable to use configuration linked using jceks://file/...
[
https://issues.apache.org/jira/browse/OOZIE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431872#comment-16431872
]
Andras Piros commented on OOZIE-3186:
-
Interesting idea [~dionusos]! The overall approach looks good to me.
Can you please create unit tests that cover the original and the modified
behavior, as well as put the patch to ReviewBoard? Thanks!
> Oozie is unable to use configuration linked using jceks://file/...
> --
>
> Key: OOZIE-3186
> URL: https://issues.apache.org/jira/browse/OOZIE-3186
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1, 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: usability
> Fix For: trunk
>
> Attachments: OOZIE-3186-001.patch, OOZIE-3186-002.patch
>
>
> When Oozie is used with Ambari, the next configuration makes Oozie fail to
> start:
> {noformat}
>
> hadoop.security.credential.provider.path
> jceks://file/.../oozie-site.jceks
>
> {noformat}
> Value should have *localjceks://* instead of *jceks://*. But Ambari does not
> let change this value. I propose change the url when Oozie loads it.
>
> Stacktrace, when the issue occurs:
> {code:java}
> org.apache.oozie.service.ServiceException: E0103: Could not load service
> classes, Could not load password for [oozie.service.JPAService.jdbc.password]
> at org.apache.oozie.service.Services.loadServices(Services.java:309) at
> org.apache.oozie.service.Services.init(Services.java:213) at
> org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
> at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
> at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780)
> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at
> org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at
> org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069) at
> org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061) at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at
> org.apache.catalina.core.StandardService.start(StandardService.java:525) at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:761) at
> org.apache.catalina.startup.Catalina.start(Catalina.java:595) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by:
> java.lang.IllegalArgumentException: Could not load password for
> [oozie.service.JPAService.jdbc.password] at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:615)
> at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:602)
> at org.apache.oozie.service.JPAService.init(JPAService.java:147) at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:386) at
> org.apache.oozie.service.Services.setService(Services.java:372) at
> org.apache.oozie.service.Services.loadServices(Services.java:305) ... 26 more
> Caused by: java.lang.reflect.InvocationTargetException at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:608)
> ... 31 more Caused by: java.lang.UnsupportedOperationException: Accessing
> local file system is not allowed at
> org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
> at
[jira] [Updated] (OOZIE-3186) Oozie is unable to use configuration linked using jceks://file/...
[
https://issues.apache.org/jira/browse/OOZIE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3186:
Description:
When Oozie is used with Ambari, the next configuration makes Oozie fail to
start:
{noformat}
hadoop.security.credential.provider.path
jceks://file/.../oozie-site.jceks
{noformat}
Value should have *localjceks://* instead of *jceks://*. But Ambari does not
let change this value. I propose change the url when Oozie loads it.
Stacktrace, when the issue occurs:
{noformat}
org.apache.oozie.service.ServiceException: E0103: Could not load service
classes, Could not load password for [oozie.service.JPAService.jdbc.password]
at org.apache.oozie.service.Services.loadServices(Services.java:309) at
org.apache.oozie.service.Services.init(Services.java:213) at
org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780) at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069) at
org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061) at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at
org.apache.catalina.core.StandardService.start(StandardService.java:525) at
org.apache.catalina.core.StandardServer.start(StandardServer.java:761) at
org.apache.catalina.startup.Catalina.start(Catalina.java:595) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by:
java.lang.IllegalArgumentException: Could not load password for
[oozie.service.JPAService.jdbc.password] at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:615)
at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:602)
at org.apache.oozie.service.JPAService.init(JPAService.java:147) at
org.apache.oozie.service.Services.setServiceInternal(Services.java:386) at
org.apache.oozie.service.Services.setService(Services.java:372) at
org.apache.oozie.service.Services.loadServices(Services.java:305) ... 26 more
Caused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:608)
... 31 more Caused by: java.lang.UnsupportedOperationException: Accessing
local file system is not allowed at
org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47) at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2795) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at
org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.(AbstractJavaKeyStoreProvider.java:82)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:49)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:41)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at
[jira] [Updated] (OOZIE-3186) Oozie is unable to use configuration linked using jceks://file/...
[
https://issues.apache.org/jira/browse/OOZIE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3186:
Description:
When Oozie is used with Ambari, the next configuration makes Oozie fail to
start:
{noformat}
hadoop.security.credential.provider.path
jceks://file/.../oozie-site.jceks
{noformat}
Value should have *localjceks://* instead of *jceks://*. But Ambari does not
let change this value. I propose change the url when Oozie loads it.
Stacktrace, when the issue occurs:
{code:java}
org.apache.oozie.service.ServiceException: E0103: Could not load service
classes, Could not load password for [oozie.service.JPAService.jdbc.password]
at org.apache.oozie.service.Services.loadServices(Services.java:309) at
org.apache.oozie.service.Services.init(Services.java:213) at
org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780) at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069) at
org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061) at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at
org.apache.catalina.core.StandardService.start(StandardService.java:525) at
org.apache.catalina.core.StandardServer.start(StandardServer.java:761) at
org.apache.catalina.startup.Catalina.start(Catalina.java:595) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by:
java.lang.IllegalArgumentException: Could not load password for
[oozie.service.JPAService.jdbc.password] at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:615)
at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:602)
at org.apache.oozie.service.JPAService.init(JPAService.java:147) at
org.apache.oozie.service.Services.setServiceInternal(Services.java:386) at
org.apache.oozie.service.Services.setService(Services.java:372) at
org.apache.oozie.service.Services.loadServices(Services.java:305) ... 26 more
Caused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:608)
... 31 more Caused by: java.lang.UnsupportedOperationException: Accessing
local file system is not allowed at
org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47) at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2795) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at
org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.(AbstractJavaKeyStoreProvider.java:82)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:49)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:41)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at
[jira] [Commented] (OOZIE-3194) Oozie should set proper permissions to sharelib after upload
[ https://issues.apache.org/jira/browse/OOZIE-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431807#comment-16431807 ] Andras Piros commented on OOZIE-3194: - Thanks for the contribution [~dionusos]! Can you please upload the latest patch to ReviewBoard, and also add a JUnit test case where the functionality the patch tries to address gets covered? Thanks! > Oozie should set proper permissions to sharelib after upload > > > Key: OOZIE-3194 > URL: https://issues.apache.org/jira/browse/OOZIE-3194 > Project: Oozie > Issue Type: Bug > Components: tools >Affects Versions: 4.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Critical > Fix For: 4.3.1 > > Attachments: OOZIE-3194-v1.patch, OOZIE-3194-v2.patch > > > When user changes HDFS umask that does not support world readability to newly > created files, upgrading sharelib causes failing Oozie jobs. > I suggest set the required permissions to sharelib after upgrading it. Files > to 544 and directories to 755. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (103 issues) Subscriber: ooziedaily Key Summary OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3194 Oozie should set proper permissions to sharelib after upload https://issues.apache.org/jira/browse/OOZIE-3194 OOZIE-3193 Applications are not killed when submitted via subworkflow https://issues.apache.org/jira/browse/OOZIE-3193 OOZIE-3186 Oozie is unable to use configuration linked using jceks://file/... https://issues.apache.org/jira/browse/OOZIE-3186 OOZIE-3185 Conflicting JARs org.apache.derby exist in Oozie https://issues.apache.org/jira/browse/OOZIE-3185 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3109 Escape log-streaming's HTML-specific characters https://issues.apache.org/jira/browse/OOZIE-3109 OOZIE-3105 testJMXInstrumentation from the org.apache.oozie.util.TestMetricsInstrumentation class is flaky https://issues.apache.org/jira/browse/OOZIE-3105 OOZIE-3094 fix for grammar mistake https://issues.apache.org/jira/browse/OOZIE-3094 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3061 Kill only those child jobs which are not already killed https://issues.apache.org/jira/browse/OOZIE-3061 OOZIE-3002 address findbugs errors in client lib https://issues.apache.org/jira/browse/OOZIE-3002 OOZIE-2975 code clean up in pig sharelib, replace Exception with more explicit, add try with resources, StringBuilder instead of StringBuffer https://issues.apache.org/jira/browse/OOZIE-2975 OOZIE-2956 Fix Findbugs warnings related to reliance on default encoding in oozie-core https://issues.apache.org/jira/browse/OOZIE-2956 OOZIE-2955 Fix Findbugs warnings related to reliance on default encoding in oozie-client https://issues.apache.org/jira/browse/OOZIE-2955 OOZIE-2954 Fix Checkstyle issues in oozie-client https://issues.apache.org/jira/browse/OOZIE-2954 OOZIE-2953 Fix Checkstyle issues in oozie-tools https://issues.apache.org/jira/browse/OOZIE-2953 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2942 Fix Findbugs warnings in oozie-examples https://issues.apache.org/jira/browse/OOZIE-2942 OOZIE-2934 Fix "Exceptional return value of java.io.File.mkdirs() ignored" Findbugs error in oozie-sharelib-spark https://issues.apache.org/jira/browse/OOZIE-2934 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2914 Consolidate Trim https://issues.apache.org/jira/browse/OOZIE-2914 OOZIE-2883 OOZIE throw the error "Missing [oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups] property" https://issues.apache.org/jira/browse/OOZIE-2883 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2867 Timezone handling for Coordinators: emphasize "Continent/City" format https://issues.apache.org/jira/browse/OOZIE-2867 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2829 Improve sharelib upload to accept multiple source folders https://issues.apache.org/jira/browse/OOZIE-2829 OOZIE-2826 Falcon feed fails to aws s3; Oozie joda time version does not meet required jar version 2.2 or later https://issues.apache.org/jira/browse/OOZIE-2826 OOZIE-2812 SparkConfigurationService should support loading configurations from
