[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146042#comment-14146042
]
Dapeng Sun commented on OOZIE-1491:
---
Thank [~rkanter] very much, refer your code like {{JaasConfiguration}} to
SENTRY-459 Security mode (Kerberos) support for SENTRY high availability.
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Fix For: 4.1.0
>
> Attachments: OOZIE-1491.patch, OOZIE-1491.patch, OOZIE-1491.patch,
> OOZIE-1491.patch, OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[ https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13837091#comment-13837091 ] Alejandro Abdelnur commented on OOZIE-1491: --- LGTM +1 > Make sure HA works with a secure ZooKeeper > -- > > Key: OOZIE-1491 > URL: https://issues.apache.org/jira/browse/OOZIE-1491 > Project: Oozie > Issue Type: Improvement > Components: HA >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Robert Kanter > Attachments: OOZIE-1491.patch, OOZIE-1491.patch, OOZIE-1491.patch, > OOZIE-1491.patch, OOZIE-1491.patch > > > We need to make sure that HA works with a secure ZooKeeper. This includes > the SASL ACL setting that will prevent someone else from deleting the oozie > znodes. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13826259#comment-13826259
]
Hadoop QA commented on OOZIE-1491:
--
Testing JIRA OOZIE-1491
Cleaning local svn workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1362
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/897/
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1491.patch, OOZIE-1491.patch, OOZIE-1491.patch,
> OOZIE-1491.patch, OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825686#comment-13825686
]
Hadoop QA commented on OOZIE-1491:
--
Testing JIRA OOZIE-1491
Cleaning local svn workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 1 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1361
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/892/
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1491.patch, OOZIE-1491.patch, OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[ https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824035#comment-13824035 ] Robert Kanter commented on OOZIE-1491: -- The javac warning is a false positive. The test issues I think are still related to it pulling in the old MiniKDC that has the dependency issue. Once the maven cache updates, it should be fine. I'll try re-uploading the patch in a few days and maybe that will fix it? > Make sure HA works with a secure ZooKeeper > -- > > Key: OOZIE-1491 > URL: https://issues.apache.org/jira/browse/OOZIE-1491 > Project: Oozie > Issue Type: Improvement > Components: HA >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Robert Kanter > Attachments: OOZIE-1491.patch, OOZIE-1491.patch > > > We need to make sure that HA works with a secure ZooKeeper. This includes > the SASL ACL setting that will prevent someone else from deleting the oozie > znodes. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823499#comment-13823499
]
Hadoop QA commented on OOZIE-1491:
--
Testing JIRA OOZIE-1491
Cleaning local svn workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 1 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1353
.Tests failed: 6
.Tests errors: 29
.The patch failed the following testcases:
. testWorkflowActionEvent(org.apache.oozie.event.TestEventGeneration)
.
testActionWithEscapedStringAndCDATA(org.apache.oozie.command.wf.TestActionStartXCommand)
. testActionStart(org.apache.oozie.command.wf.TestActionStartXCommand)
.
testActionReuseWfJobAppPath(org.apache.oozie.command.wf.TestActionStartXCommand)
.
testWorkflowActionRecoveryUserRetry(org.apache.oozie.service.TestRecoveryService)
. testRerun(org.apache.oozie.action.hadoop.TestRerun)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/887/
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1491.patch, OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822029#comment-13822029
]
Robert Kanter commented on OOZIE-1491:
--
The {{apacheds-all}} artifact includes packages for all kinds of things. I've
created HADOOP-10100 to fix this. I've verified that with my fix, the ehcache
issue goes away.
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821793#comment-13821793
]
Robert Kanter commented on OOZIE-1491:
--
Looks like the problem might be that
{{org.apache.directory.server:apacheds-all:jar}} includes all 3rd party
classes, so it probably has the ehcache jar (and possibly other stuff we don't
want). I'll have to go through the apacheds pom and fix this :(
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821770#comment-13821770
]
Robert Kanter commented on OOZIE-1491:
--
The test failures are all because of a {{NoSuchMethodError}} when trying to use
ehcache:
{noformat}
java.lang.NoSuchMethodError:
net.sf.ehcache.CacheManager.newInstance(Ljava/net/URL;)Lnet/sf/ehcache/CacheManager;
at
org.apache.oozie.dependency.hcat.EhcacheHCatDependencyCache.init(EhcacheHCatDependencyCache.java:89)
{noformat}
I tried removing the {{hadoop-minikdc}} dependency because that's the only
dependency I added/changed, and the test passes. I'm not sure why that's
happening. {{hadoop-minikdc}} doesn't use ehcache and I check with and without
the {{mini-kdc}} dependency and the ehcache jar is the same version.
{noformat}
[INFO] +- org.apache.hadoop:hadoop-minikdc:jar:2.3.0-SNAPSHOT:test
[INFO] | +- commons-io:commons-io:jar:2.1:test
[INFO] | \- org.apache.directory.server:apacheds-all:jar:2.0.0-M15:test
{noformat}
I even did a diff on the {{dependency:tree}} with and without
{{hadoop-minikdc}} and the only difference was the above.
I'll keep looking, but anyone have any ideas?
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[
https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820966#comment-13820966
]
Hadoop QA commented on OOZIE-1491:
--
Testing JIRA OOZIE-1491
Cleaning local svn workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1352
.Tests failed: 0
.Tests errors: 6
.The patch failed the following testcases:
.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/883/
> Make sure HA works with a secure ZooKeeper
> --
>
> Key: OOZIE-1491
> URL: https://issues.apache.org/jira/browse/OOZIE-1491
> Project: Oozie
> Issue Type: Improvement
> Components: HA
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1491.patch
>
>
> We need to make sure that HA works with a secure ZooKeeper. This includes
> the SASL ACL setting that will prevent someone else from deleting the oozie
> znodes.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (OOZIE-1491) Make sure HA works with a secure ZooKeeper
[ https://issues.apache.org/jira/browse/OOZIE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13816960#comment-13816960 ] Robert Kanter commented on OOZIE-1491: -- I've mostly finished the patch for this. I may decide to make a separate JIRA to do the unit tests because that might be a lot of work to do properly (we'll have to use the MiniKDC from Hadoop); though I have tested it manually. The only remaining issue is that the znodes created for the locks always have open ACLs. With help from Jordan Zimmerman on the on the Curator-User list, I'm fairly certain the problem is due to CURATOR-58, which I'll see if I can fix. Even if I fix that, we won't have it until the next Curator release, so until then, there will be a small security hole here where a malicious user could acquire a lock to prevent Oozie from processing that job. > Make sure HA works with a secure ZooKeeper > -- > > Key: OOZIE-1491 > URL: https://issues.apache.org/jira/browse/OOZIE-1491 > Project: Oozie > Issue Type: Improvement > Components: HA >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Robert Kanter > > We need to make sure that HA works with a secure ZooKeeper. This includes > the SASL ACL setting that will prevent someone else from deleting the oozie > znodes. -- This message was sent by Atlassian JIRA (v6.1#6144)
