[
https://issues.apache.org/jira/browse/FLUME-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235435#comment-13235435
]
[email protected] commented on FLUME-1020:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4360/
-----------------------------------------------------------
(Updated 2012-03-22 08:06:39.589736)
Review request for Flume.
Changes
-------
Brock, thanks for all the feedback!
I am now looking for the bigtop JAVA_HOME detection script and calling it if
it's there.
I've also incorporated more suggestions from Roman, including using slf4j 1.6.1
which Hadoop and Zookeeper are using. I'm also excluding slf4j from the hadoop
classpath when it's injected into Flume's classpath to avoid warnings in the
log when it's an older version of Hadoop.
Also incorporated the suggestions regarding not checking twice and incorporated
some debug messages to indicate overall success or failure.
I tested this all on a Kerberos cluster and it seems to work well.
Summary
-------
This is an initial pass at an implementation of HDFS security. I think it will
probably work. Currently trying to get Kerberos to play nice with the cluster
on my VM though, so I haven't successfully tested it yet. It still works when
used on HDFS with security disabled. :)
The only thing I don't like is in configure() when authentication fails I throw
a FlumeException. I'll trace up and see how bad that would be but it seems
likely to break something. Just logging the error is kind of a bummer as well,
though ... need to ensure process() doesn't fill up the disk while spewing
copious error messages into the logs. Maybe this is a use case for some kind of
FatalException type thing.
This addresses bug FLUME-1020.
https://issues.apache.org/jira/browse/FLUME-1020
Diffs (updated)
-----
bin/flume-ng 0796a5b
flume-ng-core/src/main/java/org/apache/flume/tools/GetJavaProperty.java
PRE-CREATION
flume-ng-sinks/flume-hdfs-sink/pom.xml 1a35baf
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
da82f7e
pom.xml 227bca8
Diff: https://reviews.apache.org/r/4360/diff
Testing
-------
Thanks,
Mike
> Implement Kerberos security for HDFS Sink
> -----------------------------------------
>
> Key: FLUME-1020
> URL: https://issues.apache.org/jira/browse/FLUME-1020
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Assignee: Mike Percy
>
> Make flume HDFS sink work with secure clusters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
