Sorry, I don't know much about this, but here are two relevant JIRA's: https://issues.apache.org/jira/browse/FLUME-1228 https://issues.apache.org/jira/browse/FLUME-951
On Fri, Mar 29, 2013 at 9:44 AM, Matthew Moore <[email protected]> wrote: > Hey there, > > I know this is a really newbish question, but I'm hoping to get a little > assistance here so I'm not stuck guess-and-checking. > > I'm trying to figure out how to configure FlumeNG (1.3.1), but I couldn't > figure out how to setup the hdfs sink to use the s3 implementations. > > I'm keeping track of my progress on this gist I made: > https://gist.github.com/crowdmatt/5256881 > > From what I've gathered, I should be using the hdfs type, which I'm > setting up as such: > > agent.sinks = s3Sink > agent.sinks.s3Sink.type = hdfs > agent.sinks.s3Sink.channel = recoverableMemoryChannel > > ... but that's where I end up hitting my head against the wall. I know I > should be specifying my s3 access key, secret, and bucket in this format: > s3n://ACCESS_KEY_ID:SECRET_ACCESS_KEY@my-hdfs/ > > However, I don't know where to specify that, or what dot notation to use. > > Can anyone point me in the right direction? > > Best, > Matt > -- > Matthew Moore > Co-Founder & CTO, CrowdMob Inc. > Mobile: (650) 888-5962 > > Need to schedule a meeting? Invite me via Google Calendar! > [email protected] > -- Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
