[
https://issues.apache.org/jira/browse/FLUME-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484664#comment-13484664
]
Hari Shreedharan commented on FLUME-1645:
-----------------------------------------
Steve - Thanks for the patch. Looks good. Could you rebase the patch, since it
does not apply cleanly on trunk. I guess you need to pull in the commit that
went in for FLUME-1631.
Also - do you see a way around not using the Clock and SystemClock you added
for testing? This line:
{code}
Assert.assertTrue("Incorrect suffix",
hdfsWriter.getOpenedFilePath().endsWith(Long.toString(testTime+1) + suffix +
BucketWriter.IN_USE_EXT));
{code}
can simply be changed to:
{code}
String fileName = hdfsWriter.getOpenedFilePath()
String[] nameParts = fileName.split(".");
Long.parseLong(nameParts[nameParts.length - 3]); //Will throw exception if this
is not a valid long - we don't need to test if it is current timestamp.
//This unit test isn't meant
to test that.
Assert.assertTrue("Incorrect Suffix", nameParts[nameParts.length -
2].equals(suffix));
//Check for ".tmp"
Assert.assertTrue("Incorrect Suffix", nameParts[nameParts.length -
1].equals(BucketWriter.IN_USE_EXT));
{code}
I am +1 on rest of the code. Please change this to let the
System.currentTimeMillis() calls stay the same.
> add hdfs.fileSuffix property to HDFSEventSink
> ---------------------------------------------
>
> Key: FLUME-1645
> URL: https://issues.apache.org/jira/browse/FLUME-1645
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Steve Hoffman
> Fix For: v1.3.0
>
> Attachments: patch.diff
>
>
> I'd like to be able to use Avro files created by flume as input to a
> MapReduce job. However, the AvroInputFormat requires that avro files end in
> .avro and there appears to be no property on the hdfs sink that allows me to
> set a suffix (default being blank, of course).
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
