[
https://issues.apache.org/jira/browse/FLUME-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484953#comment-13484953
]
Steve Hoffman commented on FLUME-1645:
--------------------------------------
In order to test that the suffix is NOT being appended, I need to know what it
should be.
Sure I could just assume that if it is a number then it is not the suffix.
That is unless I make my suffix to be a number in which case I can't tell if is
the roll number or the time. It is easy to test for the existence of the
suffix 'cause I know exactly what I'm looking for. Testing for the absence of
something is harder. I could do as you suggest, but I think the test would be
weaker/less precise. I could also remove the "nosuffix" test, but that too
would be incomplete.
The idea of a Clock interface I've seen used in many other projects for which
there is a dependency on System.currentTimeMillis basically to give the test
the ability to control time. For instance, if you trying to test that
something happens every hour, you aren't going to sit there for an hour waiting
for the test to finish running. You give the test the ability to advance the
clock, hence the notion of an overridable Clock (wouldn't want to do something
like a creating a mock for System). Since it could (and should) be used in
other time dependent code/tests I moved it to the common package in the hopes
others would make use of it. The SystemClock is just the default convenience
class which wraps the System class call (so everybody doesn't create the same
anonymous inner class). Can you tell I'm a fan of IOC over static methods?
If you ask for the test, I'm gonna try and make a good one ;)
I'll try and create patches for the other branches and upload today assuming
I've convinced you (and others hopefully) about the Clock notion.
> add hdfs.fileSuffix property to HDFSEventSink
> ---------------------------------------------
>
> Key: FLUME-1645
> URL: https://issues.apache.org/jira/browse/FLUME-1645
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Steve Hoffman
> Fix For: v1.3.0
>
> Attachments: patch.diff
>
>
> I'd like to be able to use Avro files created by flume as input to a
> MapReduce job. However, the AvroInputFormat requires that avro files end in
> .avro and there appears to be no property on the hdfs sink that allows me to
> set a suffix (default being blank, of course).
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
