[
https://issues.apache.org/jira/browse/FLUME-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maciej Karaś updated FLUME-2710:
--------------------------------
Description:
Timestamp extension is added to every file in HDFS sink. For a matter, it is
added to every sink using -BucketWriter.java-. Line 247 -249 adds this
timestamp counter to filename and it cannot be configured in opposite to what
[~ashar103] said in parent task.
{code:java}
long counter = fileExtensionCounter.incrementAndGet();
String fullFileName = fileName + "." + counter;
[...]
bucketPath = filePath + "/" + inUsePrefix + fullFileName + inUseSuffix;
targetPath = filePath + "/" + fullFileName;
{code}
Event creation time is not necessary a time of event saving into hdfs,
obviously events can be delayed. Thus I don't know why file creation timestamp
is mandatory - it can be confusing.
was:
Timestamp extension is added to every file in HDFS sink. For a matter, it is
added to every sink using --BucketWriter.java--. Line 247 -249 adds this
timestamp counter to filename and it cannot be configured in opposite to what
[~ashar103] said in parent task.
{code:java}
long counter = fileExtensionCounter.incrementAndGet();
String fullFileName = fileName + "." + counter;
[...]
bucketPath = filePath + "/" + inUsePrefix + fullFileName + inUseSuffix;
targetPath = filePath + "/" + fullFileName;
{code}
Event creation time is not necessary a time of event saving into hdfs,
obviously events can be delayed. Thus I don't know why file creation timestamp
is mandatory - it can be confusing.
> Timestamp file extension
> ------------------------
>
> Key: FLUME-2710
> URL: https://issues.apache.org/jira/browse/FLUME-2710
> Project: Flume
> Issue Type: Sub-task
> Components: Configuration, Sinks+Sources
> Affects Versions: v1.5.1, 1.6
> Reporter: Maciej Karaś
>
> Timestamp extension is added to every file in HDFS sink. For a matter, it is
> added to every sink using -BucketWriter.java-. Line 247 -249 adds this
> timestamp counter to filename and it cannot be configured in opposite to what
> [~ashar103] said in parent task.
> {code:java}
> long counter = fileExtensionCounter.incrementAndGet();
> String fullFileName = fileName + "." + counter;
> [...]
> bucketPath = filePath + "/" + inUsePrefix + fullFileName +
> inUseSuffix;
> targetPath = filePath + "/" + fullFileName;
> {code}
> Event creation time is not necessary a time of event saving into hdfs,
> obviously events can be delayed. Thus I don't know why file creation
> timestamp is mandatory - it can be confusing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
