Konstantine Karantasis created KAFKA-5046:
---------------------------------------------
Summary: Support file rotation in FileStreamSource Connector
Key: KAFKA-5046
URL: https://issues.apache.org/jira/browse/KAFKA-5046
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Affects Versions: 0.10.2.0
Reporter: Konstantine Karantasis
Assignee: Konstantine Karantasis
Priority: Minor
Currently when a source file is moved (for file rotation purposes, or between
restarts of Kafka Connect) the FileStreamSource Connector can not detect the
change, because it only uses the filename as key to its offset tracking.
Nevertheless, file rotation can be detected easily by checking basic file
attributes such as the {{fileKey}} in platforms that this attribute is
supported (for instance file key includes the device id and the inode in unix
based filesystems) and the file's creation time.
Such checks need to take place when the task starts and when no more records
are read during a call to {{poll}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
