67287685 created FLUME-3233:
-------------------------------
Summary: log roll to a new name,the flume collect repeat
Key: FLUME-3233
URL: https://issues.apache.org/jira/browse/FLUME-3233
Project: Flume
Issue Type: Bug
Affects Versions: 1.7.0
Reporter: 67287685
we use the flume 1.7 to collect our log send to kafka.
our java application compressed the log at every day 00:00,the log name change
aaa.log -> aaa.20180503.tmp -> aaa.20180503.log.gz,the change while complete in
1 second time.
this have a problem ,when have some data into log at 23:59:59,the log while
compress fast,the flume while not collect the new data,the log is become zip
file,the data is lost.
To solve this problem，we Modify the java application and flume
1.modify java application
the log is compress delay 5 minutes，aaa.log --> aaa.20180503.tmp -> after 5 min
-> aaa.20180503.log.gz,the flume while complete collect the data ,not lost.
2.modify flume 1.7
use inode judge the file uniqueness，not use inode + filename ,when use inode +
filename，the flume while repeate collect when the aaa.log -->
aaa.20180503.tmp,this two file have the same inode
Please help to see if our plan is feasible，thanks every .
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
