[jira] [Commented] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.

2019-05-22 Thread Jeff Yang (JIRA) 


[ 
https://issues.apache.org/jira/browse/FLUME-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16846387#comment-16846387
 ] 

Jeff Yang commented on FLUME-3216:
--

I have resubmitted this patch.

> When I rename the file, it causes flume to read the data repeatedly.
> 
>
> Key: FLUME-3216
> URL: https://issues.apache.org/jira/browse/FLUME-3216
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
> Environment: centos-6.8
>Reporter: Jeff Yang
>Priority: Critical
> Fix For: 1.7.0, 1.8.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When I use Taildir source, whenever I rename a file, it causes flume to read 
> the data repeatedly.This resulted in duplication of uploaded data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org
For additional commands, e-mail: issues-h...@flume.apache.org

[jira] [Commented] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.

2018-02-10 Thread jifei_yang (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359326#comment-16359326
 ] 

jifei_yang commented on FLUME-3216:
---

I modified the method of judgment: Judge the log file only by inode.

> When I rename the file, it causes flume to read the data repeatedly.
> 
>
> Key: FLUME-3216
> URL: https://issues.apache.org/jira/browse/FLUME-3216
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
> Environment: centos-6.8
>Reporter: jifei_yang
>Priority: Critical
> Fix For: 1.7.0, 1.8.0
>
>
> When I use Taildir source, whenever I rename a file, it causes flume to read 
> the data repeatedly.This resulted in duplication of uploaded data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org
For additional commands, e-mail: issues-h...@flume.apache.org

[jira] [Commented] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.

2018-02-10 Thread jifei_yang (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359324#comment-16359324
 ] 

jifei_yang commented on FLUME-3216:
---

flume monitor files in the specified directory, rename the file in the 
directory, flume will think the renamed file is a new file, will be read from 
the first log, resulting in the duplication of the log upload.

> When I rename the file, it causes flume to read the data repeatedly.
> 
>
> Key: FLUME-3216
> URL: https://issues.apache.org/jira/browse/FLUME-3216
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
> Environment: centos-6.8
>Reporter: jifei_yang
>Priority: Critical
> Fix For: 1.7.0, 1.8.0
>
>
> When I use Taildir source, whenever I rename a file, it causes flume to read 
> the data repeatedly.This resulted in duplication of uploaded data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org
For additional commands, e-mail: issues-h...@flume.apache.org

[jira] [Commented] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.

2018-02-08 Thread Ferenc Szabo (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356682#comment-16356682
 ] 

Ferenc Szabo commented on FLUME-3216:
-

Hi [~highfei2...@126.com], 

could you please describe the scenario where this happens?
What is the use case where someone would move out and move back the same file 
into the tailed folder and would not expect them to be read again?




> When I rename the file, it causes flume to read the data repeatedly.
> 
>
> Key: FLUME-3216
> URL: https://issues.apache.org/jira/browse/FLUME-3216
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
> Environment: centos-6.8
>Reporter: jifei_yang
>Priority: Critical
> Fix For: 1.7.0, 1.8.0
>
>
> When I use Taildir source, whenever I rename a file, it causes flume to read 
> the data repeatedly.This resulted in duplication of uploaded data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org
For additional commands, e-mail: issues-h...@flume.apache.org

[jira] [Commented] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.

2018-02-02 Thread Ferenc Szabo (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350072#comment-16350072
 ] 

Ferenc Szabo commented on FLUME-3216:
-

Renaming a file creates a new file, so the problem you describe here is the 
expected behavior.

> When I rename the file, it causes flume to read the data repeatedly.
> 
>
> Key: FLUME-3216
> URL: https://issues.apache.org/jira/browse/FLUME-3216
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
> Environment: centos-6.8
>Reporter: jifei_yang
>Priority: Critical
> Fix For: 1.7.0, 1.8.0
>
>
> When I use Taildir source, whenever I rename a file, it causes flume to read 
> the data repeatedly.This resulted in duplication of uploaded data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org
For additional commands, e-mail: issues-h...@flume.apache.org

[jira] [Commented] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.

2018-02-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349776#comment-16349776
 ] 

ASF GitHub Bot commented on FLUME-3216:
---

GitHub user highfei2011 opened a pull request:

https://github.com/apache/flume/pull/196

FLUME-3216.Avoid reading content repeatedly.

When I use Taildir source, whenever I rename a file, it causes flume to 
read the data repeatedly.This resulted in duplication of uploaded data.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/highfei2011/flume trunk

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flume/pull/196.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #196


commit db7996ca0e287aa78d68e460ee9287b731f0e5f1
Author: yangjifei 
Date:   2018-02-02T04:32:40Z

FLUME-3216.Avoid reading content repeatedly.

When I use Taildir source, whenever I rename a file, it causes flume to 
read the data repeatedly.This resulted in duplication of uploaded data.

commit 574db14d91cdc10c6333ccfdf20c543d64303abc
Author: yangjifei 
Date:   2018-02-02T04:38:02Z

FLUME-3216.Avoid reading content repeatedly.

When I use Taildir source, whenever I rename a file, it causes flume to 
read the data repeatedly.This resulted in duplication of uploaded data.




> When I rename the file, it causes flume to read the data repeatedly.
> 
>
> Key: FLUME-3216
> URL: https://issues.apache.org/jira/browse/FLUME-3216
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
> Environment: centos-6.8
>Reporter: jifei_yang
>Priority: Critical
> Fix For: 1.7.0, 1.8.0
>
>
> When I use Taildir source, whenever I rename a file, it causes flume to read 
> the data repeatedly.This resulted in duplication of uploaded data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org
For additional commands, e-mail: issues-h...@flume.apache.org