[jira] [Commented] (FLUME-2905) NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Siddharth Ahuja (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058799#comment-16058799
 ] 

Siddharth Ahuja commented on FLUME-2905:


Hey [~sati], just added the review request on the Review Board. Not sure if I 
have done everything as per the process. Would be great if you could check. 
Thanks in advance!

> NetcatSource - Socket not closed when an exception is encountered during 
> start() leading to file descriptor leaks
> -
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: 1.6.0
>Reporter: Siddharth Ahuja
>Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch, 
> FLUME-2905-2.patch, FLUME-2905-3.patch, FLUME-2905-4.patch, FLUME-2905-5.patch
>
>
> During the flume agent start-up, the flume configuration containing the 
> NetcatSource is parsed and the source's start() is called. If there is an 
> issue while binding the channel's socket to a local address to configure the 
> socket to listen for connections following exception is thrown but the socket 
> open just before is not closed. 
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: 
> Unable to start EventDrivenSourceRunner: { 
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } - 
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in 
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at 
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at 
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at 
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:167)
> ... 9 more
> {code}
> The source's start() is then called again leading to another socket being 
> opened but not closed and so on. This leads to file descriptor (socket) leaks.
> This can be easily reproduced as follows:
> 1. Set Netcat as the source in flume agent configuration.
> 2. Set the bind port for the netcat source to a port which is already in use. 
> e.g. in my case I used 50010 which is the port for DataNode's XCeiver 
> Protocol in use by the HDFS service.
> 3. Start flume agent and perform "lsof -p  | wc -l". Notice 
> the file descriptors keep on growing due to socket leaks with errors like: 
> "can't identify protocol".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 60357: NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Siddharth Ahuja 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60357/
---

Review request for Flume and Attila Simon.


Repository: flume-git


Description
---

Review request for: https://issues.apache.org/jira/browse/FLUME-2905 which is 
trying to prevent socket leaks when a netcat port is already bound to an 
existing process.


Diffs
-

  flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 9513902 
  flume-ng-core/src/test/java/org/apache/flume/source/TestNetcatSource.java 
99d413a 


Diff: https://reviews.apache.org/r/60357/diff/1/


Testing
---

I have tested flume-ng executable generated from my changes and I can confirm 
from the lsof output that the sockets do not keep increasing if the port to 
which netcat source is trying to bind to is already in use.
The junits are also passing for me for the NetcatSource.


Thanks,

Siddharth Ahuja

[jira] [Updated] (FLUME-2905) NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Siddharth Ahuja (JIRA) 

 [ 
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Ahuja updated FLUME-2905:
---
Attachment: FLUME-2905-5.patch

> NetcatSource - Socket not closed when an exception is encountered during 
> start() leading to file descriptor leaks
> -
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: 1.6.0
>Reporter: Siddharth Ahuja
>Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch, 
> FLUME-2905-2.patch, FLUME-2905-3.patch, FLUME-2905-4.patch, FLUME-2905-5.patch
>
>
> During the flume agent start-up, the flume configuration containing the 
> NetcatSource is parsed and the source's start() is called. If there is an 
> issue while binding the channel's socket to a local address to configure the 
> socket to listen for connections following exception is thrown but the socket 
> open just before is not closed. 
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: 
> Unable to start EventDrivenSourceRunner: { 
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } - 
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in 
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at 
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at 
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at 
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:167)
> ... 9 more
> {code}
> The source's start() is then called again leading to another socket being 
> opened but not closed and so on. This leads to file descriptor (socket) leaks.
> This can be easily reproduced as follows:
> 1. Set Netcat as the source in flume agent configuration.
> 2. Set the bind port for the netcat source to a port which is already in use. 
> e.g. in my case I used 50010 which is the port for DataNode's XCeiver 
> Protocol in use by the HDFS service.
> 3. Start flume agent and perform "lsof -p  | wc -l". Notice 
> the file descriptors keep on growing due to socket leaks with errors like: 
> "can't identify protocol".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (FLUME-2905) NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Siddharth Ahuja (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058745#comment-16058745
 ] 

Siddharth Ahuja edited comment on FLUME-2905 at 6/22/17 5:02 AM:
-

Hi [~sati], thanks a lot for your review.

Please find my answers for your points:

* For point 1. - "calling stop() after writing out the exception", I have moved 
stop() after logging the exception but just before it gets thrown.
* For point 2. - We should possibly have a dedicated JIRA for removing the 
"return" statement from the stop() method as this would be a different issue to 
what I am trying to fix in this JIRA which is to prevent socket leaks if a port 
is already bound. Also, it would make tracking easier with a new JIRA as 
otherwise any issues (if any) arising from this removal will be discussed in 
this JIRA which is a side-track from the original issue that is already 
potentially resolved. What do you think?
* For point 3 - I believe I have nothing to do here.

I have gone on and created a new patch - FLUME-2905-5.patch for your review. I 
have tested that to ensure that there are no leaks and the junit also passes 
for me.

I will try and add this to review board (haven't done that yet) soon.

Thanks once again.


was (Author: sahuja):
Hi [~sati], thanks a lot for your review.

Please find my answers for your points:

* For point 1. - "calling stop() after writing out the exception", I have moved 
stop() after logging the exception but just before it gets thrown.
* For point 2. - We should possibly have a dedicated JIRA for removing the 
"return" statement from the stop() method as this would be a different issue to 
what I am trying to fix in this JIRA which is to prevent socket leaks if a port 
is already bound. Also, it would make tracking easier with a new JIRA as 
otherwise any issues (if any) arising from this removal will be discussed in 
this JIRA which is a side-track from the original issue that is already 
potentially resolved. What do you think?
* For point 3 - I believe I have nothing to do here.

I have gone on and created a new patch - FLUME-2905-5.patch for your review and 
I have also tested that to ensure that there are no leaks.

I will try and add this to review board (haven't done that yet) soon.

Thanks once again.

> NetcatSource - Socket not closed when an exception is encountered during 
> start() leading to file descriptor leaks
> -
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: 1.6.0
>Reporter: Siddharth Ahuja
>Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch, 
> FLUME-2905-2.patch, FLUME-2905-3.patch, FLUME-2905-4.patch
>
>
> During the flume agent start-up, the flume configuration containing the 
> NetcatSource is parsed and the source's start() is called. If there is an 
> issue while binding the channel's socket to a local address to configure the 
> socket to listen for connections following exception is thrown but the socket 
> open just before is not closed. 
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: 
> Unable to start EventDrivenSourceRunner: { 
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } - 
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in 
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at 
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at 
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at 
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at

[jira] [Comment Edited] (FLUME-2905) NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Siddharth Ahuja (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058745#comment-16058745
 ] 

Siddharth Ahuja edited comment on FLUME-2905 at 6/22/17 4:55 AM:
-

Hi [~sati], thanks a lot for your review.

Please find my answers for your points:

* For point 1. - "calling stop() after writing out the exception", I have moved 
stop() after logging the exception but just before it gets thrown.
* For point 2. - We should possibly have a dedicated JIRA for removing the 
"return" statement from the stop() method as this would be a different issue to 
what I am trying to fix in this JIRA which is to prevent socket leaks if a port 
is already bound. Also, it would make tracking easier with a new JIRA as 
otherwise any issues (if any) arising from this removal will be discussed in 
this JIRA which is a side-track from the original issue that is already 
potentially resolved. What do you think?
* For point 3 - I believe I have nothing to do here.

I have gone on and created a new patch - FLUME-2905-5.patch for your review and 
I have also tested that to ensure that there are no leaks.

I will try and add this to review board (haven't done that yet) soon.

Thanks once again.


was (Author: sahuja):
Hi [~sati], thanks a lot for your review.

Please find my answers for your points:

* For point 1. - "calling stop() after writing out the exception", I have moved 
stop() after logging the exception but just before it gets thrown.
* For point 2. - We should possibly have a dedicated JIRA for removing the 
"return" statement from the stop() method as this would be a different issue to 
what I am trying to fix in this JIRA which is to prevent socket leaks if a port 
is already bound. Also, it would make tracking easier with a new JIRA as 
otherwise any issues (if any) arising from this removal will be discussed in 
this JIRA which is a side-track from the original issue that is already 
potentially resolved. What do you think?
* For point 3 - I believe I have nothing to do here.

I have gone on and created a new patch - FLUME-2905-5.patch for your review and 
I have also tested that.

I will try and add this to review board (haven't done that yet) soon.

Thanks once again.

> NetcatSource - Socket not closed when an exception is encountered during 
> start() leading to file descriptor leaks
> -
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: 1.6.0
>Reporter: Siddharth Ahuja
>Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch, 
> FLUME-2905-2.patch, FLUME-2905-3.patch, FLUME-2905-4.patch
>
>
> During the flume agent start-up, the flume configuration containing the 
> NetcatSource is parsed and the source's start() is called. If there is an 
> issue while binding the channel's socket to a local address to configure the 
> socket to listen for connections following exception is thrown but the socket 
> open just before is not closed. 
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: 
> Unable to start EventDrivenSourceRunner: { 
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } - 
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in 
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at 
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at 
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at 
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
>

[jira] [Commented] (FLUME-2905) NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Siddharth Ahuja (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058745#comment-16058745
 ] 

Siddharth Ahuja commented on FLUME-2905:


Hi [~sati], thanks a lot for your review.

Please find my answers for your points:

* For point 1. - "calling stop() after writing out the exception", I have moved 
stop() after logging the exception but just before it gets thrown.
* For point 2. - We should possibly have a dedicated JIRA for removing the 
"return" statement from the stop() method as this would be a different issue to 
what I am trying to fix in this JIRA which is to prevent socket leaks if a port 
is already bound. Also, it would make tracking easier with a new JIRA as 
otherwise any issues (if any) arising from this removal will be discussed in 
this JIRA which is a side-track from the original issue that is already 
potentially resolved. What do you think?
* For point 3 - I believe I have nothing to do here.

I have gone on and created a new patch - FLUME-2905-5.patch for your review.

I will try and add this to review board (haven't done that yet) soon.

Thanks once again.

> NetcatSource - Socket not closed when an exception is encountered during 
> start() leading to file descriptor leaks
> -
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: 1.6.0
>Reporter: Siddharth Ahuja
>Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch, 
> FLUME-2905-2.patch, FLUME-2905-3.patch, FLUME-2905-4.patch
>
>
> During the flume agent start-up, the flume configuration containing the 
> NetcatSource is parsed and the source's start() is called. If there is an 
> issue while binding the channel's socket to a local address to configure the 
> socket to listen for connections following exception is thrown but the socket 
> open just before is not closed. 
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: 
> Unable to start EventDrivenSourceRunner: { 
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } - 
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in 
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at 
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at 
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at 
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:167)
> ... 9 more
> {code}
> The source's start() is then called again leading to another socket being 
> opened but not closed and so on. This leads to file descriptor (socket) leaks.
> This can be easily reproduced as follows:
> 1. Set Netcat as the source in flume agent configuration.
> 2. Set the bind port for the netcat source to a port which is already in use. 
> e.g. in my case I used 50010 which is the port for DataNode's XCeiver 
> Protocol in use by the HDFS service.
> 3. Start flume agent and perform "lsof -p  | wc -l". Notice 
> the file descriptors keep on growing due to socket leaks with errors like: 
> "can't identify protocol".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (FLUME-2905) NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Siddharth Ahuja (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058745#comment-16058745
 ] 

Siddharth Ahuja edited comment on FLUME-2905 at 6/22/17 4:54 AM:
-

Hi [~sati], thanks a lot for your review.

Please find my answers for your points:

* For point 1. - "calling stop() after writing out the exception", I have moved 
stop() after logging the exception but just before it gets thrown.
* For point 2. - We should possibly have a dedicated JIRA for removing the 
"return" statement from the stop() method as this would be a different issue to 
what I am trying to fix in this JIRA which is to prevent socket leaks if a port 
is already bound. Also, it would make tracking easier with a new JIRA as 
otherwise any issues (if any) arising from this removal will be discussed in 
this JIRA which is a side-track from the original issue that is already 
potentially resolved. What do you think?
* For point 3 - I believe I have nothing to do here.

I have gone on and created a new patch - FLUME-2905-5.patch for your review and 
I have also tested that.

I will try and add this to review board (haven't done that yet) soon.

Thanks once again.


was (Author: sahuja):
Hi [~sati], thanks a lot for your review.

Please find my answers for your points:

* For point 1. - "calling stop() after writing out the exception", I have moved 
stop() after logging the exception but just before it gets thrown.
* For point 2. - We should possibly have a dedicated JIRA for removing the 
"return" statement from the stop() method as this would be a different issue to 
what I am trying to fix in this JIRA which is to prevent socket leaks if a port 
is already bound. Also, it would make tracking easier with a new JIRA as 
otherwise any issues (if any) arising from this removal will be discussed in 
this JIRA which is a side-track from the original issue that is already 
potentially resolved. What do you think?
* For point 3 - I believe I have nothing to do here.

I have gone on and created a new patch - FLUME-2905-5.patch for your review.

I will try and add this to review board (haven't done that yet) soon.

Thanks once again.

> NetcatSource - Socket not closed when an exception is encountered during 
> start() leading to file descriptor leaks
> -
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: 1.6.0
>Reporter: Siddharth Ahuja
>Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch, 
> FLUME-2905-2.patch, FLUME-2905-3.patch, FLUME-2905-4.patch
>
>
> During the flume agent start-up, the flume configuration containing the 
> NetcatSource is parsed and the source's start() is called. If there is an 
> issue while binding the channel's socket to a local address to configure the 
> socket to listen for connections following exception is thrown but the socket 
> open just before is not closed. 
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: 
> Unable to start EventDrivenSourceRunner: { 
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } - 
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in 
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at 
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at 
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at 
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at

[jira] [Commented] (FLUME-2905) NetcatSource - Socket not closed when an exception is encountered during start() leading to file descriptor leaks

2017-06-21 Thread Attila Simon (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058287#comment-16058287
 ] 

Attila Simon commented on FLUME-2905:
-

Hi [~sahuja],
Junit passed for me but hasn't run exhaustively yet. I checked the 
FLUME-2905-4.patch and would have some minor suggestions. Would you mind 
uploading to https://reviews.apache.org/ for a code review? Please let me know 
if you have trouble with it (I'm also happy to upload it there for you). 

In a nutshell:
* I would recommend calling stop() after writing out the exception. 
* I would recommend removing the return statement from that part of the stop 
function which catches an Exception from Socket close. That currently prevents 
the rest of the stop() to be executed including super.stop() which then can't 
set the Lifecycle state.
* Test currently checks that stop was executed properly expecting the source to 
be in LifecycleState.STOP state. I think that is fine since it is internal to 
the NetcatSource that it opens a ServerSocket. What we have to check whether 
the Source is stopped on error. And this is indeed checked by the test. 

As a side note: It seems like that startSource() doesn't behave as expected 
(should pick a new port on error). This is a separate issue so I guess it 
should be fixed with a separate jira/PR.

> NetcatSource - Socket not closed when an exception is encountered during 
> start() leading to file descriptor leaks
> -
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: 1.6.0
>Reporter: Siddharth Ahuja
>Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch, 
> FLUME-2905-2.patch, FLUME-2905-3.patch, FLUME-2905-4.patch
>
>
> During the flume agent start-up, the flume configuration containing the 
> NetcatSource is parsed and the source's start() is called. If there is an 
> issue while binding the channel's socket to a local address to configure the 
> socket to listen for connections following exception is thrown but the socket 
> open just before is not closed. 
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: 
> Unable to start EventDrivenSourceRunner: { 
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } - 
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in 
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at 
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at 
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at 
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:167)
> ... 9 more
> {code}
> The source's start() is then called again leading to another socket being 
> opened but not closed and so on. This leads to file descriptor (socket) leaks.
> This can be easily reproduced as follows:
> 1. Set Netcat as the source in flume agent configuration.
> 2. Set the bind port for the netcat source to a port which is already in use. 
> e.g. in my case I used 50010 which is the port for DataNode's XCeiver 
> Protocol in use by the HDFS service.
> 3. Start flume agent and perform "lsof -p  | wc -l". Notice 
> the file descriptors keep on growing due to socket leaks with errors like: 
> "can't identify protocol".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLUME-2579) JMS source support durable subscriptions and message listening

2017-06-21 Thread Denes Arvay (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057669#comment-16057669
 ] 

Denes Arvay commented on FLUME-2579:


Thanks for the patch [~andrasbeni], thanks for the review [~sati]

> JMS source support durable subscriptions and message listening
> --
>
> Key: FLUME-2579
> URL: https://issues.apache.org/jira/browse/FLUME-2579
> Project: Flume
>  Issue Type: Wish
>  Components: Sinks+Sources
>Affects Versions: 1.5.2
>Reporter: zhangshengbing
>Assignee: Andras Beni
> Fix For: 1.8.0
>
>
> best wishes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLUME-2945) Bump java target version to 1.8

2017-06-21 Thread Denes Arvay (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057675#comment-16057675
 ] 

Denes Arvay commented on FLUME-2945:


Thanks for the patch [~liorze], thank you for the review [~sati]

> Bump java target version to 1.8
> ---
>
> Key: FLUME-2945
> URL: https://issues.apache.org/jira/browse/FLUME-2945
> Project: Flume
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Lior Zeno
>Assignee: Lior Zeno
> Fix For: 1.8.0
>
> Attachments: FLUME-2945-0.patch
>
>
> We should move to Java 8 as a minimum requirement.
> # Java 7 is EOL'ed http://www.oracle.com/technetwork/java/eol-135779.html.
> # Many projects are adopting java 8 as a minimum requirement, for instance:
> * Solr 6: https://issues.apache.org/jira/browse/LUCENE-6722
> * Hbase 2: https://issues.apache.org/jira/browse/HBASE-15624
> * elasticsearch 5: 
> https://www.elastic.co/guide/en/elasticsearch/reference/master/setup.html
> The attached patch was tested on Ubuntu 16.04 with openjdk version "1.8.0_91".
> All unit tests, except for the known flaky ones, successfully passed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLUME-3072) Add IP address to headers in flume log4j appender

2017-06-21 Thread Denes Arvay (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057671#comment-16057671
 ] 

Denes Arvay commented on FLUME-3072:


Thanks for the patch [~andrasbeni], thanks for the review [~sati]

> Add IP address to headers in flume log4j appender
> -
>
> Key: FLUME-3072
> URL: https://issues.apache.org/jira/browse/FLUME-3072
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
>Reporter: Andras Beni
>Assignee: Andras Beni
> Fix For: 1.8.0
>
>
>  Add client's host address to log event headers in Flume NG Log4j Appender. 
> By receiving such events through AvroSource, one can route events based on 
> origin host in the receiving Flume agent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLUME-2579) JMS source support durable subscriptions and message listening

2017-06-21 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057625#comment-16057625
 ] 

Hudson commented on FLUME-2579:
---

FAILURE: Integrated in Jenkins build Flume-trunk-hbase-1 #251 (See 
[https://builds.apache.org/job/Flume-trunk-hbase-1/251/])
FLUME-2579 Support durable subscription in JMSSource (denes: 
[http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git=commit=857df3fe9b211a0e34f17cc6d776a1296951134a])
* (edit) 
flume-ng-sources/flume-jms-source/src/test/java/org/apache/flume/source/jms/JMSMessageConsumerTestBase.java
* (delete) 
flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/JMSMessageConsumerFactory.java
* (edit) 
flume-ng-sources/flume-jms-source/src/test/java/org/apache/flume/source/jms/TestIntegrationActiveMQ.java
* (edit) 
flume-ng-sources/flume-jms-source/src/test/java/org/apache/flume/source/jms/TestJMSMessageConsumer.java
* (edit) 
flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/JMSMessageConsumer.java
* (edit) 
flume-ng-sources/flume-jms-source/src/test/java/org/apache/flume/source/jms/TestJMSSource.java
* (edit) flume-ng-doc/sphinx/FlumeUserGuide.rst
* (edit) 
flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/JMSSource.java
* (edit) 
flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/JMSSourceConfiguration.java


> JMS source support durable subscriptions and message listening
> --
>
> Key: FLUME-2579
> URL: https://issues.apache.org/jira/browse/FLUME-2579
> Project: Flume
>  Issue Type: Wish
>  Components: Sinks+Sources
>Affects Versions: 1.5.2
>Reporter: zhangshengbing
>Assignee: Andras Beni
> Fix For: 1.8.0
>
>
> best wishes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLUME-3072) Add IP address to headers in flume log4j appender

2017-06-21 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057624#comment-16057624
 ] 

Hudson commented on FLUME-3072:
---

FAILURE: Integrated in Jenkins build Flume-trunk-hbase-1 #251 (See 
[https://builds.apache.org/job/Flume-trunk-hbase-1/251/])
FLUME-3072. Add IP address to headers in Flume Log4j appender (denes: 
[http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git=commit=f2996cca55bbb852dc10512ca3369ae7fd56c44f])
* (edit) 
flume-ng-clients/flume-ng-log4jappender/src/test/java/org/apache/flume/clients/log4jappender/TestLoadBalancingLog4jAppender.java
* (edit) 
flume-ng-clients/flume-ng-log4jappender/src/main/java/org/apache/flume/clients/log4jappender/LoadBalancingLog4jAppender.java
* (edit) 
flume-ng-clients/flume-ng-log4jappender/src/main/java/org/apache/flume/clients/log4jappender/Log4jAppender.java
* (edit) 
flume-ng-clients/flume-ng-log4jappender/src/main/java/org/apache/flume/clients/log4jappender/Log4jAvroHeaders.java
* (edit) 
flume-ng-clients/flume-ng-log4jappender/src/test/java/org/apache/flume/clients/log4jappender/TestLog4jAppender.java


> Add IP address to headers in flume log4j appender
> -
>
> Key: FLUME-3072
> URL: https://issues.apache.org/jira/browse/FLUME-3072
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
>Reporter: Andras Beni
>Assignee: Andras Beni
> Fix For: 1.8.0
>
>
>  Add client's host address to log event headers in Flume NG Log4j Appender. 
> By receiving such events through AvroSource, one can route events based on 
> origin host in the receiving Flume agent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: Flume-trunk-hbase-1 #251

2017-06-21 Thread Apache Jenkins Server 
See 


Changes:

[denes] FLUME-3072. Add IP address to headers in Flume Log4j appender

[denes] FLUME-2579 Support durable subscription in JMSSource

--
[...truncated 261.54 KB...]
[INFO] Exclude: **/*.diff
[INFO] Exclude: **/*.patch
[INFO] Exclude: **/*.avsc
[INFO] Exclude: **/*.avro
[INFO] Exclude: **/docs/**
[INFO] Exclude: **/test/resources/**
[INFO] Exclude: **/.settings/*
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/target/**
[INFO] Exclude: **/derby.log
[INFO] Exclude: **/metastore_db/
[INFO] 1 resources included (use -debug for more details)
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
approved: 1 licence.
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (verify) @ 
flume-ng-legacy-sources ---
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
flume-ng-legacy-sources ---
[INFO] Installing 

 to 

[INFO] 
[INFO] 
[INFO] Skipping Flume legacy Avro source
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Flume legacy Thrift Source
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Building Flume NG Clients 1.8.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ flume-ng-clients ---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) 
@ flume-ng-clients ---
[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
flume-ng-clients ---
[INFO] 
[INFO] --- apache-rat-plugin:0.11:check (verify.rat) @ flume-ng-clients ---
[INFO] 52 implicit excludes (use -debug for more details).
[INFO] Exclude: **/.idea/
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/nb-configuration.xml
[INFO] Exclude: .git/
[INFO] Exclude: patchprocess/
[INFO] Exclude: .gitignore
[INFO] Exclude: .repository/
[INFO] Exclude: **/*.diff
[INFO] Exclude: **/*.patch
[INFO] Exclude: **/*.avsc
[INFO] Exclude: **/*.avro
[INFO] Exclude: **/docs/**
[INFO] Exclude: **/test/resources/**
[INFO] Exclude: **/.settings/*
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/target/**
[INFO] Exclude: **/derby.log
[INFO] Exclude: **/metastore_db/
[INFO] 1 resources included (use -debug for more details)
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
approved: 1 licence.
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (verify) @ flume-ng-clients ---
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
flume-ng-clients ---
[INFO] Installing 
 
to 

[INFO] 
[INFO] 
[INFO] Skipping Flume NG Log4j Appender
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Flume NG Tools
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Flume NG distribution
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO]

[jira] [Resolved] (FLUME-2579) JMS source support durable subscriptions and message listening

2017-06-21 Thread Denes Arvay (JIRA) 

 [ 
https://issues.apache.org/jira/browse/FLUME-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denes Arvay resolved FLUME-2579.

Resolution: Fixed

> JMS source support durable subscriptions and message listening
> --
>
> Key: FLUME-2579
> URL: https://issues.apache.org/jira/browse/FLUME-2579
> Project: Flume
>  Issue Type: Wish
>  Components: Sinks+Sources
>Affects Versions: 1.5.2
>Reporter: zhangshengbing
>Assignee: Andras Beni
> Fix For: 1.8.0
>
>
> best wishes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLUME-2579) JMS source support durable subscriptions and message listening

2017-06-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057582#comment-16057582
 ] 

ASF GitHub Bot commented on FLUME-2579:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flume/pull/120


> JMS source support durable subscriptions and message listening
> --
>
> Key: FLUME-2579
> URL: https://issues.apache.org/jira/browse/FLUME-2579
> Project: Flume
>  Issue Type: Wish
>  Components: Sinks+Sources
>Affects Versions: 1.5.2
>Reporter: zhangshengbing
>Assignee: Andras Beni
> Fix For: 1.8.0
>
>
> best wishes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flume pull request #120: FLUME-2579 Support durable subscription in JMSSourc...

2017-06-21 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/flume/pull/120


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flume pull request #121: FLUME-3072 Add IP address to headers in flume log4j...

2017-06-21 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/flume/pull/121


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLUME-3072) Add IP address to headers in flume log4j appender

2017-06-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057568#comment-16057568
 ] 

ASF subversion and git services commented on FLUME-3072:


Commit f2996cca55bbb852dc10512ca3369ae7fd56c44f in flume's branch 
refs/heads/trunk from [~andrasbeni]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=f2996cc ]

FLUME-3072. Add IP address to headers in Flume Log4j appender

Log4jAppender and LoadBalancingLog4jAppender resolve local hosts address at 
startup and
add it to each event's header with the key "flume.client.log4j.address".

This closes #121.

(Andras Beni via Denes Arvay)


> Add IP address to headers in flume log4j appender
> -
>
> Key: FLUME-3072
> URL: https://issues.apache.org/jira/browse/FLUME-3072
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources
>Affects Versions: 1.8.0
>Reporter: Andras Beni
>Assignee: Andras Beni
>
>  Add client's host address to log event headers in Flume NG Log4j Appender. 
> By receiving such events through AvroSource, one can route events based on 
> origin host in the receiving Flume agent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLUME-3099) TestSpoolingFileLineReader failing

2017-06-21 Thread salamani (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057226#comment-16057226
 ] 

salamani commented on FLUME-3099:
-

any updates?

> TestSpoolingFileLineReader failing
> --
>
> Key: FLUME-3099
> URL: https://issues.apache.org/jira/browse/FLUME-3099
> Project: Flume
>  Issue Type: Bug
>  Components: Test
>Affects Versions: 1.7.0
> Environment: s390x/ub16.10
> openjdk1.8
>Reporter: salamani
> Attachments: TestSpoolingFileLineReader_tests.out
>
>
> Failed tests:   
> testDestinationExistsAndSameFileWindows(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Platform not recognized
> Tests in error:
>   
> testDestinationExistsAndSameFileNotOnWindows(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Unexpected exception, expected but 
> was
>   
> testBasicCommitFailure(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testBasicCommitFailureAndBufferSizeChanges(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testDestinationExistsAndDifferentFile(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Unexpected exception, expected but 
> was
>   
> testBehaviorWithEmptyFile(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testBatchedReadsWithinAFile(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testBatchedReadsAcrossFileBoundary(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testEmptyDirectoryAfterCommittingFile(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testLineExceedsMaxLineLength(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testNameCorrespondsToLatestRead(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   testBasicSpooling(org.apache.flume.client.avro.TestSpoolingFileLineReader): 
> Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testInitiallyEmptyDirectory(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
>   
> testFileChangesDuringRead(org.apache.flume.client.avro.TestSpoolingFileLineReader):
>  Unexpected exception, expected but 
> was
> Tests run: 14, Failures: 1, Errors: 13, Skipped: 0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)