Hello everyone,
I am facing an issue with Flume 1.6 using the log4j flume appender and I am
looking for some help.
The picture is:
[CLIENT JVM : LOG4J -> FLUME-LOG4J-APPENDER ]---> [FLUME JVM : AVRO-SRC -> MEM
CHANNEL -> LOGGER SINK]
After all the logs are sent by the client's log4j appender - and received and
processed correctly by the flume agent (using avro source) - the client JVM
(which is a simple test) hangs forever and never exits.
When the client is recompiled and executed with the jars from Flume 1.5, it
executes and terminates correctly...
An extract of the client execution with jdb seems to indicate that, with Flume
1.6 jars, after the main thread terminates, some threads related to Avro /
NettyTransceiver remain alive hung in some deadlock.
Has anyone hit this kind of issue ?
I am copying below the jdb session output, my client's java code and
log4j.properties and flume.conf files.
Thanks in advance for any help you might provide !
Hedi
JDB SESSION:
---
$ jdb Main
Initializing jdb ...
> run
run Main
Set uncaught java.lang.Throwable
Set deferred uncaught java.lang.Throwable
>
VM Started: Using default maxIOWorkers
Start of main
This is a 1st log message
This is a 2nd log message
This is the 3rd and last message
End of main
suspend
All threads suspended.
> threads
Group system:
(java.lang.ref.Reference$ReferenceHandler)0x128 Reference Handler
cond. waiting
(java.lang.ref.Finalizer$FinalizerThread)0x127 Finalizer
cond. waiting
(java.lang.Thread)0x126 Signal Dispatcher
running
Group main:
(java.lang.Thread)0x293 New I/O worker #1
running
(java.lang.Thread)0x29b New I/O worker #2
running
(java.lang.Thread)0x29c New I/O worker #3
running
(java.lang.Thread)0x29d New I/O worker #4
running
(java.lang.Thread)0x29e New I/O worker #5
running
(java.lang.Thread)0x29f New I/O worker #6
running
(java.lang.Thread)0x2a0 New I/O worker #7
running
(java.lang.Thread)0x2a1 New I/O worker #8
running
(java.lang.Thread)0x31f Avro NettyTransceiver Boss 1
cond. waiting
(java.lang.Thread)0x320 Hashed wheel timer #1
sleeping
(java.lang.Thread)0x4bd Flume Avro RPC Client Call
Invoker 1 cond. waiting
(java.lang.Thread)0x535 Flume Avro RPC Client Call
Invoker 2 cond. waiting
(java.lang.Thread)0x536 DestroyJavaVM
running
---
Client source code and log4j.properties:
---
import org.apache.log4j.Logger;
class Main {
final static Logger logger = Logger.getLogger(Main.class);
public static void main(String[] args) {
System.out.println("Start of main");
logger.warn("This is a 1st log message");
logger.warn("This is a 2nd log message");
logger.warn("This is the 3rd and last log message");
System.out.println("End of main");
}
}
---
log4j.rootLogger=INFO, stdout, flume
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%m%n
log4j.appender.flume=org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Port=5554
log4j.appender.flume.Hostname=localhost
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
log4j.appender.flume.layout.ConversionPattern=%m/%n
---
Flume.conf file:
---
agent.sources = avroSource
agent.channels = logChannel
agent.sinks = loggerSink
agent.sources.avroSource.type = avro
agent.sources.avroSource.bind = 0.0.0.0
agent.sources.avroSource.port = 5554
agent.sources.avroSource.channels = logChannel
agent.sinks.loggerSink.type = logger
agent.sinks.loggerSink.channel = logChannel
agent.channels.logChannel.type = memory
agent.channels.logChannel.capacity = 100
---
END
