[
https://issues.apache.org/jira/browse/FLUME-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tinawenqiao updated FLUME-2961:
-------------------------------
Description:
TaidirSource defaults to LINE, this has issue when multiline log events like
stack traces and have request/responses. Following part is Java traceback logs.
We expect to have log line start regex Key to aggregate all the log lines till
the next regex key is found.
2016-07-16 14:59:43,956 ERROR lifecycleSupervisor-1-7 LifecycleSupervisor.run -
Unable to start EventDrivenSourceRunner: {
source:cn.yottabyte.flume.source.http.HTTPSource{name:sourceHttp,state:IDLE} }
- Exception follows.
java.lang.IllegalStateException: Running HTTP Server found in source:
sourceHttp before I started one. Will not attempt to start.
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at
cn.yottabyte.flume.source.http.HTTPSource.startHttpSourceServer(HTTPSource.java:170)
at cn.yottabyte.flume.source.http.HTTPSource.start(HTTPSource.java:166)
at
org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
was:
1. TaidirSource defaults to LINE, this has issue when multiline log events like
stack traces and have request/responses.
2. Proposal to have log line start regex Key to aggregate all the log lines
till the next regex key is found.
> Make TaildirSource work with multiline
> --------------------------------------
>
> Key: FLUME-2961
> URL: https://issues.apache.org/jira/browse/FLUME-2961
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.7.0
> Reporter: tinawenqiao
> Assignee: tinawenqiao
> Fix For: v1.7.0
>
> Attachments: FLUME-2961_1.patch
>
>
> TaidirSource defaults to LINE, this has issue when multiline log events like
> stack traces and have request/responses. Following part is Java traceback
> logs. We expect to have log line start regex Key to aggregate all the log
> lines till the next regex key is found.
> 2016-07-16 14:59:43,956 ERROR lifecycleSupervisor-1-7 LifecycleSupervisor.run
> - Unable to start EventDrivenSourceRunner: {
> source:cn.yottabyte.flume.source.http.HTTPSource{name:sourceHttp,state:IDLE}
> } - Exception follows.
> java.lang.IllegalStateException: Running HTTP Server found in source:
> sourceHttp before I started one. Will not attempt to start.
> at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at
> cn.yottabyte.flume.source.http.HTTPSource.startHttpSourceServer(HTTPSource.java:170)
> at cn.yottabyte.flume.source.http.HTTPSource.start(HTTPSource.java:166)
> at
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
