[
https://issues.apache.org/jira/browse/FLUME-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491201#comment-13491201
]
Ralph Goers commented on FLUME-1502:
------------------------------------
Arvind, it would be hard for me to disagree with your point of view since it
matches exactly with how the embedded agent is currently implemented in Log4j
2. The current incarnation allows any channels or sinks, but all channels are
automatically connected to the source (which is really the appender itself).
As for simplicity, my intention with Log4j 2 is to have the configuration end
up looking like
<Flume name="eventLogger" suppressExceptions="false" compress="true"
embedded="true" dataDir="${sys:flumeDir}">
<Agent host="192.168.10.101" port="8800"/>
<Agent host="192.168.10.102" port="8800"/>
<RFC5424Layout enterpriseNumber="18060" includeMDC="true" appName="MyApp"/>
</Flume>
and then maybe something to configure encryption. Although Log4j 2 will
support configuration by Flume properties, my guess is that most people would
prefer the default configuration. If they don't want the FileChannel it would
be fairly simple to add channel="Memory|File" as an attribute.
If you look at the embedded Appender you will see that it creates the
FlumeEvent (using code common to both the embedded and non-embedded versions)
and then does
public void send(FlumeEvent event) {
sourceCounter.incrementAppendReceivedCount();
sourceCounter.incrementEventReceivedCount();
try {
getChannelProcessor().processEvent(event);
} catch (ChannelException ex) {
logger.warn("Unabled to process event {}" + event, ex);
throw ex;
}
sourceCounter.incrementAppendAcceptedCount();
sourceCounter.incrementEventAcceptedCount();
}
in a class that extends AbstractSource. The only real challenge I had was in
getting it into the Flume configuration so Flume could create the Source object
and then obtaining a reference to the object so the Appender could call it. To
do that I had to do
SourceRunner runner =
node.getConfiguration().getSourceRunners().get(SOURCE_NAME);
if (runner == null || runner.getSource() == null) {
throw new IllegalStateException("No Source has been created for
Appender " + shortName);
}
source = (Log4jEventSource) runner.getSource();
It would be much better if I could pass the Source class to the configuration
processor and get back a Source instance when the embedded agent is started.
In this csae I would recommend the source has to implement something like
public interface EmbeddedSource {
void send(FlumeEvent event);
}
OTOH, you could just provide EmbeddedSource that implements send() as shown
above.
> Support for running simple configurations embedded in host process
> ------------------------------------------------------------------
>
> Key: FLUME-1502
> URL: https://issues.apache.org/jira/browse/FLUME-1502
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.2.0
> Reporter: Arvind Prabhakar
> Assignee: Brock Noland
> Attachments: embeeded-agent-1.pdf
>
>
> Flume should provide a light-weight embeddable node manager that can be
> started in process where necessary. This will allow the users to embed
> light-weight agents within the host process where necessary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
