Hi,
I have been trying unsuccessfully to have a karaf distribution connect
to elastic search via flume using the FlumeLog4jAppender.
I followed the steps listed here
http://blog.nanthrax.net/2012/12/create-custom-log4j-appender-for-karaf-and-pax-logging/
for adding a custom appender and applied them to flume's source log4j
appender module and the karaf logging service configuration files. From
karaf console I see that the flume-log4jappender module appears as
resolved and see that the logging-service is in active state.
In the logs I see the following that doesn't give much information..
being new to Karaf not sure what to make out of those messages.
---
| 20 - org.apache.aries.blueprint.core - 1.4.0 | No blueprint
application found in bundle org.ops4j.pax.logging.pax-logging-service
2014-10-20 11:37:11,305 | DEBUG | Event Dispatcher |
BlueprintExtender | 20 - org.apache.aries.blueprint.core
- 1.4.0 | Starting BlueprintContainer destruction process for bundle
org.apache.flume.log4jappender
2014-10-20 11:37:11,305 | DEBUG | Event Dispatcher |
BlueprintExtender | 20 - org.apache.aries.blueprint.core
- 1.4.0 | Not a blueprint bundle or destruction of BlueprintContainer
already finished for org.apache.flume.log4jappender.
----
Please see my detailed configuration below.
thx
-Syed
Here is how my modified flume log4j appender module pom.xml looks
-----
1.
<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.company.one</groupId>
<artifactId>commons.parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
<relativePath>../../commons/parent</relativePath>
</parent>
<groupId>org.apache.flume.flume-ng-clients</groupId>
<artifactId>flume-ng-log4jappender</artifactId>
<name>Flume NG Log4j Appender</name>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.3.5</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>bundle</goal>
</goals>
</execution>
</executions>
<extensions>true</extensions>
<configuration>
<instructions>
<Export-Package>
org.apache.flume.clients.log4jappender
</Export-Package>
<Import-Package></Import-Package>
<Fragment-Host>org.ops4j.pax.logging.pax-logging-service</Fragment-Host>
<Bundle-SymbolicName>org.apache.flume.log4jappender</Bundle-SymbolicName>
<_failok>true</_failok>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>apache-log4j-extras</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-ng-sdk</artifactId>
<version>1.6.0-SNAPSHOT</version>
</dependency>
<!-- pull in flume core only for unit tests. TODO: not ideal -->
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-ng-core</artifactId>
<version>1.6.0-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-ng-configuration</artifactId>
<version>1.6.0-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.ops4j.pax.logging</groupId>
<artifactId>pax-logging-service</artifactId>
<version>1.7.2</version>
</dependency>
</dependencies>
</project>
------
2. I have placed the flume-ng-log4jappender-1.0.0-SNAPSHOT.jar in karaf
sytem's folder and the path looks like this
system/org/apache/flume/flume-ng-clients/flume-ng-log4jappender/1.0.0-SNAPSHOT/
3. I have modified the /etc/startup.properties as
mvn\:org.ops4j.pax.url/pax-url-wrap/1.6.0 = 5
*mvn\:org.ops4j.pax.logging/pax-logging-api/1.7.2 = 8**
**mvn\:org.ops4j.pax.logging/pax-logging-service/1.7.2 = 8**
**mvn\:org.apache.flume.flume-ng-clients/flume-ng-log4jappender/1.0.0-SNAPSHOT
= 8*
mvn\:org.apache.karaf.service/org.apache.karaf.service.guard/3.0.1 = 10
mvn\:org.apache.felix/org.apache.felix.configadmin/1.6.0 = 10
mvn\:org.apache.felix/org.apache.felix.fileinstall/3.2.8 = 11
4. I have modified the etc/org.ops4j.pax.logging.cfg to include the
flume appender as follows
# Root logger
log4j.rootLogger = DEBUG, out, *flume,* osgi:*
# flume appender
log4j.appender.flume=org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Port=7070
log4j.appender.flume.Hostname=<ip of the elastic search/flume>
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
