This is an automated email from the ASF dual-hosted git repository.
rgoers pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/logging-flume.git
The following commit(s) were added to refs/heads/trunk by this push:
new 1ff090a31 Drop Mina and Netty from flume-ng-core (#448)
1ff090a31 is described below
commit 1ff090a313f61454751e92e9df3fd67954f39a6b
Author: Piotr P. Karwasz <[email protected]>
AuthorDate: Fri Jun 12 15:08:22 2026 +0200
Drop Mina and Netty from flume-ng-core (#448)
* Drop Mina and Netty from flume-ng-core
Move the sources that pull in Apache Mina and Netty out of
flume-ng-core into two new modules under flume-ng-sources:
- flume-syslog-source (org.apache.flume.source.syslog):
SyslogTcpSource, SyslogUDPSource, MultiportSyslogTCPSource and
their shared parser/utils
- flume-netcat-source (org.apache.flume.source.netcat):
NetcatUdpSource
The classes are repackaged so each module owns a distinct package,
keeping the modules JPMS-friendly (no split packages). The short
source aliases in SourceType keep working; only the target FQCNs
change. flume-ng-core no longer depends on io.netty:netty-all or
org.apache.mina:mina-core.
The TLS unit tests no longer rely on checked-in keystore/PEM blobs:
the X509Certificates helper moves to the flume-ng-sdk test-jar
alongside a new TestKeyStores helper, and the stores are generated
in memory with Bouncy Castle. The orphaned test resources are
deleted.
Also remove the dead Netty PatternRule class (no references).
Assisted-By: Claude Opus 4.8 <[email protected]>
* Open java.base packages for flume-ng-sdk SSLUtil tests
AbstractSSLUtilTest injects environment variables by reflecting into
java.lang.ProcessEnvironment, which fails with an
InaccessibleObjectException on Java 16+ unless the relevant java.base
packages are opened. Add the required --add-opens to the flume-ng-sdk
surefire argLine. This is a pre-existing failure, unrelated to the
Mina/Netty removal.
Assisted-By: Claude Opus 4.8 <[email protected]>
* Add source-factory tests for the syslog and netcat modules
TestDefaultSourceFactory in flume-ng-core used to verify that the
short aliases (syslogtcp, multiport_syslogtcp, syslogudp, netcatudp)
resolve to their source classes. Those classes moved out of core, so
restore the coverage in the modules that now own them, following the
TestHTTPSourceFactory pattern.
Assisted-By: Claude Opus 4.8 <[email protected]>
* Adjust spotbugs violation ceilings after the source move
The syslog and netcat sources accounted for 11 spotbugs violations
(8 + 3) in flume-ng-core. Move those allowances to the new modules
and tighten flume-ng-core to its current counts, which also absorbs
the headroom left by earlier module extractions:
- flume-ng-core: spotbugs 199 -> 152, pmd 121 -> 64
- flume-syslog-source: spotbugs 8
- flume-netcat-source: spotbugs 3
The moved code has no pmd violations, so only spotbugs ceilings are
added to the new modules.
Assisted-By: Claude Opus 4.8 <[email protected]>
* Formatting
---
flume-bom/pom.xml | 10 ++
.../org/apache/flume/conf/source/SourceType.java | 8 +-
flume-ng-core/pom.xml | 14 +-
.../org/apache/flume/netty/filter/PatternRule.java | 160 ---------------------
.../flume/source/TestDefaultSourceFactory.java | 4 -
flume-ng-core/src/test/resources/certs/gencerts.sh | 31 ----
flume-ng-core/src/test/resources/certs/rootca.conf | 9 --
flume-ng-core/src/test/resources/certs/server.conf | 9 --
flume-ng-core/src/test/resources/keystorefile.jks | Bin 3518 -> 0 bytes
.../src/test/resources/server.flume-crt.pem | 17 ---
.../src/test/resources/server.flume-keystore.p12 | Bin 2447 -> 0 bytes
flume-ng-core/src/test/resources/server.flume.pem | 23 ---
flume-ng-core/src/test/resources/server.p12 | Bin 1637 -> 0 bytes
flume-ng-core/src/test/resources/truststore.jks | Bin 687 -> 0 bytes
.../src/test/resources/truststorefile.jks | Bin 775 -> 0 bytes
flume-ng-dist/pom.xml | 8 ++
flume-ng-sdk/pom.xml | 25 ++++
.../java/org/apache/flume/util/TestKeyStores.java | 77 ++++++++++
.../org/apache/flume/util}/X509Certificates.java | 8 +-
.../apache/flume/source/http/TestHTTPSource.java | 26 +---
flume-ng-sources/{ => flume-netcat-source}/pom.xml | 41 ++++--
.../flume/source/netcat}/NetcatUdpSource.java | 3 +-
.../flume/source/netcat}/TestNetcatUdpSource.java | 2 +-
.../source/netcat/TestNetcatUdpSourceFactory.java | 41 ++++++
.../flume-syslog-source}/pom.xml | 86 ++++++-----
.../source/syslog}/MultiportSyslogTCPSource.java | 3 +-
.../apache/flume/source/syslog}/SyslogParser.java | 2 +-
.../SyslogSourceConfigurationConstants.java | 2 +-
.../flume/source/syslog}/SyslogTcpSource.java | 3 +-
.../flume/source/syslog}/SyslogUDPSource.java | 3 +-
.../apache/flume/source/syslog}/SyslogUtils.java | 2 +-
.../syslog}/TestMultiportSyslogTCPSource.java | 18 ++-
.../flume/source/syslog}/TestSyslogParser.java | 2 +-
.../source/syslog/TestSyslogSourceFactory.java | 55 +++++++
.../flume/source/syslog}/TestSyslogTcpSource.java | 20 ++-
.../flume/source/syslog}/TestSyslogUdpSource.java | 2 +-
.../flume/source/syslog}/TestSyslogUtils.java | 2 +-
flume-ng-sources/pom.xml | 2 +
38 files changed, 357 insertions(+), 361 deletions(-)
diff --git a/flume-bom/pom.xml b/flume-bom/pom.xml
index 774b2cab9..97cf0ad7c 100644
--- a/flume-bom/pom.xml
+++ b/flume-bom/pom.xml
@@ -217,6 +217,16 @@
<artifactId>flume-taildir-source</artifactId>
<version>${flume-taildir.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.flume.flume-ng-sources</groupId>
+ <artifactId>flume-syslog-source</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flume.flume-ng-sources</groupId>
+ <artifactId>flume-netcat-source</artifactId>
+ <version>${project.version}</version>
+ </dependency>
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-shared-kafka</artifactId>
diff --git
a/flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
b/flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
index f26bb5946..5792d8dce 100644
---
a/flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
+++
b/flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
@@ -61,21 +61,21 @@ public enum SourceType implements ComponentWithClassName {
*
* @see org.apache.flume.source.SyslogTcpSource
*/
- SYSLOGTCP("org.apache.flume.source.SyslogTcpSource"),
+ SYSLOGTCP("org.apache.flume.source.syslog.SyslogTcpSource"),
/**
* MultiportSyslogTCPSource
*
* @see org.apache.flume.source.MultiportSyslogTCPSource
*/
- MULTIPORT_SYSLOGTCP("org.apache.flume.source.MultiportSyslogTCPSource"),
+
MULTIPORT_SYSLOGTCP("org.apache.flume.source.syslog.MultiportSyslogTCPSource"),
/**
* SyslogUDPSource
*
* @see org.apache.flume.source.SyslogUDPSource
*/
- SYSLOGUDP("org.apache.flume.source.SyslogUDPSource"),
+ SYSLOGUDP("org.apache.flume.source.syslog.SyslogUDPSource"),
/**
* Spool directory source
@@ -117,7 +117,7 @@ public enum SourceType implements ComponentWithClassName {
*
* @see org.apache.flume.source.NetcatUdpSource
*/
- NETCATUDP("org.apache.flume.source.NetcatUdpSource");
+ NETCATUDP("org.apache.flume.source.netcat.NetcatUdpSource");
private final String sourceClassName;
diff --git a/flume-ng-core/pom.xml b/flume-ng-core/pom.xml
index b33f5c71c..33d269607 100644
--- a/flume-ng-core/pom.xml
+++ b/flume-ng-core/pom.xml
@@ -32,8 +32,8 @@
<properties>
<!-- TODO fix spotbugs violations -->
- <spotbugs.maxAllowedViolations>199</spotbugs.maxAllowedViolations>
- <pmd.maxAllowedViolations>121</pmd.maxAllowedViolations>
+ <spotbugs.maxAllowedViolations>152</spotbugs.maxAllowedViolations>
+ <pmd.maxAllowedViolations>64</pmd.maxAllowedViolations>
<module.name>org.apache.flume.core</module.name>
</properties>
@@ -133,11 +133,6 @@
<artifactId>commons-lang</artifactId>
</dependency>
- <dependency>
- <groupId>io.netty</groupId>
- <artifactId>netty-all</artifactId>
- </dependency>
-
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
@@ -156,11 +151,6 @@
<scope>test</scope>
</dependency>
- <dependency>
- <groupId>org.apache.mina</groupId>
- <artifactId>mina-core</artifactId>
- </dependency>
-
</dependencies>
<build>
<plugins>
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/netty/filter/PatternRule.java
b/flume-ng-core/src/main/java/org/apache/flume/netty/filter/PatternRule.java
deleted file mode 100644
index 31cc2b67c..000000000
--- a/flume-ng-core/src/main/java/org/apache/flume/netty/filter/PatternRule.java
+++ /dev/null
@@ -1,160 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to you under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.flume.netty.filter;
-
-import io.netty.handler.ipfilter.IpFilterRule;
-import io.netty.handler.ipfilter.IpFilterRuleType;
-import java.net.InetAddress;
-import java.net.InetSocketAddress;
-import java.net.UnknownHostException;
-import java.util.regex.Pattern;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-/**
- * The Class PatternRule represents an IP filter rule using string patterns.
- * <br>
- * Rule Syntax:
- * <br>
- * <pre>
- * Rule ::= [n|i]:address n stands for computer name, i for ip address
- * address ::= <regex> | localhost
- * regex is a regular expression with '*' as multi character and '?' as single
character wild card
- * </pre>
- * <br>
- * Example: allow localhost:
- * <br>
- * new PatternRule(true, "n:localhost")
- * <br>
- * Example: allow local lan:
- * <br>
- * new PatternRule(true, "i:192.168.0.*")
- * <br>
- * Example: block all
- * <br>
- * new PatternRule(false, "n:*")
- * <br>
- * <p>
- * For some reason Netty 4 didn't copy this from Netty 3. The code was copied
from the Netty 3 PatternRule
- * and modifed as required to match the new version of IpFilterRule.
- */
-public class PatternRule implements IpFilterRule {
-
- private static final Logger logger =
LoggerFactory.getLogger(PatternRule.class);
- private static final String LOCALHOST = "127.0.0.1";
- private final IpFilterRuleType ruleType;
- private Pattern ipPattern;
- private Pattern namePattern;
- private boolean localhost;
-
- /**
- * Construct the IpFilterRule from a pattern.
- *
- * @param ruleType The RuleType (accept or deny)
- * @param pattern The pattern.
- */
- public PatternRule(IpFilterRuleType ruleType, String pattern) {
- this.ruleType = ruleType;
- parse(pattern);
- }
-
- private static String addRule(String pattern, String rule) {
- if (rule == null || rule.length() == 0) {
- return pattern;
- }
- if (pattern.length() != 0) {
- pattern += "|";
- }
- rule = rule.replaceAll("\\.", "\\\\.");
- rule = rule.replaceAll("\\*", ".*");
- rule = rule.replaceAll("\\?", ".");
- pattern += '(' + rule + ')';
- return pattern;
- }
-
- private static boolean isLocalhost(InetAddress address) {
- try {
- if (address.equals(InetAddress.getLocalHost())) {
- return true;
- }
- } catch (UnknownHostException e) {
- if (logger.isInfoEnabled()) {
- logger.info("error getting ip of localhost", e);
- }
- }
- try {
- InetAddress[] addrs = InetAddress.getAllByName(LOCALHOST);
- for (InetAddress addr : addrs) {
- if (addr.equals(address)) {
- return true;
- }
- }
- } catch (UnknownHostException e) {
- if (logger.isInfoEnabled()) {
- logger.info("error getting ip of localhost", e);
- }
- }
- return false;
- }
-
- @Override
- public boolean matches(InetSocketAddress inetSocketAddress) {
- InetAddress inetAddress = inetSocketAddress.getAddress();
- if (localhost && isLocalhost(inetAddress)) {
- return true;
- }
- if (ipPattern != null &&
ipPattern.matcher(inetAddress.getHostAddress()).matches()) {
- return true;
- }
- if (namePattern != null) {
- return namePattern.matcher(inetAddress.getHostName()).matches();
- }
- return false;
- }
-
- @Override
- public IpFilterRuleType ruleType() {
- return ruleType;
- }
-
- private void parse(String pattern) {
- if (pattern == null) {
- return;
- }
-
- String[] acls = pattern.split(",");
-
- String ip = "";
- String name = "";
- for (String c : acls) {
- c = c.trim();
- if ("n:localhost".equals(c)) {
- localhost = true;
- } else if (c.startsWith("n:")) {
- name = addRule(name, c.substring(2));
- } else if (c.startsWith("i:")) {
- ip = addRule(ip, c.substring(2));
- }
- }
- if (ip.length() != 0) {
- ipPattern = Pattern.compile(ip);
- }
- if (name.length() != 0) {
- namePattern = Pattern.compile(name);
- }
- }
-}
diff --git
a/flume-ng-core/src/test/java/org/apache/flume/source/TestDefaultSourceFactory.java
b/flume-ng-core/src/test/java/org/apache/flume/source/TestDefaultSourceFactory.java
index df445b6cc..08897f919 100644
---
a/flume-ng-core/src/test/java/org/apache/flume/source/TestDefaultSourceFactory.java
+++
b/flume-ng-core/src/test/java/org/apache/flume/source/TestDefaultSourceFactory.java
@@ -60,11 +60,7 @@ public class TestDefaultSourceFactory {
public void testSourceCreation() throws Exception {
verifySourceCreation("seq-src", "seq", SequenceGeneratorSource.class);
verifySourceCreation("netcat-src", "netcat", NetcatSource.class);
- verifySourceCreation("netcat-udp-src", "netcatudp",
NetcatUdpSource.class);
verifySourceCreation("exec-src", "exec", ExecSource.class);
- verifySourceCreation("syslogtcp-src", "syslogtcp",
SyslogTcpSource.class);
- verifySourceCreation("multiport_syslogtcp-src", "multiport_syslogtcp",
MultiportSyslogTCPSource.class);
- verifySourceCreation("syslogudp-src", "syslogudp",
SyslogUDPSource.class);
// verifySourceCreation("spooldir-src", "spooldir",
SpoolDirectorySource.class);
// verifySourceCreation("thrift-src", "thrift", ThriftSource.class);
verifySourceCreation("custom-src",
MockSource.class.getCanonicalName(), MockSource.class);
diff --git a/flume-ng-core/src/test/resources/certs/gencerts.sh
b/flume-ng-core/src/test/resources/certs/gencerts.sh
deleted file mode 100755
index 0fb0263de..000000000
--- a/flume-ng-core/src/test/resources/certs/gencerts.sh
+++ /dev/null
@@ -1,31 +0,0 @@
-mkdir tmp
-rm ../truststorefile.jks
-rm ../keystorefile.jks
-rm ../server.flume-keystore.p12
-# Create the CA key and certificate
-openssl req -config rootca.conf -new -x509 -nodes -keyout tmp/flume-cacert.key
-out tmp/flume-ca.crt -days 10960
-# Create the trust store and import the certificate
-keytool -keystore ../truststorefile.jks -storetype jks -importcert -file
'tmp/flume-ca.crt' -keypass password -storepass password -alias flume-cacert
-noprompt
-#Import the root certificate
-keytool -keystore ../keystorefile.jks -alias flume-ca -importcert -file
tmp/flume-ca.crt -keypass password -storepass password -noprompt
-# Create the client private key in the client key store
-keytool -genkeypair -keyalg RSA -alias client -keystore ../keystorefile.jks
-storepass password -keypass password -validity 10960 -keysize 2048 -dname
"CN=client.flume, C=US"
-# Create a signing request for the client #
-keytool -keystore ../keystorefile.jks -alias client -certreq -file
tmp/client.csr -keypass password -storepass password
-# Sign the client certificate
-openssl x509 -req -CA 'tmp/flume-ca.crt' -CAkey 'tmp/flume-cacert.key' -in
tmp/client.csr -out tmp/client.crt_signed -days 10960 -CAcreateserial -passin
pass:password
-# Verify the signed certificate
-openssl verify -CAfile 'tmp/flume-ca.crt' tmp/client.crt_signed
-#Import the client's signed certificate
-keytool -keystore ../keystorefile.jks -alias client -importcert -file
tmp/client.crt_signed -keypass password -storepass password -noprompt
-#Verify the keystore
-keytool -list -v -keystore ../keystorefile.jks -storepass password
-# Create the server private key in the server key store
-keytool -genkeypair -keyalg RSA -alias server -keystore
../server.flume-keystore.p12 -storepass password -storetype PKCS12 -keypass
password -validity 10960 -keysize 2048 -dname "CN=server.flume, C=US"
-# Create a signing request for the server #
-keytool -keystore ../server.flume-keystore.p12 -alias server -certreq -file
tmp/server.csr -keypass password -storepass password
-# Sign the server certificate
-openssl x509 -req -CA 'tmp/flume-ca.crt' -CAkey 'tmp/flume-cacert.key' -in
tmp/server.csr -out ../server.flume-crt.pem -days 10960 -CAcreateserial -passin
pass:password
-# Extract the private key
-openssl pkcs12 -in ../server.flume-keystore.p12 -passin pass:password -nokeys
-out ../server.flume.pem
-rm -rf tmp
diff --git a/flume-ng-core/src/test/resources/certs/rootca.conf
b/flume-ng-core/src/test/resources/certs/rootca.conf
deleted file mode 100644
index 722e9c39b..000000000
--- a/flume-ng-core/src/test/resources/certs/rootca.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-[ req ]
-distinguished_name = CA_DN
-prompt = no
-output_password = password
-default_bits = 2048
-
-[ CA_DN ]
-C = US
-CN = flume-ca
diff --git a/flume-ng-core/src/test/resources/certs/server.conf
b/flume-ng-core/src/test/resources/certs/server.conf
deleted file mode 100644
index 7a9fb5835..000000000
--- a/flume-ng-core/src/test/resources/certs/server.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-[ req ]
-distinguished_name = CA_DN
-prompt = no
-output_password = password
-default_bits = 2048
-
-[ CA_DN ]
-C = US
-CN = server.flume
diff --git a/flume-ng-core/src/test/resources/keystorefile.jks
b/flume-ng-core/src/test/resources/keystorefile.jks
deleted file mode 100644
index d62e95792..000000000
Binary files a/flume-ng-core/src/test/resources/keystorefile.jks and /dev/null
differ
diff --git a/flume-ng-core/src/test/resources/server.flume-crt.pem
b/flume-ng-core/src/test/resources/server.flume-crt.pem
deleted file mode 100644
index 9e3774427..000000000
--- a/flume-ng-core/src/test/resources/server.flume-crt.pem
+++ /dev/null
@@ -1,17 +0,0 @@
------BEGIN CERTIFICATE-----
-MIICwjCCAaoCCQD8AQyla5FbDjANBgkqhkiG9w0BAQsFADAgMQswCQYDVQQGEwJV
-UzERMA8GA1UEAwwIZmx1bWUtY2EwIBcNMjMwMzIyMDAyODM0WhgPMjA1MzAzMjQw
-MDI4MzRaMCQxCzAJBgNVBAYTAlVTMRUwEwYDVQQDEwxzZXJ2ZXIuZmx1bWUwggEi
-MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDEEAUShlX2lWDYiEgcZL1JzWFw
-auNkdxOGo1DtD3YaZdrU1GS2UwspLm0qcSFEF1Sx6uVTABdgjZGnxJLSTJkabVu/
-iMuP+EQrE/4AWJoyuuaYMSG0EGeP+mttTtLHYyt/k9NJkABYxFZkNqdDuo+lF/Vs
-QdW3icym0vqMwiIvE+VKw9j3F+zFZizfx6MiBH0uuNrXHFCNUg52/cbeyiXO1mks
-yruOV+PF8/44zAepjLWiJgp7Wo6ejXmLvR+k68RwB5V7fXrzPYueM9GQDmLffkdO
-ZhrcafiRFGlzWSmC820Eb2b5+cVnm96XAlUXE5ao5o58oMmcufMnJ5k2UbFJAgMB
-AAEwDQYJKoZIhvcNAQELBQADggEBAHBGpqraT39aw/HVrJdmpsw8CwSmdiir+NYk
-5PprbIKAyf/P/9ObKcqesO8d8CQZVvzzm+Ok2rgcALDIl/TbbAVUPizIrN4AiH+Z
-BPOqDFF4taWkw73iMDiq61QS8SpJIOxxmL8PsK5eefuABrpumnVgW5X9BT/uMIqW
-NOwiyII3NVvtlErcdAL/ZTYWc3S8CEWVRc88ZpIBSLB4/tqQbPM+m+ZtYMSKi3Sh
-ugOjonPuteeQqu6R7HRYOajepKdGe048Moq90v0IrDI8v+rbezLFpEWOnG0fUDEq
-LfA9l7e/q1ukXRW4ccJWZWXLrZbEbX5hJeTlyYhHciwB5jfueMM=
------END CERTIFICATE-----
diff --git a/flume-ng-core/src/test/resources/server.flume-keystore.p12
b/flume-ng-core/src/test/resources/server.flume-keystore.p12
deleted file mode 100644
index da51f355d..000000000
Binary files a/flume-ng-core/src/test/resources/server.flume-keystore.p12 and
/dev/null differ
diff --git a/flume-ng-core/src/test/resources/server.flume.pem
b/flume-ng-core/src/test/resources/server.flume.pem
deleted file mode 100644
index 6bc71ed79..000000000
--- a/flume-ng-core/src/test/resources/server.flume.pem
+++ /dev/null
@@ -1,23 +0,0 @@
-Bag Attributes
- friendlyName: server
- localKeyID: 54 69 6D 65 20 31 36 37 39 34 34 34 39 31 33 33 33 39
-subject=/C=US/CN=server.flume
-issuer=/C=US/CN=server.flume
------BEGIN CERTIFICATE-----
-MIIC6TCCAdGgAwIBAgIEMlDgqzANBgkqhkiG9w0BAQsFADAkMQswCQYDVQQGEwJV
-UzEVMBMGA1UEAxMMc2VydmVyLmZsdW1lMCAXDTIzMDMyMjAwMjgzM1oYDzIwNTMw
-MzI0MDAyODMzWjAkMQswCQYDVQQGEwJVUzEVMBMGA1UEAxMMc2VydmVyLmZsdW1l
-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxBAFEoZV9pVg2IhIHGS9
-Sc1hcGrjZHcThqNQ7Q92GmXa1NRktlMLKS5tKnEhRBdUserlUwAXYI2Rp8SS0kyZ
-Gm1bv4jLj/hEKxP+AFiaMrrmmDEhtBBnj/prbU7Sx2Mrf5PTSZAAWMRWZDanQ7qP
-pRf1bEHVt4nMptL6jMIiLxPlSsPY9xfsxWYs38ejIgR9Lrja1xxQjVIOdv3G3sol
-ztZpLMq7jlfjxfP+OMwHqYy1oiYKe1qOno15i70fpOvEcAeVe3168z2LnjPRkA5i
-335HTmYa3Gn4kRRpc1kpgvNtBG9m+fnFZ5velwJVFxOWqOaOfKDJnLnzJyeZNlGx
-SQIDAQABoyEwHzAdBgNVHQ4EFgQUClD6FZ+qPIFrtBaz0swRTtWt1WEwDQYJKoZI
-hvcNAQELBQADggEBALuX9E+tRWvvA9uULj9Iq+k9iUNMQzkmyzXGu7hY46ZU9lx+
-fNnLZq82zz9rHq8IhK4HsLIsPCLfNeXwG/TNR4zUHKI53lzkburxgu76soMUDbHX
-8udyUgrs0YjQcppw6IOOmlZtgNeF2nu7jeoXrCaA07yXzehAqukHv7glWaV3oORc
-rDkZvHfJ2G7hPbUYYIeouJsbG9rNukNPOY9JEYGFYzDxZ8hlFt7Lp/icbdpjFGDV
-tkMtPpVz59B47j/Kk/k5zxaDLnD42svL8GByyM5UxvAqAlYnfMKiZqXfY0JbCpMC
-e9Z5xOyt9F8NLFyjRsmBlJD61LuLb8hAZK/Ho70=
------END CERTIFICATE-----
diff --git a/flume-ng-core/src/test/resources/server.p12
b/flume-ng-core/src/test/resources/server.p12
deleted file mode 100644
index 07eef5168..000000000
Binary files a/flume-ng-core/src/test/resources/server.p12 and /dev/null differ
diff --git a/flume-ng-core/src/test/resources/truststore.jks
b/flume-ng-core/src/test/resources/truststore.jks
deleted file mode 100644
index 8c08349e0..000000000
Binary files a/flume-ng-core/src/test/resources/truststore.jks and /dev/null
differ
diff --git a/flume-ng-core/src/test/resources/truststorefile.jks
b/flume-ng-core/src/test/resources/truststorefile.jks
deleted file mode 100644
index a0c3a49ac..000000000
Binary files a/flume-ng-core/src/test/resources/truststorefile.jks and
/dev/null differ
diff --git a/flume-ng-dist/pom.xml b/flume-ng-dist/pom.xml
index 7bfe27428..0f328cb81 100644
--- a/flume-ng-dist/pom.xml
+++ b/flume-ng-dist/pom.xml
@@ -150,6 +150,14 @@
<groupId>org.apache.flume.flume-ng-sources</groupId>
<artifactId>flume-taildir-source</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.flume.flume-ng-sources</groupId>
+ <artifactId>flume-syslog-source</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flume.flume-ng-sources</groupId>
+ <artifactId>flume-netcat-source</artifactId>
+ </dependency>
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-ng-environment-variable-config-filter</artifactId>
diff --git a/flume-ng-sdk/pom.xml b/flume-ng-sdk/pom.xml
index dc47f42b1..8ddc7e3c5 100644
--- a/flume-ng-sdk/pom.xml
+++ b/flume-ng-sdk/pom.xml
@@ -44,6 +44,19 @@
<scope>test</scope>
</dependency>
+ <!-- Used by the shared X509Certificates test helper (packaged in the
test-jar). -->
+ <dependency>
+ <groupId>org.bouncycastle</groupId>
+ <artifactId>bcpkix-jdk18on</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.bouncycastle</groupId>
+ <artifactId>bcprov-jdk18on</artifactId>
+ <scope>test</scope>
+ </dependency>
+
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
@@ -93,6 +106,18 @@
</executions>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <!-- AbstractSSLUtilTest injects environment variables by reflecting
into JDK internals,
+ which requires these opens on Java 16+. -->
+ <argLine>-Djava.net.preferIPv4Stack=true
+ --add-opens java.base/java.lang=ALL-UNNAMED
+ --add-opens java.base/java.util=ALL-UNNAMED</argLine>
+ </configuration>
+ </plugin>
+
</plugins>
</build>
</project>
diff --git
a/flume-ng-sdk/src/test/java/org/apache/flume/util/TestKeyStores.java
b/flume-ng-sdk/src/test/java/org/apache/flume/util/TestKeyStores.java
new file mode 100644
index 000000000..7be650027
--- /dev/null
+++ b/flume-ng-sdk/src/test/java/org/apache/flume/util/TestKeyStores.java
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flume.util;
+
+import java.io.OutputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.security.KeyPair;
+import java.security.KeyStore;
+import java.security.cert.X509Certificate;
+
+/**
+ * Test helper that builds key and trust stores backed by a single self-signed
certificate,
+ * so SSL tests can generate their TLS material in memory instead of shipping
keystore files.
+ */
+public final class TestKeyStores {
+
+ private final KeyPair keyPair;
+ private final X509Certificate certificate;
+
+ private TestKeyStores(KeyPair keyPair, X509Certificate certificate) {
+ this.keyPair = keyPair;
+ this.certificate = certificate;
+ }
+
+ /**
+ * Generates a self-signed credential for the given distinguished name
(for example {@code CN=localhost}).
+ */
+ public static TestKeyStores selfSigned(String dname) throws Exception {
+ KeyPair keyPair = X509Certificates.generateKeyPair();
+ X509Certificate certificate =
X509Certificates.generateSelfSignedCertificate(keyPair, dname);
+ return new TestKeyStores(keyPair, certificate);
+ }
+
+ public X509Certificate certificate() {
+ return certificate;
+ }
+
+ /** A keystore of the given type holding the private key and certificate
under alias {@code key}. */
+ public KeyStore keyStore(String type, String password) throws Exception {
+ KeyStore ks = KeyStore.getInstance(type);
+ ks.load(null, null);
+ ks.setKeyEntry("key", keyPair.getPrivate(), password.toCharArray(),
new X509Certificate[] {certificate});
+ return ks;
+ }
+
+ /** A truststore of the given type holding only the certificate under
alias {@code cert}. */
+ public KeyStore trustStore(String type) throws Exception {
+ KeyStore ts = KeyStore.getInstance(type);
+ ts.load(null, null);
+ ts.setCertificateEntry("cert", certificate);
+ return ts;
+ }
+
+ /** Writes {@link #keyStore} to {@code file} and returns its path. */
+ public Path writeKeyStore(Path file, String type, String password) throws
Exception {
+ KeyStore ks = keyStore(type, password);
+ try (OutputStream out = Files.newOutputStream(file)) {
+ ks.store(out, password.toCharArray());
+ }
+ return file;
+ }
+}
diff --git
a/flume-ng-sources/flume-http-source/src/test/java/org/apache/flume/source/http/X509Certificates.java
b/flume-ng-sdk/src/test/java/org/apache/flume/util/X509Certificates.java
similarity index 94%
rename from
flume-ng-sources/flume-http-source/src/test/java/org/apache/flume/source/http/X509Certificates.java
rename to flume-ng-sdk/src/test/java/org/apache/flume/util/X509Certificates.java
index c98ece50f..ac3845002 100644
---
a/flume-ng-sources/flume-http-source/src/test/java/org/apache/flume/source/http/X509Certificates.java
+++ b/flume-ng-sdk/src/test/java/org/apache/flume/util/X509Certificates.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source.http;
+package org.apache.flume.util;
import java.math.BigInteger;
import java.security.KeyPair;
@@ -37,7 +37,7 @@ import
org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
/**
* Utility class to generate X.509 certificates for testing purposes.
*/
-final class X509Certificates {
+public final class X509Certificates {
private static final long MINUTE_IN_MILLIS = 60_000L;
private static final long YEAR_IN_MILLIS = 365L * 24 * 60 *
MINUTE_IN_MILLIS;
@@ -54,7 +54,7 @@ final class X509Certificates {
}
}
- static KeyPair generateKeyPair() {
+ public static KeyPair generateKeyPair() {
return RSA_GENERATOR.generateKeyPair();
}
@@ -66,7 +66,7 @@ final class X509Certificates {
* @return a self-signed X.509 server certificate
* @throws Exception if certificate creation or signing fails
*/
- static X509Certificate generateSelfSignedCertificate(KeyPair keyPair,
String subjectDn) throws Exception {
+ public static X509Certificate generateSelfSignedCertificate(KeyPair
keyPair, String subjectDn) throws Exception {
long now = System.currentTimeMillis();
Date notBefore = new Date(now - MINUTE_IN_MILLIS);
Date notAfter = new Date(now + YEAR_IN_MILLIS);
diff --git
a/flume-ng-sources/flume-http-source/src/test/java/org/apache/flume/source/http/TestHTTPSource.java
b/flume-ng-sources/flume-http-source/src/test/java/org/apache/flume/source/http/TestHTTPSource.java
index fc950b8d7..9c674cbc8 100644
---
a/flume-ng-sources/flume-http-source/src/test/java/org/apache/flume/source/http/TestHTTPSource.java
+++
b/flume-ng-sources/flume-http-source/src/test/java/org/apache/flume/source/http/TestHTTPSource.java
@@ -24,8 +24,6 @@ import com.google.common.collect.Maps;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import jakarta.servlet.http.HttpServletResponse;
-import java.io.File;
-import java.io.FileOutputStream;
import java.io.IOException;
import java.lang.management.ManagementFactory;
import java.lang.reflect.Type;
@@ -35,9 +33,7 @@ import java.net.ServerSocket;
import java.net.Socket;
import java.net.URL;
import java.net.UnknownHostException;
-import java.security.KeyPair;
import java.security.KeyStore;
-import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
@@ -65,6 +61,7 @@ import org.apache.flume.channel.ReplicatingChannelSelector;
import org.apache.flume.conf.Configurables;
import org.apache.flume.event.JSONEvent;
import org.apache.flume.instrumentation.SourceCounter;
+import org.apache.flume.util.TestKeyStores;
import org.apache.flume.util.Whitebox;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
@@ -147,23 +144,12 @@ public class TestHTTPSource {
private static KeyStore trustStore;
private static void generateSslStores() throws Exception {
- KeyPair keyPair = X509Certificates.generateKeyPair();
- X509Certificate certificate =
X509Certificates.generateSelfSignedCertificate(keyPair, "CN=localhost");
-
- KeyStore keyStore = KeyStore.getInstance("JKS");
- keyStore.load(null, null);
- keyStore.setKeyEntry(
- "jetty", keyPair.getPrivate(),
KEYSTORE_PASSWORD.toCharArray(), new X509Certificate[] {certificate});
- File keystoreFile = TEMP_FOLDER.newFile("keystore.jks");
- try (FileOutputStream out = new FileOutputStream(keystoreFile)) {
- keyStore.store(out, KEYSTORE_PASSWORD.toCharArray());
- }
- serverKeystorePath = keystoreFile.getAbsolutePath();
-
+ TestKeyStores credentials = TestKeyStores.selfSigned("CN=localhost");
+ serverKeystorePath = credentials
+ .writeKeyStore(TEMP_FOLDER.newFile("keystore.jks").toPath(),
"JKS", KEYSTORE_PASSWORD)
+ .toString();
// The test client only needs to trust the self-signed server
certificate.
- trustStore = KeyStore.getInstance("JKS");
- trustStore.load(null, null);
- trustStore.setCertificateEntry("server", certificate);
+ trustStore = credentials.trustStore("JKS");
}
@BeforeClass
diff --git a/flume-ng-sources/pom.xml
b/flume-ng-sources/flume-netcat-source/pom.xml
similarity index 57%
copy from flume-ng-sources/pom.xml
copy to flume-ng-sources/flume-netcat-source/pom.xml
index 683ea2a53..91aca106c 100644
--- a/flume-ng-sources/pom.xml
+++ b/flume-ng-sources/flume-netcat-source/pom.xml
@@ -21,19 +21,40 @@
<parent>
<groupId>org.apache.flume</groupId>
- <artifactId>flume-parent</artifactId>
+ <artifactId>flume-ng-sources</artifactId>
<version>2.0.0-SNAPSHOT</version>
- <relativePath>../flume-parent/pom.xml</relativePath>
</parent>
- <groupId>org.apache.flume</groupId>
- <artifactId>flume-ng-sources</artifactId>
- <packaging>pom</packaging>
- <name>Flume Sources</name>
+ <groupId>org.apache.flume.flume-ng-sources</groupId>
+ <artifactId>flume-netcat-source</artifactId>
+ <name>Flume Netcat Source</name>
- <modules>
- <module>flume-http-source</module>
- <module>flume-taildir-source</module>
- </modules>
+ <properties>
+ <!-- TODO fix spotbugs violations -->
+ <spotbugs.maxAllowedViolations>3</spotbugs.maxAllowedViolations>
+ <module.name>org.apache.flume.source.netcat</module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.flume</groupId>
+ <artifactId>flume-ng-sdk</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flume</groupId>
+ <artifactId>flume-ng-core</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-all</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
</project>
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/source/NetcatUdpSource.java
b/flume-ng-sources/flume-netcat-source/src/main/java/org/apache/flume/source/netcat/NetcatUdpSource.java
similarity index 98%
rename from
flume-ng-core/src/main/java/org/apache/flume/source/NetcatUdpSource.java
rename to
flume-ng-sources/flume-netcat-source/src/main/java/org/apache/flume/source/netcat/NetcatUdpSource.java
index be044b508..f16c3ab8f 100644
--- a/flume-ng-core/src/main/java/org/apache/flume/source/NetcatUdpSource.java
+++
b/flume-ng-sources/flume-netcat-source/src/main/java/org/apache/flume/source/netcat/NetcatUdpSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.netcat;
import com.google.common.annotations.VisibleForTesting;
import io.netty.bootstrap.Bootstrap;
@@ -40,6 +40,7 @@ import org.apache.flume.conf.Configurable;
import org.apache.flume.conf.Configurables;
import org.apache.flume.event.EventBuilder;
import org.apache.flume.exception.ChannelException;
+import org.apache.flume.source.AbstractSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
diff --git
a/flume-ng-core/src/test/java/org/apache/flume/source/TestNetcatUdpSource.java
b/flume-ng-sources/flume-netcat-source/src/test/java/org/apache/flume/source/netcat/TestNetcatUdpSource.java
similarity index 99%
rename from
flume-ng-core/src/test/java/org/apache/flume/source/TestNetcatUdpSource.java
rename to
flume-ng-sources/flume-netcat-source/src/test/java/org/apache/flume/source/netcat/TestNetcatUdpSource.java
index 7266086a4..3af5910dc 100644
---
a/flume-ng-core/src/test/java/org/apache/flume/source/TestNetcatUdpSource.java
+++
b/flume-ng-sources/flume-netcat-source/src/test/java/org/apache/flume/source/netcat/TestNetcatUdpSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.netcat;
import com.google.common.base.Charsets;
import java.io.IOException;
diff --git
a/flume-ng-sources/flume-netcat-source/src/test/java/org/apache/flume/source/netcat/TestNetcatUdpSourceFactory.java
b/flume-ng-sources/flume-netcat-source/src/test/java/org/apache/flume/source/netcat/TestNetcatUdpSourceFactory.java
new file mode 100644
index 000000000..855733ca1
--- /dev/null
+++
b/flume-ng-sources/flume-netcat-source/src/test/java/org/apache/flume/source/netcat/TestNetcatUdpSourceFactory.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flume.source.netcat;
+
+import org.apache.flume.Source;
+import org.apache.flume.SourceFactory;
+import org.apache.flume.source.DefaultSourceFactory;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+public class TestNetcatUdpSourceFactory {
+
+ private SourceFactory sourceFactory;
+
+ @Before
+ public void setUp() {
+ sourceFactory = new DefaultSourceFactory();
+ }
+
+ @Test
+ public void testNetcatUdpSourceCreation() throws Exception {
+ Source src = sourceFactory.create("netcat-udp-src", "netcatudp");
+ Assert.assertNotNull(src);
+ Assert.assertTrue(src instanceof NetcatUdpSource);
+ }
+}
diff --git a/flume-ng-sdk/pom.xml b/flume-ng-sources/flume-syslog-source/pom.xml
similarity index 58%
copy from flume-ng-sdk/pom.xml
copy to flume-ng-sources/flume-syslog-source/pom.xml
index dc47f42b1..8f85d836f 100644
--- a/flume-ng-sdk/pom.xml
+++ b/flume-ng-sources/flume-syslog-source/pom.xml
@@ -16,27 +16,55 @@
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.flume</groupId>
- <artifactId>flume-parent</artifactId>
+ <artifactId>flume-ng-sources</artifactId>
<version>2.0.0-SNAPSHOT</version>
- <relativePath>../flume-parent/pom.xml</relativePath>
</parent>
- <artifactId>flume-ng-sdk</artifactId>
- <name>Flume NG SDK</name>
- <description>Flume Software Development Kit: Stable public API for
integration with Flume 1.x</description>
+ <groupId>org.apache.flume.flume-ng-sources</groupId>
+ <artifactId>flume-syslog-source</artifactId>
+ <name>Flume Syslog Source</name>
<properties>
- <!-- TODO fix spotbugs/pmd violations -->
- <spotbugs.maxAllowedViolations>69</spotbugs.maxAllowedViolations>
- <pmd.maxAllowedViolations>170</pmd.maxAllowedViolations>
- <module.name>org.apache.flume.sdk</module.name>
+ <!-- TODO fix spotbugs violations -->
+ <spotbugs.maxAllowedViolations>8</spotbugs.maxAllowedViolations>
+ <module.name>org.apache.flume.source.syslog</module.name>
</properties>
<dependencies>
+ <dependency>
+ <groupId>org.apache.flume</groupId>
+ <artifactId>flume-ng-sdk</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flume</groupId>
+ <artifactId>flume-ng-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flume</groupId>
+ <artifactId>flume-ng-sdk</artifactId>
+ <classifier>tests</classifier>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-all</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.mina</groupId>
+ <artifactId>mina-core</artifactId>
+ </dependency>
<dependency>
<groupId>junit</groupId>
@@ -45,54 +73,34 @@
</dependency>
<dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-api</artifactId>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
</dependency>
<dependency>
- <groupId>com.google.guava</groupId>
- <artifactId>guava</artifactId>
+ <groupId>org.bouncycastle</groupId>
+ <artifactId>bcpkix-jdk18on</artifactId>
+ <scope>test</scope>
</dependency>
<dependency>
- <groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-slf4j-impl</artifactId>
+ <groupId>org.bouncycastle</groupId>
+ <artifactId>bcprov-jdk18on</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-jul</artifactId>
+ <artifactId>log4j-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-1.2-api</artifactId>
+ <artifactId>log4j-slf4j-impl</artifactId>
<scope>test</scope>
</dependency>
-
- <dependency>
- <groupId>commons-lang</groupId>
- <artifactId>commons-lang</artifactId>
- </dependency>
-
</dependencies>
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- <executions>
- <execution>
- <goals>
- <goal>test-jar</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
-
- </plugins>
- </build>
</project>
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/source/MultiportSyslogTCPSource.java
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/MultiportSyslogTCPSource.java
similarity index 99%
rename from
flume-ng-core/src/main/java/org/apache/flume/source/MultiportSyslogTCPSource.java
rename to
flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/MultiportSyslogTCPSource.java
index 8853857cc..2a92c7468 100644
---
a/flume-ng-core/src/main/java/org/apache/flume/source/MultiportSyslogTCPSource.java
+++
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/MultiportSyslogTCPSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import com.google.common.base.Preconditions;
import com.google.common.base.Throwables;
@@ -38,6 +38,7 @@ import org.apache.flume.conf.Configurable;
import org.apache.flume.conf.LogPrivacyUtil;
import org.apache.flume.event.EventBuilder;
import org.apache.flume.instrumentation.SourceCounter;
+import org.apache.flume.source.SslContextAwareAbstractSource;
import org.apache.mina.core.buffer.IoBuffer;
import org.apache.mina.core.service.IoHandlerAdapter;
import org.apache.mina.core.session.IdleStatus;
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogParser.java
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogParser.java
similarity index 99%
rename from
flume-ng-core/src/main/java/org/apache/flume/source/SyslogParser.java
rename to
flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogParser.java
index 8e84a7cc8..97cd8af10 100644
--- a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogParser.java
+++
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogParser.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import com.google.common.base.Preconditions;
import com.google.common.cache.CacheBuilder;
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogSourceConfigurationConstants.java
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogSourceConfigurationConstants.java
similarity index 98%
rename from
flume-ng-core/src/main/java/org/apache/flume/source/SyslogSourceConfigurationConstants.java
rename to
flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogSourceConfigurationConstants.java
index 8378f1056..e791a0162 100644
---
a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogSourceConfigurationConstants.java
+++
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogSourceConfigurationConstants.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
public final class SyslogSourceConfigurationConstants {
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogTcpSource.java
similarity index 98%
rename from
flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java
rename to
flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogTcpSource.java
index 09bfa4952..48c587a7b 100644
--- a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java
+++
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogTcpSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import com.google.common.annotations.VisibleForTesting;
import io.netty.bootstrap.ServerBootstrap;
@@ -45,6 +45,7 @@ import org.apache.flume.conf.Configurable;
import org.apache.flume.conf.Configurables;
import org.apache.flume.exception.ChannelException;
import org.apache.flume.instrumentation.SourceCounter;
+import org.apache.flume.source.SslContextAwareAbstractSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogUDPSource.java
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogUDPSource.java
similarity index 98%
rename from
flume-ng-core/src/main/java/org/apache/flume/source/SyslogUDPSource.java
rename to
flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogUDPSource.java
index d5d159b02..cb4862d39 100644
--- a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogUDPSource.java
+++
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogUDPSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import com.google.common.annotations.VisibleForTesting;
import io.netty.bootstrap.Bootstrap;
@@ -37,6 +37,7 @@ import org.apache.flume.conf.Configurable;
import org.apache.flume.conf.Configurables;
import org.apache.flume.exception.ChannelException;
import org.apache.flume.instrumentation.SourceCounter;
+import org.apache.flume.source.AbstractSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
diff --git
a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogUtils.java
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogUtils.java
similarity index 99%
rename from flume-ng-core/src/main/java/org/apache/flume/source/SyslogUtils.java
rename to
flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogUtils.java
index 049020b4a..ac65eea27 100644
--- a/flume-ng-core/src/main/java/org/apache/flume/source/SyslogUtils.java
+++
b/flume-ng-sources/flume-syslog-source/src/main/java/org/apache/flume/source/syslog/SyslogUtils.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import io.netty.buffer.ByteBuf;
import java.io.ByteArrayOutputStream;
diff --git
a/flume-ng-core/src/test/java/org/apache/flume/source/TestMultiportSyslogTCPSource.java
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestMultiportSyslogTCPSource.java
similarity index 97%
rename from
flume-ng-core/src/test/java/org/apache/flume/source/TestMultiportSyslogTCPSource.java
rename to
flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestMultiportSyslogTCPSource.java
index 49c74421d..8a63720de 100644
---
a/flume-ng-core/src/test/java/org/apache/flume/source/TestMultiportSyslogTCPSource.java
+++
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestMultiportSyslogTCPSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
@@ -33,6 +33,8 @@ import java.net.SocketAddress;
import java.net.UnknownHostException;
import java.nio.charset.CharacterCodingException;
import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.Path;
import java.security.cert.X509Certificate;
import java.text.ParseException;
import java.time.ZonedDateTime;
@@ -59,10 +61,11 @@ import org.apache.flume.channel.ReplicatingChannelSelector;
import org.apache.flume.conf.Configurables;
import org.apache.flume.exception.ChannelException;
import org.apache.flume.instrumentation.SourceCounter;
-import org.apache.flume.source.MultiportSyslogTCPSource.LineSplitter;
-import org.apache.flume.source.MultiportSyslogTCPSource.MultiportSyslogHandler;
-import org.apache.flume.source.MultiportSyslogTCPSource.ParsedBuffer;
-import org.apache.flume.source.MultiportSyslogTCPSource.ThreadSafeDecoder;
+import org.apache.flume.source.syslog.MultiportSyslogTCPSource.LineSplitter;
+import
org.apache.flume.source.syslog.MultiportSyslogTCPSource.MultiportSyslogHandler;
+import org.apache.flume.source.syslog.MultiportSyslogTCPSource.ParsedBuffer;
+import
org.apache.flume.source.syslog.MultiportSyslogTCPSource.ThreadSafeDecoder;
+import org.apache.flume.util.TestKeyStores;
import org.apache.flume.util.Whitebox;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -231,9 +234,12 @@ public class TestMultiportSyslogTCPSource {
SocketFactory socketFactory = sslContext.getSocketFactory();
+ Path keystore = TestKeyStores.selfSigned("CN=localhost")
+ .writeKeyStore(Files.createTempFile("server", ".p12"),
"PKCS12", "password");
+
Context context = new Context();
context.put("ssl", "true");
- context.put("keystore",
"src/test/resources/server.flume-keystore.p12");
+ context.put("keystore", keystore.toString());
context.put("keystore-password", "password");
context.put("keystore-type", "PKCS12");
diff --git
a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogParser.java
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogParser.java
similarity index 99%
rename from
flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogParser.java
rename to
flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogParser.java
index 07200383a..ca6e53ed7 100644
--- a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogParser.java
+++
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogParser.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import com.google.common.base.Charsets;
import com.google.common.collect.Lists;
diff --git
a/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogSourceFactory.java
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogSourceFactory.java
new file mode 100644
index 000000000..1c559ac3b
--- /dev/null
+++
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogSourceFactory.java
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flume.source.syslog;
+
+import org.apache.flume.Source;
+import org.apache.flume.SourceFactory;
+import org.apache.flume.source.DefaultSourceFactory;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+public class TestSyslogSourceFactory {
+
+ private SourceFactory sourceFactory;
+
+ @Before
+ public void setUp() {
+ sourceFactory = new DefaultSourceFactory();
+ }
+
+ @Test
+ public void testSyslogTcpSourceCreation() throws Exception {
+ Source src = sourceFactory.create("syslogtcp-src", "syslogtcp");
+ Assert.assertNotNull(src);
+ Assert.assertTrue(src instanceof SyslogTcpSource);
+ }
+
+ @Test
+ public void testMultiportSyslogTcpSourceCreation() throws Exception {
+ Source src = sourceFactory.create("multiport_syslogtcp-src",
"multiport_syslogtcp");
+ Assert.assertNotNull(src);
+ Assert.assertTrue(src instanceof MultiportSyslogTCPSource);
+ }
+
+ @Test
+ public void testSyslogUdpSourceCreation() throws Exception {
+ Source src = sourceFactory.create("syslogudp-src", "syslogudp");
+ Assert.assertNotNull(src);
+ Assert.assertTrue(src instanceof SyslogUDPSource);
+ }
+}
diff --git
a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogTcpSource.java
similarity index 94%
rename from
flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java
rename to
flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogTcpSource.java
index feb955629..fbfd87403 100644
---
a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java
+++
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogTcpSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
@@ -28,6 +28,7 @@ import java.io.OutputStream;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
+import java.nio.file.Path;
import java.security.cert.X509Certificate;
import java.time.LocalDateTime;
import java.util.ArrayList;
@@ -47,8 +48,12 @@ import org.apache.flume.channel.MemoryChannel;
import org.apache.flume.channel.ReplicatingChannelSelector;
import org.apache.flume.conf.Configurables;
import org.apache.flume.exception.ChannelException;
+import org.apache.flume.util.TestKeyStores;
import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
import org.mockito.Mockito;
import org.slf4j.LoggerFactory;
@@ -68,6 +73,17 @@ public class TestSyslogTcpSource {
private final String bodyWithTimestamp = stamp1 + " " + data1;
private final String bodyWithTandH = "<10>" + stamp1 + " " + host1 + " " +
data1 + "\n";
+ @ClassRule
+ public static final TemporaryFolder TEMP_FOLDER = new TemporaryFolder();
+
+ private static Path serverKeystore;
+
+ @BeforeClass
+ public static void generateKeystore() throws Exception {
+ serverKeystore = TestKeyStores.selfSigned("CN=localhost")
+ .writeKeyStore(TEMP_FOLDER.newFile("server.p12").toPath(),
"PKCS12", "password");
+ }
+
private void init(String keepFields) {
init(keepFields, new Context());
}
@@ -96,7 +112,7 @@ public class TestSyslogTcpSource {
private void initSsl() {
Context context = new Context();
context.put("ssl", "true");
- context.put("keystore", "src/test/resources/server.p12");
+ context.put("keystore", serverKeystore.toString());
context.put("keystore-password", "password");
context.put("keystore-type", "PKCS12");
init("none", context);
diff --git
a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogUdpSource.java
similarity index 99%
rename from
flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java
rename to
flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogUdpSource.java
index 9979906cf..7d8ce85a2 100644
---
a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java
+++
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogUdpSource.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
diff --git
a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUtils.java
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogUtils.java
similarity index 99%
rename from
flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUtils.java
rename to
flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogUtils.java
index fd8206cdd..9834ca3ee 100644
--- a/flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUtils.java
+++
b/flume-ng-sources/flume-syslog-source/src/test/java/org/apache/flume/source/syslog/TestSyslogUtils.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.flume.source;
+package org.apache.flume.source.syslog;
import static io.netty.buffer.Unpooled.*;
import static org.junit.Assert.assertEquals;
diff --git a/flume-ng-sources/pom.xml b/flume-ng-sources/pom.xml
index 683ea2a53..3318e3fd2 100644
--- a/flume-ng-sources/pom.xml
+++ b/flume-ng-sources/pom.xml
@@ -34,6 +34,8 @@
<modules>
<module>flume-http-source</module>
<module>flume-taildir-source</module>
+ <module>flume-syslog-source</module>
+ <module>flume-netcat-source</module>
</modules>
</project>
