[jira] [Commented] (NIFI-1831) Allow encrypted passwords in configuration files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416516#comment-15416516
 ] 

ASF GitHub Bot commented on NIFI-1831:
--

GitHub user alopresto opened a pull request:

https://github.com/apache/nifi/pull/834

NIFI-1831 Implemented encrypted configuration capabilities

This is a "beta" PR -- there are still some edge cases I want to clean up 
but I wanted to get eyes on it because it will be a large change. Specific 
areas of focus for review:
* LICENSE
* `AESSensitivePropertyProvider` -- class which actually performs 
encryption/decryption
* `ConfigEncryptionTool` -- command-line class to make the process of 
encrypting the configuration values easy

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alopresto/nifi NIFI-1831

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/834.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #834


commit b75cdb0742fd814e0c9c7831e8e0bcbdc48c1223
Author: Andy LoPresto 
Date:   2016-06-11T04:06:55Z

NIFI-1831 Cleaned up trailing whitespace. (+1 squashed commit)
Squashed commits:
[7fc7e0a] NIFI-1831 Adapted Apache Commons Hex and Base64 encoding/decoding 
classes and implemented in nifi-properties (avoiding extraneous Maven 
dependencies).
Added sanity checks in unit tests.
Added utility join method in StringUtils.

commit f9f173099b026eb78b2fe16c6ad15ab1bdeed8d3
Author: Andy LoPresto 
Date:   2016-06-11T04:18:37Z

NIFI-1831 Added interface and first implementation of 
SensitivePropertyProvider.

commit 354596a2e3ea9cde41bd12c0ff4d145e1ab910c4
Author: Andy LoPresto 
Date:   2016-06-11T04:19:13Z

NIFI-1831 Added skeleton for regression tests and new unit tests for 
sensitive property loading.

commit 836ee1e84c2b352ee96c8831e02178d95396400c
Author: Andy LoPresto 
Date:   2016-07-14T20:31:17Z

NIFI-1831 Added initial logic in NiFiProperties to detect sensitive 
property keys and return list of sensitive properties.
Added unit tests (and resource).

commit dafa426d36e2bb86361ee8c14681addfa31677f9
Author: Andy LoPresto 
Date:   2016-07-14T20:39:17Z

NIFI-1831 Moved Base64 and Hex sanity tests into correct package.

commit eee44e107ddbecbacd8cb44b9c94fd0bbef5ec60
Author: Andy LoPresto 
Date:   2016-07-14T22:58:01Z

NIFI-1831 Added BouncyCastle dependency to nifi-properties module.

commit a0286b55eb1c1f469ae9612a51a3aadb4198f175
Author: Andy LoPresto 
Date:   2016-07-14T22:59:06Z

NIFI-1831 Added SensitivePropertyProtectionException.
Updated unprotect method contract in interface and implementation with new 
exception.
Added unprotect logic.

commit 064b2d14c088f7e984af625658011fe9310a759b
Author: Andy LoPresto 
Date:   2016-07-14T22:59:34Z

NIFI-1831 Added unit tests for sensitive property encryption/decryption.

commit 46a7b1b902672843f10758edbe763dcd35f1ffc7
Author: Andy LoPresto 
Date:   2016-07-15T19:11:50Z

NIFI-1831 Finished logic for sensitive property encryption/decryption.
Added negative/edge case unit tests for sensitive property 
encryption/decryption.

commit 2d8ec30adfb6f46e1a4302c0fe074d7b132a3cda
Author: Andy LoPresto 
Date:   2016-07-15T19:17:28Z

NIFI-1831 Excluded additional keys property from self-inclusion to avoid 
recursive/protection issue.

commit 723523059a1c8fffe90ddbf29a9eb8ac3e2ec32a
Author: Andy LoPresto 
Date:   2016-07-15T19:33:20Z

NIFI-1831 Removed unnecessary duplication of Base64 and Hex encoders as 
BouncyCastle includes implementations.

commit b81f89fa8bb20d96d4d2ff1a45d15c2271e88ad6
Author: Andy LoPresto 
Date:   2016-07-15T19:35:53Z

NIFI-1831 Removed unnecessary reference to commons-codec from NOTICE files 
as the duplicate code is removed.

commit 323756119e6841e4b6bab234f793386ea1ce13a5
Author: Andy LoPresto 
Date:   2016-07-15T19:36:57Z

NIFI-1831 Removed unnecessary nifi-properties NOTICE file as the duplicate 
code is removed.

commit 0798cd12d2712132285284209be51ac40bfd970b
Author: Andy LoPresto 
Date:   2016-07-15T21:25:32Z

NIFI-1831 Fixed tests which were not deterministic (shuffling IV and 
ciphertext could occasionally result in same value, rendering successful 
decryption and lacking expected exception).
Added license to AESSensitivePropertyProviderTest.

commit 7d9c3a8159a4d06d0a96364e87141de10a298e33
Author: Andy LoPresto 
Date:   2016-07-15T23:20:59Z

[GitHub] nifi pull request #834: NIFI-1831 Implemented encrypted configuration capabi...

[alopresto]

GitHub user alopresto opened a pull request:

https://github.com/apache/nifi/pull/834

NIFI-1831 Implemented encrypted configuration capabilities

This is a "beta" PR -- there are still some edge cases I want to clean up 
but I wanted to get eyes on it because it will be a large change. Specific 
areas of focus for review:
* LICENSE
* `AESSensitivePropertyProvider` -- class which actually performs 
encryption/decryption
* `ConfigEncryptionTool` -- command-line class to make the process of 
encrypting the configuration values easy

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alopresto/nifi NIFI-1831

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/834.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #834


commit b75cdb0742fd814e0c9c7831e8e0bcbdc48c1223
Author: Andy LoPresto 
Date:   2016-06-11T04:06:55Z

NIFI-1831 Cleaned up trailing whitespace. (+1 squashed commit)
Squashed commits:
[7fc7e0a] NIFI-1831 Adapted Apache Commons Hex and Base64 encoding/decoding 
classes and implemented in nifi-properties (avoiding extraneous Maven 
dependencies).
Added sanity checks in unit tests.
Added utility join method in StringUtils.

commit f9f173099b026eb78b2fe16c6ad15ab1bdeed8d3
Author: Andy LoPresto 
Date:   2016-06-11T04:18:37Z

NIFI-1831 Added interface and first implementation of 
SensitivePropertyProvider.

commit 354596a2e3ea9cde41bd12c0ff4d145e1ab910c4
Author: Andy LoPresto 
Date:   2016-06-11T04:19:13Z

NIFI-1831 Added skeleton for regression tests and new unit tests for 
sensitive property loading.

commit 836ee1e84c2b352ee96c8831e02178d95396400c
Author: Andy LoPresto 
Date:   2016-07-14T20:31:17Z

NIFI-1831 Added initial logic in NiFiProperties to detect sensitive 
property keys and return list of sensitive properties.
Added unit tests (and resource).

commit dafa426d36e2bb86361ee8c14681addfa31677f9
Author: Andy LoPresto 
Date:   2016-07-14T20:39:17Z

NIFI-1831 Moved Base64 and Hex sanity tests into correct package.

commit eee44e107ddbecbacd8cb44b9c94fd0bbef5ec60
Author: Andy LoPresto 
Date:   2016-07-14T22:58:01Z

NIFI-1831 Added BouncyCastle dependency to nifi-properties module.

commit a0286b55eb1c1f469ae9612a51a3aadb4198f175
Author: Andy LoPresto 
Date:   2016-07-14T22:59:06Z

NIFI-1831 Added SensitivePropertyProtectionException.
Updated unprotect method contract in interface and implementation with new 
exception.
Added unprotect logic.

commit 064b2d14c088f7e984af625658011fe9310a759b
Author: Andy LoPresto 
Date:   2016-07-14T22:59:34Z

NIFI-1831 Added unit tests for sensitive property encryption/decryption.

commit 46a7b1b902672843f10758edbe763dcd35f1ffc7
Author: Andy LoPresto 
Date:   2016-07-15T19:11:50Z

NIFI-1831 Finished logic for sensitive property encryption/decryption.
Added negative/edge case unit tests for sensitive property 
encryption/decryption.

commit 2d8ec30adfb6f46e1a4302c0fe074d7b132a3cda
Author: Andy LoPresto 
Date:   2016-07-15T19:17:28Z

NIFI-1831 Excluded additional keys property from self-inclusion to avoid 
recursive/protection issue.

commit 723523059a1c8fffe90ddbf29a9eb8ac3e2ec32a
Author: Andy LoPresto 
Date:   2016-07-15T19:33:20Z

NIFI-1831 Removed unnecessary duplication of Base64 and Hex encoders as 
BouncyCastle includes implementations.

commit b81f89fa8bb20d96d4d2ff1a45d15c2271e88ad6
Author: Andy LoPresto 
Date:   2016-07-15T19:35:53Z

NIFI-1831 Removed unnecessary reference to commons-codec from NOTICE files 
as the duplicate code is removed.

commit 323756119e6841e4b6bab234f793386ea1ce13a5
Author: Andy LoPresto 
Date:   2016-07-15T19:36:57Z

NIFI-1831 Removed unnecessary nifi-properties NOTICE file as the duplicate 
code is removed.

commit 0798cd12d2712132285284209be51ac40bfd970b
Author: Andy LoPresto 
Date:   2016-07-15T21:25:32Z

NIFI-1831 Fixed tests which were not deterministic (shuffling IV and 
ciphertext could occasionally result in same value, rendering successful 
decryption and lacking expected exception).
Added license to AESSensitivePropertyProviderTest.

commit 7d9c3a8159a4d06d0a96364e87141de10a298e33
Author: Andy LoPresto 
Date:   2016-07-15T23:20:59Z

NIFI-1831 Added logic to retrieve protected keys and their protection 
mechanism from properties.
Added logic to determine percentage of sensitive keys currently protected.
Added unit tests.
Added failing unit test for 

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416445#comment-15416445
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74364045
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java
 ---
@@ -13,307 +13,174 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
+import static org.junit.Assert.assertTrue;
+
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
 import org.apache.commons.mail.Email;
 import org.apache.commons.mail.EmailException;
 import org.apache.commons.mail.SimpleEmail;
-import org.apache.nifi.processor.ProcessContext;
-import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.remote.io.socket.NetworkUtils;
+import org.apache.nifi.ssl.SSLContextService;
 import org.apache.nifi.ssl.StandardSSLContextService;
-import org.apache.nifi.util.MockFlowFile;
 import org.apache.nifi.util.TestRunner;
 import org.apache.nifi.util.TestRunners;
-
-import org.apache.nifi.ssl.SSLContextService;
-
-import org.junit.Assert;
+import org.junit.After;
+import org.junit.Before;
 import org.junit.Test;
 
-import java.util.List;
-import java.util.concurrent.atomic.AtomicBoolean;
-
 public class TestListenSMTP {
 
-@Test(timeout=15000)
-public void ValidEmailTls() throws Exception {
-boolean[] failed = {false};
-ListenSMTP listenSmtp = new ListenSMTP();
-final TestRunner runner = TestRunners.newTestRunner(listenSmtp);
-
-runner.setProperty(ListenSMTP.SMTP_PORT, "0");
-runner.setProperty(ListenSMTP.SMTP_HOSTNAME, "bermudatriangle");
-runner.setProperty(ListenSMTP.SMTP_MAXIMUM_CONNECTIONS, "3");
-runner.setProperty(ListenSMTP.SMTP_TIMEOUT, "10 seconds");
-
-// Setup the SSL Context
-final SSLContextService sslContextService = new 
StandardSSLContextService();
-runner.addControllerService("ssl-context", sslContextService);
-runner.setProperty(sslContextService, 
StandardSSLContextService.TRUSTSTORE, "src/test/resources/localhost-ts.jks");
-runner.setProperty(sslContextService, 
StandardSSLContextService.TRUSTSTORE_PASSWORD, "localtest");
-runner.setProperty(sslContextService, 
StandardSSLContextService.TRUSTSTORE_TYPE, "JKS");
-runner.setProperty(sslContextService, 
StandardSSLContextService.KEYSTORE, "src/test/resources/localhost-ks.jks");
-runner.setProperty(sslContextService, 
StandardSSLContextService.KEYSTORE_PASSWORD, "localtest");
-runner.setProperty(sslContextService, 
StandardSSLContextService.KEYSTORE_TYPE, "JKS");
-runner.enableControllerService(sslContextService);
-
-// and add the SSL context to the runner
-runner.setProperty(ListenSMTP.SSL_CONTEXT_SERVICE, "ssl-context");
-runner.setProperty(ListenSMTP.CLIENT_AUTH, 
SSLContextService.ClientAuth.NONE.name());
-
+private ScheduledExecutorService executor;
 
+/**
+ *
+ */
+@Before
+public void before() {
+this.executor = Executors.newScheduledThreadPool(2);
+}
 
-final ProcessSessionFactory processSessionFactory = 
runner.getProcessSessionFactory();
-final ProcessContext context = runner.getProcessContext();
-
-// NOTE: This test routine uses  the same strategy used by 
TestListenAndPutSyslog
-// where listenSmtp method calls are used to allow the processor 
to be started using
-// port "0" without triggering a violation of PORT_VALIDATOR
-
-listenSmtp.onScheduled(context);
-listenSmtp.initializeSMTPServer(context);
-
-final int port = listenSmtp.getPort();
-
-try {
-final Thread clientThread = new Thread(new Runnable() {
-@Override
-public void run() {
-try {
-
-
-System.setProperty("mail.smtp.ssl.trust", "*");
-System.setProperty("javax.net.ssl.keyStore", 
"src/test/resources/localhost-ks.jks");
-

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74364045
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java
 ---
@@ -13,307 +13,174 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
+import static org.junit.Assert.assertTrue;
+
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
 import org.apache.commons.mail.Email;
 import org.apache.commons.mail.EmailException;
 import org.apache.commons.mail.SimpleEmail;
-import org.apache.nifi.processor.ProcessContext;
-import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.remote.io.socket.NetworkUtils;
+import org.apache.nifi.ssl.SSLContextService;
 import org.apache.nifi.ssl.StandardSSLContextService;
-import org.apache.nifi.util.MockFlowFile;
 import org.apache.nifi.util.TestRunner;
 import org.apache.nifi.util.TestRunners;
-
-import org.apache.nifi.ssl.SSLContextService;
-
-import org.junit.Assert;
+import org.junit.After;
+import org.junit.Before;
 import org.junit.Test;
 
-import java.util.List;
-import java.util.concurrent.atomic.AtomicBoolean;
-
 public class TestListenSMTP {
 
-@Test(timeout=15000)
-public void ValidEmailTls() throws Exception {
-boolean[] failed = {false};
-ListenSMTP listenSmtp = new ListenSMTP();
-final TestRunner runner = TestRunners.newTestRunner(listenSmtp);
-
-runner.setProperty(ListenSMTP.SMTP_PORT, "0");
-runner.setProperty(ListenSMTP.SMTP_HOSTNAME, "bermudatriangle");
-runner.setProperty(ListenSMTP.SMTP_MAXIMUM_CONNECTIONS, "3");
-runner.setProperty(ListenSMTP.SMTP_TIMEOUT, "10 seconds");
-
-// Setup the SSL Context
-final SSLContextService sslContextService = new 
StandardSSLContextService();
-runner.addControllerService("ssl-context", sslContextService);
-runner.setProperty(sslContextService, 
StandardSSLContextService.TRUSTSTORE, "src/test/resources/localhost-ts.jks");
-runner.setProperty(sslContextService, 
StandardSSLContextService.TRUSTSTORE_PASSWORD, "localtest");
-runner.setProperty(sslContextService, 
StandardSSLContextService.TRUSTSTORE_TYPE, "JKS");
-runner.setProperty(sslContextService, 
StandardSSLContextService.KEYSTORE, "src/test/resources/localhost-ks.jks");
-runner.setProperty(sslContextService, 
StandardSSLContextService.KEYSTORE_PASSWORD, "localtest");
-runner.setProperty(sslContextService, 
StandardSSLContextService.KEYSTORE_TYPE, "JKS");
-runner.enableControllerService(sslContextService);
-
-// and add the SSL context to the runner
-runner.setProperty(ListenSMTP.SSL_CONTEXT_SERVICE, "ssl-context");
-runner.setProperty(ListenSMTP.CLIENT_AUTH, 
SSLContextService.ClientAuth.NONE.name());
-
+private ScheduledExecutorService executor;
 
+/**
+ *
+ */
+@Before
+public void before() {
+this.executor = Executors.newScheduledThreadPool(2);
+}
 
-final ProcessSessionFactory processSessionFactory = 
runner.getProcessSessionFactory();
-final ProcessContext context = runner.getProcessContext();
-
-// NOTE: This test routine uses  the same strategy used by 
TestListenAndPutSyslog
-// where listenSmtp method calls are used to allow the processor 
to be started using
-// port "0" without triggering a violation of PORT_VALIDATOR
-
-listenSmtp.onScheduled(context);
-listenSmtp.initializeSMTPServer(context);
-
-final int port = listenSmtp.getPort();
-
-try {
-final Thread clientThread = new Thread(new Runnable() {
-@Override
-public void run() {
-try {
-
-
-System.setProperty("mail.smtp.ssl.trust", "*");
-System.setProperty("javax.net.ssl.keyStore", 
"src/test/resources/localhost-ks.jks");
-
System.setProperty("javax.net.ssl.keyStorePassword", "localtest");
-
-Email email = new SimpleEmail();
-
-email.setHostName("127.0.0.1");
-email.setSmtpPort(port);
  

[GitHub] nifi issue #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/827
  
@olegz - Tested the processor and other than the view that attributes 
should not be removed (as they are useful for many auditing and downstream 
applications) and a few remarks on SubEthaSMTP intricacies  I am amazed by the 
refactoring. 

Truly awesome work in here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416429#comment-15416429
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74363521
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void process(final 

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74363458
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void process(final OutputStream out) throws 
IOException {
+size.set(IOUtils.copy(inputDataStream, out));
+}
+});
+

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416420#comment-15416420
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74363226
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

Got it


> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416410#comment-15416410
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362934
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

Perhaps I am not fully in line with that due to not fully understanding it, 
but if it's only used for "display" I am ok with putting it back in. Please 
confirm


> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416411#comment-15416411
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362947
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
--- End diff --

Will put it back


> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416413#comment-15416413
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362949
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416409#comment-15416409
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362870
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362947
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
--- End diff --

Will put it back


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362949
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 
description = "The principal for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.from", description = "The value 
used during MAIL FROM (i.e. envelope)"),
-   

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362934
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

Perhaps I am not fully in line with that due to not fully understanding it, 
but if it's only used for "display" I am ok with putting it back in. Please 
confirm


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362870
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 
description = "The principal for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.from", description = "The value 
used during MAIL FROM (i.e. envelope)"),
-

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416403#comment-15416403
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362705
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
--- End diff --

Sorry not used to post comments. But yes, custom validator. :-)


> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416398#comment-15416398
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362612
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362680
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

Because I may have a system using AWS's weird naming structures but look to 
display a fancy name like mx.trixpanmailservices.com




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362612
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 
description = "The principal for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.from", description = "The value 
used during MAIL FROM (i.e. envelope)"),
-   

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416394#comment-15416394
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362408
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void process(final 

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362408
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void process(final OutputStream out) throws 
IOException {
+size.set(IOUtils.copy(inputDataStream, out));
+}
+});
+

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416384#comment-15416384
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74362012
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void 

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361967
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void process(final OutputStream out) throws 
IOException {
+size.set(IOUtils.copy(inputDataStream, out));
+}
+});
+

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416383#comment-15416383
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361967
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void process(final 

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416381#comment-15416381
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361899
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
-protected Collection customValidate(final 
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("All new messages will be routed as FlowFiles to 
this relationship")
+.build();
 
-final String clientAuth = 
validationContext.getProperty(CLIENT_AUTH).getValue();
-final SSLContextService sslContextService = 
validationContext.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
+private final static List propertyDescriptors;
 
-if (sslContextService != null && StringUtils.isBlank(clientAuth)) {
-results.add(new ValidationResult.Builder()
-.explanation("Client Auth must be provided when using 
TLS/SSL")
-.valid(false).subject("Client Auth").build());
-}
+private final static Set relationships;
 
-return results;
+static {
+List _propertyDescriptors = new ArrayList<>();
+_propertyDescriptors.add(SMTP_PORT);
+_propertyDescriptors.add(SMTP_MAXIMUM_CONNECTIONS);
+_propertyDescriptors.add(SMTP_TIMEOUT);
+_propertyDescriptors.add(SMTP_MAXIMUM_MSG_SIZE);
+_propertyDescriptors.add(SSL_CONTEXT_SERVICE);
+_propertyDescriptors.add(CLIENT_AUTH);
+propertyDescriptors = 
Collections.unmodifiableList(_propertyDescriptors);
 
+Set _relationships = new HashSet<>();
+_relationships.add(REL_SUCCESS);
+relationships = Collections.unmodifiableSet(_relationships);
 }
 
+private volatile SMTPServer smtp;
 
-public static final Relationship REL_SUCCESS = new 
Relationship.Builder()
-.name("success")
-.description("Extraction was successful")
-.build();
+private volatile SmtpConsumer smtpConsumer;
 
-private Set relationships;
-private List propertyDescriptors;
-private volatile LinkedBlockingQueue incomingMessages;
+/**
+ *
+ */
+@Override
+public void onTrigger(ProcessContext context, ProcessSessionFactory 
sessionFactory) throws ProcessException {
+ProcessSession processSession = sessionFactory.createSession();
+if (this.smtp == null) {
+this.setupSmtpIfNecessary(context, processSession);
+}
+
+if (this.smtpConsumer.hasMessage()) {
+try {
+/*
+ * Will consume incoming message directly from the wire 
and into
+ * FlowFile/Content repository before exiting. This 
essentially
+ * limits any potential data loss by allowing SMTPServer 
thread
+ * to actually commit NiFi session if all good. However in 
the
+ * event of exception, such exception will be propagated 
back to
+ * the email sender via "undeliverable message" allowing 
such
+ * user to re-send the message
+ */
+this.smtpConsumer.consumeUsing((inputDataStream) -> {
+FlowFile flowFile = processSession.create();
+AtomicInteger size = new AtomicInteger();
+flowFile = processSession.write(flowFile, new 
OutputStreamCallback() {
+@Override
+public void 

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416377#comment-15416377
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361819
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
--- End diff --

I believe you are talking about customValidate() you had there before. I 
think it was an oversight on my end. Will put it back 


> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361819
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
--- End diff --

I believe you are talking about customValidate() you had there before. I 
think it was an oversight on my end. Will put it back 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416375#comment-15416375
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361725
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361725
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 
description = "The principal for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.from", description = "The value 
used during MAIL FROM (i.e. envelope)"),
-

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416371#comment-15416371
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361570
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
--- End diff --

Although harmless, this should not be a valid configuration:


![image](https://cloud.githubusercontent.com/assets/3108527/17576962/e5e0414a-5fbb-11e6-9050-9234dc1fe7d1.png)



> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361570
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -166,317 +108,158 @@
 .identifiesControllerService(SSLContextService.class)
 .build();
 
-public static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
+static final PropertyDescriptor CLIENT_AUTH = new 
PropertyDescriptor.Builder()
 .name("CLIENT_AUTH")
 .displayName("Client Auth")
 .description("The client authentication policy to use for the 
SSL Context. Only used if an SSL Context Service is provided.")
 .required(false)
 .allowableValues(SSLContextService.ClientAuth.NONE.toString(), 
SSLContextService.ClientAuth.REQUIRED.toString())
 .build();
 
-@Override
--- End diff --

Although harmless, this should not be a valid configuration:


![image](https://cloud.githubusercontent.com/assets/3108527/17576962/e5e0414a-5fbb-11e6-9050-9234dc1fe7d1.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[olegz]

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361483
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

True, so my question is whatever it defaults to is based on the network 
settings of your machine. Exposing it means I can set it to any name, so I am 
wondering why would this be important?
Don't get me wrong, putting it back is trivial, but I am looking for 
justification.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416366#comment-15416366
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user olegz commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361483
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

True, so my question is whatever it defaults to is based on the network 
settings of your machine. Exposing it means I can set it to any name, so I am 
wondering why would this be important?
Don't get me wrong, putting it back is trivial, but I am looking for 
justification.


> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2546) NPE while creating a template

[Oleg Zhurakousky (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416363#comment-15416363
 ] 

Oleg Zhurakousky commented on NIFI-2546:


[~joewitt] any chance you can describe what you did? The commit you mention 
does not touch any part of UUID of any component and was related to *only* 
un-scrubbing PropertyDescriptors and the reason for such un-scrubbing being 
ControllerServices and I have tested it with Mark's work. So While I'll be 
looking into it in the morning (stack trace does help a lot) I can't figure out 
scenario for NPE, so any input would help.

> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150] 
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at 
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
>  ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
>  ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
>  ~[classes/:na]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
>  ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
>  ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_66]
> at 
> 

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416360#comment-15416360
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361191
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

Without this SubEtha seems to default to localhost

```
$ telnet 10.0.2.15 2525
Trying 10.0.2.15...
Connected to 10.0.2.15.
Escape character is '^]'.
220 localhost ESMTP Apache NiFi

```


> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74361191
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -106,26 +69,17 @@
 .addValidator(StandardValidators.PORT_VALIDATOR)
 .build();
 
-protected static final PropertyDescriptor SMTP_HOSTNAME = new 
PropertyDescriptor.Builder()
-.name("SMTP_HOSTNAME")
--- End diff --

Without this SubEtha seems to default to localhost

```
$ telnet 10.0.2.15 2525
Trying 10.0.2.15...
Connected to 10.0.2.15.
Escape character is '^]'.
220 localhost ESMTP Apache NiFi

```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-2546) NPE while creating a template

[Oleg Zhurakousky (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky reassigned NIFI-2546:
--

Assignee: Oleg Zhurakousky

> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150] 
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at 
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
>  ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
>  ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
>  ~[classes/:na]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
>  ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
>  ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>  ~[jersey-server-1.19.jar:1.19]
> at 
> 

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[Andre (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416356#comment-15416356
 ] 

Andre commented on NIFI-2519:
-

[~joewitt] port 25 would require NiFi to be running as root.

> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416335#comment-15416335
 ] 

ASF GitHub Bot commented on NIFI-2519:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74359964
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 

[GitHub] nifi pull request #827: NIFI-2519 Fixed and refactored ListenSMTP processor

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/827#discussion_r74359964
  
--- Diff: 
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ListenSMTP.java
 ---
@@ -13,89 +13,52 @@
  *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
implied.
  *  See the License for the specific language governing permissions and
  *  limitations under the License.
-*/
+ */
 package org.apache.nifi.processors.email;
 
-import javax.net.ssl.SSLContext;
-import javax.net.ssl.SSLSocket;
-import javax.net.ssl.SSLSocketFactory;
 import java.io.IOException;
-import java.io.InputStream;
+import java.io.OutputStream;
 import java.net.InetSocketAddress;
 import java.net.Socket;
 import java.util.ArrayList;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Set;
-import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
-
-import org.apache.commons.lang3.StringUtils;
-
-import org.subethamail.smtp.server.SMTPServer;
-
-
-import org.apache.nifi.annotation.lifecycle.OnStopped;
-import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
-import org.apache.nifi.flowfile.attributes.CoreAttributes;
-import org.apache.nifi.processor.DataUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSocket;
+import javax.net.ssl.SSLSocketFactory;
 
+import org.apache.commons.io.IOUtils;
 import org.apache.nifi.annotation.behavior.InputRequirement;
-import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.WritesAttributes;
 import org.apache.nifi.annotation.documentation.CapabilityDescription;
 import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.DataUnit;
 import org.apache.nifi.processor.ProcessContext;
 import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
 import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.components.ValidationContext;
-import org.apache.nifi.components.ValidationResult;
-import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
 import org.apache.nifi.ssl.SSLContextService;
-
-import org.apache.nifi.processors.email.smtp.event.SmtpEvent;
-import org.apache.nifi.processors.email.smtp.handler.SMTPResultCode;
-import 
org.apache.nifi.processors.email.smtp.handler.SMTPMessageHandlerFactory;
+import org.subethamail.smtp.server.SMTPServer;
 
 @Tags({"listen", "email", "smtp"})
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
-@CapabilityDescription("This processor implements a lightweight SMTP 
server to an arbitrary port, " +
-"allowing nifi to listen for incoming email. " +
-"" +
-"Note this server does not perform any email validation. If direct 
exposure to the internet is sought," +
-"it may be a better idea to use the combination of NiFi and an 
industrial scale MTA (e.g. Postfix)")
-@WritesAttributes({
-@WritesAttribute(attribute = "mime.type", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.helo", description = "The value 
used during HELO"),
-@WritesAttribute(attribute = "smtp.certificates.*.serial", 
description = "The serial numbers for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.certificates.*.principal", 
description = "The principal for each of the " +
-"certificates used by an TLS peer"),
-@WritesAttribute(attribute = "smtp.from", description = "The value 
used during MAIL FROM (i.e. envelope)"),
-   

[jira] [Commented] (NIFI-2547) Add DeleteHDFS Processor

[Ricky Saltzer (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416182#comment-15416182
 ] 

Ricky Saltzer commented on NIFI-2547:
-

I created a pretty polished prototype of this processor and deployed it onto my 
own production cluster. I swapped the original pipeline which was effectively 
stuck on over 2,000 directories (containing thousands of files per directory) 
for the new {{DeleteHDFS}} processor based pipeline. The new pipeline was able 
to delete all of the remaining directories in about 5 minutes. 

I will clean up the code and write some tests before submitting a PR. 

> Add DeleteHDFS Processor 
> -
>
> Key: NIFI-2547
> URL: https://issues.apache.org/jira/browse/NIFI-2547
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Ricky Saltzer
>Assignee: Ricky Saltzer
>
> There are times where a user may want to remove a file or directory from 
> HDFS. The reasons for this vary, but to provide some context, I currently 
> have a pipeline where I need to periodically delete files that my NiFi 
> pipeline is producing. In my case, it's a "Delete files after they are 7 days 
> old". 
> Currently, I have to use the {{ExecuteStreamCommand}} processor and manually 
> call {{hdfs dfs -rm}}, which is awful when dealing with a large amount of 
> files. For one, an entire JVM is spun up for each delete, and two, when 
> deleting directories with thousands of files, it can sometimes cause the 
> command to hang indefinitely. 
> With that being said, I am proposing we add a {{DeleteHDFS}} processor which 
> meets the following criteria. 
> * Can delete both directories and files
> * Can delete directories recursively
> * Supports the dynamic expression language 
> * Supports using glob paths (e.g. /data/for/2017/08/*)
> * Capable of being a downstream processor as well as a standalone processor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2547) Add DeleteHDFS Processor

[Ricky Saltzer (JIRA)]

Ricky Saltzer created NIFI-2547:
---

 Summary: Add DeleteHDFS Processor 
 Key: NIFI-2547
 URL: https://issues.apache.org/jira/browse/NIFI-2547
 Project: Apache NiFi
  Issue Type: New Feature
Reporter: Ricky Saltzer
Assignee: Ricky Saltzer


There are times where a user may want to remove a file or directory from HDFS. 
The reasons for this vary, but to provide some context, I currently have a 
pipeline where I need to periodically delete files that my NiFi pipeline is 
producing. In my case, it's a "Delete files after they are 7 days old". 

Currently, I have to use the {{ExecuteStreamCommand}} processor and manually 
call {{hdfs dfs -rm}}, which is awful when dealing with a large amount of 
files. For one, an entire JVM is spun up for each delete, and two, when 
deleting directories with thousands of files, it can sometimes cause the 
command to hang indefinitely. 

With that being said, I am proposing we add a {{DeleteHDFS}} processor which 
meets the following criteria. 

* Can delete both directories and files
* Can delete directories recursively
* Supports the dynamic expression language 
* Supports using glob paths (e.g. /data/for/2017/08/*)
* Capable of being a downstream processor as well as a standalone processor





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2536) Update Documentation for Templates

[James Wing (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Wing reassigned NIFI-2536:


Assignee: James Wing

> Update Documentation for Templates
> --
>
> Key: NIFI-2536
> URL: https://issues.apache.org/jira/browse/NIFI-2536
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: James Wing
>Assignee: James Wing
>Priority: Minor
>
> The documentation for importing a template does not reflect recent UI changes 
> for 1.0.  Required updates include:
> * Template features moved to the Operate Palette (create, upload)
> ** Association between template actions and current process group permissions
> * Template features still in the Templates dialog (list, filter, export, 
> delete)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2536) Update Documentation for Templates

[James Wing (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416151#comment-15416151
 ] 

James Wing commented on NIFI-2536:
--

I updated the ticket to expand the scope to template-related sections.  After 
looking at the User Guide text more closely, I realized it didn't make any 
sense to just change the Import Template section alone.

> Update Documentation for Templates
> --
>
> Key: NIFI-2536
> URL: https://issues.apache.org/jira/browse/NIFI-2536
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: James Wing
>Priority: Minor
>
> The documentation for importing a template does not reflect recent UI changes 
> for 1.0.  Required updates include:
> * Template features moved to the Operate Palette (create, upload)
> ** Association between template actions and current process group permissions
> * Template features still in the Templates dialog (list, filter, export, 
> delete)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2536) Update Documentation for Templates

[James Wing (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Wing updated NIFI-2536:
-
Description: 
The documentation for importing a template does not reflect recent UI changes 
for 1.0.  Required updates include:
* Template features moved to the Operate Palette (create, upload)
** Association between template actions and current process group permissions
* Template features still in the Templates dialog (list, filter, export, delete)

  was:
The documentation for importing a template does not reflect recent UI changes 
for 1.0.  The current User Guide text reflects uploading templates from the 
Template Management dialog.  It should describe the new Upload Template button 
on the Actions Toolbar (or "Operate Palette"?).

{quote}
*Importing a Template*
After receiving a Template that has been exported from another NiFi, the first 
step needed to use the template is to import the template into this instance of 
NiFi. From the Template Management dialog (see Managing Templates), click the 
“Browse” button. This provides a File Selection dialog that allows the user to 
choose which template file to upload. Select the file and click Open. The 
“Browse” button is replaced by an “Import” button and a “Cancel” button. 
Clicking the “Import” button will attempt to import the Template into this 
instance of NiFi. If unable to import the Template, an error message will 
appear in red next to the “Browse” button. Otherwise, the Template will be 
added to the table and the “Browse” button will reappear.
{quote}


> Update Documentation for Templates
> --
>
> Key: NIFI-2536
> URL: https://issues.apache.org/jira/browse/NIFI-2536
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: James Wing
>Priority: Minor
>
> The documentation for importing a template does not reflect recent UI changes 
> for 1.0.  Required updates include:
> * Template features moved to the Operate Palette (create, upload)
> ** Association between template actions and current process group permissions
> * Template features still in the Templates dialog (list, filter, export, 
> delete)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2536) Update Documentation for Templates

[James Wing (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Wing updated NIFI-2536:
-
Summary: Update Documentation for Templates  (was: Update Documentation for 
Importing a Template)

> Update Documentation for Templates
> --
>
> Key: NIFI-2536
> URL: https://issues.apache.org/jira/browse/NIFI-2536
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: James Wing
>Priority: Minor
>
> The documentation for importing a template does not reflect recent UI changes 
> for 1.0.  The current User Guide text reflects uploading templates from the 
> Template Management dialog.  It should describe the new Upload Template 
> button on the Actions Toolbar (or "Operate Palette"?).
> {quote}
> *Importing a Template*
> After receiving a Template that has been exported from another NiFi, the 
> first step needed to use the template is to import the template into this 
> instance of NiFi. From the Template Management dialog (see Managing 
> Templates), click the “Browse” button. This provides a File Selection dialog 
> that allows the user to choose which template file to upload. Select the file 
> and click Open. The “Browse” button is replaced by an “Import” button and a 
> “Cancel” button. Clicking the “Import” button will attempt to import the 
> Template into this instance of NiFi. If unable to import the Template, an 
> error message will appear in red next to the “Browse” button. Otherwise, the 
> Template will be added to the table and the “Browse” button will reappear.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2546) NPE while creating a template

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416120#comment-15416120
 ] 

Joseph Witt commented on NIFI-2546:
---

[~markap14] and [~ozhurakousky] related to recent commit for NIFI-2540?

> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Joseph Witt
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150] 
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at 
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
>  ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
>  ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
>  ~[classes/:na]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
>  ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
>  ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
>  ~[classes/:1.0.0-SNAPSHOT]
> at 
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
>  ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[na:1.8.0_66]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>  ~[jersey-server-1.19.jar:1.19]
> at 

[jira] [Created] (NIFI-2546) NPE while creating a template

[Joseph Witt (JIRA)]

Joseph Witt created NIFI-2546:
-

 Summary: NPE while creating a template
 Key: NIFI-2546
 URL: https://issues.apache.org/jira/browse/NIFI-2546
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Reporter: Joseph Witt
Priority: Critical


Attempted to create a template on latest master.

{quote}
2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150] 
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
java.lang.NullPointerException. Returning Internal Server Error response.
java.lang.NullPointerException: null
at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
at 
org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
 ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
 ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at 
org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
 ~[classes/:na]
at 
org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
 ~[classes/:1.0.0-SNAPSHOT]
at 
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
 ~[classes/:1.0.0-SNAPSHOT]
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
 ~[classes/:1.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[na:1.8.0_66]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[na:1.8.0_66]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[na:1.8.0_66]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
 ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at 
org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
 ~[classes/:1.0.0-SNAPSHOT]
at 
org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
 ~[classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[na:1.8.0_66]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[na:1.8.0_66]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[na:1.8.0_66]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
at 
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
 ~[jersey-server-1.19.jar:1.19]
at 
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
 ~[jersey-server-1.19.jar:1.19]
at 
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
 ~[jersey-server-1.19.jar:1.19]
at 

[GitHub] nifi issue #833: Add String Escape Utils to Nifi Expression Language (redo)

[devin-fisher]

Github user devin-fisher commented on the issue:

https://github.com/apache/nifi/pull/833
  
This a redo for PR 777


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416080#comment-15416080
 ] 

Joseph Witt commented on NIFI-2519:
---

[~ozhurakousky] I think listening port should default to 25, no?

> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #777: Add String Escape Utils to Nifi Expression Language

[devin-fisher]

Github user devin-fisher commented on the issue:

https://github.com/apache/nifi/pull/777
  
Re-did the PR at PR #833


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416073#comment-15416073
 ] 

Joseph Witt commented on NIFI-2519:
---

[~ozhurakousky] Looks like some healthy changes to stabilize the processor and 
importantly the tests as well which were making the build brittle.  I think 
we'll need to account for the attributes that [~trixpan] had been nicely 
extracting.  I think the other processors that now exist can be used after 
ListenSMTP instead to capture those attributes if needed but perhaps there are 
generic attributes we can grab that aren't protocol specific?

> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #777: Add String Escape Utils to Nifi Expression Language

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/777
  
Either is good, a new PR on a new branch would just get you in the habit of 
using a "feature branch" for development versus the local master branch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2523) Add an "Undo" to the User Interface

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-2523:
--
Fix Version/s: (was: 1.1.0)

> Add an "Undo" to the User Interface
> ---
>
> Key: NIFI-2523
> URL: https://issues.apache.org/jira/browse/NIFI-2523
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Reporter: Alexander Kell
>  Labels: usability
>
> I am not sure whether this feature has been forgotten ... 
> As a NiFi user, I'd like to be able to quickly undo an action, or series of 
> actions, in case I accidentally delete or move one or more processors while 
> editing them.
> This feature is nessesary to work with nifi alot more effectivly



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #777: Add String Escape Utils to Nifi Expression Language

[devin-fisher]

Github user devin-fisher commented on a diff in the pull request:

https://github.com/apache/nifi/pull/777#discussion_r74333276
  
--- Diff: nifi-docs/src/main/asciidoc/expression-language-guide.adoc ---
@@ -993,6 +954,229 @@ Expressions will provide the following results:
 
 
 
+[[escape]]
--- End diff --

That could be a good idea.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2544) Create integration tests for Zero-Master Clustering

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416035#comment-15416035
 ] 

ASF GitHub Bot commented on NIFI-2544:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/832

NIFI-2544: Created integration tests for clustering and addressed a f…

…ew minor bugs that were found in doing so

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2544

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/832.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #832


commit 944543ed3bfb17166648d0db85283ca262c450de
Author: Mark Payne 
Date:   2016-08-09T20:50:04Z

NIFI-2544: Created integration tests for clustering and addressed a few 
minor bugs that were found in doing so




> Create integration tests for Zero-Master Clustering
> ---
>
> Key: NIFI-2544
> URL: https://issues.apache.org/jira/browse/NIFI-2544
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> With the new Zero-Master Clustering, we have been testing a lot using manual 
> testing, but we should have automated integration tests, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #832: NIFI-2544: Created integration tests for clustering ...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/832

NIFI-2544: Created integration tests for clustering and addressed a f…

…ew minor bugs that were found in doing so

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2544

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/832.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #832


commit 944543ed3bfb17166648d0db85283ca262c450de
Author: Mark Payne 
Date:   2016-08-09T20:50:04Z

NIFI-2544: Created integration tests for clustering and addressed a few 
minor bugs that were found in doing so




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #777: Add String Escape Utils to Nifi Expression Language

[devin-fisher]

Github user devin-fisher commented on the issue:

https://github.com/apache/nifi/pull/777
  
Looks like my editor removes whitespace at the end of lines. So just clean 
up. Sorry, about thatI did not know it did that.

I'm still a little green with Git and GitHub PR model. I'll have to do some 
research into how to that.  I'll see what I can figure out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416008#comment-15416008
 ] 

Joseph Witt commented on NIFI-2519:
---

btw [~ozhurakousky] can you address these

[INFO] --- maven-checkstyle-plugin:2.15:check (check-style) @ 
nifi-email-processors ---
[WARNING] src/main/java/org/apache/nifi/processors/email/ListenSMTP.java[220] 
(javadoc) NonEmptyAtclauseDescription: At-clause should have a non-empty 
description.
[WARNING] src/main/java/org/apache/nifi/processors/email/ListenSMTP.java[221] 
(javadoc) NonEmptyAtclauseDescription: At-clause should have a non-empty 
description.
[INFO] 
[INFO] Reactor Summary:

> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
>
> While running a full NiFi parallel build received the following. So there is 
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
>  [deprecation] stop() in Thread has been deprecated
> [WARNING] 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
>  [deprecation] stop() in Thread has been deprecated
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ 
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO] 
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ 
> nifi-email-processors ---
> [INFO] Surefire report directory: 
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider 
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< 
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP)  Time elapsed: 
> 0.038 sec  <<< FAILURE!
> java.lang.AssertionError: Sending email failed
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertFalse(Assert.java:64)
>   at 
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in 
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - 
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests: 
>   TestListenSMTP.ValidEmail:188 Sending email failed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2544) Create integration tests for Zero-Master Clustering

[Mark Payne (JIRA)]

Mark Payne created NIFI-2544:


 Summary: Create integration tests for Zero-Master Clustering
 Key: NIFI-2544
 URL: https://issues.apache.org/jira/browse/NIFI-2544
 Project: Apache NiFi
  Issue Type: Task
  Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
 Fix For: 1.0.0


With the new Zero-Master Clustering, we have been testing a lot using manual 
testing, but we should have automated integration tests, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2460) Add StringEscapeUtils function to Nifi Expression Language

[Matt Burgess (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415990#comment-15415990
 ] 

Matt Burgess commented on NIFI-2460:


Absolutely! This looks great, I volunteer to review :)

> Add StringEscapeUtils function to Nifi Expression Language
> --
>
> Key: NIFI-2460
> URL: https://issues.apache.org/jira/browse/NIFI-2460
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Devin Fisher
>
> When using Nifi Expression Language to build up a transport format (like JSON 
> or XML) there is no go way to make sure that it has been properly escaped.  
> Example Use Case:
> When using PutHTTP to a Web Service that requires JSON input.  If the 
> incoming data could contain String with quotes or other disallow character in 
> JSON there is no reliable way to Escape the inputs as you put them into a 
> larger JSON object.
> Proposed Change: 
> StringEscapeUtils is part of apache commons-lang which is already a 
> dependency for nifi base POM. The functions in StringEscapeUtils can easily 
> be mapped into functions in nifi expression language.
> Work Done:
> I have the work done on a fork and am ready to submit a PR. That PR should 
> get good review since it is my first contribution (outside of a fixed typo) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2460) Add StringEscapeUtils function to Nifi Expression Language

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-2460:
---
Status: Patch Available  (was: Open)

> Add StringEscapeUtils function to Nifi Expression Language
> --
>
> Key: NIFI-2460
> URL: https://issues.apache.org/jira/browse/NIFI-2460
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Devin Fisher
>
> When using Nifi Expression Language to build up a transport format (like JSON 
> or XML) there is no go way to make sure that it has been properly escaped.  
> Example Use Case:
> When using PutHTTP to a Web Service that requires JSON input.  If the 
> incoming data could contain String with quotes or other disallow character in 
> JSON there is no reliable way to Escape the inputs as you put them into a 
> larger JSON object.
> Proposed Change: 
> StringEscapeUtils is part of apache commons-lang which is already a 
> dependency for nifi base POM. The functions in StringEscapeUtils can easily 
> be mapped into functions in nifi expression language.
> Work Done:
> I have the work done on a fork and am ready to submit a PR. That PR should 
> get good review since it is my first contribution (outside of a fixed typo) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2541) Update Multi-tenant authorization doc for UI change to Policy page

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415971#comment-15415971
 ] 

ASF subversion and git services commented on NIFI-2541:
---

Commit 916292994c6253e5be673804abb2e659ff755f5d in nifi's branch 
refs/heads/master from [~andrewmlim]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9162929 ]

NIFI-2541 This closes #831. Update Multi-tenant authorization doc for UI change 
to Policy page


> Update Multi-tenant authorization doc for UI change to Policy page
> --
>
> Key: NIFI-2541
> URL: https://issues.apache.org/jira/browse/NIFI-2541
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
> Fix For: 1.0.0
>
>
> Updating doc for NIFI-2475.  Want to make it clear to user that view/modify 
> policies do not support inheritance override.
> Will add to end of  Access Policy Inheritance section:
> Note:   “View the policies” and “modify the policies” component-level access 
> policies are an exception to this inherited behavior.  When a user is added 
> to either policy, they are added to the current list of administrators.  They 
> do not override higher level administrators.  For this reason, only component 
> specific administrators are displayed for the “view the policies” and “modify 
> the policies" access policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2541) Update Multi-tenant authorization doc for UI change to Policy page

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415973#comment-15415973
 ] 

ASF GitHub Bot commented on NIFI-2541:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/831


> Update Multi-tenant authorization doc for UI change to Policy page
> --
>
> Key: NIFI-2541
> URL: https://issues.apache.org/jira/browse/NIFI-2541
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
> Fix For: 1.0.0
>
>
> Updating doc for NIFI-2475.  Want to make it clear to user that view/modify 
> policies do not support inheritance override.
> Will add to end of  Access Policy Inheritance section:
> Note:   “View the policies” and “modify the policies” component-level access 
> policies are an exception to this inherited behavior.  When a user is added 
> to either policy, they are added to the current list of administrators.  They 
> do not override higher level administrators.  For this reason, only component 
> specific administrators are displayed for the “view the policies” and “modify 
> the policies" access policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2460) Add StringEscapeUtils function to Nifi Expression Language

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415949#comment-15415949
 ] 

Joseph Witt commented on NIFI-2460:
---

This looks pretty cool.  Haven't checked details but if we can do this it would 
be helpful.  Anyone available to review for 1.0?

> Add StringEscapeUtils function to Nifi Expression Language
> --
>
> Key: NIFI-2460
> URL: https://issues.apache.org/jira/browse/NIFI-2460
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Devin Fisher
>
> When using Nifi Expression Language to build up a transport format (like JSON 
> or XML) there is no go way to make sure that it has been properly escaped.  
> Example Use Case:
> When using PutHTTP to a Web Service that requires JSON input.  If the 
> incoming data could contain String with quotes or other disallow character in 
> JSON there is no reliable way to Escape the inputs as you put them into a 
> larger JSON object.
> Proposed Change: 
> StringEscapeUtils is part of apache commons-lang which is already a 
> dependency for nifi base POM. The functions in StringEscapeUtils can easily 
> be mapped into functions in nifi expression language.
> Work Done:
> I have the work done on a fork and am ready to submit a PR. That PR should 
> get good review since it is my first contribution (outside of a fixed typo) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #777: Add String Escape Utils to Nifi Expression Language

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/777#discussion_r74322615
  
--- Diff: nifi-docs/src/main/asciidoc/expression-language-guide.adoc ---
@@ -993,6 +954,229 @@ Expressions will provide the following results:
 
 
 
+[[escape]]
--- End diff --

This section might better be called "Encode/Decode", since escaping is a 
form of encoding and the urlEncode/Decode functions have been moved here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-2543) UI - Refresh Controller Service Referencing Components

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-2543:
-

 Summary: UI - Refresh Controller Service Referencing Components
 Key: NIFI-2543
 URL: https://issues.apache.org/jira/browse/NIFI-2543
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Reporter: Matt Gilman
Priority: Critical
 Fix For: 1.0.0


After updating a Controller Service, we attempt to refresh all components that 
reference it. It appears that the UI is attempting to refresh a referencing 
component that the user does not have access to. This is resulting in a 
confusing error message to the user as it appears the save was unsuccessful. 

The updating should be done more selectively or through an endpoint that 
returns the services in a filtered form (like listing all services).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2449) Need to document Variable Registry work in 1.0

[Andrew Lim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lim updated NIFI-2449:
-
Description: Need to capture work done in NIFI-2208.  Admin and Getting 
Started guides should be updated.  (was: Need to capture work done in 
NIFI-2208.  Admin and User Guides should be updated.)

> Need to document Variable Registry work in 1.0
> --
>
> Key: NIFI-2449
> URL: https://issues.apache.org/jira/browse/NIFI-2449
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
> Fix For: 1.0.0
>
>
> Need to capture work done in NIFI-2208.  Admin and Getting Started guides 
> should be updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2542) Revisions for Controller Service Referencing Components

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-2542:
-

 Summary: Revisions for Controller Service Referencing Components
 Key: NIFI-2542
 URL: https://issues.apache.org/jira/browse/NIFI-2542
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework, Core UI
Reporter: Matt Gilman
Priority: Critical
 Fix For: 1.0.0


The revisions for components that reference controller services transitively 
are not available when generating the response DTOs.

Additionally, it doesn't appear that the transitive references are being 
returned or rendered correctly. It is unclear which is the case due to the 
issue described above.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #831: NIFI-2541 Update Multi-tenant authorization doc for ...

[andrewmlim]

GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi/pull/831

NIFI-2541 Update Multi-tenant authorization doc for UI change to Poli…

…cy page

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi NIFI-2541

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/831.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #831


commit 01ad4940d358b6ed62b04ec93b9ad8c2577c8d87
Author: Andrew Lim 
Date:   2016-08-10T19:25:25Z

NIFI-2541 Update Multi-tenant authorization doc for UI change to Policy page




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2531) SQL-to-Avro processors do not convert BIGINT correctly

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-2531:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> SQL-to-Avro processors do not convert BIGINT correctly
> --
>
> Key: NIFI-2531
> URL: https://issues.apache.org/jira/browse/NIFI-2531
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> For the SQL to Avro processors that use JdbcCommon (such as ExecuteSQL), if a 
> BigInteger object is being put into an Avro record, it is being put in as a 
> String. However when the Avro schema is created and the SQL type of the 
> column is BIGINT, the schema contains the expected type "long" (actually a 
> union between null and long to allow for null values). This causes errors 
> such as:
> UnresolvedUnionException: not in union: ["null", "long"]
> If a BigInteger is retrieved from the result set and the SQL type is BIGINT, 
> then its value is expected to fit into 8 bytes and should thus be converted 
> to a long before storing in the Avro record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2531) SQL-to-Avro processors do not convert BIGINT correctly

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415828#comment-15415828
 ] 

ASF subversion and git services commented on NIFI-2531:
---

Commit d544274881f738d387ffad51b85c344a8f097238 in nifi's branch 
refs/heads/master from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d544274 ]

NIFI-2531: Fixed JdbcCommon handling of BigInteger objects for Avro

This closes #823.

Signed-off-by: Bryan Bende 


> SQL-to-Avro processors do not convert BIGINT correctly
> --
>
> Key: NIFI-2531
> URL: https://issues.apache.org/jira/browse/NIFI-2531
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> For the SQL to Avro processors that use JdbcCommon (such as ExecuteSQL), if a 
> BigInteger object is being put into an Avro record, it is being put in as a 
> String. However when the Avro schema is created and the SQL type of the 
> column is BIGINT, the schema contains the expected type "long" (actually a 
> union between null and long to allow for null values). This causes errors 
> such as:
> UnresolvedUnionException: not in union: ["null", "long"]
> If a BigInteger is retrieved from the result set and the SQL type is BIGINT, 
> then its value is expected to fit into 8 bytes and should thus be converted 
> to a long before storing in the Avro record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2531) SQL-to-Avro processors do not convert BIGINT correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415829#comment-15415829
 ] 

ASF GitHub Bot commented on NIFI-2531:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/823


> SQL-to-Avro processors do not convert BIGINT correctly
> --
>
> Key: NIFI-2531
> URL: https://issues.apache.org/jira/browse/NIFI-2531
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> For the SQL to Avro processors that use JdbcCommon (such as ExecuteSQL), if a 
> BigInteger object is being put into an Avro record, it is being put in as a 
> String. However when the Avro schema is created and the SQL type of the 
> column is BIGINT, the schema contains the expected type "long" (actually a 
> union between null and long to allow for null values). This causes errors 
> such as:
> UnresolvedUnionException: not in union: ["null", "long"]
> If a BigInteger is retrieved from the result set and the SQL type is BIGINT, 
> then its value is expected to fit into 8 bytes and should thus be converted 
> to a long before storing in the Avro record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #823: NIFI-2531: Fixed JdbcCommon handling of BigInteger objects ...

[bbende]

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/823
  
Latest update looks good, nice work! Will merge to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2531) SQL-to-Avro processors do not convert BIGINT correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415826#comment-15415826
 ] 

ASF GitHub Bot commented on NIFI-2531:
--

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/823
  
Latest update looks good, nice work! Will merge to master


> SQL-to-Avro processors do not convert BIGINT correctly
> --
>
> Key: NIFI-2531
> URL: https://issues.apache.org/jira/browse/NIFI-2531
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> For the SQL to Avro processors that use JdbcCommon (such as ExecuteSQL), if a 
> BigInteger object is being put into an Avro record, it is being put in as a 
> String. However when the Avro schema is created and the SQL type of the 
> column is BIGINT, the schema contains the expected type "long" (actually a 
> union between null and long to allow for null values). This causes errors 
> such as:
> UnresolvedUnionException: not in union: ["null", "long"]
> If a BigInteger is retrieved from the result set and the SQL type is BIGINT, 
> then its value is expected to fit into 8 bytes and should thus be converted 
> to a long before storing in the Avro record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2541) Update Multi-tenant authorization doc for UI change to Policy page

[Andrew Lim (JIRA)]

Andrew Lim created NIFI-2541:


 Summary: Update Multi-tenant authorization doc for UI change to 
Policy page
 Key: NIFI-2541
 URL: https://issues.apache.org/jira/browse/NIFI-2541
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Documentation & Website
Affects Versions: 1.0.0
Reporter: Andrew Lim
Priority: Minor
 Fix For: 1.0.0


Updating doc for NIFI-2475.  Want to make it clear to user that view/modify 
policies do not support inheritance override.

Will add to end of  Access Policy Inheritance section:

Note:   “View the policies” and “modify the policies” component-level access 
policies are an exception to this inherited behavior.  When a user is added to 
either policy, they are added to the current list of administrators.  They do 
not override higher level administrators.  For this reason, only component 
specific administrators are displayed for the “view the policies” and “modify 
the policies" access policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2541) Update Multi-tenant authorization doc for UI change to Policy page

[Andrew Lim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lim reassigned NIFI-2541:


Assignee: Andrew Lim

> Update Multi-tenant authorization doc for UI change to Policy page
> --
>
> Key: NIFI-2541
> URL: https://issues.apache.org/jira/browse/NIFI-2541
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Affects Versions: 1.0.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
> Fix For: 1.0.0
>
>
> Updating doc for NIFI-2475.  Want to make it clear to user that view/modify 
> policies do not support inheritance override.
> Will add to end of  Access Policy Inheritance section:
> Note:   “View the policies” and “modify the policies” component-level access 
> policies are an exception to this inherited behavior.  When a user is added 
> to either policy, they are added to the current list of administrators.  They 
> do not override higher level administrators.  For this reason, only component 
> specific administrators are displayed for the “view the policies” and “modify 
> the policies" access policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1663) Add support for ORC format

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-1663:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Add support for ORC format
> --
>
> Key: NIFI-1663
> URL: https://issues.apache.org/jira/browse/NIFI-1663
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> From the Hive/ORC wiki 
> (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC): 
> The Optimized Row Columnar (ORC) file format provides a highly efficient way 
> to store Hive data ... Using ORC files improves performance when Hive is 
> reading, writing, and processing data.
> As users are interested in NiFi integrations with Hive (NIFI-981, NIFI-1193, 
> etc.), NiFi should be able to support ORC file format to enable users to 
> efficiently store flow files for use by Hive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #830: NIFI-1663: Adding attribution for derived work for O...

[mattyb149]

Github user mattyb149 closed the pull request at:

https://github.com/apache/nifi/pull/830


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1663) Add support for ORC format

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415775#comment-15415775
 ] 

ASF subversion and git services commented on NIFI-1663:
---

Commit 2938a5353afe29d43ed3275e7d84525d60b72ec8 in nifi's branch 
refs/heads/master from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2938a53 ]

NIFI-1663: Adding attribution for derived work for OrcFlowFileWriter


> Add support for ORC format
> --
>
> Key: NIFI-1663
> URL: https://issues.apache.org/jira/browse/NIFI-1663
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> From the Hive/ORC wiki 
> (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC): 
> The Optimized Row Columnar (ORC) file format provides a highly efficient way 
> to store Hive data ... Using ORC files improves performance when Hive is 
> reading, writing, and processing data.
> As users are interested in NiFi integrations with Hive (NIFI-981, NIFI-1193, 
> etc.), NiFi should be able to support ORC file format to enable users to 
> efficiently store flow files for use by Hive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1663) Add support for ORC format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415776#comment-15415776
 ] 

ASF GitHub Bot commented on NIFI-1663:
--

Github user mattyb149 closed the pull request at:

https://github.com/apache/nifi/pull/830


> Add support for ORC format
> --
>
> Key: NIFI-1663
> URL: https://issues.apache.org/jira/browse/NIFI-1663
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> From the Hive/ORC wiki 
> (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC): 
> The Optimized Row Columnar (ORC) file format provides a highly efficient way 
> to store Hive data ... Using ORC files improves performance when Hive is 
> reading, writing, and processing data.
> As users are interested in NiFi integrations with Hive (NIFI-981, NIFI-1193, 
> etc.), NiFi should be able to support ORC file format to enable users to 
> efficiently store flow files for use by Hive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2535) SSL context service property in Publish/ConsumeKafka is causing clustering issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415719#comment-15415719
 ] 

ASF GitHub Bot commented on NIFI-2535:
--

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/829
  
Reviewing...


> SSL context service property in Publish/ConsumeKafka is causing clustering 
> issue 
> -
>
> Key: NIFI-2535
> URL: https://issues.apache.org/jira/browse/NIFI-2535
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Haimo Liu
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.0.0
>
>
> NIFI build Aug.9th
> I have a flow with the new kafka processors (publish, consume). it seems like 
> the newly added SSL context service property is causing clustering issue (one 
> of the nodes cannot join cluster), logs below (tried to copy the flow.xml.gz 
> file from node 1 to the other nodes so that they are completely identical, 
> doesn't fix the issue):
> 2016-08-09 16:52:38,893 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator Resetting cluster node statuses from 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=8], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=4]} to 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=30], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Primary Node], 
> updateId=32]}
> 2016-08-09 16:52:40,071 ERROR [Reconnect to Cluster] 
> o.a.nifi.controller.StandardFlowService Handling reconnection request failed 
> due to: org.apache.nifi.controller.UninheritableFlowException: Failed to 
> connect node to cluster because local flow is different than cluster flow.
> org.apache.nifi.controller.UninheritableFlowException: Failed to connect node 
> to cluster because local flow is different than cluster flow.
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:863)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.handleReconnectionRequest(StandardFlowService.java:589)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.access$300(StandardFlowService.java:97)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService$2.run(StandardFlowService.java:401)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed 
> configuration is not inheritable by the flow controller because of flow 
> differences: Found difference in Flows:
> Local Fingerprint:   
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEssl.context.serviceNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.pr
> Cluster Fingerprint: 
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.processors.standard.GenerateF
>   at 
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:240)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1429)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:81)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:668)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:839)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   ... 4 common frames omitted
> 2016-08-09 16:52:40,071 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator localhost:8891 requested disconnection 
> from cluster due to org.apache.nifi.controller.UninheritableFlowException: 
> Failed to connect node to cluster because local flow is different than 
> cluster flow.
> 2016-08-09 16:52:40,071 INFO [Reconnect to Cluster] 
> 

[GitHub] nifi issue #829: NIFI-2535: Do not include properties that are unset in flow...

[bbende]

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/829
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1663) Add support for ORC format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415712#comment-15415712
 ] 

ASF GitHub Bot commented on NIFI-1663:
--

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/830
  
+1


> Add support for ORC format
> --
>
> Key: NIFI-1663
> URL: https://issues.apache.org/jira/browse/NIFI-1663
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> From the Hive/ORC wiki 
> (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC): 
> The Optimized Row Columnar (ORC) file format provides a highly efficient way 
> to store Hive data ... Using ORC files improves performance when Hive is 
> reading, writing, and processing data.
> As users are interested in NiFi integrations with Hive (NIFI-981, NIFI-1193, 
> etc.), NiFi should be able to support ORC file format to enable users to 
> efficiently store flow files for use by Hive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #830: NIFI-1663: Adding attribution for derived work for OrcFlowF...

[olegz]

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/830
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #823: NIFI-2531: Fixed JdbcCommon handling of BigInteger objects ...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/823
  
Good point, let me see if I can use the column width (for BIGINT types) to 
determine whether to expect a long or string in the value.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1663) Add support for ORC format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415700#comment-15415700
 ] 

ASF GitHub Bot commented on NIFI-1663:
--

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/830

NIFI-1663: Adding attribution for derived work for OrcFlowFileWriter



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi old_orc

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/830.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #830


commit d30f5afb342f8c4226147d6e3da1f59f787410d9
Author: Matt Burgess 
Date:   2016-08-10T18:11:15Z

NIFI-1663: Adding attribution for derived work for OrcFlowFileWriter




> Add support for ORC format
> --
>
> Key: NIFI-1663
> URL: https://issues.apache.org/jira/browse/NIFI-1663
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.0.0
>
>
> From the Hive/ORC wiki 
> (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC): 
> The Optimized Row Columnar (ORC) file format provides a highly efficient way 
> to store Hive data ... Using ORC files improves performance when Hive is 
> reading, writing, and processing data.
> As users are interested in NiFi integrations with Hive (NIFI-981, NIFI-1193, 
> etc.), NiFi should be able to support ORC file format to enable users to 
> efficiently store flow files for use by Hive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2535) SSL context service property in Publish/ConsumeKafka is causing clustering issue

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415677#comment-15415677
 ] 

ASF GitHub Bot commented on NIFI-2535:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/829

NIFI-2535: Do not include properties that are unset in flow fingerpri…

…nt. This allows a new property to be added to a processor without 
affecting the fingerprint, if the value is never set

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2535

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/829.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #829


commit f676eaaee2ba98361fac2d10ef524eb9d9335bdf
Author: Mark Payne 
Date:   2016-08-10T17:57:45Z

NIFI-2535: Do not include properties that are unset in flow fingerprint. 
This allows a new property to be added to a processor without affecting the 
fingerprint, if the value is never set




> SSL context service property in Publish/ConsumeKafka is causing clustering 
> issue 
> -
>
> Key: NIFI-2535
> URL: https://issues.apache.org/jira/browse/NIFI-2535
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Haimo Liu
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.0.0
>
>
> NIFI build Aug.9th
> I have a flow with the new kafka processors (publish, consume). it seems like 
> the newly added SSL context service property is causing clustering issue (one 
> of the nodes cannot join cluster), logs below (tried to copy the flow.xml.gz 
> file from node 1 to the other nodes so that they are completely identical, 
> doesn't fix the issue):
> 2016-08-09 16:52:38,893 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator Resetting cluster node statuses from 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=8], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=4]} to 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=30], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Primary Node], 
> updateId=32]}
> 2016-08-09 16:52:40,071 ERROR [Reconnect to Cluster] 
> o.a.nifi.controller.StandardFlowService Handling reconnection request failed 
> due to: org.apache.nifi.controller.UninheritableFlowException: Failed to 
> connect node to cluster because local flow is different than cluster flow.
> org.apache.nifi.controller.UninheritableFlowException: Failed to connect node 
> to cluster because local flow is different than cluster flow.
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:863)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.handleReconnectionRequest(StandardFlowService.java:589)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.access$300(StandardFlowService.java:97)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService$2.run(StandardFlowService.java:401)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed 
> configuration is not inheritable by the flow controller because of flow 
> differences: Found difference in Flows:
> Local Fingerprint:   
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEssl.context.serviceNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.pr
> Cluster Fingerprint: 
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.processors.standard.GenerateF
>   at 
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:240)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1429)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> 

[jira] [Updated] (NIFI-2535) SSL context service property in Publish/ConsumeKafka is causing clustering issue

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne updated NIFI-2535:
-
Status: Patch Available  (was: Open)

> SSL context service property in Publish/ConsumeKafka is causing clustering 
> issue 
> -
>
> Key: NIFI-2535
> URL: https://issues.apache.org/jira/browse/NIFI-2535
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Haimo Liu
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.0.0
>
>
> NIFI build Aug.9th
> I have a flow with the new kafka processors (publish, consume). it seems like 
> the newly added SSL context service property is causing clustering issue (one 
> of the nodes cannot join cluster), logs below (tried to copy the flow.xml.gz 
> file from node 1 to the other nodes so that they are completely identical, 
> doesn't fix the issue):
> 2016-08-09 16:52:38,893 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator Resetting cluster node statuses from 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=8], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=4]} to 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=30], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Primary Node], 
> updateId=32]}
> 2016-08-09 16:52:40,071 ERROR [Reconnect to Cluster] 
> o.a.nifi.controller.StandardFlowService Handling reconnection request failed 
> due to: org.apache.nifi.controller.UninheritableFlowException: Failed to 
> connect node to cluster because local flow is different than cluster flow.
> org.apache.nifi.controller.UninheritableFlowException: Failed to connect node 
> to cluster because local flow is different than cluster flow.
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:863)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.handleReconnectionRequest(StandardFlowService.java:589)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.access$300(StandardFlowService.java:97)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService$2.run(StandardFlowService.java:401)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed 
> configuration is not inheritable by the flow controller because of flow 
> differences: Found difference in Flows:
> Local Fingerprint:   
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEssl.context.serviceNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.pr
> Cluster Fingerprint: 
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.processors.standard.GenerateF
>   at 
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:240)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1429)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:81)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:668)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:839)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   ... 4 common frames omitted
> 2016-08-09 16:52:40,071 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator localhost:8891 requested disconnection 
> from cluster due to org.apache.nifi.controller.UninheritableFlowException: 
> Failed to connect node to cluster because local flow is different than 
> cluster flow.
> 2016-08-09 16:52:40,071 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator Status of localhost:8891 changed from 
> NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33] to 

[GitHub] nifi pull request #829: NIFI-2535: Do not include properties that are unset ...

[markap14]

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/829

NIFI-2535: Do not include properties that are unset in flow fingerpri…

…nt. This allows a new property to be added to a processor without 
affecting the fingerprint, if the value is never set

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-2535

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/829.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #829


commit f676eaaee2ba98361fac2d10ef524eb9d9335bdf
Author: Mark Payne 
Date:   2016-08-10T17:57:45Z

NIFI-2535: Do not include properties that are unset in flow fingerprint. 
This allows a new property to be added to a processor without affecting the 
fingerprint, if the value is never set




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2540) Templates should be scrubbed for Property Descriptors' descriptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415620#comment-15415620
 ] 

ASF GitHub Bot commented on NIFI-2540:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/828


> Templates should be scrubbed for Property Descriptors' descriptions
> ---
>
> Key: NIFI-2540
> URL: https://issues.apache.org/jira/browse/NIFI-2540
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> In NIFI-2150, we removed a lot of unused information from templates. The 
> Property Descriptors, in particular, were removed. However, these were later 
> added back in, as it was necessary to have the 'Identifies Controller 
> Service' flag. Bringing this back in, however, brings in the descriptions 
> which are very verbose, unused, and may change over time, which would cause 
> problems with the deterministic templating effort. We should remove 
> everything from the Property Descriptors except for the name and whether or 
> not the property identifies a controller service.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2540) Templates should be scrubbed for Property Descriptors' descriptions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415618#comment-15415618
 ] 

ASF subversion and git services commented on NIFI-2540:
---

Commit 85877a73dc454a4b7f2b567517d345332ccb3b52 in nifi's branch 
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=85877a7 ]

NIFI-2540: Exclude from templates the parts of property descriptors that are 
not necessary. Also ensure that Property Descriptors are not completely removed 
from Controller Services.

This closes #828


> Templates should be scrubbed for Property Descriptors' descriptions
> ---
>
> Key: NIFI-2540
> URL: https://issues.apache.org/jira/browse/NIFI-2540
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> In NIFI-2150, we removed a lot of unused information from templates. The 
> Property Descriptors, in particular, were removed. However, these were later 
> added back in, as it was necessary to have the 'Identifies Controller 
> Service' flag. Bringing this back in, however, brings in the descriptions 
> which are very verbose, unused, and may change over time, which would cause 
> problems with the deterministic templating effort. We should remove 
> everything from the Property Descriptors except for the name and whether or 
> not the property identifies a controller service.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2540) Templates should be scrubbed for Property Descriptors' descriptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415617#comment-15415617
 ] 

ASF GitHub Bot commented on NIFI-2540:
--

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/828
  
Looks pretty straight forward. Validated that it did not break the 
correctness of template import/export. Merging


> Templates should be scrubbed for Property Descriptors' descriptions
> ---
>
> Key: NIFI-2540
> URL: https://issues.apache.org/jira/browse/NIFI-2540
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> In NIFI-2150, we removed a lot of unused information from templates. The 
> Property Descriptors, in particular, were removed. However, these were later 
> added back in, as it was necessary to have the 'Identifies Controller 
> Service' flag. Bringing this back in, however, brings in the descriptions 
> which are very verbose, unused, and may change over time, which would cause 
> problems with the deterministic templating effort. We should remove 
> everything from the Property Descriptors except for the name and whether or 
> not the property identifies a controller service.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #828: NIFI-2540: Exclude from templates the parts of property des...

[olegz]

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/828
  
Looks pretty straight forward. Validated that it did not break the 
correctness of template import/export. Merging


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-2514) Update documentation on handling of Controller Services

[Sarah Olson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarah Olson reassigned NIFI-2514:
-

Assignee: Sarah Olson

> Update documentation on handling of Controller Services
> ---
>
> Key: NIFI-2514
> URL: https://issues.apache.org/jira/browse/NIFI-2514
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Affects Versions: 1.0.0-Beta
>Reporter: Aldrin Piri
>Assignee: Sarah Olson
> Fix For: 1.0.0
>
>
> Controller services work as anticipated but the way they are structured is a 
> bit of a departure from how they used to work.  Would be helpful to have docs 
> that inform users of how each is structured.
> It seems that Flow Controller level controller services are only there for 
> other controller services and/or reporting tasks.
> These are separate from controller services that are for an associated 
> process group (or parent) which are nested.
> {quote}
> The access/visibility of controller services seems a bit confusing to me and 
> don't see any docs that clarify. It seems that for nested process groups, I 
> can view all controller services in parent process groups.  The configuration 
> on the "Operate" palette seems to control these which seems to be completely 
> disjoint from the flow level controller settings, which are not listed in any 
> of the nested process groups.  I would imagine the flow level controller 
> settings would also be present in each of the child PGs regardless of nesting 
> level but these do not seem to be visible to any of my components.  Are these 
> just to support reporting tasks?  Would reporting tasks also have a similar 
> hierarchy where they could also be contained within a given PG chain?
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-1849) servers do not show up in user list

[Joe Skora (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Skora resolved NIFI-1849.
-
Resolution: Not A Bug
  Assignee: Joe Skora

This was caused by a timing difference between when corrections were made to 
the REST API endpoint URI and when the corresponding client code was updated.  
Older systems could talk to each other as could newer systems, but a mix of 
older and newer would cause this behavior which is resolved in the current 0.x 
and 1.x branches. 

> servers do not show up in user list
> ---
>
> Key: NIFI-1849
> URL: https://issues.apache.org/jira/browse/NIFI-1849
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.5.1
>Reporter: Ben Icore
>Assignee: Joe Skora
>
> when new server attempts to make a site-to-site connection, the sending 
> server does not show up in the recieving servers user list.  The only way to 
> authorize the new server is to add it to the authorized user file and restart 
> the receiving instance



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2535) SSL context service property in Publish/ConsumeKafka is causing clustering issue

[Mark Payne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Payne reassigned NIFI-2535:


Assignee: Mark Payne

> SSL context service property in Publish/ConsumeKafka is causing clustering 
> issue 
> -
>
> Key: NIFI-2535
> URL: https://issues.apache.org/jira/browse/NIFI-2535
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Haimo Liu
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.0.0
>
>
> NIFI build Aug.9th
> I have a flow with the new kafka processors (publish, consume). it seems like 
> the newly added SSL context service property is causing clustering issue (one 
> of the nodes cannot join cluster), logs below (tried to copy the flow.xml.gz 
> file from node 1 to the other nodes so that they are completely identical, 
> doesn't fix the issue):
> 2016-08-09 16:52:38,893 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator Resetting cluster node statuses from 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=8], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=4]} to 
> {localhost:8893=NodeConnectionStatus[state=CONNECTED, roles=[Cluster 
> Coordinator], updateId=30], 
> localhost:8891=NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33], 
> localhost:8892=NodeConnectionStatus[state=CONNECTED, roles=[Primary Node], 
> updateId=32]}
> 2016-08-09 16:52:40,071 ERROR [Reconnect to Cluster] 
> o.a.nifi.controller.StandardFlowService Handling reconnection request failed 
> due to: org.apache.nifi.controller.UninheritableFlowException: Failed to 
> connect node to cluster because local flow is different than cluster flow.
> org.apache.nifi.controller.UninheritableFlowException: Failed to connect node 
> to cluster because local flow is different than cluster flow.
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:863)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.handleReconnectionRequest(StandardFlowService.java:589)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.access$300(StandardFlowService.java:97)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService$2.run(StandardFlowService.java:401)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed 
> configuration is not inheritable by the flow controller because of flow 
> differences: Found difference in Flows:
> Local Fingerprint:   
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEssl.context.serviceNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.pr
> Cluster Fingerprint: 
> he.nifi.processors.kafka.pubsub.Partitioners$RoundRobinPartitionersasl.kerberos.service.nameNO_VALUEtopicWARNINGsuccess04015ca0-0156-1000--4224fe88org.apache.nifi.processors.standard.GenerateF
>   at 
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:240)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1429)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:81)
>  ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:668)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:839)
>  [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   ... 4 common frames omitted
> 2016-08-09 16:52:40,071 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator localhost:8891 requested disconnection 
> from cluster due to org.apache.nifi.controller.UninheritableFlowException: 
> Failed to connect node to cluster because local flow is different than 
> cluster flow.
> 2016-08-09 16:52:40,071 INFO [Reconnect to Cluster] 
> o.a.n.c.c.node.NodeClusterCoordinator Status of localhost:8891 changed from 
> NodeConnectionStatus[state=CONNECTING, roles=[], updateId=33] to 
> 

[jira] [Commented] (NIFI-2540) Templates should be scrubbed for Property Descriptors' descriptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415577#comment-15415577
 ] 

ASF GitHub Bot commented on NIFI-2540:
--

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/828
  
Reviewing.


> Templates should be scrubbed for Property Descriptors' descriptions
> ---
>
> Key: NIFI-2540
> URL: https://issues.apache.org/jira/browse/NIFI-2540
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> In NIFI-2150, we removed a lot of unused information from templates. The 
> Property Descriptors, in particular, were removed. However, these were later 
> added back in, as it was necessary to have the 'Identifies Controller 
> Service' flag. Bringing this back in, however, brings in the descriptions 
> which are very verbose, unused, and may change over time, which would cause 
> problems with the deterministic templating effort. We should remove 
> everything from the Property Descriptors except for the name and whether or 
> not the property identifies a controller service.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #828: NIFI-2540: Exclude from templates the parts of property des...

[olegz]

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/828
  
Reviewing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---