date:20180221

[jira] [Commented] (NIFI-4898) Remote Process Group in a SSL setup generates Java Exception

2018-02-21 Thread Josef Zahner (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372517#comment-16372517
 ] 

Josef Zahner commented on NIFI-4898:


You were right, I was using a lot of custom NARs/JARs. So I've removed them and 
finally it turns out that the Splunk JAR (splunk-library-javalogging-1.5.2) 
caused the issue. I'll investigate now why this happens. I was confused because 
with NiFi 1.4.0 it worked without any issues or at least without error messages 
in the log. To be honest I haven't checked whether Splunk gets the log 
messages. I'll try to investigate now where the error comes from. Thanks a lot!

> Remote Process Group in a SSL setup generates Java Exception
> 
>
> Key: NIFI-4898
> URL: https://issues.apache.org/jira/browse/NIFI-4898
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.5.0
> Environment: NiFi Version 1.5.0
> Java 1.8.0_161-b12
> CentOS Linux release 7.4.1708
>Reporter: Josef Zahner
>Priority: Major
>
> In a SSL secured NiFi setup, doesn't matter whether it is a cluster or not, 
> NiFi creates a Java exception as soon as I create a "Remote Process Group". 
> It doesn't mater which URL (even one which doesn't exists) I insert or if I 
> choose RAW or HTTP, the error is always the same and occurs as soon as I 
> click "add" on the "Remote Process Group".
> On NiFi 1.4.0 this works without any issues.
> Error:
> {code:java}
> 2018-02-21 10:42:10,006 ERROR [Remote Process Group 
> b7bde0cc-0161-1000-2e7f-3167a78d8386 Thread-1] 
> org.apache.nifi.engine.FlowEngine A flow controller task execution stopped 
> abnormally
> java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at java.util.concurrent.FutureTask.report(Unknown Source)
> at java.util.concurrent.FutureTask.get(Unknown Source)
> at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1237)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.fetchController(SiteToSiteRestApiClient.java:419)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:394)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:361)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:346)
> at 
> org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:842)
> at 
> org.apache.nifi.remote.StandardRemoteProcessGroup.lambda$initialize$0(StandardRemoteProcessGroup.java:193)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown
>  Source)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>  Source)
> ... 3 common frames omitted
> 2018-02-21 10:42:10,009 ERROR [Remote Process Group 
> b7bde0cc-0161-1000-2e7f-3167a78d8386 Thread-1] 
> org.apache.nifi.engine.FlowEngine A flow controller task execution stopped 
> abnormally
> java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at java.util.concurrent.FutureTask.report(Unknown Source)
> at java.util.concurrent.FutureTask.get(Unknown Source)
> at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at 
>

[jira] [Commented] (NIFI-4833) NIFI-4833 Add ScanHBase processor

2018-02-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371889#comment-16371889
 ] 

ASF GitHub Bot commented on NIFI-4833:
--

Github user bdesert commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2478#discussion_r169749383
  
--- Diff: 
nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/main/java/org/apache/nifi/hbase/ScanHBase.java
 ---
@@ -0,0 +1,562 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.hbase;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.hbase.io.JsonFullRowSerializer;
+import org.apache.nifi.hbase.io.JsonQualifierAndValueRowSerializer;
+import org.apache.nifi.hbase.io.RowSerializer;
+import org.apache.nifi.hbase.scan.Column;
+import org.apache.nifi.hbase.scan.ResultCell;
+import org.apache.nifi.hbase.scan.ResultHandler;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.Tuple;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.regex.Pattern;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"hbase", "scan", "fetch", "get"})
+@CapabilityDescription("Scans and fetches rows from an HBase table. This 
processor may be used to fetch rows from hbase table by specifying a range of 
rowkey values (start and/or end ),"
++ "by time range, by filter expression, or any combination of 
them. \n"
++ "Order of records can be controlled by a property 
Reversed"
++ "Number of rows retrieved by the processor can be limited.")
+@WritesAttributes({
+@WritesAttribute(attribute = "hbase.table", description = "The 
name of the HBase table that the row was fetched from"),
+@WritesAttribute(attribute = "hbase.resultset", description = "A 
JSON document/s representing the row/s. This property is only written when a 
Destination of flowfile-attributes is selected."),
+@WritesAttribute(attribute = "mime.type", description = "Set to 
application/json when using a Destination of flowfile-content, not set or 
modified otherwise"),
+@WritesAttribute(attribute = "hbase.rows.count", description = 
"Number of rows in the content of given flow file"),
+@WritesAttribute(attribute = "scanhbase.results.found", 
description = "Indicates whether at least one row has been found in given hbase 
table with provided conditions. "
++ "Could be null (not present) if transfered to FAILURE")
+})
+public class

[GitHub] nifi pull request #2478: NIFI-4833 Add scanHBase Processor

2018-02-21 Thread bdesert

Github user bdesert commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2478#discussion_r169749383
  
--- Diff: 
nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/main/java/org/apache/nifi/hbase/ScanHBase.java
 ---
@@ -0,0 +1,562 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.hbase;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.hbase.io.JsonFullRowSerializer;
+import org.apache.nifi.hbase.io.JsonQualifierAndValueRowSerializer;
+import org.apache.nifi.hbase.io.RowSerializer;
+import org.apache.nifi.hbase.scan.Column;
+import org.apache.nifi.hbase.scan.ResultCell;
+import org.apache.nifi.hbase.scan.ResultHandler;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.Tuple;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.regex.Pattern;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"hbase", "scan", "fetch", "get"})
+@CapabilityDescription("Scans and fetches rows from an HBase table. This 
processor may be used to fetch rows from hbase table by specifying a range of 
rowkey values (start and/or end ),"
++ "by time range, by filter expression, or any combination of 
them. \n"
++ "Order of records can be controlled by a property 
Reversed"
++ "Number of rows retrieved by the processor can be limited.")
+@WritesAttributes({
+@WritesAttribute(attribute = "hbase.table", description = "The 
name of the HBase table that the row was fetched from"),
+@WritesAttribute(attribute = "hbase.resultset", description = "A 
JSON document/s representing the row/s. This property is only written when a 
Destination of flowfile-attributes is selected."),
+@WritesAttribute(attribute = "mime.type", description = "Set to 
application/json when using a Destination of flowfile-content, not set or 
modified otherwise"),
+@WritesAttribute(attribute = "hbase.rows.count", description = 
"Number of rows in the content of given flow file"),
+@WritesAttribute(attribute = "scanhbase.results.found", 
description = "Indicates whether at least one row has been found in given hbase 
table with provided conditions. "
++ "Could be null (not present) if transfered to FAILURE")
+})
+public class ScanHBase extends AbstractProcessor {
+//enhanced regex for columns to allow "-" in column qualifier names
+static final Pattern COLUMNS_PATTERN = 
Pattern.compile("\\w+(:(\\w|-)+)?(?:,\\w+(:(\\w|-)+)?)*");
+static

[GitHub] nifi issue #2443: NIFI-4827 Added support for reading queries from the flowf...

2018-02-21 Thread MikeThomsen

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2443
  
@mattyb149 Passed the test and it says it's ready.


---

[GitHub] nifi issue #2416: NIFI 4774: Provide alternate implementation of Write-Ahead...

2018-02-21 Thread devriesb

Github user devriesb commented on the issue:

https://github.com/apache/nifi/pull/2416
  
@markap14 yes... under this somewhat unusual circumstance, my proposed 
solution would sacrifice data for consistency.  However, if alwaysSync is set 
false, there isn't a guarantee of no loss anyway.  So, we'd really be no worse 
off than we are, except that the plug getting kicked out of the wall wouldn't 
kill your repo.

Also, I fully agree that an implementation that DOES guarantee no loss of 
CREATEs would be for the best.  My proposal above doesn't address that in any 
way.  It lives inside the "all or nothing" guarantee of the current 
(/previous?) WriteAheadFFR.


---

[GitHub] nifi pull request #2478: NIFI-4833 Add scanHBase Processor

2018-02-21 Thread bdesert

Github user bdesert commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2478#discussion_r169732184
  
--- Diff: 
nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/main/java/org/apache/nifi/hbase/ScanHBase.java
 ---
@@ -0,0 +1,562 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.hbase;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.hbase.io.JsonFullRowSerializer;
+import org.apache.nifi.hbase.io.JsonQualifierAndValueRowSerializer;
+import org.apache.nifi.hbase.io.RowSerializer;
+import org.apache.nifi.hbase.scan.Column;
+import org.apache.nifi.hbase.scan.ResultCell;
+import org.apache.nifi.hbase.scan.ResultHandler;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.Tuple;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.regex.Pattern;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
--- End diff --

I was thinking about that before. Couldn't really decide, and then I took a 
look at DeleteHBaseRow and FetchHBaseRow and decided to keep it consistent.


---

[GitHub] nifi issue #2416: NIFI 4774: Provide alternate implementation of Write-Ahead...

2018-02-21 Thread devriesb

Github user devriesb commented on the issue:

https://github.com/apache/nifi/pull/2416
  
Again, I would submit that it isn't violating the data loss guarantees of 
the original implementation.  Yes, some data could potentially be lost, but 
none that it guaranteed to keep, and in doing so it prevents corruption... in a 
handful of easily understandable lines, that don't fundamentally alter the 
implementation.  

I want to be clear... I'm in no way opposed to improvements.  However, for 
operational data flows there is going to be some trepidation about large 
changes to such a key component... especially one that we've had previous 
issues with.  The smaller / more incremental the changes, and the more time 
allowed for evaluation under load, the more comfortable I would imagine people 
would be.


---

[jira] [Updated] (NIFIREG-147) Add Keycloak authentication method

2018-02-21 Thread Bryan Bende (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFIREG-147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFIREG-147:

Issue Type: Improvement  (was: Bug)

> Add Keycloak authentication method
> --
>
> Key: NIFIREG-147
> URL: https://issues.apache.org/jira/browse/NIFIREG-147
> Project: NiFi Registry
>  Issue Type: Improvement
>Reporter: Gregory Reshetniak
>Priority: Major
>
> Keycloak does implement a lot of related functionality, including groups, 
> users and such. It would be great to have first-class integration available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4775) Allow FlowFile Repository to optionally perform fsync when writing CREATE events but not other events

2018-02-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371874#comment-16371874
 ] 

ASF GitHub Bot commented on NIFI-4775:
--

Github user devriesb commented on the issue:

https://github.com/apache/nifi/pull/2416
  
I'll grant NIFI-4775 may raise issues with my proposed solution. However, 
there is a problem right now.  My proposed solution addresses the problem right 
now.  Future modification may require adjustments to previous assumptions.  
That, however, is a problem for the future.  

In any case, after doing some experimentation, I'm not sure the current 
version of NIFI-4775 is the correct approach.  And whatever the eventual 
approach is, it may more appropriately be a new implementation (as discussed 
above).  I don't think we should put off correcting current bugs because they 
may complicate potential future features.


> Allow FlowFile Repository to optionally perform fsync when writing CREATE 
> events but not other events
> -
>
> Key: NIFI-4775
> URL: https://issues.apache.org/jira/browse/NIFI-4775
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Reporter: Mark Payne
>Priority: Major
>
> Currently, when a FlowFile is written to the FlowFile Repository, the repo 
> can either fsync or not, depending on nifi.properties. We should allow a 
> third option, of fsync only for CREATE events. In this case, if we receive 
> new data from a source we can fsync the update to the FlowFile Repository 
> before ACK'ing the data from the source. This allows us to guarantee data 
> persistence without the overhead of an fsync for every FlowFile Repository 
> update.
> It may make sense, though, to be a bit more selective about when do this. For 
> example if the source is a system that does not allow us to acknowledge the 
> receipt of data, such as a ListenUDP processor, this doesn't really buy us 
> much. In such a case, we could be smart about avoiding the high cost of an 
> fsync. However, for something like GetSFTP where we have to remove the file 
> in order to 'acknowledge receipt' we can ensure that we wait for the fsync 
> before proceeding.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4898) Remote Process Group in a SSL setup generates Java Exception

2018-02-21 Thread Pierre Villard (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371958#comment-16371958
 ] 

Pierre Villard commented on NIFI-4898:
--

Hi [~jzahner], I'm not able to reproduce the issue on my side. Any custom NAR 
or non-default jars in /lib directory?

> Remote Process Group in a SSL setup generates Java Exception
> 
>
> Key: NIFI-4898
> URL: https://issues.apache.org/jira/browse/NIFI-4898
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.5.0
> Environment: NiFi Version 1.5.0
> Java 1.8.0_161-b12
> CentOS Linux release 7.4.1708
>Reporter: Josef Zahner
>Priority: Major
>
> In a SSL secured NiFi setup, doesn't matter whether it is a cluster or not, 
> NiFi creates a Java exception as soon as I create a "Remote Process Group". 
> It doesn't mater which URL (even one which doesn't exists) I insert or if I 
> choose RAW or HTTP, the error is always the same and occurs as soon as I 
> click "add" on the "Remote Process Group".
> On NiFi 1.4.0 this works without any issues.
> Error:
> {code:java}
> 2018-02-21 10:42:10,006 ERROR [Remote Process Group 
> b7bde0cc-0161-1000-2e7f-3167a78d8386 Thread-1] 
> org.apache.nifi.engine.FlowEngine A flow controller task execution stopped 
> abnormally
> java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at java.util.concurrent.FutureTask.report(Unknown Source)
> at java.util.concurrent.FutureTask.get(Unknown Source)
> at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1237)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.fetchController(SiteToSiteRestApiClient.java:419)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:394)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:361)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:346)
> at 
> org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:842)
> at 
> org.apache.nifi.remote.StandardRemoteProcessGroup.lambda$initialize$0(StandardRemoteProcessGroup.java:193)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown
>  Source)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>  Source)
> ... 3 common frames omitted
> 2018-02-21 10:42:10,009 ERROR [Remote Process Group 
> b7bde0cc-0161-1000-2e7f-3167a78d8386 Thread-1] 
> org.apache.nifi.engine.FlowEngine A flow controller task execution stopped 
> abnormally
> java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at java.util.concurrent.FutureTask.report(Unknown Source)
> at java.util.concurrent.FutureTask.get(Unknown Source)
> at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.NoSuchMethodError: 
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219)
> at 
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189)
> at 
>

[jira] [Commented] (NIFI-4899) Unable to find valid certification path to requested target

2018-02-21 Thread Pierre Villard (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371966#comment-16371966
 ] 

Pierre Villard commented on NIFI-4899:
--

This looks like as a truststore issue. Is it a cluster setup? How has SSL been 
enabled on the cluster, manually or using the toolkit?

Nevertheless, it's kind of weird this is happening only once after a NiFi 
restart... [~alopresto] may have an idea about it.

> Unable to find valid certification path to requested target
> ---
>
> Key: NIFI-4899
> URL: https://issues.apache.org/jira/browse/NIFI-4899
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.5.0
> Environment: NiFi Version 1.5.0 
> Java 1.8.0_161-b12 
> CentOS Linux release 7.4.1708
>Reporter: Josef Zahner
>Priority: Minor
>  Labels: certificate, login, ssl
> Attachments: Screen Shot 2018-02-21 at 11.08.13.png
>
>
> In my clustered ssl environment, if I start the webgui the first time, enter 
> my login credentials (verified via LDAP) and go ahead (click "LOG IN") I'm 
> getting the error below:
> !Screen Shot 2018-02-21 at 11.08.13.png!
> {code:java}
> javax.ws.rs.ProcessingException: javax.net.ssl.SSLHandshakeException: 
> sun.security.validator.ValidatorException: PKIX path building failed: 
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find 
> valid certification path to requested target
> at 
> org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:284)
> at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:278)
> at 
> org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:753)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:229)
> at 
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:414)
> at 
> org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:752)
> at 
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:661)
> at 
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:875)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: javax.net.ssl.SSLHandshakeException: 
> sun.security.validator.ValidatorException: PKIX path building failed: 
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find 
> valid certification path to requested target
> at sun.security.ssl.Alerts.getSSLException(Unknown Source)
> at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
> at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
> at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
> at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source)
> at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
> at sun.security.ssl.Handshaker.processLoop(Unknown Source)
> at sun.security.ssl.Handshaker.process_record(Unknown Source)
> at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
> at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
> at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
> at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
> at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
> at 
> sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown 
> Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
> at java.net.HttpURLConnection.getResponseCode(Unknown Source)
> at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown 
> Source)
> at 
> org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:390)
> at 
> org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:282)
> ... 14 common frames omitted
> Caused by: sun.security.validator.ValidatorException: PKIX path building 
> failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to 
> find valid certification path to requested target
> at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
> at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
> at

[GitHub] nifi issue #2416: NIFI 4774: Provide alternate implementation of Write-Ahead...

2018-02-21 Thread devriesb

Github user devriesb commented on the issue:

https://github.com/apache/nifi/pull/2416

I'm glad there's support for making this opt in. One point on @joewitt 's
comment : "The claim of a simple fix being available to close the previous gaps
doesn't appear to be backed with a suggested implementation though it does look
like you received a good response to why that wasn't feasible." ... the reasons
given were reasons why @markap14 's original proposed solution wouldn't work,
not why another solution might not. For example (as suggested previously in
email):

> the immediate fix that I see needing is that when building the
transaction map[1], we need to keep track of the *lowest* encountered corrupt
transaction. Then, when iterating over the transaction map to rebuild the repo,
we need to stop when that transaction is reached... after that we can't trust
that there are no missing transactions which could lead to repo corruption or
incorrect state.

This is the relatively simple fix that I believe would be more appropriate
to the WhriteAheadFlowFileRepository, as it would only prevent a known bug from
causing corruption, as opposed to a major rewrite. I never got any response
suggesting this wouldn't work, rather simply that another solution was
preferred... which turned out not to be feasible. At that point, instead of
circling back and trying to fix the bug, it appears as though the rewrite began.

If there's a reason the author of the original implementation doesn't
believe my suggestion would work to prevent the observed corruption, I'd be
happy to take another look.

[1]
https://github.com/apache/nifi/blob/0f2ac39f69c1a744f151f0d924c9978f6790b7f7/nifi-commons/nifi-write-ahead-log/src/main/java/org/wali/MinimalLockingWriteAheadLog.java#L444

1 2 >

1 - 100 of 153 matches

Mail list logo