Re: SSL_ERROR_NO_CYPHER_OVERLAP

[Phil H]

No worries Andy,

I have another issue - I'm sure it's a much simpler one, but it has me
stumped.

I still get the insufficient permissions error, even after adding myself to
the authorizers.xml file as per the admin guide.

I added the "EMAILADDRESS=" part of the subject after receiving an unkown
user error with that as part of the DN (checking the certificate and the
subject contains the email address part).

Output from nifi-user.log

2018-05-05 14:10:29,034 INFO [NiFi Web Server-21]
o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException:
Kerberos ticket login not supported by this NiFi.. Returning Conflict
response.
2018-05-05 14:10:29,249 INFO [NiFi Web Server-19]
o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException:
OpenId Connect is not configured.. Returning Conflict response.
2018-05-05 14:10:29,310 INFO [NiFi Web Server-76]
o.a.n.w.s.NiFiAuthenticationFilter Attempting request for
(EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah) GET
https://nifi1.blah/nifi-api/flow/current-user (source ip: 192.168.10.192)
2018-05-05 14:10:29,310 INFO [NiFi Web Server-76]
o.a.n.w.s.NiFiAuthenticationFilter Authentication success for
EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah
2018-05-05 14:10:29,312 INFO [NiFi Web Server-76]
o.a.n.w.a.c.AccessDeniedExceptionMapper identity[EMAILADDRESS=phil@blah,
CN=Phil H, OU=Blah], groups[] does not have permission to access the
requested resource. Unable to view the user interface. Returning Forbidden
response.

Partial authorizers.xml (the only changes I have made from the install are
in red)


file-user-group-provider
org.apache.nifi.authorization.FileUserGroupProvider
./conf/users.xml


EMAILADDRESS=phil@blah,
CN=Phil H, OU=Blah



file-access-policy-provider

org.apache.nifi.authorization.FileAccessPolicyProvider
file-user-group-provider
./conf/authorizations.xml
EMAILADDRESS=phil@blah,
CN=Phil H, OU=Blah













On Sat, May 5, 2018 at 11:35 AM, Andy LoPresto  wrote:

> Phil,
>
> Thanks for documenting the steps you took to get it working. This will
> definitely be helpful to anyone who has a similar problem in the future. I
> suspected it was the missing private key and I forgot to explain that yes,
> to import a PEM-encoded public key and private key into a JKS keystore, you
> do have to go through the PKCS12 intermediary.
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On May 4, 2018, at 4:36 PM, Phil H  wrote:
>
> Sorry for talking to myself so much.  I have solved the problem!
>
> Once I realised that I had never supplied the private key to keytool when I
> imported the certificate, I started googling along those lines.
>
> I found this link from 2008 which provided the solution
> http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html
>
> Basically you use OpenSSL to create a PKCS12 file from the cert and key,
> and then keytool to import that (as an existing keystore, not as a
> certificate)
>
> From there, everything else just worked (including client authentication
> through the browser).  Now I have the "Insufficient permissions" problem,
> which there is loads of documentation as to how to move forwards.
>
> Thanks!
> Phil
>
> On Sat, May 5, 2018 at 8:27 AM, Phil H  wrote:
>
> Further info
>
> In the httpd installation, I need the private key for the certificate and
> it's paraphrase. That private key has not played a part of the NiFi install
> thus far (seems like an important thing - the "keystore" only has a
> certificate, not a key)
>
> The reason why I used PKCS12 for the keystore was some Jetty debug output
> telling me that was preferred to JKS. Interestingly it gave me contrary
> advice for the trust store.
>
> On Sat, 5 May 2018 at 08:22, Phil H  wrote:
>
> Hi Andy,
>
> Sorry - poor use of words (it was late when I wrote the email). When I
> said "client" certificate I meant the certificate for the NiFi server (no
> idea why I wrote "client")
>
> I'm not trying to use certificate authentication yet - simply to get NiFi
> operating over TLS.
>
> When I test with OpenSSL, I get the same Nifi server log output as seen
> with a connection attempt from a browser.
>
> On the Cipher issue, when I use OpenSSL to connect using said certificate
> using httpd, it lists the successful cipher as one of those ignored by
> Jetty, albeit with less underscores in the onscreen output:
> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
>
> All machines involved are CentOS 7 - the only add-ons being Java 1.8
> (current release) and NiFi 1.6.0 (also current release)
>
> On Sat, 5 May 2018 at 01:26, Andy LoPresto  wrote:
>
> Hi Phil,
>
> Sorry to hear you are having this problem. I have a couple steps you 

Re: SSL_ERROR_NO_CYPHER_OVERLAP

[Andy LoPresto]

Phil,

Thanks for documenting the steps you took to get it working. This will 
definitely be helpful to anyone who has a similar problem in the future. I 
suspected it was the missing private key and I forgot to explain that yes, to 
import a PEM-encoded public key and private key into a JKS keystore, you do 
have to go through the PKCS12 intermediary.


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On May 4, 2018, at 4:36 PM, Phil H  wrote:
> 
> Sorry for talking to myself so much.  I have solved the problem!
> 
> Once I realised that I had never supplied the private key to keytool when I
> imported the certificate, I started googling along those lines.
> 
> I found this link from 2008 which provided the solution
> http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html
> 
> Basically you use OpenSSL to create a PKCS12 file from the cert and key,
> and then keytool to import that (as an existing keystore, not as a
> certificate)
> 
> From there, everything else just worked (including client authentication
> through the browser).  Now I have the "Insufficient permissions" problem,
> which there is loads of documentation as to how to move forwards.
> 
> Thanks!
> Phil
> 
> On Sat, May 5, 2018 at 8:27 AM, Phil H  wrote:
> 
>> Further info
>> 
>> In the httpd installation, I need the private key for the certificate and
>> it's paraphrase. That private key has not played a part of the NiFi install
>> thus far (seems like an important thing - the "keystore" only has a
>> certificate, not a key)
>> 
>> The reason why I used PKCS12 for the keystore was some Jetty debug output
>> telling me that was preferred to JKS. Interestingly it gave me contrary
>> advice for the trust store.
>> 
>> On Sat, 5 May 2018 at 08:22, Phil H  wrote:
>> 
>>> Hi Andy,
>>> 
>>> Sorry - poor use of words (it was late when I wrote the email). When I
>>> said "client" certificate I meant the certificate for the NiFi server (no
>>> idea why I wrote "client")
>>> 
>>> I'm not trying to use certificate authentication yet - simply to get NiFi
>>> operating over TLS.
>>> 
>>> When I test with OpenSSL, I get the same Nifi server log output as seen
>>> with a connection attempt from a browser.
>>> 
>>> On the Cipher issue, when I use OpenSSL to connect using said certificate
>>> using httpd, it lists the successful cipher as one of those ignored by
>>> Jetty, albeit with less underscores in the onscreen output:
>>> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
>>> 
>>> All machines involved are CentOS 7 - the only add-ons being Java 1.8
>>> (current release) and NiFi 1.6.0 (also current release)
>>> 
>>> On Sat, 5 May 2018 at 01:26, Andy LoPresto  wrote:
>>> 
 Hi Phil,
 
 Sorry to hear you are having this problem. I have a couple steps you can
 try to resolve this.
 
 First, to clarify the terminology for NiFi, a “client certificate”
 refers to a public certificate and private key which in combination allow a
 client to uniquely identify itself and authenticate on a mutual
 authentication TLS connection. In NiFi terminology, the client certificate
 identifies a user or service which connects to NiFi. The “server
 certificate” identifies the NiFi service, and the CA is what signs one (or
 both) of those certificates.
 
 The “no cipher suites in common” error can occur when there are
 legitimately no cipher suites that both the client and server support. This
 can be verified by using the OpenSSL s_client tool to make a connection
 from the client to the server. I’ve pasted a sample invocation below.
 
 $ openssl s_client -connect  -debug -state -cert
  -key  -CAfile
 
 
 However, that error can also appear when the keystore does not contain a
 valid private key to be used. I suspect the keystore you generated for NiFi
 does not have the private key. You can verify this by examining the
 nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see
 a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output.
 If you do not see this, you have only the public certificate in the file.
 Importing that into a keystore means that NiFi (or any other service using
 that keystore) will not be able to sign or decrypt any information
 encrypted with the public key, so it won’t be able to support any cipher
 suites that rely on RSA encryption or signatures.
 
 The nifi1.crt you imported into the keystore may also not have the
 complete certificate chain encoded, in which case when the server presents
 that certificate on an incoming connection, the client (command-line or
 browser) won’t be able to verify and trust it. You’ll get a different
 error, but it is something to be aware of.
 
 Is there a 

Re: SSL_ERROR_NO_CYPHER_OVERLAP

[Phil H]

Sorry for talking to myself so much.  I have solved the problem!

Once I realised that I had never supplied the private key to keytool when I
imported the certificate, I started googling along those lines.

I found this link from 2008 which provided the solution
http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html

Basically you use OpenSSL to create a PKCS12 file from the cert and key,
and then keytool to import that (as an existing keystore, not as a
certificate)

>From there, everything else just worked (including client authentication
through the browser).  Now I have the "Insufficient permissions" problem,
which there is loads of documentation as to how to move forwards.

Thanks!
Phil

On Sat, May 5, 2018 at 8:27 AM, Phil H  wrote:

> Further info
>
> In the httpd installation, I need the private key for the certificate and
> it's paraphrase. That private key has not played a part of the NiFi install
> thus far (seems like an important thing - the "keystore" only has a
> certificate, not a key)
>
> The reason why I used PKCS12 for the keystore was some Jetty debug output
> telling me that was preferred to JKS. Interestingly it gave me contrary
> advice for the trust store.
>
> On Sat, 5 May 2018 at 08:22, Phil H  wrote:
>
>> Hi Andy,
>>
>> Sorry - poor use of words (it was late when I wrote the email). When I
>> said "client" certificate I meant the certificate for the NiFi server (no
>> idea why I wrote "client")
>>
>> I'm not trying to use certificate authentication yet - simply to get NiFi
>> operating over TLS.
>>
>> When I test with OpenSSL, I get the same Nifi server log output as seen
>> with a connection attempt from a browser.
>>
>> On the Cipher issue, when I use OpenSSL to connect using said certificate
>> using httpd, it lists the successful cipher as one of those ignored by
>> Jetty, albeit with less underscores in the onscreen output:
>> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
>>
>> All machines involved are CentOS 7 - the only add-ons being Java 1.8
>> (current release) and NiFi 1.6.0 (also current release)
>>
>> On Sat, 5 May 2018 at 01:26, Andy LoPresto  wrote:
>>
>>> Hi Phil,
>>>
>>> Sorry to hear you are having this problem. I have a couple steps you can
>>> try to resolve this.
>>>
>>> First, to clarify the terminology for NiFi, a “client certificate”
>>> refers to a public certificate and private key which in combination allow a
>>> client to uniquely identify itself and authenticate on a mutual
>>> authentication TLS connection. In NiFi terminology, the client certificate
>>> identifies a user or service which connects to NiFi. The “server
>>> certificate” identifies the NiFi service, and the CA is what signs one (or
>>> both) of those certificates.
>>>
>>> The “no cipher suites in common” error can occur when there are
>>> legitimately no cipher suites that both the client and server support. This
>>> can be verified by using the OpenSSL s_client tool to make a connection
>>> from the client to the server. I’ve pasted a sample invocation below.
>>>
>>> $ openssl s_client -connect  -debug -state -cert
>>>  -key  -CAfile
>>> 
>>>
>>> However, that error can also appear when the keystore does not contain a
>>> valid private key to be used. I suspect the keystore you generated for NiFi
>>> does not have the private key. You can verify this by examining the
>>> nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see
>>> a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output.
>>> If you do not see this, you have only the public certificate in the file.
>>> Importing that into a keystore means that NiFi (or any other service using
>>> that keystore) will not be able to sign or decrypt any information
>>> encrypted with the public key, so it won’t be able to support any cipher
>>> suites that rely on RSA encryption or signatures.
>>>
>>> The nifi1.crt you imported into the keystore may also not have the
>>> complete certificate chain encoded, in which case when the server presents
>>> that certificate on an incoming connection, the client (command-line or
>>> browser) won’t be able to verify and trust it. You’ll get a different
>>> error, but it is something to be aware of.
>>>
>>> Is there a reason you chose to use a PKCS12 keystore in this scenario?
>>> Usually we recommend using JKS for both the keystore and the truststore.
>>>
>>> I hope this helps. If none of this resolves your issues, please let us
>>> know and we can continue to help.
>>>
>>> Andy LoPresto
>>> alopre...@apache.org
>>> *alopresto.apa...@gmail.com *
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On May 4, 2018, at 4:03 AM, Phil H  wrote:
>>>
>>> Hi all,
>>>
>>> I am trying to secure my NiFi installation.  I have a client certificate
>>> (nifi1.crt) and the CA for the intranet (ca.crt).  I created the trust
>>> and
>>> 

Re: SSL_ERROR_NO_CYPHER_OVERLAP

[Phil H]

Further info

In the httpd installation, I need the private key for the certificate and
it's paraphrase. That private key has not played a part of the NiFi install
thus far (seems like an important thing - the "keystore" only has a
certificate, not a key)

The reason why I used PKCS12 for the keystore was some Jetty debug output
telling me that was preferred to JKS. Interestingly it gave me contrary
advice for the trust store.

On Sat, 5 May 2018 at 08:22, Phil H  wrote:

> Hi Andy,
>
> Sorry - poor use of words (it was late when I wrote the email). When I
> said "client" certificate I meant the certificate for the NiFi server (no
> idea why I wrote "client")
>
> I'm not trying to use certificate authentication yet - simply to get NiFi
> operating over TLS.
>
> When I test with OpenSSL, I get the same Nifi server log output as seen
> with a connection attempt from a browser.
>
> On the Cipher issue, when I use OpenSSL to connect using said certificate
> using httpd, it lists the successful cipher as one of those ignored by
> Jetty, albeit with less underscores in the onscreen output:
> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
>
> All machines involved are CentOS 7 - the only add-ons being Java 1.8
> (current release) and NiFi 1.6.0 (also current release)
>
> On Sat, 5 May 2018 at 01:26, Andy LoPresto  wrote:
>
>> Hi Phil,
>>
>> Sorry to hear you are having this problem. I have a couple steps you can
>> try to resolve this.
>>
>> First, to clarify the terminology for NiFi, a “client certificate” refers
>> to a public certificate and private key which in combination allow a client
>> to uniquely identify itself and authenticate on a mutual authentication TLS
>> connection. In NiFi terminology, the client certificate identifies a user
>> or service which connects to NiFi. The “server certificate” identifies the
>> NiFi service, and the CA is what signs one (or both) of those certificates.
>>
>> The “no cipher suites in common” error can occur when there are
>> legitimately no cipher suites that both the client and server support. This
>> can be verified by using the OpenSSL s_client tool to make a connection
>> from the client to the server. I’ve pasted a sample invocation below.
>>
>> $ openssl s_client -connect  -debug -state -cert
>>  -key  -CAfile
>> 
>>
>> However, that error can also appear when the keystore does not contain a
>> valid private key to be used. I suspect the keystore you generated for NiFi
>> does not have the private key. You can verify this by examining the
>> nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see
>> a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output.
>> If you do not see this, you have only the public certificate in the file.
>> Importing that into a keystore means that NiFi (or any other service using
>> that keystore) will not be able to sign or decrypt any information
>> encrypted with the public key, so it won’t be able to support any cipher
>> suites that rely on RSA encryption or signatures.
>>
>> The nifi1.crt you imported into the keystore may also not have the
>> complete certificate chain encoded, in which case when the server presents
>> that certificate on an incoming connection, the client (command-line or
>> browser) won’t be able to verify and trust it. You’ll get a different
>> error, but it is something to be aware of.
>>
>> Is there a reason you chose to use a PKCS12 keystore in this scenario?
>> Usually we recommend using JKS for both the keystore and the truststore.
>>
>> I hope this helps. If none of this resolves your issues, please let us
>> know and we can continue to help.
>>
>> Andy LoPresto
>> alopre...@apache.org
>> *alopresto.apa...@gmail.com *
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On May 4, 2018, at 4:03 AM, Phil H  wrote:
>>
>> Hi all,
>>
>> I am trying to secure my NiFi installation.  I have a client certificate
>> (nifi1.crt) and the CA for the intranet (ca.crt).  I created the trust and
>> keystores as below:
>>
>> keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore
>> server_keystore.p12 -storetype PKCS12
>>
>> keytool -import -file ca.crt -alias cacert -keystore truststore.jks
>>
>> And the relevant nifi.properties are set as follows
>>
>> nifi.security.keystore=./conf/server_keystore.p12
>> nifi.security.keystoreType=PKCS12
>> nifi.security.keystorePasswd=
>> nifi.security.keyPasswd=
>> nifi.security.truststore=./conf/truststore.jks
>> nifi.security.truststoreType=JKS
>> nifi.security.truststorePasswd=
>>
>> When I try and access the site via https, I receive the above error in
>> Firefox, and the following in the nifi-bootstrap.log (I have enabled
>> additional debugging).
>>
>> Using both of these certificates inside Apache httpd works on the client
>> as
>> expected, so the certificates are fine.  I have seen some references to
>> bugs/features 

Re: SSL_ERROR_NO_CYPHER_OVERLAP

[Phil H]

Hi Andy,

Sorry - poor use of words (it was late when I wrote the email). When I said
"client" certificate I meant the certificate for the NiFi server (no idea
why I wrote "client")

I'm not trying to use certificate authentication yet - simply to get NiFi
operating over TLS.

When I test with OpenSSL, I get the same Nifi server log output as seen
with a connection attempt from a browser.

On the Cipher issue, when I use OpenSSL to connect using said certificate
using httpd, it lists the successful cipher as one of those ignored by
Jetty, albeit with less underscores in the onscreen output:
TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384

All machines involved are CentOS 7 - the only add-ons being Java 1.8
(current release) and NiFi 1.6.0 (also current release)

On Sat, 5 May 2018 at 01:26, Andy LoPresto  wrote:

> Hi Phil,
>
> Sorry to hear you are having this problem. I have a couple steps you can
> try to resolve this.
>
> First, to clarify the terminology for NiFi, a “client certificate” refers
> to a public certificate and private key which in combination allow a client
> to uniquely identify itself and authenticate on a mutual authentication TLS
> connection. In NiFi terminology, the client certificate identifies a user
> or service which connects to NiFi. The “server certificate” identifies the
> NiFi service, and the CA is what signs one (or both) of those certificates.
>
> The “no cipher suites in common” error can occur when there are
> legitimately no cipher suites that both the client and server support. This
> can be verified by using the OpenSSL s_client tool to make a connection
> from the client to the server. I’ve pasted a sample invocation below.
>
> $ openssl s_client -connect  -debug -state -cert
>  -key  -CAfile
> 
>
> However, that error can also appear when the keystore does not contain a
> valid private key to be used. I suspect the keystore you generated for NiFi
> does not have the private key. You can verify this by examining the
> nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see
> a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output.
> If you do not see this, you have only the public certificate in the file.
> Importing that into a keystore means that NiFi (or any other service using
> that keystore) will not be able to sign or decrypt any information
> encrypted with the public key, so it won’t be able to support any cipher
> suites that rely on RSA encryption or signatures.
>
> The nifi1.crt you imported into the keystore may also not have the
> complete certificate chain encoded, in which case when the server presents
> that certificate on an incoming connection, the client (command-line or
> browser) won’t be able to verify and trust it. You’ll get a different
> error, but it is something to be aware of.
>
> Is there a reason you chose to use a PKCS12 keystore in this scenario?
> Usually we recommend using JKS for both the keystore and the truststore.
>
> I hope this helps. If none of this resolves your issues, please let us
> know and we can continue to help.
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On May 4, 2018, at 4:03 AM, Phil H  wrote:
>
> Hi all,
>
> I am trying to secure my NiFi installation.  I have a client certificate
> (nifi1.crt) and the CA for the intranet (ca.crt).  I created the trust and
> keystores as below:
>
> keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore
> server_keystore.p12 -storetype PKCS12
>
> keytool -import -file ca.crt -alias cacert -keystore truststore.jks
>
> And the relevant nifi.properties are set as follows
>
> nifi.security.keystore=./conf/server_keystore.p12
> nifi.security.keystoreType=PKCS12
> nifi.security.keystorePasswd=
> nifi.security.keyPasswd=
> nifi.security.truststore=./conf/truststore.jks
> nifi.security.truststoreType=JKS
> nifi.security.truststorePasswd=
>
> When I try and access the site via https, I receive the above error in
> Firefox, and the following in the nifi-bootstrap.log (I have enabled
> additional debugging).
>
> Using both of these certificates inside Apache httpd works on the client as
> expected, so the certificates are fine.  I have seen some references to
> bugs/features in Jetty under Java 1.8 related to older TLS versions, but
> I'm at a loss to explain this! Help!!
>
> Thanks,
> Phil
>
> 2018-05-04 20:57:17,406 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Using SSLEngineImpl.
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Allow unsafe renegotiation: false
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Allow legacy hello messages: true
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Is initial handshake: true
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] 

Re: SSL_ERROR_NO_CYPHER_OVERLAP

[Andy LoPresto]

Hi Phil,

Sorry to hear you are having this problem. I have a couple steps you can try to 
resolve this.

First, to clarify the terminology for NiFi, a “client certificate” refers to a 
public certificate and private key which in combination allow a client to 
uniquely identify itself and authenticate on a mutual authentication TLS 
connection. In NiFi terminology, the client certificate identifies a user or 
service which connects to NiFi. The “server certificate” identifies the NiFi 
service, and the CA is what signs one (or both) of those certificates.

The “no cipher suites in common” error can occur when there are legitimately no 
cipher suites that both the client and server support. This can be verified by 
using the OpenSSL s_client tool to make a connection from the client to the 
server. I’ve pasted a sample invocation below.

$ openssl s_client -connect  -debug -state -cert 
 -key  -CAfile 


However, that error can also appear when the keystore does not contain a valid 
private key to be used. I suspect the keystore you generated for NiFi does not 
have the private key. You can verify this by examining the nifi1.crt file you 
imported. If you run `$ more nifi1.crt`, you should see a line “-BEGIN 
PRIVATE KEY-“ and then some Base64-encoded output. If you do not see this, 
you have only the public certificate in the file. Importing that into a 
keystore means that NiFi (or any other service using that keystore) will not be 
able to sign or decrypt any information encrypted with the public key, so it 
won’t be able to support any cipher suites that rely on RSA encryption or 
signatures.

The nifi1.crt you imported into the keystore may also not have the complete 
certificate chain encoded, in which case when the server presents that 
certificate on an incoming connection, the client (command-line or browser) 
won’t be able to verify and trust it. You’ll get a different error, but it is 
something to be aware of.

Is there a reason you chose to use a PKCS12 keystore in this scenario? Usually 
we recommend using JKS for both the keystore and the truststore.

I hope this helps. If none of this resolves your issues, please let us know and 
we can continue to help.

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On May 4, 2018, at 4:03 AM, Phil H  wrote:
> 
> Hi all,
> 
> I am trying to secure my NiFi installation.  I have a client certificate
> (nifi1.crt) and the CA for the intranet (ca.crt).  I created the trust and
> keystores as below:
> 
> keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore
> server_keystore.p12 -storetype PKCS12
> 
> keytool -import -file ca.crt -alias cacert -keystore truststore.jks
> 
> And the relevant nifi.properties are set as follows
> 
> nifi.security.keystore=./conf/server_keystore.p12
> nifi.security.keystoreType=PKCS12
> nifi.security.keystorePasswd=
> nifi.security.keyPasswd=
> nifi.security.truststore=./conf/truststore.jks
> nifi.security.truststoreType=JKS
> nifi.security.truststorePasswd=
> 
> When I try and access the site via https, I receive the above error in
> Firefox, and the following in the nifi-bootstrap.log (I have enabled
> additional debugging).
> 
> Using both of these certificates inside Apache httpd works on the client as
> expected, so the certificates are fine.  I have seen some references to
> bugs/features in Jetty under Java 1.8 related to older TLS versions, but
> I'm at a loss to explain this! Help!!
> 
> Thanks,
> Phil
> 
> 2018-05-04 20:57:17,406 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Using SSLEngineImpl.
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Allow unsafe renegotiation: false
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Allow legacy hello messages: true
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Is initial handshake: true
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Is secure renegotiation: false
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
> for TLSv1
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
> for TLSv1
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
> for TLSv1
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
> Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for
> TLSv1
> 2018-05-04 20:57:17,407 INFO [NiFi logging handler] 

Re: ReplaceText Flow File Processing Count

[Bryan Bende]

I don't know the history of this particular processor, but I think the
purpose of the session.get() with batches is similar to the concept of
@SupportsBatching. Basically both of them should have better
performance because you are handling multiple flow files in a single
session. The supports batching concept is a bit more flexible as it is
configurable by the user, where as this case is hard-coded into the
processor.

I suppose if there is some reason why you need to process 1 flow file
at a time, you could set the back-pressure threshold to 1 on the queue
leading into ReplaceText.

On Fri, May 4, 2018 at 3:50 AM, Peter Wicks (pwicks)  wrote:
> Had a user notice today that a ReplaceText processor, scheduled to run every 
> 20 minutes, had processed all 14 files in queue at once. I looked at the code 
> and see that ReplaceText does not do a standard session.get, but instead 
> calls:
>
> final List flowFiles = 
> session.get(FlowFileFilters.newSizeBasedFilter(1, DataUnit.MB, 100));
>
> Was there a design reason behind this? To us it was just really confusing 
> that we didn't have full control over how quickly FlowFile's move through 
> this processor.
>
> Thanks,
>   Peter

SSL_ERROR_NO_CYPHER_OVERLAP

[Phil H]

Hi all,

I am trying to secure my NiFi installation.  I have a client certificate
(nifi1.crt) and the CA for the intranet (ca.crt).  I created the trust and
keystores as below:

keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore
server_keystore.p12 -storetype PKCS12

keytool -import -file ca.crt -alias cacert -keystore truststore.jks

And the relevant nifi.properties are set as follows

nifi.security.keystore=./conf/server_keystore.p12
nifi.security.keystoreType=PKCS12
nifi.security.keystorePasswd=
nifi.security.keyPasswd=
nifi.security.truststore=./conf/truststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=

When I try and access the site via https, I receive the above error in
Firefox, and the following in the nifi-bootstrap.log (I have enabled
additional debugging).

Using both of these certificates inside Apache httpd works on the client as
expected, so the certificates are fine.  I have seen some references to
bugs/features in Jetty under Java 1.8 related to older TLS versions, but
I'm at a loss to explain this! Help!!

Thanks,
Phil

2018-05-04 20:57:17,406 INFO [NiFi logging handler] org.apache.nifi.StdOut
Using SSLEngineImpl.
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Allow unsafe renegotiation: false
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Allow legacy hello messages: true
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Is initial handshake: true
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Is secure renegotiation: false
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_GCM_SHA384 for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384
for TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 for
TLSv1
2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut

ReplaceText Flow File Processing Count

[Peter Wicks (pwicks)]

Had a user notice today that a ReplaceText processor, scheduled to run every 20 
minutes, had processed all 14 files in queue at once. I looked at the code and 
see that ReplaceText does not do a standard session.get, but instead calls:

final List flowFiles = 
session.get(FlowFileFilters.newSizeBasedFilter(1, DataUnit.MB, 100));

Was there a design reason behind this? To us it was just really confusing that 
we didn't have full control over how quickly FlowFile's move through this 
processor.

Thanks,
  Peter

Re: GetMongoDB : How to pass parameters as input to GetMongoDB processor

[Sivaprasanna]

As Bryan mentioned in the actual ‘GetSplunk’ thread, it is not available
but it certainly makes sense to have that feature. If you’re interested in
having that, please raise a Jira at https://issues.apache.org/jira

-
Sivaprasanna

On Fri, 4 May 2018 at 12:56 PM, Brajendra Mishra <
brajendra_mis...@persistent.com> wrote:

> Hi Mike,
>
> Thanks a lot for your valuable inputs.
> We tried GetMongoDB Template which you shared in previous mail with new
> version of Apache NiFi (NiFi-1.6) , Its working fine and we are able to get
> mongoDB data in desired format.
>
> One query: Is such input flow functionality available with 'GetSplunk'
> processor too with NiFI 1.6.0? if yes, please share sample template or if
> not, When can expect such implementation with 'GetSplunk' processor too?
>
> Brajendra Mishra
> Persistent Systems Ltd.
>
> -Original Message-
> From: Mike Thomsen 
> Sent: Thursday, May 03, 2018 6:56 PM
> To: dev@nifi.apache.org
> Subject: Re: GetMongoDB : How to pass parameters as input to GetMongoDB
> processor
>
> Brajendra,
>
> I would recommend an update to 1.6.0. It'll make your life a lot easier on
> this. I did that patch to GetMongo because I had a client that had an
> explosion of GetMongos due to that inflexibility. With that said, *be aware
> of this bug* in 1.6.0 w/ PutMongo if you use it and upgrade. It is fixed in
> 1.7.0 (still in development):
>
> Migrating from 1.5.0 to 1.6.0
>
>- PutMongo can fail in insert mode. Will be fixed in next release. In
>the mean time you can set query keys for insert even though they'll be
>ignored it should workaround the validation bug.
>
>
> What it means is there is a validator function that is broken in PutMongo
> when one is using the "insert mode" instead of "update mode." You can do a
> work around by putting a dummy value in the "query key" field to make it
> happy.
>
> On Thu, May 3, 2018 at 8:26 AM Pierre Villard  >
> wrote:
>
> > Hi,
> >
> > As Mike said: incoming relationship has been added for NiFi 1.6.0.
> > https://issues.apache.org/jira/browse/NIFI-4827
> >
> > Pierre
> >
> > 2018-05-03 14:09 GMT+02:00 Brajendra Mishra <
> > brajendra_mis...@persistent.com
> > >:
> >
> > > Hi Mike,
> > >
> > > I did attach the same in my previous mail. Well reattaching it again.
> > > Well error is at GetMongoDB Processor and error text is : "Upstream
> > > Connections is invalid because Processor does not allow upstream
> > > connections but currently has 1"
> > >
> > > Brajendra Mishra
> > > Persistent Systems Ltd.
> > >
> > > -Original Message-
> > > From: Mike Thomsen 
> > > Sent: Thursday, May 03, 2018 5:20 PM
> > > To: dev@nifi.apache.org
> > > Subject: Re: GetMongoDB : How to pass parameters as input to
> > > GetMongoDB processor
> > >
> > > Brajendra,
> > >
> > > Looks like the image didn't make it.
> > >
> > > On Wed, May 2, 2018 at 11:36 PM Brajendra Mishra <
> > > brajendra_mis...@persistent.com> wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > Thanks for responding.
> > > >
> > > > Here, I have attached missing image attachment.
> > > >
> > > >
> > > >
> > > > Brajendra Mishra
> > > >
> > > > Persistent Systems Ltd.
> > > >
> > > >
> > > >
> > > > *From:* Mike Thomsen 
> > > > *Sent:* Wednesday, May 02, 2018 6:24 PM
> > > >
> > > >
> > > > *To:* dev@nifi.apache.org
> > > > *Subject:* Re: GetMongoDB : How to pass parameters as input to
> > > > GetMongoDB processor
> > > >
> > > >
> > > >
> > > > That might require 1.6.0. Also, your image didn't come through in
> > > > your response to Sivaprasanna so resend that too.
> > > >
> > > > On Wed, May 2, 2018 at 8:37 AM Brajendra Mishra <
> > > > brajendra_mis...@persistent.com> wrote:
> > > >
> > > > Hi Mike,
> > > >
> > > > Thanks a lot for responding.
> > > >
> > > > On your statement
> > > > "That is its new default behavior if you leave the query field
> > > > blank and have an incoming connection from another processor. That
> > > > would be a good way to integrate the flow with another application"
> > > >
> > > > Could you please share a sample template for the same?
> > > >
> > > >
> > > > Brajendra Mishra
> > > > Persistent Systems Ltd.
> > > >
> > > > -Original Message-
> > > > From: Mike Thomsen 
> > > > Sent: Wednesday, May 02, 2018 5:58 PM
> > > > To: dev@nifi.apache.org
> > > > Subject: Re: GetMongoDB : How to pass parameters as input to
> > > > GetMongoDB processor
> > > >
> > > > GetMongo can also use the body of a flowfile for the query. That
> > > > is its new default behavior if you leave the query field blank and
> > > > have an incoming connection from another processor. That would be
> > > > a good way to integrate the flow with another application. For
> > > > example, you could add FetchKafka to the flow and have your
> > > > applications post messages to Kafka with the queries they want it
> > > > to run and FetchKafka 

RE: GetMongoDB : How to pass parameters as input to GetMongoDB processor

[Brajendra Mishra]

Hi Mike, 

Thanks a lot for your valuable inputs. 
We tried GetMongoDB Template which you shared in previous mail with new version 
of Apache NiFi (NiFi-1.6) , Its working fine and we are able to get mongoDB 
data in desired format.

One query: Is such input flow functionality available with 'GetSplunk' 
processor too with NiFI 1.6.0? if yes, please share sample template or if not, 
When can expect such implementation with 'GetSplunk' processor too?

Brajendra Mishra
Persistent Systems Ltd.

-Original Message-
From: Mike Thomsen  
Sent: Thursday, May 03, 2018 6:56 PM
To: dev@nifi.apache.org
Subject: Re: GetMongoDB : How to pass parameters as input to GetMongoDB 
processor

Brajendra,

I would recommend an update to 1.6.0. It'll make your life a lot easier on 
this. I did that patch to GetMongo because I had a client that had an explosion 
of GetMongos due to that inflexibility. With that said, *be aware of this bug* 
in 1.6.0 w/ PutMongo if you use it and upgrade. It is fixed in
1.7.0 (still in development):

Migrating from 1.5.0 to 1.6.0

   - PutMongo can fail in insert mode. Will be fixed in next release. In
   the mean time you can set query keys for insert even though they'll be
   ignored it should workaround the validation bug.


What it means is there is a validator function that is broken in PutMongo when 
one is using the "insert mode" instead of "update mode." You can do a work 
around by putting a dummy value in the "query key" field to make it happy.

On Thu, May 3, 2018 at 8:26 AM Pierre Villard 
wrote:

> Hi,
>
> As Mike said: incoming relationship has been added for NiFi 1.6.0.
> https://issues.apache.org/jira/browse/NIFI-4827
>
> Pierre
>
> 2018-05-03 14:09 GMT+02:00 Brajendra Mishra < 
> brajendra_mis...@persistent.com
> >:
>
> > Hi Mike,
> >
> > I did attach the same in my previous mail. Well reattaching it again.
> > Well error is at GetMongoDB Processor and error text is : "Upstream 
> > Connections is invalid because Processor does not allow upstream 
> > connections but currently has 1"
> >
> > Brajendra Mishra
> > Persistent Systems Ltd.
> >
> > -Original Message-
> > From: Mike Thomsen 
> > Sent: Thursday, May 03, 2018 5:20 PM
> > To: dev@nifi.apache.org
> > Subject: Re: GetMongoDB : How to pass parameters as input to 
> > GetMongoDB processor
> >
> > Brajendra,
> >
> > Looks like the image didn't make it.
> >
> > On Wed, May 2, 2018 at 11:36 PM Brajendra Mishra < 
> > brajendra_mis...@persistent.com> wrote:
> >
> > > Hi Mike,
> > >
> > > Thanks for responding.
> > >
> > > Here, I have attached missing image attachment.
> > >
> > >
> > >
> > > Brajendra Mishra
> > >
> > > Persistent Systems Ltd.
> > >
> > >
> > >
> > > *From:* Mike Thomsen 
> > > *Sent:* Wednesday, May 02, 2018 6:24 PM
> > >
> > >
> > > *To:* dev@nifi.apache.org
> > > *Subject:* Re: GetMongoDB : How to pass parameters as input to 
> > > GetMongoDB processor
> > >
> > >
> > >
> > > That might require 1.6.0. Also, your image didn't come through in 
> > > your response to Sivaprasanna so resend that too.
> > >
> > > On Wed, May 2, 2018 at 8:37 AM Brajendra Mishra < 
> > > brajendra_mis...@persistent.com> wrote:
> > >
> > > Hi Mike,
> > >
> > > Thanks a lot for responding.
> > >
> > > On your statement
> > > "That is its new default behavior if you leave the query field 
> > > blank and have an incoming connection from another processor. That 
> > > would be a good way to integrate the flow with another application"
> > >
> > > Could you please share a sample template for the same?
> > >
> > >
> > > Brajendra Mishra
> > > Persistent Systems Ltd.
> > >
> > > -Original Message-
> > > From: Mike Thomsen 
> > > Sent: Wednesday, May 02, 2018 5:58 PM
> > > To: dev@nifi.apache.org
> > > Subject: Re: GetMongoDB : How to pass parameters as input to 
> > > GetMongoDB processor
> > >
> > > GetMongo can also use the body of a flowfile for the query. That 
> > > is its new default behavior if you leave the query field blank and 
> > > have an incoming connection from another processor. That would be 
> > > a good way to integrate the flow with another application. For 
> > > example, you could add FetchKafka to the flow and have your 
> > > applications post messages to Kafka with the queries they want it 
> > > to run and FetchKafka would send that JSON to GetMongo as it comes 
> > > in. Or you could build a REST service that writes the JSON to disk and 
> > > use GetFile to load it.
> > Lots of ways to do this.
> > >
> > > On Wed, May 2, 2018 at 6:42 AM Sivaprasanna 
> > > 
> > > wrote:
> > >
> > > > Since I'm not so sure about your exact use case, I have just 
> > > > created a rough template based on the simple example flow that I 
> > > > had posted earlier which is GenerateFlowfile -> UpdateAttribute 
> > > > -> GetMongo. I have attached the template