Re: How to run NiFi on HTTPS

[Andy LoPresto]

Apache NiFi does not support Basic Authentication in any scenario. There are 
multiple options for user authentication to the NiFi UI/API, including LDAP, 
Kerberos, client certificates, Apache Knox, and OpenID Connect. More details 
about configuring these options are available in the Admin Guide [1].

As for your TLS error, my guess is that there is an error with the certificate 
you generated. The error “No overlapping cipher suites available” can occur 
when the certificate is expired or otherwise invalid, and all the available 
cipher suites require an RSA key for signing or encryption. To further debug 
this, you can use the OpenSSL s_client tool to attempt to make a connection via 
the command line, and enable the JSSE SSL debugging via a modification to 
bootstrap.conf. Once you restart, you should see additional TLS/SSL debug 
output in logs/nifi-bootstrap.conf.

For us to be able to offer further advice, you’ll need to provide more 
information, like stacktraces from your logs, or the openssl output from 
examining the certificates. Images do not come through on the list, so please 
copy and paste text output instead. There are other possible explanations, such 
as OS-limited cipher suites available, older browser versions, etc. but these 
are much less common.

Add this line to bootstrap.conf:

java.arg.15=-Djavax.net.debug=ssl,handshake

[1] 
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On May 22, 2018, at 5:53 AM, Brajendra Mishra 
>  wrote:
> 
> Team I need to know the implementation of basic authentication with HTTPS as 
> well.
> 
> Brajendra Mishra
> Persistent Systems Ltd.
> 
> From: Brajendra Mishra 
> Sent: Tuesday, May 22, 2018 6:22 PM
> To: dev@nifi.apache.org
> Subject: How to run NiFi on HTTPS
> 
> Hi Team,
> 
> I have used tlstoolkit to create required files (nifi.properties, keystore 
> and truststore files) to run NiFi on HTTPS.
> I also configured successfully and ran the NiFi service correctly which show 
> it is running on Https protocol.
> But once I tried to see its UI I am facing following error on all browsers 
> (IE, Firefox and Chrome):
> 
> "Secure Connection Failed - An error occurred during a connection to 
> localhost:9090. Cannot communicate securely with peer: no common encryption 
> algorithm(s). Error code: SSL_ERROR_NO_CYPHER_OVERLAP"
> 
> [cid:image001.png@01D3F1F9.9EC1D450]
> 
> Could you please let me know how can I see NiFi UI in this case? I have 
> already tried all possible options (spread on internet) to get rid this issue 
> on browsers but no luc
> 
> 
> Brajendra Mishra
> Persistent Systems Ltd.
> 
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the 
> property of Persistent Systems Ltd. It is intended only for the use of the 
> individual or entity to which it is addressed. If you are not the intended 
> recipient, you are not authorized to read, retain, copy, print, distribute or 
> use this message. If you have received this communication in error, please 
> notify the sender and delete all copies of this message. Persistent Systems 
> Ltd. does not accept any liability for virus infected mails.



signature.asc
Description: Message signed with OpenPGP using GPGMail

[ANNOUNCE] CVE Announcement for Apache NiFi 1.0.0 - 1.5.0

[Andy LoPresto]

The Apache NiFi PMC would like to announce the following CVE discoveries in 
Apache NiFi 1.0.0 - 1.5.0. These issues were resolved with the release of NiFi 
1.6.0 on April 8, 2018. NiFi is an easy to use, powerful, and reliable system 
to process and distribute data. It supports powerful and scalable directed 
graphs of data routing, transformation, and system mediation logic. For more 
information, see https://nifi.apache.org/security.html 
.

CVE-2018-1309 : Apache 
NiFi External XML Entity issue in SplitXML processor

Severity: Moderate

Versions Affected:

Apache NiFi 0.1.0 - 1.5.0
Description: Malicious XML content could cause information disclosure or remote 
code execution.

Mitigation: The fix to disable external general entity parsing and disallow 
doctype declarations was applied on the Apache NiFi 1.6.0 release. Users 
running a prior 1.x release should upgrade to the appropriate release.

Credit: This issue was discovered by 圆珠笔.

CVE Link: Mitre Database: CVE-2018-1309 


CVE-2018-1310 : Apache 
NiFi JMS Deserialization issue because of ActiveMQ client vulnerability

Severity: Moderate

Versions Affected:

Apache NiFi 0.1.0 - 1.5.0
Description: Malicious JMS content could cause denial of service. See ActiveMQ 
CVE-2015-5254 announcement 

 for more information.

Mitigation: The fix to upgrade the activemq-client library to 5.15.3 was 
applied on the Apache NiFi 1.6.0 release. Users running a prior 1.x release 
should upgrade to the appropriate release.

Credit: This issue was discovered by 圆珠笔.

CVE Link: Mitre Database: CVE-2018-1310 


CVE-2017-8028 : Apache 
NiFi LDAP TLS issue because of Spring Security LDAP vulnerability

Severity: Severe

Versions Affected:

Apache NiFi 0.1.0 - 1.5.0
Description: Spring Security LDAP library was not enforcing credential 
authentication after TLS handshake negotiation. See NVD CVE-2017-8028 
disclosure  for more 
information.

Mitigation: The fix to upgrade the spring-ldap library to 2.3.2.RELEASE+ was 
applied on the Apache NiFi 1.6.0 release. Users running a prior 1.x release 
should upgrade to the appropriate release.

Credit: This issue was discovered by Matthew Elder.

CVE Link: Mitre Database: CVE-2017-8028 


CVE-2018-1324 : Apache 
NiFi Denial of service issue because of commons-compress vulnerability

Severity: Low

Versions Affected:

Apache NiFi 0.1.0 - 1.5.0
Description: A vulnerability in the commons-compress library could cause denial 
of service. See commons-compress CVE-2018-1324 announcement 
 for 
more information.

Mitigation: The fix to upgrade the commons-compress library to 1.16.1 was 
applied on the Apache NiFi 1.6.0 release. Users running a prior 1.x release 
should upgrade to the appropriate release.

Credit: This issue was discovered by Joe Witt.

CVE Link: Mitre Database: CVE-2018-1324 



Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69



signature.asc
Description: Message signed with OpenPGP using GPGMail

minificpp AppVeyor

[Andy Christianson]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

minificpp devs:

The AppVeyor builds are broken 100% of the time and don't seem to be adding any
value. Are there any objections to the removal of AppVeyor?

Regards,

Andy I.C.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJbBD+9AAoJEG1+mBKNMpIDo1kIAKhADE74kaQSJKFDW66uQNr8
a7Us6hev4PuRgy79oAyTR8ykMXXmkPAqRskA7bScgP7GV5KWQGLfaQ0cigDsQed1
8DwtfTLVhUutkV5jz7PcQxKgWFXW4dL3oxV8farFfrev5CQ8xNugA+7wRFWI1zxJ
4qC2CzCTd10p3VEBbIhaaGEVg62tSjuXPt81SdJkGfB/rRRzwgYyipfBEEsQLyI5
aN/Qh/r7umeJlZ4un7xDMCTa/eZCHDuGQnpi2HOA3qbEwupXPib22sGKumUnGn5v
jQM/zsCyB52nR6nBFytA6uGwfyrmazXHmDOBVEkd997PaVnUrKFZT6Mnn4uQmTE=
=jj0/
-END PGP SIGNATURE-

Sent from [ProtonMail](https://protonmail.com), Swiss-based encrypted email.

Re: [DISCUSS] Apache NiFi MiNiFi C++ 0.5.0

[Jeremy Dyer]

I’m happy to handle RM duties



Thanks - Jeremy Dyer





On Tue, May 22, 2018 at 10:25 AM -0400, "Marc"  wrote:










Reviving this thread -- Sorry for the delay. I will start the release
process with the completion of [1], likely in the next few days or
early next week. I will take RM duties unless someone else would like
to take that opportunity.

[1] https://issues.apache.org/jira/browse/MINIFICPP-457

Thanks,
Marc

On Sun, Feb 18, 2018 at 9:17 AM, Joe Witt  wrote:
> sounds good!
>
> On Feb 18, 2018 9:15 AM, "Kevin Doran"  wrote:
>
>> Hi Marc,
>>
>> Thanks for kicking off this discuss thread. I agree we should start
>> planning the next release to get these new and improved capabilities into a
>> stable version for users.
>>
>> Thanks!
>> Kevin
>>
>> On 2/14/18, 13:42, "Marc"  wrote:
>>
>> Hello Everyone,
>>   I wanted to discuss releasing 0.5.0 in the coming weeks. We have some
>> really useful features and bug fixes in place or nearly up for review.
>> I am
>> proposing that we release Apache NiFi MiNiFi C++ 0.5.0 when these
>> activities are complete.
>>
>>We've had some very useful processors added (UpdateAttribute added
>> RouteOnAttribute coming ) , MQTT security, support for SUSE/SLES, C2
>> updates, and various bug fixes. If everyone is favorable to begin the
>> release process I am happy to act as RM if no one else is interested.
>>Thanks for your consideration,
>>Marc
>>
>>
>>
>>

Re: [DISCUSS] Apache NiFi MiNiFi C++ 0.5.0

[Marc]

Reviving this thread -- Sorry for the delay. I will start the release
process with the completion of [1], likely in the next few days or
early next week. I will take RM duties unless someone else would like
to take that opportunity.

[1] https://issues.apache.org/jira/browse/MINIFICPP-457

Thanks,
Marc

On Sun, Feb 18, 2018 at 9:17 AM, Joe Witt  wrote:
> sounds good!
>
> On Feb 18, 2018 9:15 AM, "Kevin Doran"  wrote:
>
>> Hi Marc,
>>
>> Thanks for kicking off this discuss thread. I agree we should start
>> planning the next release to get these new and improved capabilities into a
>> stable version for users.
>>
>> Thanks!
>> Kevin
>>
>> On 2/14/18, 13:42, "Marc"  wrote:
>>
>> Hello Everyone,
>>   I wanted to discuss releasing 0.5.0 in the coming weeks. We have some
>> really useful features and bug fixes in place or nearly up for review.
>> I am
>> proposing that we release Apache NiFi MiNiFi C++ 0.5.0 when these
>> activities are complete.
>>
>>We've had some very useful processors added (UpdateAttribute added
>> RouteOnAttribute coming ) , MQTT security, support for SUSE/SLES, C2
>> updates, and various bug fixes. If everyone is favorable to begin the
>> release process I am happy to act as RM if no one else is interested.
>>Thanks for your consideration,
>>Marc
>>
>>
>>
>>

Re: A user's experience with a bake off

[Mike Thomsen]

I think the first thing would be to expose as much data about the record
sets as possible in the flow. The second would be to consider some UI work
to make things a little flashier in the way that attracted his eye to
Streamsets.

I don't think we need the live monitoring because with s2s you can easily
integrate ELK and get something as flashy as Streamsets, but we need
something that can be used to help ELK visualize what record sets were
processed.

On Tue, May 22, 2018 at 12:37 AM Joe Witt  wrote:

> Mike,
>
> What specifically do you have in mind?  There are a lot of good/useful
> things we can do of course.  For instance we can make records a first
> class part of the nifi-api and then provide live monitoring over them.
> We have tons of good data in that regard.  But, we must also be
> mindful of the need to make progress on the extension registry as
> well.
>
> Thanks
>
> On Mon, May 21, 2018 at 4:17 PM, Mike Thomsen 
> wrote:
> > https://statsbot.co/blog/open-source-etl/
> >
> > Someone on my team shared that. Overall, it's a very fair take. I think
> it
> > has some direct action items, particularly on the UX side that could be
> > considered for 1.8 to close some gaps.
> >
> > (Not trying to start a flame war here, as once again I think it's a fair
> > article)
> >
> > Mike
>

Re: Put data to Elastic with static settings or index template

[Mike Thomsen]

Bobby,

You need to use ElasticSearch templates for this.

https://www.elastic.co/guide/en/elasticsearch/reference/5.4/indices-templates.html

That's the official Elastic-sanctioned way of doing this.

On Tue, May 22, 2018 at 4:23 AM Koji Kawamura 
wrote:

> Hi Bobby,
>
> Elasticsearch creates index if it doesn't exist.
>
> I haven't tried it myself yet, but Elasticsearch's Index template
> might be useful to tweak default settings for indices those are
> created automatically.
>
> https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
>
> Thanks,
> Koji
>
> On Tue, May 22, 2018 at 3:41 PM, Bobby  wrote:
> > Siva,
> >
> > In my putElastic processor i only state below properties:
> >
> > <
> http://apache-nifi-developer-list.39713.n7.nabble.com/file/t921/2018-05-22_13_36_48-NiFi.png
> >
> >
> > Given the index name is using expression language, i assume it will be
> > created if it is not exist; In my example, i tend to create new index per
> > day. My team also said, he didn't create index first, the processor take
> > care of it.
> >
> > Thanks
> >
> >
> >
> > -
> >
> > -
> > Bobby
> > --
> > Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>

RE: How to run NiFi on HTTPS

[Brajendra Mishra]

Team I need to know the implementation of basic authentication with HTTPS as 
well.

Brajendra Mishra
Persistent Systems Ltd.

From: Brajendra Mishra 
Sent: Tuesday, May 22, 2018 6:22 PM
To: dev@nifi.apache.org
Subject: How to run NiFi on HTTPS

Hi Team,

I have used tlstoolkit to create required files (nifi.properties, keystore and 
truststore files) to run NiFi on HTTPS.
I also configured successfully and ran the NiFi service correctly which show it 
is running on Https protocol.
But once I tried to see its UI I am facing following error on all browsers (IE, 
Firefox and Chrome):

"Secure Connection Failed - An error occurred during a connection to 
localhost:9090. Cannot communicate securely with peer: no common encryption 
algorithm(s). Error code: SSL_ERROR_NO_CYPHER_OVERLAP"

[cid:image001.png@01D3F1F9.9EC1D450]

Could you please let me know how can I see NiFi UI in this case? I have already 
tried all possible options (spread on internet) to get rid this issue on 
browsers but no luc


Brajendra Mishra
Persistent Systems Ltd.

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.

How to run NiFi on HTTPS

[Brajendra Mishra]

Hi Team,

I have used tlstoolkit to create required files (nifi.properties, keystore and 
truststore files) to run NiFi on HTTPS.
I also configured successfully and ran the NiFi service correctly which show it 
is running on Https protocol.
But once I tried to see its UI I am facing following error on all browsers (IE, 
Firefox and Chrome):

"Secure Connection Failed - An error occurred during a connection to 
localhost:9090. Cannot communicate securely with peer: no common encryption 
algorithm(s). Error code: SSL_ERROR_NO_CYPHER_OVERLAP"

[cid:image001.png@01D3F1F9.9EC1D450]

Could you please let me know how can I see NiFi UI in this case? I have already 
tried all possible options (spread on internet) to get rid this issue on 
browsers but no luc


Brajendra Mishra
Persistent Systems Ltd.

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.

Re: Put data to Elastic with static settings or index template

[Koji Kawamura]

Hi Bobby,

Elasticsearch creates index if it doesn't exist.

I haven't tried it myself yet, but Elasticsearch's Index template
might be useful to tweak default settings for indices those are
created automatically.
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html

Thanks,
Koji

On Tue, May 22, 2018 at 3:41 PM, Bobby  wrote:
> Siva,
>
> In my putElastic processor i only state below properties:
>
> 
>
> Given the index name is using expression language, i assume it will be
> created if it is not exist; In my example, i tend to create new index per
> day. My team also said, he didn't create index first, the processor take
> care of it.
>
> Thanks
>
>
>
> -
>
> -
> Bobby
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Put data to Elastic with static settings or index template

[Bobby]

Siva,

In my putElastic processor i only state below properties:


 

Given the index name is using expression language, i assume it will be
created if it is not exist; In my example, i tend to create new index per
day. My team also said, he didn't create index first, the processor take
care of it.

Thanks



-

-
Bobby
--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Put data to Elastic with static settings or index template

[Sivaprasanna]

Bobby,

If I'm correct, this setting is done during index creation and the
PutElasticsearch processors doesn't create index. It primarily works with
the assumption that the configured index already exists (people correct me,
if I'm wrong). If that's the case, there is no need to do anything on the
NiFi side. Rather while creating the index through ES APIs, you set the
"static" setting. Hope that helps.

-
Sivaprasanna

On Tue, May 22, 2018 at 8:21 AM, Bobby  wrote:

> Hi, when inserting data to elastic using nifi's processor (putElastic), i
> need to apply static setting for the index..like mentioned in
> https://www.elastic.co/guide/en/elasticsearch/reference/
> current/index-modules.html
>  current/index-modules.html>
> , this must be applied in index creation..
>
> With the processor, will it be possible to use this utility? I need to do
> this in order to save the space...or in other word, changing the
> compression
> type...
>
> As for last resort, i might need to write custom processor extended from
> putElastic
>
>
> Any suggestion?
>
> Thank you
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>