Re: [VOTE] Release Apache NiFi MiNiFi 0.3.0

[Joe Percivall]

There are some really cool new features in this release and I'm really
looking forward to seeing NiFi + MiNiFi + Registry all working together.
Thanks everyone for their hard work!

I was able to verify the hashes and the normal build + contrib check work
on OSX 10.12.5. Also, a simple S2S flow communicating to NiFi 1.4.0 works
as expected. Unfortunately, I am continually hitting MINIFI-416[1] when
running the tests in minifi-integration-tests (either using the command
line or IDE). I believe the cause of which is the switch to Alpine Linux as
the base docker image. That said, I'd like to get verification that this
isn't just a configuration issue on my end before voting "-1".

Also, I was surprised we haven't upgraded past NiFi-1.2.0 yet. I went ahead
and created a ticket for it[2] and put up a PR[3]. Not something that needs
to be in the release, I just wanted to bring attention to it.

[1] https://issues.apache.org/jira/browse/MINIFI-416
[2] https://issues.apache.org/jira/browse/MINIFI-417
[3] https://github.com/apache/nifi-minifi/pull/103

Joe

On Sun, Dec 10, 2017 at 6:42 PM, Aldrin Piri  wrote:

> Hello,
>
> I am pleased to call this vote for the source release of Apache NiFi
> MiNiFi, minifi-0.3.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1112
>
> The Git tag is minifi-0.3.0-RC1
> The Git commit ID is 930c3fb0112c043614158018d0d8f1ca31a4f1d8
> * https://git-wip-us.apache.org/repos/asf?p=nifi-minifi.git;a=
> commit;h=930c3fb0112c043614158018d0d8f1ca31a4f1d8
> * https://github.com/apache/nifi-minifi/commit/930c3fb0112c043
> 614158018d0d8f1ca31a4f1d8
>
> Checksums of minifi-0.3.0-source-release.zip:
> MD5:  69f2e8b90af7a7bdec45684406c7b83d
> SHA1:  bc0e8e1edb13f9f6ea69f19cfb70d18443b6604f
> SHA256:  b11c6df7c82479d77c4c245cfb737fe15cb2bd4742c962b08375485bbbe2894e
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/aldrin.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 18 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
> ctId=12319921=12340598
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/MINIFI/Release+N
> otes#ReleaseNotes-Version0.3.0
>
> The vote will be open for at 72 hours and close at 7:30PM EDT, 13 December
> 2017 [1].
>
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test.  Then please vote:
>
> [ ] +1 Release this package as minifi-0.3.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> Thanks!
>
> [1] You can determine this time for your local time zone at
> https://s.apache.org/minifi-0.3.0-rc1-close
>
>


-- 
*Joe Percivall*
linkedin.com/in/Percivall
e: jperciv...@apache.com

Re: Unable to checkpoint FlowFile Repository (No space left on device)

[尹文才]

Hi Michael, I checked the system available inodes by running df -i command
and there're quite enough inodes in the system. I then removed all the
files in all repository folders and restarted
the system, I couldn't see the error again. I will continue to track the
problem to see what's causing it, but it seems not relevant to the inode
use-up reason you mentioned. Thanks.

/Ben

2017-12-12 23:45 GMT+08:00 Michael Moser :

> Greetings Ben,
>
> The "No space left on device" error can also be caused by running out of
> inodes on your device.  You can check this with "df -i".
>
> -- Mike
>
>
> On Tue, Dec 12, 2017 at 1:36 AM, 尹文才  wrote:
>
> > sorry that I forgot to mention the environment that caused this problem,
> > I'm using the latest nifi 1.4.0 release and installed it on centos 7.
> >
> > 2017-12-12 14:35 GMT+08:00 尹文才 :
> >
> > > Hi guys, I'm running into a very weird problem, I wrote a processor
> > > specifically to extract some data
> > > and I found starting from yesterday it kept showing errors in the log,
> as
> > > below:
> > >
> > > 2017-12-12 14:01:04,661 INFO [pool-10-thread-1] o.a.n.c.r.
> > WriteAheadFlowFileRepository
> > > Initiating checkpoint of FlowFile Repository
> > > 2017-12-12 14:01:04,676 ERROR [pool-10-thread-1] o.a.n.c.r.
> > WriteAheadFlowFileRepository
> > > Unable to checkpoint FlowFile Repository due to
> > > java.io.FileNotFoundException: ../flowfile_repository/
> > partition-5/96.journal
> > > (No space left on device)
> > > java.io.FileNotFoundException: ../flowfile_repository/
> > partition-5/96.journal
> > > (No space left on device)
> > > at java.io.FileOutputStream.open0(Native Method)
> > > at java.io.FileOutputStream.open(FileOutputStream.java:270)
> > > at java.io.FileOutputStream.(FileOutputStream.java:213)
> > > at java.io.FileOutputStream.(FileOutputStream.java:162)
> > > at org.wali.MinimalLockingWriteAheadLog$Partition.rollover(
> > > MinimalLockingWriteAheadLog.java:779)
> > > at org.wali.MinimalLockingWriteAheadLog.checkpoint(
> > > MinimalLockingWriteAheadLog.java:528)
> > > at org.apache.nifi.controller.repository.
> > > WriteAheadFlowFileRepository.checkpoint(WriteAheadFlowFileRepository.
> > > java:451)
> > > at org.apache.nifi.controller.repository.
> > > WriteAheadFlowFileRepository$1.run(WriteAheadFlowFileRepository.
> > java:423)
> > > at java.util.concurrent.Executors$RunnableAdapter.
> > > call(Executors.java:511)
> > > at java.util.concurrent.FutureTask.runAndReset(
> > > FutureTask.java:308)
> > > at java.util.concurrent.ScheduledThreadPoolExecutor$
> > > ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > at java.util.concurrent.ScheduledThreadPoolExecutor$
> > > ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > ThreadPoolExecutor.java:1142)
> > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > ThreadPoolExecutor.java:617)
> > > at java.lang.Thread.run(Thread.java:745)
> > >
> > >
> > > I noticed the log mentioned no space left on device and I went to check
> > > the available space and found 33G left. Does anyone know what could
> > > possibly cause this and how to resolve this problem, thanks
> > >
> > > /Ben
> > >
> >
>

NiFI 1.4.0 UI can't be displayed in an IFrame?

[tanezavm]

Hi,

I tried to display NiFi 1.4.0 UI in an IFrame but it failed to load with
error below:

Refused to display 'https://172.16.0.33:8443/nifi/' in a frame because it
set 'X-Frame-Options' to 'sameorigin'.

Note: This setup works using NiFi 1.1.2.

Kindly advise. 


Thanks,
Virgil



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: About NIFI-3620: Multipart support in invokeHTTP.java

[Adam Taft]

Multipart is just a set of related content types (multipart/form-data,
multipart/mixed).  InvokeHTTP doesn't care too much about content types, it
just sends bytes verbatim from the flowfile payload.

What should be considered is for an upstream processor to create the
multipart payload in the flowfile.  Then InvokeHTTP can be used to deliver
those bytes.  This would likely be the easiest thing to do, in order to
keep changes to InvokeHTTP to a minimum.  And this could potentially be
used in PostHTTP, and potentially other transports too (smtp maybe).

A multipart message is really a lot like the output from MergeContent.
It's a "tar" format, of sorts.  I wouldn't muddy MergeContent either, but
the concept is closer aligned with packing multiple flowfiles together.

Anyway, I think a separate processor would be ideal, since multipart is
really not related to transport but more of content.

Thanks,

Adam


On Tue, Dec 12, 2017 at 3:45 AM, Damiano Giampaoli 
wrote:

> Ciao Pierre,
>
> I can spend some hours tomorrow looking deep in the invokeHTTP
> implementation and figure out a way to properly support Multipart messages,
> then write back in the ML asking for feedback before implementing it.
> @Andre in case you have already some code or ideas I'll be glad to follow
> it!
>
>
> Best regards
> Damiano
>
> -Original Message-
> From: Pierre Villard [mailto:pierre.villard...@gmail.com]
> Sent: Tuesday, December 12, 2017 11:25 AM
> To: dev 
> Subject: Re: About NIFI-3620: Multipart support in invokeHTTP.java
>
> Hey Damiano,
>
> Andre will correct me in case I'm wrong but he is not working on
> https://issues.apache.org/jira/browse/NIFI-3620 at the moment. If you
> want to give it a try, that would be more than appreciated.
>
> Pierre
>
> 2017-12-11 18:01 GMT+01:00 Joe Witt :
>
> > Hello
> >
> > Sounds great.  I think i might have dropped the ball on that review by
> > commenting on it and then it made others who might be able to help
> > avoid it.  Just commented on the PR again but we're of course happy to
> > work with you to improve as needed.
> >
> > Thanks
> > Joe
> >
> > On Mon, Dec 11, 2017 at 11:10 AM, Damiano Giampaoli
> >  wrote:
> > > Hi list,
> > >
> > >
> > > We are planning to move from our in-house-built workflow engine to
> > NiFi... needless to say, we love this project.
> > >
> > > We created a PoC of some our production workflows using NiFi but in
> > order to move entirely to NiFi we need a full multipart support in
> > order to be able to invoke some of our internal microservices.
> > >
> > >
> > > I saw there is already a pending PR for the ListenHTTP multipart
> > > support<
> > https://github.com/apache/nifi/pull/1795> and an issue already opened
> > about the InvokeHTTP
> > which is marked as in progress.
> > >
> > >
> > > We are willing to contribute to the developments starting from
> > > adding
> > the multipart support but before we would like to ensure that there
> > are no other developers who are already working on this, in case we
> > can provide support in testing, complete or bugfix a work not ready
> > yet to be merged in the master branch.
> > >
> > >
> > >
> > > We are looking forward to hearing from you!
> > >
> > >
> > > Best regards,
> > >
> > > Damiano
> > >
> > >
> > >
> > > SearchInk
> > >
> > >
> > >
> > > Damiano Giampaoli
> > >
> > > Software Engineer
> > >
> > >
> > >
> > > mobile:  +49 1719956912
> > >
> > > email:  
> > > 
> > dami...@searchink.com
> > >
> > >
> > >
> > > #execcircle17 event: Connecting
> > Innovators & Insurers
> > >
> > > Join our team: searchink.com/careers
> > >
> > >
> > >
> > > Koppenplatz 10, D-10115 Berlin
> > > +49 30 220 560 730
> > >
> > > www.searchink.com
> > >
> > >
> > >
> > > HRB 171236 B, Amtsgericht Charlottenburg, Berlin |  UID: DE302404693
> > >
> > > This e-mail and any attached files are confidential and may be
> > > legally
> > privileged. If you are not the addressee, any disclosure,
> > reproduction, copying, distribution, or other dissemination or use of
> > this communication is strictly prohibited. If you have received this
> > transmission in error please notify the sender immediately and then
> delete this mail.
> > >
> > > E-mail transmission cannot be guaranteed to be secure or error free
> > > as
> > information could be intercepted, corrupted, lost, destroyed, arrive
> > late or incomplete, or contain viruses. The sender therefore does not
> > accept liability for any errors or omissions in the contents of this
> > message which arise as a result of e-mail transmission or changes to
> > transmitted date not specifically approved by the sender.
> > >
> > > If this e-mail or attached files contain 

Nifi on Apache Mesos-DC/OS

[Anil Rai]

Hi,

Has anyone tried to deploy nifi on Mesos or DC/OS and is there a
recommended nifi deployment architecture for mesos or DC/OS?

Regards
Anil

Re: Unable to checkpoint FlowFile Repository (No space left on device)

[Michael Moser]

Greetings Ben,

The "No space left on device" error can also be caused by running out of
inodes on your device.  You can check this with "df -i".

-- Mike


On Tue, Dec 12, 2017 at 1:36 AM, 尹文才  wrote:

> sorry that I forgot to mention the environment that caused this problem,
> I'm using the latest nifi 1.4.0 release and installed it on centos 7.
>
> 2017-12-12 14:35 GMT+08:00 尹文才 :
>
> > Hi guys, I'm running into a very weird problem, I wrote a processor
> > specifically to extract some data
> > and I found starting from yesterday it kept showing errors in the log, as
> > below:
> >
> > 2017-12-12 14:01:04,661 INFO [pool-10-thread-1] o.a.n.c.r.
> WriteAheadFlowFileRepository
> > Initiating checkpoint of FlowFile Repository
> > 2017-12-12 14:01:04,676 ERROR [pool-10-thread-1] o.a.n.c.r.
> WriteAheadFlowFileRepository
> > Unable to checkpoint FlowFile Repository due to
> > java.io.FileNotFoundException: ../flowfile_repository/
> partition-5/96.journal
> > (No space left on device)
> > java.io.FileNotFoundException: ../flowfile_repository/
> partition-5/96.journal
> > (No space left on device)
> > at java.io.FileOutputStream.open0(Native Method)
> > at java.io.FileOutputStream.open(FileOutputStream.java:270)
> > at java.io.FileOutputStream.(FileOutputStream.java:213)
> > at java.io.FileOutputStream.(FileOutputStream.java:162)
> > at org.wali.MinimalLockingWriteAheadLog$Partition.rollover(
> > MinimalLockingWriteAheadLog.java:779)
> > at org.wali.MinimalLockingWriteAheadLog.checkpoint(
> > MinimalLockingWriteAheadLog.java:528)
> > at org.apache.nifi.controller.repository.
> > WriteAheadFlowFileRepository.checkpoint(WriteAheadFlowFileRepository.
> > java:451)
> > at org.apache.nifi.controller.repository.
> > WriteAheadFlowFileRepository$1.run(WriteAheadFlowFileRepository.
> java:423)
> > at java.util.concurrent.Executors$RunnableAdapter.
> > call(Executors.java:511)
> > at java.util.concurrent.FutureTask.runAndReset(
> > FutureTask.java:308)
> > at java.util.concurrent.ScheduledThreadPoolExecutor$
> > ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > at java.util.concurrent.ScheduledThreadPoolExecutor$
> > ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> > I noticed the log mentioned no space left on device and I went to check
> > the available space and found 33G left. Does anyone know what could
> > possibly cause this and how to resolve this problem, thanks
> >
> > /Ben
> >
>

Re: Adding user in NIFI-1.4.0

[tanezavm]

Hi Pallavi,

You need to configure nifi for a secure access (https) so you can login to
NiFi as a specific user.
There is a nifi tls toolkit that you can download for a start to create
certificates and keys for your nifi server as well as for a user.

https://community.hortonworks.com/articles/58233/using-the-tls-toolkit-to-simplify-security.html
https://community.hortonworks.com/articles/81184/understanding-the-initial-admin-identity-access-po.html

Regards,
Virgil



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

RE: About NIFI-3620: Multipart support in invokeHTTP.java

[Damiano Giampaoli]

Ciao Pierre,

I can spend some hours tomorrow looking deep in the invokeHTTP implementation 
and figure out a way to properly support Multipart messages, then write back in 
the ML asking for feedback before implementing it. 
@Andre in case you have already some code or ideas I'll be glad to follow it!


Best regards
Damiano

-Original Message-
From: Pierre Villard [mailto:pierre.villard...@gmail.com] 
Sent: Tuesday, December 12, 2017 11:25 AM
To: dev 
Subject: Re: About NIFI-3620: Multipart support in invokeHTTP.java

Hey Damiano,

Andre will correct me in case I'm wrong but he is not working on
https://issues.apache.org/jira/browse/NIFI-3620 at the moment. If you want to 
give it a try, that would be more than appreciated.

Pierre

2017-12-11 18:01 GMT+01:00 Joe Witt :

> Hello
>
> Sounds great.  I think i might have dropped the ball on that review by 
> commenting on it and then it made others who might be able to help 
> avoid it.  Just commented on the PR again but we're of course happy to 
> work with you to improve as needed.
>
> Thanks
> Joe
>
> On Mon, Dec 11, 2017 at 11:10 AM, Damiano Giampaoli 
>  wrote:
> > Hi list,
> >
> >
> > We are planning to move from our in-house-built workflow engine to
> NiFi... needless to say, we love this project.
> >
> > We created a PoC of some our production workflows using NiFi but in
> order to move entirely to NiFi we need a full multipart support in 
> order to be able to invoke some of our internal microservices.
> >
> >
> > I saw there is already a pending PR for the ListenHTTP multipart 
> > support<
> https://github.com/apache/nifi/pull/1795> and an issue already opened 
> about the InvokeHTTP
> which is marked as in progress.
> >
> >
> > We are willing to contribute to the developments starting from 
> > adding
> the multipart support but before we would like to ensure that there 
> are no other developers who are already working on this, in case we 
> can provide support in testing, complete or bugfix a work not ready 
> yet to be merged in the master branch.
> >
> >
> >
> > We are looking forward to hearing from you!
> >
> >
> > Best regards,
> >
> > Damiano
> >
> >
> >
> > SearchInk
> >
> >
> >
> > Damiano Giampaoli
> >
> > Software Engineer
> >
> >
> >
> > mobile:  +49 1719956912
> >
> > email:   
> > 
> dami...@searchink.com
> >
> >
> >
> > #execcircle17 event: Connecting
> Innovators & Insurers
> >
> > Join our team: searchink.com/careers
> >
> >
> >
> > Koppenplatz 10, D-10115 Berlin
> > +49 30 220 560 730
> >
> > www.searchink.com
> >
> >
> >
> > HRB 171236 B, Amtsgericht Charlottenburg, Berlin |  UID: DE302404693
> >
> > This e-mail and any attached files are confidential and may be 
> > legally
> privileged. If you are not the addressee, any disclosure, 
> reproduction, copying, distribution, or other dissemination or use of 
> this communication is strictly prohibited. If you have received this 
> transmission in error please notify the sender immediately and then delete 
> this mail.
> >
> > E-mail transmission cannot be guaranteed to be secure or error free 
> > as
> information could be intercepted, corrupted, lost, destroyed, arrive 
> late or incomplete, or contain viruses. The sender therefore does not 
> accept liability for any errors or omissions in the contents of this 
> message which arise as a result of e-mail transmission or changes to 
> transmitted date not specifically approved by the sender.
> >
> > If this e-mail or attached files contain information which do not 
> > relate
> to our professional activity we do not accept liability for such 
> information.
> >
>

Re: About NIFI-3620: Multipart support in invokeHTTP.java

[Pierre Villard]

Hey Damiano,

Andre will correct me in case I'm wrong but he is not working on
https://issues.apache.org/jira/browse/NIFI-3620 at the moment. If you want
to give it a try, that would be more than appreciated.

Pierre

2017-12-11 18:01 GMT+01:00 Joe Witt :

> Hello
>
> Sounds great.  I think i might have dropped the ball on that review by
> commenting on it and then it made others who might be able to help
> avoid it.  Just commented on the PR again but we're of course happy to
> work with you to improve as needed.
>
> Thanks
> Joe
>
> On Mon, Dec 11, 2017 at 11:10 AM, Damiano Giampaoli
>  wrote:
> > Hi list,
> >
> >
> > We are planning to move from our in-house-built workflow engine to
> NiFi... needless to say, we love this project.
> >
> > We created a PoC of some our production workflows using NiFi but in
> order to move entirely to NiFi we need a full multipart support in order to
> be able to invoke some of our internal microservices.
> >
> >
> > I saw there is already a pending PR for the ListenHTTP multipart support<
> https://github.com/apache/nifi/pull/1795> and an issue already opened
> about the InvokeHTTP
> which is marked as in progress.
> >
> >
> > We are willing to contribute to the developments starting from adding
> the multipart support but before we would like to ensure that there are no
> other developers who are already working on this, in case we can provide
> support in testing, complete or bugfix a work not ready yet to be merged in
> the master branch.
> >
> >
> >
> > We are looking forward to hearing from you!
> >
> >
> > Best regards,
> >
> > Damiano
> >
> >
> >
> > SearchInk
> >
> >
> >
> > Damiano Giampaoli
> >
> > Software Engineer
> >
> >
> >
> > mobile:  +49 1719956912
> >
> > email:   
> dami...@searchink.com
> >
> >
> >
> > #execcircle17 event: Connecting
> Innovators & Insurers
> >
> > Join our team: searchink.com/careers
> >
> >
> >
> > Koppenplatz 10, D-10115 Berlin
> > +49 30 220 560 730
> >
> > www.searchink.com
> >
> >
> >
> > HRB 171236 B, Amtsgericht Charlottenburg, Berlin |  UID: DE302404693
> >
> > This e-mail and any attached files are confidential and may be legally
> privileged. If you are not the addressee, any disclosure, reproduction,
> copying, distribution, or other dissemination or use of this communication
> is strictly prohibited. If you have received this transmission in error
> please notify the sender immediately and then delete this mail.
> >
> > E-mail transmission cannot be guaranteed to be secure or error free as
> information could be intercepted, corrupted, lost, destroyed, arrive late
> or incomplete, or contain viruses. The sender therefore does not accept
> liability for any errors or omissions in the contents of this message which
> arise as a result of e-mail transmission or changes to transmitted date not
> specifically approved by the sender.
> >
> > If this e-mail or attached files contain information which do not relate
> to our professional activity we do not accept liability for such
> information.
> >
>