答复： Re: Failed to open the page

[luby]

Thanks a lot for help.

I've deleted the work directory and restart service, but it still doesn't 
work.






发件人: "Mark Payne" 
收件人: "dev@nifi.apache.org" 
日期:   2022/01/13 10:06
主题:   Re: Failed to open the page



Boying,

You can safely delete the work directory while nifi is stopped. It will 
generate the directory again on start. That being said, 1.6.0 is extremely 
old. Recommend you upgrade to 1.15.2. 

Thanks
-Mark

Sent from my iPhone

> On Jan 12, 2022, at 9:03 PM, l...@china-inv.cn wrote:
> 
> Hi, All,
> 
> We are using NiFi 1.6.0.
> 
> After restart nifi service (using 'service nifi start'), the page 
> xxx:8079/nifi
> failed to be shown and an error page is shown with following info:
> 
> HTTP ERROR 404
> 
> Problem accessing /index.jsp. Reason:
> 
>/index.jsp
> 
> I found the root cause is that some files failed to be created under the 

> work/jetty/xxx/jsp/
> 
> Does anyone know the location of the source codes that create those 
files?
> 
> Thanks
> 
> Boying
> 
> 
> 
> 
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
> 
> 
> This email message may contain confidential and/or privileged 
information. 
> If you are not the intended recipient, please do not read, save, 
forward, 
> disclose or copy the contents of this email or open any file attached to 

> this email. We will be grateful if you could advise the sender 
immediately 
> by replying this email, and delete this email and any attachment or 
links 
> to this email completely and immediately from your computer system. 
> 
> 
> 






 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

Failed to open the page

[luby]

Hi, All,

We are using NiFi 1.6.0.

After restart nifi service (using 'service nifi start'), the page 
xxx:8079/nifi
failed to be shown and an error page is shown with following info:

HTTP ERROR 404

Problem accessing /index.jsp. Reason:

/index.jsp

I found the root cause is that some files failed to be created under the 
work/jetty/xxx/jsp/

Does anyone know the location of the source codes that create those files?

Thanks

Boying



 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

Failed to open NIFI page

[luby]

Hi, experts,

I'm using NiFi 1.6.0 and after stoping and restarting nifi service on my 
linux box (as root),

I can't open http://localhost:8079, I was told:
HTTP ERROR 404
Problem accessing /index.jsp. Reason:
 /index.jsp

Someone told me to delete following subdirectries under /opt/nifi to fix 
the issue:
content_repository
database_repository
flowfile_repository
provenance_repository

But after deleting these directories and restart nifi, I got following 
error in browser:
HTTP ERROR 404

Problem accessing /nifi/canvas.jsp. Reason:

/nifi/canvas.jsp

Does anyone know how to fix this issue?

Thanks a lot in advance.






 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

How to enable processors after changing a controller service automatically?

[luby]

Hi, All,

I'm using NiFi 1.6.0. I need to change a setting of a contoller service, 
NiFi stopped all the processors that reference this service then disable 
the service before I could make changes.
After I made the change and re-enable the service, I found that the 
processors were NOT
re-enabled automatically.

I have to enable these processors manually. Since there are thousands of 
such processors
this operation is very time-consuming. 

Does anyone know how to tell NiFi to start those processors automatically?

Thanks 



陆伯鹰
中国投资有限责任公司信息技术部
电话：+86 (0)10 84096521
传真：+86 (0)10 64086851 
北京市东城区朝阳门北大街1号新保利大厦8层 100010
网站：www.china-inv.cn 
 




 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: Re: A question to copy files to other folders

[luby]

oh, yes. I mixed the two replied emails. sorry :)



发件人: 
"Sivaprasanna" 
收件人:
dev@nifi.apache.org
日期:
2018/03/01 13:46
主题:
Re: Re: A question to copy files to other folders



Yep. And that’s what Matt suggested.

On Thu, 1 Mar 2018 at 8:16 AM,  wrote:

> I found that I can set the 'Directory' property of 'PutFile' processor 
to
> B/${path} and C/${path}
> to solve this issue. So the patch is not needed I think
>
>
>
>
> 发件人:
> l...@china-inv.cn
> 收件人:
>
> 抄送:
> dev@nifi.apache.org
> 日期:
> 2018/03/01 09:17
> 主题:
> 答复:  Re: A question to copy files to other folders
>
>
>
> I haven't tried it yet.
>
> I just find that the PutFile doesn't meet my requirements.
>
> I'll create a patch for PutFile.
>
>
>
> 发件人:
> Matt Burgess 
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/28 21:10
> 主题:
> Re: A question to copy files to other folders
>
>
>
> Looking at the code, it appears that PutFile should support the
> creation of arbitrary directories if the Create Missing Directories
> property is set to true. With that and setting the Directory property
> to ${path}, I would think that would create the subdirectories
> properly. If not, what error are you getting?
>
> Regards,
> Matt
>
> On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen 
> wrote:
> > That seems like a pretty easy thing to fix with PutFile. Could be done
> with
> > a patch to add an attribute that provides a relative path.
> >
> > On Wed, Feb 28, 2018 at 7:00 AM,  wrote:
> >
> >> Hi, team,
> >>
> >> I'm writing a data flow template to copy files from directory A to
> other
> >> two directories B and C
> >>
> >> There are sub directories under A and I need to copy all files under
> those
> >> sub directories to the same
> >> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
> >> C/foo/a.txt
> >>
> >> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't 
support
> >> creating sub directories under B and C
> >> (i.e. B/foo and C/foo in above example).
> >>
> >> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
> >> attribute of flowfile. But the 'PutFile' doesn't use it
> >> to create.
> >>
> >> I don't want to create those folders manually and create data flow by
> >> using 'GetFile' --> 'PutFile' for each folder because there
> >> will be too many data flows.
> >>
> >>
> >> Is there any processor that support creating folder and copying 
files?
> >>
> >> Thanks
> >>
> >> Boying
> >>
> >>
> >>
> >>
> >> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存
、
> 对外
> >> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告
知
> 发件
> >> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
> >>
> >>
> >> This email message may contain confidential and/or privileged
> information.
> >> If you are not the intended recipient, please do not read, save,
> forward,
> >> disclose or copy the contents of this email or open any file attached
> to
> >> this email. We will be grateful if you could advise the sender
> immediately
> >> by replying this email, and delete this email and any attachment or
> links
> >> to this email completely and immediately from your computer system.
> >>
> >>
> >>
> >>
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged 
information.
>
> If you are not the intended recipient, please do not read, save, 
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender 
immediately
>
> by replying this email, and delete this email and any attachment or 
links
> to this email completely and immediately from your computer system.
>
>
>
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged 
information.
> If you are not the intended recipient, please do not read, save, 
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender 
immediately
> by replying this email, and delete this email and any attachment or 
links
> to this email completely and immediately from your computer system.
>
>
>
>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: A question to copy files to other folders

[luby]

I found that I can set the 'Directory' property of 'PutFile' processor to 
B/${path} and C/${path}
to solve this issue. So the patch is not needed I think




发件人: 
l...@china-inv.cn
收件人:

抄送:
dev@nifi.apache.org
日期:
2018/03/01 09:17
主题:
答复:  Re: A question to copy files to other folders



I haven't tried it yet.

I just find that the PutFile doesn't meet my requirements.

I'll create a patch for PutFile.



发件人: 
Matt Burgess 
收件人:
dev@nifi.apache.org
日期:
2018/02/28 21:10
主题:
Re: A question to copy files to other folders



Looking at the code, it appears that PutFile should support the
creation of arbitrary directories if the Create Missing Directories
property is set to true. With that and setting the Directory property
to ${path}, I would think that would create the subdirectories
properly. If not, what error are you getting?

Regards,
Matt

On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen  
wrote:
> That seems like a pretty easy thing to fix with PutFile. Could be done 
with
> a patch to add an attribute that provides a relative path.
>
> On Wed, Feb 28, 2018 at 7:00 AM,  wrote:
>
>> Hi, team,
>>
>> I'm writing a data flow template to copy files from directory A to 
other
>> two directories B and C
>>
>> There are sub directories under A and I need to copy all files under 
those
>> sub directories to the same
>> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
>> C/foo/a.txt
>>
>> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
>> creating sub directories under B and C
>> (i.e. B/foo and C/foo in above example).
>>
>> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
>> attribute of flowfile. But the 'PutFile' doesn't use it
>> to create.
>>
>> I don't want to create those folders manually and create data flow by
>> using 'GetFile' --> 'PutFile' for each folder because there
>> will be too many data flows.
>>
>>
>> Is there any processor that support creating folder and copying files?
>>
>> Thanks
>>
>> Boying
>>
>>
>>
>>
>> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、
对外
>> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知
发件
>> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>>
>>
>> This email message may contain confidential and/or privileged 
information.
>> If you are not the intended recipient, please do not read, save, 
forward,
>> disclose or copy the contents of this email or open any file attached 
to
>> this email. We will be grateful if you could advise the sender 
immediately
>> by replying this email, and delete this email and any attachment or 
links
>> to this email completely and immediately from your computer system.
>>
>>
>>
>>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 

If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 

by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 








 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: A question to copy files to other folders

[luby]

I haven't tried it yet.

I just find that the PutFile doesn't meet my requirements.

I'll create a patch for PutFile.



发件人: 
Matt Burgess 
收件人:
dev@nifi.apache.org
日期:
2018/02/28 21:10
主题:
Re: A question to copy files to other folders



Looking at the code, it appears that PutFile should support the
creation of arbitrary directories if the Create Missing Directories
property is set to true. With that and setting the Directory property
to ${path}, I would think that would create the subdirectories
properly. If not, what error are you getting?

Regards,
Matt

On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen  
wrote:
> That seems like a pretty easy thing to fix with PutFile. Could be done 
with
> a patch to add an attribute that provides a relative path.
>
> On Wed, Feb 28, 2018 at 7:00 AM,  wrote:
>
>> Hi, team,
>>
>> I'm writing a data flow template to copy files from directory A to 
other
>> two directories B and C
>>
>> There are sub directories under A and I need to copy all files under 
those
>> sub directories to the same
>> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
>> C/foo/a.txt
>>
>> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
>> creating sub directories under B and C
>> (i.e. B/foo and C/foo in above example).
>>
>> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
>> attribute of flowfile. But the 'PutFile' doesn't use it
>> to create.
>>
>> I don't want to create those folders manually and create data flow by
>> using 'GetFile' --> 'PutFile' for each folder because there
>> will be too many data flows.
>>
>>
>> Is there any processor that support creating folder and copying files?
>>
>> Thanks
>>
>> Boying
>>
>>
>>
>>
>> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、
对外
>> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知
发件
>> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>>
>>
>> This email message may contain confidential and/or privileged 
information.
>> If you are not the intended recipient, please do not read, save, 
forward,
>> disclose or copy the contents of this email or open any file attached 
to
>> this email. We will be grateful if you could advise the sender 
immediately
>> by replying this email, and delete this email and any attachment or 
links
>> to this email completely and immediately from your computer system.
>>
>>
>>
>>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

A question to copy files to other folders

[luby]

Hi, team,

I'm writing a data flow template to copy files from directory A to other 
two directories B and C

There are sub directories under A and I need to copy all files under those 
sub directories to the same
palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and 
C/foo/a.txt

I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support 
creating sub directories under B and C
(i.e. B/foo and C/foo in above example).

The 'GetFile' processor saves the relative path 'foo' in the 'PATH' 
attribute of flowfile. But the 'PutFile' doesn't use it
to create.

I don't want to create those folders manually and create data flow by 
using 'GetFile' --> 'PutFile' for each folder because there
will be too many data flows.


Is there any processor that support creating folder and copying files?

Thanks 

Boying



 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

NiFi opens too many jar files

[luby]

Hi, Team,

We are using NiFi 1.3.0 and yesterday our NiFi server doesn't response 
anymore.
The root cause is that the NiFi service opens too many files (about 
1,000,000+ files)

Further investigation shows that most opened files are jar files in nar 
bundles (about 980,000+ files)
Same jar file in a nar file was opened multiple times without closing any 
of them. e.g.
I found file 
/opt/nifi/nifi-1.3.0/work/nar/extensions/nifi-websocket-services-api-nar-1.3.0.nar-unpacked/META-INF/bundled-dependencies/nifi-utils-1.3.0.jar
was opened several times without closing.

Are those jar files loaded by NarClassLoader and why they are opened so 
many times?

Any solution to this issue?

Thanks 

Boying




 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?

[luby]

Yes, that is what I do currently.

But I think it will be better if NiFi can support this feature natively.



发件人: 
"Andrew Grande" 
收件人:
dev@nifi.apache.org
日期:
2018/02/23 09:07
主题:
Re: Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?



One could write a script and call it in 1 step. I don't believe there is
anything available OOTB.

Andrew

On Thu, Feb 22, 2018, 7:58 PM  wrote:

>  Thanks a lot for your help.
>
> Yes. that is what I do to trigger a dataflow on demand.
> But I want to know if there is an API that I can do this in one step.
>
>
>
> 发件人:
> "Daniel Chaffelson" 
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/23 04:46
> 主题:
> Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
>
>
>
> Hi Boying,
>
> I have been working on a NiFi Python Client SDK that might help you 
here,
> as the goal is to be able to replicate everyday actions taken in the 
NiFi
> GUI as well as extending it for CICD/SDLC work.
> For example with the following commands you would:
>
>1. get the reference object for a processor
>2. stop it if it is running
>3. change the scheduling period to 3s (or most other parameters)
>4. start it again
>
>
> import nipyapi
> processor_state_1 = nipyapi.canvas.get_processor('MyProcessor')
> nipyapi.canvas.schedule_processor(processor, scheduled=False)
> update = nipyapi.nifi.ProcessorConfigDTO(
> scheduling_period='3s'
> )
> processor_state_2 = nipyapi.canvas.update_processor(processor, update)
> nipyapi.canvas.schedule_processor(processor, scheduled=True)
>
> If you need a different set of steps then please let me know and perhaps 
I
> can help.
> Those commands are currently in the master branch awaiting release:
> https://github.com/Chaffelson/nipyapi
>
> Thanks,
> Dan
>
> On Thu, Feb 22, 2018 at 7:41 AM  wrote:
>
> > Thanks very much, I'll try your suggestions.
> >
> >
> >
> > 发件人:
> > James Wing 
> > 收件人:
> > NiFi Dev List 
> > 日期:
> > 2018/02/22 14:05
> > 主题:
> > Re: 答复: Re: Is there a REST API to run a dataflow on demand?
> >
> >
> >
> > The NiFi API can be used to start and stop processors or process 
groups,
> > and this might solve your use case.  But NiFi does not have an API to
> run
> > a
> > processor only once, immediately, separate from its configured 
schedule.
> I
> > have solved similar problems in the past by creating two separate
> upstream
> > sources - one for scheduled operation, and one for ad-hoc operation.
> > GenerateFlowFile, GetFile, or similar processors can be used to inject 
a
> > flowfile where you need to kick off the flow.
> >
> > Thanks,
> >
> > James
> >
> > On Wed, Feb 21, 2018 at 5:57 PM,  wrote:
> >
> > > Thanks a lot.
> > >
> > > But I want to know if there is a REST API that triggers a dataflow 
on
> > > demand?
> > > I don't find the API in the page.
> > >
> > >
> > >
> > >
> > > 发件人:
> > > Charlie Meyer 
> > > 收件人:
> > > dev@nifi.apache.org
> > > 日期:
> > > 2018/02/22 09:36
> > > 主题:
> > > Re: Is there a REST API to run a dataflow on demand?
> > >
> > >
> > >
> > > Yep, when you make the changes in the UI, open developer tools in 
your
> > > browser and see what calls to the nifi api it is making then mimic
> those
> > > with code.
> > >
> > > The nifi team also kindly publishes
> > > https://nifi.apache.org/docs/nifi-docs/rest-api/index.html which 
help
> a
> > > lot.
> > >
> > > Best of luck!
> > >
> > > -Charlie
> > >
> > > On Wed, Feb 21, 2018 at 7:34 PM,  wrote:
> > >
> > > > Hi, team,
> > > >
> > > > We set up several NiFi dataflows for data processing.
> > > > These dataflows are configured to run once per day in the 
midnight.
> > > >
> > > > But sometimes, some dataflows are failed,I want to run the 
dataflow
> > > again
> > > > immediately after fixing the issue instead of waiting for running 
it
> > in
> > > > the midnight to
> > > > make sure that the issue is really fixed.
> > > >
> > > > The only way I know to do this is to change the time of running 
the
> > > > dataflow to the 5 mintutes from now for example
> > > > and then change it back to midnight.
> > > >
> > > > It's a little inconvenient.
> > > >
> > > > Is there any REST API that I can use to trigger the dataflow on
> demand
> > > > i.e. without change the time back and forth?
> > > >
> > > > Thanks
> > > >
> > > > Boying
> > > >
> > > >
> > > >
> > > > 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保
存
> 、
> > 对
> > > 外
> > > > 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件
告
> 知
> > 发
> > > 件
> > > > 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
> > > >
> > > >
> > > > This email message may contain confidential and/or privileged
> > > information.
> > > > If you are not the intended recipient, please do not read, save,
> > > forward,
> > > > disclose or copy the contents of this email or open any file
> attached
> > to
> > > > this email. We will be grateful if you 

答复: Re: 答复: Re: Is there a REST API to run a dataflow on demand?

[luby]

Thanks very much, I'll try your suggestions.



发件人: 
James Wing 
收件人:
NiFi Dev List 
日期:
2018/02/22 14:05
主题:
Re: 答复: Re: Is there a REST API to run a dataflow on demand?



The NiFi API can be used to start and stop processors or process groups,
and this might solve your use case.  But NiFi does not have an API to run 
a
processor only once, immediately, separate from its configured schedule. I
have solved similar problems in the past by creating two separate upstream
sources - one for scheduled operation, and one for ad-hoc operation.
GenerateFlowFile, GetFile, or similar processors can be used to inject a
flowfile where you need to kick off the flow.

Thanks,

James

On Wed, Feb 21, 2018 at 5:57 PM,  wrote:

> Thanks a lot.
>
> But I want to know if there is a REST API that triggers a dataflow on
> demand?
> I don't find the API in the page.
>
>
>
>
> 发件人:
> Charlie Meyer 
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/22 09:36
> 主题:
> Re: Is there a REST API to run a dataflow on demand?
>
>
>
> Yep, when you make the changes in the UI, open developer tools in your
> browser and see what calls to the nifi api it is making then mimic those
> with code.
>
> The nifi team also kindly publishes
> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html which help a
> lot.
>
> Best of luck!
>
> -Charlie
>
> On Wed, Feb 21, 2018 at 7:34 PM,  wrote:
>
> > Hi, team,
> >
> > We set up several NiFi dataflows for data processing.
> > These dataflows are configured to run once per day in the midnight.
> >
> > But sometimes, some dataflows are failed,I want to run the dataflow
> again
> > immediately after fixing the issue instead of waiting for running it 
in
> > the midnight to
> > make sure that the issue is really fixed.
> >
> > The only way I know to do this is to change the time of running the
> > dataflow to the 5 mintutes from now for example
> > and then change it back to midnight.
> >
> > It's a little inconvenient.
> >
> > Is there any REST API that I can use to trigger the dataflow on demand
> > i.e. without change the time back and forth?
> >
> > Thanks
> >
> > Boying
> >
> >
> >
> > 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、
对
> 外
> > 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知
发
> 件
> > 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
> >
> >
> > This email message may contain confidential and/or privileged
> information.
> > If you are not the intended recipient, please do not read, save,
> forward,
> > disclose or copy the contents of this email or open any file attached 
to
> > this email. We will be grateful if you could advise the sender
> immediately
> > by replying this email, and delete this email and any attachment or
> links
> > to this email completely and immediately from your computer system.
> >
> >
> >
> >
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged 
information.
> If you are not the intended recipient, please do not read, save, 
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender 
immediately
> by replying this email, and delete this email and any attachment or 
links
> to this email completely and immediately from your computer system.
>
>
>
>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: Is there a REST API to run a dataflow on demand?

[luby]

Thanks a lot.

But I want to know if there is a REST API that triggers a dataflow on 
demand?
I don't find the API in the page.




发件人: 
Charlie Meyer 
收件人:
dev@nifi.apache.org
日期:
2018/02/22 09:36
主题:
Re: Is there a REST API to run a dataflow on demand?



Yep, when you make the changes in the UI, open developer tools in your
browser and see what calls to the nifi api it is making then mimic those
with code.

The nifi team also kindly publishes
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html which help a 
lot.

Best of luck!

-Charlie

On Wed, Feb 21, 2018 at 7:34 PM,  wrote:

> Hi, team,
>
> We set up several NiFi dataflows for data processing.
> These dataflows are configured to run once per day in the midnight.
>
> But sometimes, some dataflows are failed,I want to run the dataflow 
again
> immediately after fixing the issue instead of waiting for running it in
> the midnight to
> make sure that the issue is really fixed.
>
> The only way I know to do this is to change the time of running the
> dataflow to the 5 mintutes from now for example
> and then change it back to midnight.
>
> It's a little inconvenient.
>
> Is there any REST API that I can use to trigger the dataflow on demand
> i.e. without change the time back and forth?
>
> Thanks
>
> Boying
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged 
information.
> If you are not the intended recipient, please do not read, save, 
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender 
immediately
> by replying this email, and delete this email and any attachment or 
links
> to this email completely and immediately from your computer system.
>
>
>
>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

Is there a REST API to run a dataflow on demand?

[luby]

Hi, team,

We set up several NiFi dataflows for data processing.
These dataflows are configured to run once per day in the midnight.

But sometimes, some dataflows are failed,I want to run the dataflow again 
immediately after fixing the issue instead of waiting for running it in 
the midnight to 
make sure that the issue is really fixed.

The only way I know to do this is to change the time of running the 
dataflow to the 5 mintutes from now for example
and then change it back to midnight.

It's a little inconvenient.

Is there any REST API that I can use to trigger the dataflow on demand 
i.e. without change the time back and forth?

Thanks 

Boying


 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: 答复: Re: NiFi starts very slowly

[luby]

Thanks for your help.

Actually we are using Kylo (https://kylo.io/) which uses NiFi to do the 
work.

There are about 10 processors in each group, so there will be 16000 
processors in total.

I understand that the processor should be validate before it starts, but 
could this validation be defered until the processor (or the group it 
belongs to) is to start for the first time
instead of doing this when NiFi service starts up?



发件人: 
Joe Witt <joe.w...@gmail.com>
收件人:
dev@nifi.apache.org
日期:
2017/12/29 09:20
主题:
Re: 答复: Re: NiFi starts very slowly



hello

the total number of processors is probably a factor.  how many are there?
can you please list how many processors of each type there are?  Any 
custom
processors?

creating 1600 disjoint flows is probably going against the grain of nifi.
cant the flows be designed to operate as a directed graph amd leverage
attributes?

keep in mind to start each processor we call validate.  this could be the
cause of the slowness.

help us collect the above info and we can give pointers.  if you could
share your flow.xml.gz that would be great.

thanks
joe

On Dec 28, 2017 8:13 PM, <l...@china-inv.cn> wrote:

> Thanks for your reply.
>
> We are not using a cluster, our NiFi 1.3 runs in the single node mode.
>
>
>
> 发件人:
> "Mohammed Nadeem" <nadeemm...@gmail.com>
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2017/12/28 20:11
> 主题:
> Re: NiFi starts very slowly
>
>
>
> Hi Luby,
>
> If you are using a cluster based setup then Nifi is unstable due to 
nodes
> getting disconnected by the cluster cordinator. Cluster coordinator
> disconnects nodes from the cluster when the nodes take a little longer 
to
> respond ( beyond timeout) . The cluster coordinator timeout based on the
> settings in the nifi.properties file.
>
> We need to adjust the following properties
>
> *nifi.cluster.node.connection.timeout * = 5 sec  ( by default )
> *nifi.cluster.node.read.timeout *=  5 sec ( by default)
>
> By Increasing the timeout ,from 5 sec to higher value would probably 
solve
> a
> problem
>
>
> Thanks,
> Nadeem
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged 
information.
> If you are not the intended recipient, please do not read, save, 
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender 
immediately
> by replying this email, and delete this email and any attachment or 
links
> to this email completely and immediately from your computer system.
>
>
>
>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: NiFi starts very slowly

[luby]

Thanks for your reply.

We are not using a cluster, our NiFi 1.3 runs in the single node mode.



发件人: 
"Mohammed Nadeem" <nadeemm...@gmail.com>
收件人:
dev@nifi.apache.org
日期:
2017/12/28 20:11
主题:
Re: NiFi starts very slowly



Hi Luby,

If you are using a cluster based setup then Nifi is unstable due to nodes
getting disconnected by the cluster cordinator. Cluster coordinator
disconnects nodes from the cluster when the nodes take a little longer to
respond ( beyond timeout) . The cluster coordinator timeout based on the
settings in the nifi.properties file.

We need to adjust the following properties 

*nifi.cluster.node.connection.timeout * = 5 sec  ( by default )
*nifi.cluster.node.read.timeout *=  5 sec ( by default)

By Increasing the timeout ,from 5 sec to higher value would probably solve 
a
problem


Thanks,
Nadeem



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

NiFi starts very slowly

[luby]

Hi, All,

We created about 1600+ processor groups based on a NiFi template, but 
after creating those groups, we found 
tht NiFi service start very slowly, it takes about 40min ~ 1 hour to 
start.

We are using NiFi 1.3.

Does anyone know how to speed it up?

Thanks 

Boying


 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

答复: Re: How to delete the data in the flowfile?

[luby]

Very appreicate for your helpl. It's very helpful. :)



发件人: 
Jeff 
收件人:
dev@nifi.apache.org
日期:
2017/11/20 22:07
主题:
Re: How to delete the data in the flowfile?



Hello Boying,

Once flowfiles have completed processing, they may still be archived 
within
the content repository for a certain period of time before they age-off. 
In
the NiFi Admin guide, there is a section on Content Repository properties
[1] you can set in nifi.properties, through which you can tweak how much
space is used to archive, how long flowfiles are archived, or to disable
archiving completely.

Lowering the "nifi.content.repository.archive.max.retention.period" and
"nifi.content.repository.archive.max.usage.percentage" properties can help
limit the amount of disk space the content repository uses for archived
flowfiles.  You can disable content archiving by setting
"nifi.content.repository.archive.enabled" to false if you prefer to have 
no
archive at all.

If your flow uses a processor like PutFile to place a flowfile in a
temporary directory to do further processing on it, or to allow "backups"
of the flowfile for various stages of processing, then your flow must be
designed to clean up those files after they are no longer needed.  There
are several ways to do this, one of them being Wait/Notify processors.
There's a blog that Koji has written [2] with some examples on how to use
the Wait and Notify processors, and the concepts covered in the blog 
should
be usable in your case where you might want to use the Wait/Notify
processors to signal that flowfiles that are no longer needed that have
been explicitly archived/copied by processors like "PutFile" can be 
removed.

Please let me know if neither of these solutions help with disk space
issues while using your flow.  If you provide your flow as an example, we
can take a look at other ways to try to minimize disk usage.

[1]
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-system-content-repository-properties

[2]
http://ijokarumawak.github.io/nifi/2017/02/02/nifi-notify-batch/#alternative-solution-waitnotify


On Mon, Nov 20, 2017 at 3:16 AM  wrote:

> Hi, All,
>
> We use NiFi to import data from Oracle database to Hive.
>
> The first step is to extract all data from the Oracle database and 
persist
> it into the flowfile
> which will then 'flow' into other processors to do further processing.
>
> After persisting the data into the Hive, we found that the data 
persisted
> in the first step were not
> deteled. This will occupied a lot of disk spaces.
>
> So is there any way to tell NiFi to delete those data after the next
> processor has finished reading the data?
>
> Thanks
>
> Boying
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged 
information.
> If you are not the intended recipient, please do not read, save, 
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender 
immediately
> by replying this email, and delete this email and any attachment or 
links
> to this email completely and immediately from your computer system.
>
>
>
>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

How to delete the data in the flowfile?

[luby]

Hi, All,

We use NiFi to import data from Oracle database to Hive.

The first step is to extract all data from the Oracle database and persist 
it into the flowfile 
which will then 'flow' into other processors to do further processing.

After persisting the data into the Hive, we found that the data persisted 
in the first step were not
deteled. This will occupied a lot of disk spaces.

So is there any way to tell NiFi to delete those data after the next 
processor has finished reading the data?

Thanks

Boying



 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 

Questions to debugging NiFi

[luby]

Hi, All,

I'm new to NiFi and deployed NiFi 1.3.0 in our test environment, I got 
some issues about NiFi and want to debug it.

I've checked out source code and want to add some logs for debug.

I've several questions:
1. I added some logs in FlowResource.java, how do I know which nar file 
contains the FlowResource.class?
2. How to generate a specific nar file with mvn without building all other 
nar files?
3. How to delete all my NiFi data manually if the UI is unaccessible?

Thanks

Boying


 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system.