Re: Re: Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
I believe even if it is where to be made available as part of NiFi REST
API, I think it is internally going to call the start and stop APIs.
On Fri, 23 Feb 2018 at 6:54 AM, <l...@china-inv.cn> wrote:
> Yes, that is what I do currently.
>
> But I think it will be better if NiFi can support this feature natively.
>
>
>
> 发件人:
> "Andrew Grande" <apere...@gmail.com>
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/23 09:07
> 主题:
> Re: Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
>
>
>
> One could write a script and call it in 1 step. I don't believe there is
> anything available OOTB.
>
> Andrew
>
> On Thu, Feb 22, 2018, 7:58 PM <l...@china-inv.cn> wrote:
>
> > Thanks a lot for your help.
> >
> > Yes. that is what I do to trigger a dataflow on demand.
> > But I want to know if there is an API that I can do this in one step.
> >
> >
> >
> > 发件人:
> > "Daniel Chaffelson" <chaffel...@gmail.com>
> > 收件人:
> > dev@nifi.apache.org
> > 日期:
> > 2018/02/23 04:46
> > 主题:
> > Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
> >
> >
> >
> > Hi Boying,
> >
> > I have been working on a NiFi Python Client SDK that might help you
> here,
> > as the goal is to be able to replicate everyday actions taken in the
> NiFi
> > GUI as well as extending it for CICD/SDLC work.
> > For example with the following commands you would:
> >
> >1. get the reference object for a processor
> >2. stop it if it is running
> >3. change the scheduling period to 3s (or most other parameters)
> >4. start it again
> >
> >
> > import nipyapi
> > processor_state_1 = nipyapi.canvas.get_processor('MyProcessor')
> > nipyapi.canvas.schedule_processor(processor, scheduled=False)
> > update = nipyapi.nifi.ProcessorConfigDTO(
> > scheduling_period='3s'
> > )
> > processor_state_2 = nipyapi.canvas.update_processor(processor, update)
> > nipyapi.canvas.schedule_processor(processor, scheduled=True)
> >
> > If you need a different set of steps then please let me know and perhaps
> I
> > can help.
> > Those commands are currently in the master branch awaiting release:
> > https://github.com/Chaffelson/nipyapi
> >
> > Thanks,
> > Dan
> >
> > On Thu, Feb 22, 2018 at 7:41 AM <l...@china-inv.cn> wrote:
> >
> > > Thanks very much, I'll try your suggestions.
> > >
> > >
> > >
> > > 发件人:
> > > James Wing <jvw...@gmail.com>
> > > 收件人:
> > > NiFi Dev List <dev@nifi.apache.org>
> > > 日期:
> > > 2018/02/22 14:05
> > > 主题:
> > > Re: 答复: Re: Is there a REST API to run a dataflow on demand?
> > >
> > >
> > >
> > > The NiFi API can be used to start and stop processors or process
> groups,
> > > and this might solve your use case. But NiFi does not have an API to
> > run
> > > a
> > > processor only once, immediately, separate from its configured
> schedule.
> > I
> > > have solved similar problems in the past by creating two separate
> > upstream
> > > sources - one for scheduled operation, and one for ad-hoc operation.
> > > GenerateFlowFile, GetFile, or similar processors can be used to inject
> a
> > > flowfile where you need to kick off the flow.
> > >
> > > Thanks,
> > >
> > > James
> > >
> > > On Wed, Feb 21, 2018 at 5:57 PM, <l...@china-inv.cn> wrote:
> > >
> > > > Thanks a lot.
> > > >
> > > > But I want to know if there is a REST API that triggers a dataflow
> on
> > > > demand?
> > > > I don't find the API in the page.
> > > >
> > > >
> > > >
> > > >
> > > > 发件人:
> > > > Charlie Meyer <charlie.me...@civitaslearning.com>
> > > > 收件人:
> > > > dev@nifi.apache.org
> > > > 日期:
> > > > 2018/02/22 09:36
> > > > 主题:
> > > > Re: Is there a REST API to run a dataflow on demand?
> > > >
> > > >
> > > >
> > > > Yep, when you make the changes in the UI, open developer tools in
> your
> > > > browser and see what calls to the nifi api it is making then mimic
> > those
> > > > with code.
> > > >
> > > > The nifi team also kindly publishes
> > > > https://nifi.apache
答复: Re: Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
Yes, that is what I do currently.
But I think it will be better if NiFi can support this feature natively.
发件人:
"Andrew Grande" <apere...@gmail.com>
收件人:
dev@nifi.apache.org
日期:
2018/02/23 09:07
主题:
Re: Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
One could write a script and call it in 1 step. I don't believe there is
anything available OOTB.
Andrew
On Thu, Feb 22, 2018, 7:58 PM <l...@china-inv.cn> wrote:
> Thanks a lot for your help.
>
> Yes. that is what I do to trigger a dataflow on demand.
> But I want to know if there is an API that I can do this in one step.
>
>
>
> 发件人:
> "Daniel Chaffelson" <chaffel...@gmail.com>
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/23 04:46
> 主题:
> Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
>
>
>
> Hi Boying,
>
> I have been working on a NiFi Python Client SDK that might help you
here,
> as the goal is to be able to replicate everyday actions taken in the
NiFi
> GUI as well as extending it for CICD/SDLC work.
> For example with the following commands you would:
>
>1. get the reference object for a processor
>2. stop it if it is running
>3. change the scheduling period to 3s (or most other parameters)
>4. start it again
>
>
> import nipyapi
> processor_state_1 = nipyapi.canvas.get_processor('MyProcessor')
> nipyapi.canvas.schedule_processor(processor, scheduled=False)
> update = nipyapi.nifi.ProcessorConfigDTO(
> scheduling_period='3s'
> )
> processor_state_2 = nipyapi.canvas.update_processor(processor, update)
> nipyapi.canvas.schedule_processor(processor, scheduled=True)
>
> If you need a different set of steps then please let me know and perhaps
I
> can help.
> Those commands are currently in the master branch awaiting release:
> https://github.com/Chaffelson/nipyapi
>
> Thanks,
> Dan
>
> On Thu, Feb 22, 2018 at 7:41 AM <l...@china-inv.cn> wrote:
>
> > Thanks very much, I'll try your suggestions.
> >
> >
> >
> > 发件人:
> > James Wing <jvw...@gmail.com>
> > 收件人:
> > NiFi Dev List <dev@nifi.apache.org>
> > 日期:
> > 2018/02/22 14:05
> > 主题:
> > Re: 答复: Re: Is there a REST API to run a dataflow on demand?
> >
> >
> >
> > The NiFi API can be used to start and stop processors or process
groups,
> > and this might solve your use case. But NiFi does not have an API to
> run
> > a
> > processor only once, immediately, separate from its configured
schedule.
> I
> > have solved similar problems in the past by creating two separate
> upstream
> > sources - one for scheduled operation, and one for ad-hoc operation.
> > GenerateFlowFile, GetFile, or similar processors can be used to inject
a
> > flowfile where you need to kick off the flow.
> >
> > Thanks,
> >
> > James
> >
> > On Wed, Feb 21, 2018 at 5:57 PM, <l...@china-inv.cn> wrote:
> >
> > > Thanks a lot.
> > >
> > > But I want to know if there is a REST API that triggers a dataflow
on
> > > demand?
> > > I don't find the API in the page.
> > >
> > >
> > >
> > >
> > > 发件人:
> > > Charlie Meyer <charlie.me...@civitaslearning.com>
> > > 收件人:
> > > dev@nifi.apache.org
> > > 日期:
> > > 2018/02/22 09:36
> > > 主题:
> > > Re: Is there a REST API to run a dataflow on demand?
> > >
> > >
> > >
> > > Yep, when you make the changes in the UI, open developer tools in
your
> > > browser and see what calls to the nifi api it is making then mimic
> those
> > > with code.
> > >
> > > The nifi team also kindly publishes
> > > https://nifi.apache.org/docs/nifi-docs/rest-api/index.html which
help
> a
> > > lot.
> > >
> > > Best of luck!
> > >
> > > -Charlie
> > >
> > > On Wed, Feb 21, 2018 at 7:34 PM, <l...@china-inv.cn> wrote:
> > >
> > > > Hi, team,
> > > >
> > > > We set up several NiFi dataflows for data processing.
> > > > These dataflows are configured to run once per day in the
midnight.
> > > >
> > > > But sometimes, some dataflows are failed,I want to run the
dataflow
> > > again
> > > > immediately after fixing the issue instead of waiting for running
it
> > in
> > > > the midnight to
> > > > make sure that the issue is really fixed.
> > > >
> > > > The only way I know to do this is to change
Re: Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
One could write a script and call it in 1 step. I don't believe there is
anything available OOTB.
Andrew
On Thu, Feb 22, 2018, 7:58 PM <l...@china-inv.cn> wrote:
> Thanks a lot for your help.
>
> Yes. that is what I do to trigger a dataflow on demand.
> But I want to know if there is an API that I can do this in one step.
>
>
>
> 发件人:
> "Daniel Chaffelson" <chaffel...@gmail.com>
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/23 04:46
> 主题:
> Re: Re: 答复: Re: Is there a REST API to run a dataflow on demand?
>
>
>
> Hi Boying,
>
> I have been working on a NiFi Python Client SDK that might help you here,
> as the goal is to be able to replicate everyday actions taken in the NiFi
> GUI as well as extending it for CICD/SDLC work.
> For example with the following commands you would:
>
>1. get the reference object for a processor
>2. stop it if it is running
>3. change the scheduling period to 3s (or most other parameters)
>4. start it again
>
>
> import nipyapi
> processor_state_1 = nipyapi.canvas.get_processor('MyProcessor')
> nipyapi.canvas.schedule_processor(processor, scheduled=False)
> update = nipyapi.nifi.ProcessorConfigDTO(
> scheduling_period='3s'
> )
> processor_state_2 = nipyapi.canvas.update_processor(processor, update)
> nipyapi.canvas.schedule_processor(processor, scheduled=True)
>
> If you need a different set of steps then please let me know and perhaps I
> can help.
> Those commands are currently in the master branch awaiting release:
> https://github.com/Chaffelson/nipyapi
>
> Thanks,
> Dan
>
> On Thu, Feb 22, 2018 at 7:41 AM <l...@china-inv.cn> wrote:
>
> > Thanks very much, I'll try your suggestions.
> >
> >
> >
> > 发件人:
> > James Wing <jvw...@gmail.com>
> > 收件人:
> > NiFi Dev List <dev@nifi.apache.org>
> > 日期:
> > 2018/02/22 14:05
> > 主题:
> > Re: 答复: Re: Is there a REST API to run a dataflow on demand?
> >
> >
> >
> > The NiFi API can be used to start and stop processors or process groups,
> > and this might solve your use case. But NiFi does not have an API to
> run
> > a
> > processor only once, immediately, separate from its configured schedule.
> I
> > have solved similar problems in the past by creating two separate
> upstream
> > sources - one for scheduled operation, and one for ad-hoc operation.
> > GenerateFlowFile, GetFile, or similar processors can be used to inject a
> > flowfile where you need to kick off the flow.
> >
> > Thanks,
> >
> > James
> >
> > On Wed, Feb 21, 2018 at 5:57 PM, <l...@china-inv.cn> wrote:
> >
> > > Thanks a lot.
> > >
> > > But I want to know if there is a REST API that triggers a dataflow on
> > > demand?
> > > I don't find the API in the page.
> > >
> > >
> > >
> > >
> > > 发件人:
> > > Charlie Meyer <charlie.me...@civitaslearning.com>
> > > 收件人:
> > > dev@nifi.apache.org
> > > 日期:
> > > 2018/02/22 09:36
> > > 主题:
> > > Re: Is there a REST API to run a dataflow on demand?
> > >
> > >
> > >
> > > Yep, when you make the changes in the UI, open developer tools in your
> > > browser and see what calls to the nifi api it is making then mimic
> those
> > > with code.
> > >
> > > The nifi team also kindly publishes
> > > https://nifi.apache.org/docs/nifi-docs/rest-api/index.html which help
> a
> > > lot.
> > >
> > > Best of luck!
> > >
> > > -Charlie
> > >
> > > On Wed, Feb 21, 2018 at 7:34 PM, <l...@china-inv.cn> wrote:
> > >
> > > > Hi, team,
> > > >
> > > > We set up several NiFi dataflows for data processing.
> > > > These dataflows are configured to run once per day in the midnight.
> > > >
> > > > But sometimes, some dataflows are failed,I want to run the dataflow
> > > again
> > > > immediately after fixing the issue instead of waiting for running it
> > in
> > > > the midnight to
> > > > make sure that the issue is really fixed.
> > > >
> > > > The only way I know to do this is to change the time of running the
> > > > dataflow to the 5 mintutes from now for example
> > > > and then change it back to midnight.
> > > >
> > > > It's a little inconvenient.
> > > >
> > > > Is there any REST API that I can use to trigger
