Re: Dynamic attributes on repeating capture groups

[wildo]

Fantastic! Thanks for the info!



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Dynamic-attributes-on-repeating-capture-groups-tp16540p16568.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Dynamic attributes on repeating capture groups

[James Wing]

An exception thrown in your script and not caught will cause the process
session to roll back, meaning that the flowfile will be go back to the
input queue and retry.  If you have the processor set to run on schedule
every 0 seconds, it might retry a lot :).  You are correct that routing to
REL_FAILURE is usually better than retrying, but I suppose it depends on
the particular flow.

ExecuteScript provides a local script variable, 'log', which you can use to
log messages to the regular NiFi log.  The messages get sent to the same
log files, following the rules outlined in conf/logback.xml, and will show
up in the UI as bulletins if the severity deserves it.  So you can do

log.error("Something bad happpened")

And it will both log and display as you would expect a processor error to
display.  It is an instance of ComponentLog, so there are also debug(),
info(), warn(), etc.

James

On Wed, Aug 2, 2017 at 12:22 PM, wildo  wrote:

> James- thanks for the comment. I've been hacking on this all day and think
> I
> have the python written to do what I need. I have just one follow up
> question as I'm new to the ExecuteScript processor.
>
>
> -What do I do on exception? Some of the cookbook example scripts seem to
> print to traceback:
>
> except:
> traceback.print_exec(file=sys.stdout)
> raise
>
>
> ...but I'm curious what you should do in a production environment. Instead
> of raising the exception, should you instead transfer the session to
> REL_FAILURE? Or maybe raising the exception automatically transfers to
> REL_FAILURE?
>
> Also- I'm guessing that in the real world you'd probably write that
> exception to some logger, rather than stdout correct?
>
>
>
>
>
> --
> View this message in context: http://apache-nifi-developer-l
> ist.39713.n7.nabble.com/Dynamic-attributes-on-repeating-
> capture-groups-tp16540p16565.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: How to add headers to CSV file

[mayank rathi]

Thanks Mark. I am almost there.

Text in Flow file: ZYZTF9y wrote:

> Hello,
>
> You should be able to do this with ReplaceText, using a Replacement
> Strategy
> of Prepend. The Replacement Value would then be the header that you want
> to add to the
> beginning of the file.
>
> Thanks
> -Mark
>
>
> > On Aug 2, 2017, at 3:36 AM, mayank rathi  wrote:
> >
> > Hello All,
> >
> > I am working on a NiFi flow to read data from a database and dump it
> into a
> > CSV file.
> >
> > I will use ExecuteSQL to read data and PutFile to generate CSV file.
> >
> > Which processor can I use to add header to this CSV file?
> >
> > Thanks in advance
> > MR
> >
> > --
> > NOTICE: This email message is for the sole use of the intended
> recipient(s)
> > and may contain confidential and privileged information. Any unauthorized
> > review, use, disclosure or distribution is prohibited. If you are not the
> > intended recipient, please contact the sender by reply email and destroy
> > all copies of the original message.
>
>


-- 
NOTICE: This email message is for the sole use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message.

Re: Dynamic attributes on repeating capture groups

[wildo]

James- thanks for the comment. I've been hacking on this all day and think I
have the python written to do what I need. I have just one follow up
question as I'm new to the ExecuteScript processor.


-What do I do on exception? Some of the cookbook example scripts seem to
print to traceback:

except:
traceback.print_exec(file=sys.stdout)
raise


...but I'm curious what you should do in a production environment. Instead
of raising the exception, should you instead transfer the session to
REL_FAILURE? Or maybe raising the exception automatically transfers to
REL_FAILURE?

Also- I'm guessing that in the real world you'd probably write that
exception to some logger, rather than stdout correct?





--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Dynamic-attributes-on-repeating-capture-groups-tp16540p16565.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Nifi Site-to-Site pre-existing deployments do not work after enabling TLS and Ldap

[Mark Payne]

Hello,

Yes, that is true. If you change the web port of the NiFi instance, it would be 
considered a 'new instance' in terms
of Remote Process Groups. So any NiFi instance that is sending to/pulling from 
that instance would have to create
a new Remote Process Group to point to the new URL. This probably should be 
improved by allowing the user to
change the URL that the Remote Process Group points to. Feel free to file a 
JIRA [1] if you think this would be a
helpful improvement.

In terms of updating policies, yes you would need to give permissions to all of 
the ports that you want users to have
access to. Typically, this is best done by using Groups so that each port can 
be configured to allow Site-to-Site for a
Group of users and then you can just change the members of that Group. This way 
you don't have to update many
Ports each time that you want to change permissions. Do you have other ideas in 
mind of how this would be made
easier?

Thanks
-Mark


[1] https://issues.apache.org/jira/projects/NIFI


On Aug 2, 2017, at 6:45 AM, nifi-san 
> wrote:

Thanks Mark,

I had a follow up question though.

Let's say you have a  nifi flow with site-to-site deployment between two
nodes node-1- source and node-2
destination on a non secure cluster.

The default http port "8080" is used in the configuration of the RPG on node
which is http://node-2:8080/nifi.

Once you configure ssl to secure your cluster,you may have the node
bootstrapping on the https port ,let's say 9966.

All the previously configured RPG's will get affected because of this since
the nodes are no longer going to listen on the http port.

How do we handle such a scenario?You cannot even manually modify an existing
RPC to listen to the new ssl port.
Also, with the ssl configuration, on the remote node where the Input Port is
configured ,you would need to modify the access policies to "Receive data
site-to-site".

Assuming you have quite a few flows using RPG, manually changing them might
be very difficult.




--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Site-to-Site-pre-existing-deployments-do-not-work-after-enabling-TLS-and-Ldap-tp16486p16560.html
Sent from the Apache NiFi Developer List mailing list archive at 
Nabble.com.

Re: How to add headers to CSV file

[Mark Payne]

Hello,

You should be able to do this with ReplaceText, using a Replacement Strategy
of Prepend. The Replacement Value would then be the header that you want to add 
to the
beginning of the file.

Thanks
-Mark


> On Aug 2, 2017, at 3:36 AM, mayank rathi  wrote:
> 
> Hello All,
> 
> I am working on a NiFi flow to read data from a database and dump it into a
> CSV file.
> 
> I will use ExecuteSQL to read data and PutFile to generate CSV file.
> 
> Which processor can I use to add header to this CSV file?
> 
> Thanks in advance
> MR
> 
> -- 
> NOTICE: This email message is for the sole use of the intended recipient(s)
> and may contain confidential and privileged information. Any unauthorized
> review, use, disclosure or distribution is prohibited. If you are not the
> intended recipient, please contact the sender by reply email and destroy
> all copies of the original message.