Re: PutSQL ERROR bulletin

[Joe Witt]

Hello Sven

LogAttributes will just show the attributes as they are understood by
the flow file itself.  But the PutSQL processor may be doing something
more specific with the data.  Can you share your configuration for
PutSQL?

Thanks
Joe

On Fri, Aug 12, 2016 at 7:05 PM, Sven Davison  wrote:
> Actualy… I’m fairly sure I found it. I sent stuff off to “logAttribute”
> processor and found the input is not escaped.
>
>
>
> http://prntscr.com/c51je3
>
>
>
>
>
>
>
> Sent from Mail for Windows 10
>
>
>
> From: Sven Davison
> Sent: Friday, August 12, 2016 7:02 PM
> To: users@nifi.apache.org
> Subject: PutSQL ERROR bulletin
>
>
>
> I’m getting several inserts but every once in a while (every 1-2 minutes or
> so)… I get this error. Anyone know what might cause this?
>
>
> PutSQL[id=b8d54aa6-567a-4686-96bc-1c00e5d43461] Failed to update database
> due to a failed batch update. There were a total of 1 FlowFiles that failed,
> 0 that succeeded, and 0 that were not execute and will be routed to retry;
>
>
>
>
>
> -Sven
>
>
>
> Sent from Mail for Windows 10
>
>
>
>

RE: PutSQL ERROR bulletin

[Sven Davison]

Actualy… I’m fairly sure I found it. I sent stuff off to “logAttribute” 
processor and found the input is not escaped.

http://prntscr.com/c51je3



Sent from Mail for Windows 10

From: Sven Davison

PutSQL ERROR bulletin

[Sven Davison]

I’m getting several inserts but every once in a while (every 1-2 minutes or 
so)… I get this error. Anyone know what might cause this?

PutSQL[id=b8d54aa6-567a-4686-96bc-1c00e5d43461] Failed to update database due 
to a failed batch update. There were a total of 1 FlowFiles that failed, 0 that 
succeeded, and 0 that were not execute and will be routed to retry;


-Sven

Sent from Mail for Windows 10

NiFi cluster error

[Sven Davison]

I had a working cluster.. now when i try to restart a couple of the child
nodes... it stops durring spin up. If i dont start the master node, the
child will run until i DO start the master, then it dies. The message is
the same in both conditions.

"Failed to connect node to cluster due to:
java.lang.illegalargumentexception: invalide attribute key: 


i removed the flow.xml.gz from the child node and restarted it, same
error... anyone know what i might try next?

RE: How to deal with decimals while they're not supported?

[Lee Laim (leelaim)]

Hi Stephane, 
I've been porting  to an Execute Stream Command (ESC) processor  with a python 
one-liner to do that math, return to the output as an attribute, and finally 
pass the original to the next processor.  It is sufficiently fast for our use 
case.

Argument: -c "from __future__ import division; print '%.1f%%' 
%(${numerator:multiply(100)}/${denominator})"
Command: /usr/bin/python
Output Destination Attribute: metric_percent

Thanks,
Lee



-Original Message-
From: Joe Percivall [mailto:joeperciv...@yahoo.com] 
Sent: Friday, August 12, 2016 8:22 AM
To: users@nifi.apache.org
Subject: Re: How to deal with decimals while they're not supported?

Not sure why my email got delayed so late but Conrad's way is definitely an 
option too. When you know that all the numbers will have the exact same number 
of decimals it is probably faster than my way (substring vs. a regex). My way 
will allow you to just grab the most significant decimal point, regardless of 
size, and compare on that.

Regardless, there seems to be a couple of different ways to accomplish this and 
I'm glad you got it working.
Joe
 
- - - - - - 
Joseph Percivall
linkedin.com/in/Percivall
e: joeperciv...@yahoo.com



On Friday, August 12, 2016 5:11 AM, Joe Percivall  
wrote:



Hey Stephane,

Currently working with Decimals in NiFi is like putting a square peg in a round 
hole. I haven't tried to do it much but for a simple use-case like yours I 
believe there are two options. 

1: Use Expression Language and a Regex to move the decimal
2: (Not 100% sure this works) Use the scripting processors to write a script to 
do the routing

For option 1 I think you can do something like this 
"${myAttr:replaceAll("0.(\d)(\d*)", "$1"):lt(1)}. This should "move" the 
decimal over one place so that you can check it against the 1. This assumes 
there is nothing else in that attribute.
For option 2, Matt Burgess would need to weigh-in.

As for NIFI-1662, it got de-prioritzed in that I got super busy with everything 
else I was doing for MiNiFi-0.0.1 and NiFi-1.0.0 (on-going) and it went to the 
back-burner. It's something I really want to get in though, along with a few 
other changes, because I think they open NiFi up to many new possibilities. I'd 
like to do it (decimals) in 1.1.0 since it wouldn't be a breaking change.

Hope that helps,
Joe
 
- - - - - - 
Joseph Percivall
linkedin.com/in/Percivall
e: joeperciv...@yahoo.com



On Thursday, August 11, 2016 8:16 PM, Stéphane Maarek 
 wrote:



Hi,

I have a flow in which I extract an attribute from json using jsonpath. That 
attribute happens to be a decimal number (0.123). I wanted to do a simple 
operation such as myAttr:lt(0.1) but obviously that won't work. What also won't 
work is myAttr:multiply(10):lt(1). I'm kinda stuck and I really need this logic 
to be working. What do you advise as a workaround?

Also, I've seen there is a JIRA for this: 
https://issues.apache.org/jira/browse/NIFI-1662 but stuff hasn't moved much 
since it first appeared. Not sure if it got de-prioritized or something

Congrats on the 1.0.0 beta, it looks great !!

Cheers,
Stephane

Re: How to deal with decimals while they're not supported?

[Joe Percivall]

Hey Stephane,
Currently working with Decimals in NiFi is like putting a square peg in a round 
hole. I haven't tried to do it much but for a simple use-case like yours I 
believe there are two options. 
1: Use Expression Language and a Regex to move the decimal2: (Not 100% sure 
this works) Use the scripting processors to write a script to do the routing
For option 1 I think you can do something like this 
"${myAttr:replaceAll("0.(\d)(\d*)", "$1"):lt(1)}. This should "move" the 
decimal over one place so that you can check it against the 1. This assumes 
there is nothing else in that attribute.For option 2, Matt Burgess would need 
to weigh-in.
As for NIFI-1662, it got de-prioritzed in that I got super busy with everything 
else I was doing for MiNiFi-0.0.1 and NiFi-1.0.0 (on-going) and it went to the 
back-burner. It's something I really want to get in though, along with a few 
other changes, because I think they open NiFi up to many new possibilities. I'd 
like to do it (decimals) in 1.1.0 since it wouldn't be a breaking change.
Hope that helps,Joe - - - - - - Joseph Percivalllinkedin.com/in/Percivalle: 
joeperciv...@yahoo.com
 

On Thursday, August 11, 2016 8:16 PM, Stéphane Maarek 
 wrote:
 

 Hi,
I have a flow in which I extract an attribute from json using jsonpath. That 
attribute happens to be a decimal number (0.123). I wanted to do a simple 
operation such as myAttr:lt(0.1) but obviously that won't work. What also won't 
work is myAttr:multiply(10):lt(1). I'm kinda stuck and I really need this logic 
to be working. What do you advise as a workaround?
Also, I've seen there is a JIRA for this: 
https://issues.apache.org/jira/browse/NIFI-1662 but stuff hasn't moved much 
since it first appeared. Not sure if it got de-prioritized or something
Congrats on the 1.0.0 beta, it looks great !!
Cheers,Stephane

  

Re: How to deal with decimals while they're not supported?

[Conrad Crampton]

Excellent.
Again, haven’t tried it, but could possibly calculate the scaling factor from 
doing a length of the string after the decimal point. Haven’t really worked 
this through and may get very complicated.
Alternatively you could do another substring on the decimal portion and limit 
it to just two decimal places for example.
Regards
Conrad

From: Stéphane Maarek 
Reply-To: "users@nifi.apache.org" 
Date: Friday, 12 August 2016 at 08:04
To: "users@nifi.apache.org" 
Subject: Re: How to deal with decimals while they're not supported?

That works :) That implies knowing a bit about the scale of numbers, but that's 
a decent workaround. It'd be so great if decimals were supported natively 
though. Or at least the casting of decimal to int without having to remove the 
decimal
i.e (1.2):toNumber() instead of 1.2:toString().substringBefore('.'):toNumber()

On Fri, Aug 12, 2016 at 4:59 PM Conrad Crampton 
> wrote:
HI,
Can you coerce to a string, take the numbers after the decimal point and use 
that? So a similar logic to multiplying by 10 then doing the comparison. 
Obviously have to revert back to number to do the actual comparison though.
Haven’t tested but something like…
myAttr:toString():substringAfter(‘.’):toNumber():lt(1)
Conrad

From: Stéphane Maarek 
>
Reply-To: "users@nifi.apache.org" 
>
Date: Friday, 12 August 2016 at 01:16
To: "users@nifi.apache.org" 
>
Subject: How to deal with decimals while they're not supported?

Hi,

I have a flow in which I extract an attribute from json using jsonpath. That 
attribute happens to be a decimal number (0.123). I wanted to do a simple 
operation such as myAttr:lt(0.1) but obviously that won't work. What also won't 
work is myAttr:multiply(10):lt(1). I'm kinda stuck and I really need this logic 
to be working. What do you advise as a workaround?

Also, I've seen there is a JIRA for this: 
https://issues.apache.org/jira/browse/NIFI-1662 but stuff hasn't moved much 
since it first appeared. Not sure if it got de-prioritized or something

Congrats on the 1.0.0 beta, it looks great !!

Cheers,
Stephane


***This email originated outside SecureData***

Click here to report 
this email as spam.

SecureData, combating cyber threats



The information contained in this message or any of its attachments may be 
privileged and confidential and intended for the exclusive use of the intended 
recipient. If you are not the intended recipient any disclosure, reproduction, 
distribution or other dissemination or use of this communications is strictly 
prohibited. The views expressed in this email are those of the individual and 
not necessarily of SecureData Europe Ltd. Any prices quoted are only valid if 
followed up by a formal written quote.

SecureData Europe Limited. Registered in England & Wales 04365896. Registered 
Address: SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent, 
ME16 9NT

Re: How to deal with decimals while they're not supported?

[Stéphane Maarek]

That works :) That implies knowing a bit about the scale of numbers, but
that's a decent workaround. It'd be so great if decimals were supported
natively though. Or at least the casting of decimal to int without having
to remove the decimal
i.e (1.2):toNumber() instead of
1.2:toString().substringBefore('.'):toNumber()

On Fri, Aug 12, 2016 at 4:59 PM Conrad Crampton 
wrote:

> HI,
>
> Can you coerce to a string, take the numbers after the decimal point and
> use that? So a similar logic to multiplying by 10 then doing the
> comparison. Obviously have to revert back to number to do the actual
> comparison though.
>
> Haven’t tested but something like…
>
> myAttr:toString():substringAfter(‘.’):toNumber():lt(1)
>
> Conrad
>
>
>
> *From: *Stéphane Maarek 
> *Reply-To: *"users@nifi.apache.org" 
> *Date: *Friday, 12 August 2016 at 01:16
> *To: *"users@nifi.apache.org" 
> *Subject: *How to deal with decimals while they're not supported?
>
>
>
> Hi,
>
>
>
> I have a flow in which I extract an attribute from json using jsonpath.
> That attribute happens to be a decimal number (0.123). I wanted to do a
> simple operation such as myAttr:lt(0.1) but obviously that won't work. What
> also won't work is myAttr:multiply(10):lt(1). I'm kinda stuck and I really
> need this logic to be working. What do you advise as a workaround?
>
>
> Also, I've seen there is a JIRA for this:
> https://issues.apache.org/jira/browse/NIFI-1662 but stuff hasn't moved
> much since it first appeared. Not sure if it got de-prioritized or something
>
>
>
> Congrats on the 1.0.0 beta, it looks great !!
>
>
>
> Cheers,
>
> Stephane
>
>
>
> ***This email originated outside SecureData***
>
> Click here  to
> report this email as spam.
>
>
> SecureData, combating cyber threats
>
> --
>
> The information contained in this message or any of its attachments may be
> privileged and confidential and intended for the exclusive use of the
> intended recipient. If you are not the intended recipient any disclosure,
> reproduction, distribution or other dissemination or use of this
> communications is strictly prohibited. The views expressed in this email
> are those of the individual and not necessarily of SecureData Europe Ltd.
> Any prices quoted are only valid if followed up by a formal written quote.
>
> SecureData Europe Limited. Registered in England & Wales 04365896.
> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
> Maidstone, Kent, ME16 9NT
>

Re: How to deal with decimals while they're not supported?

[Conrad Crampton]

HI,
Can you coerce to a string, take the numbers after the decimal point and use 
that? So a similar logic to multiplying by 10 then doing the comparison. 
Obviously have to revert back to number to do the actual comparison though.
Haven’t tested but something like…
myAttr:toString():substringAfter(‘.’):toNumber():lt(1)
Conrad

From: Stéphane Maarek 
Reply-To: "users@nifi.apache.org" 
Date: Friday, 12 August 2016 at 01:16
To: "users@nifi.apache.org" 
Subject: How to deal with decimals while they're not supported?

Hi,

I have a flow in which I extract an attribute from json using jsonpath. That 
attribute happens to be a decimal number (0.123). I wanted to do a simple 
operation such as myAttr:lt(0.1) but obviously that won't work. What also won't 
work is myAttr:multiply(10):lt(1). I'm kinda stuck and I really need this logic 
to be working. What do you advise as a workaround?

Also, I've seen there is a JIRA for this: 
https://issues.apache.org/jira/browse/NIFI-1662 but stuff hasn't moved much 
since it first appeared. Not sure if it got de-prioritized or something

Congrats on the 1.0.0 beta, it looks great !!

Cheers,
Stephane


***This email originated outside SecureData***

Click here to report 
this email as spam.


SecureData, combating cyber threats
__ 
The information contained in this message or any of its attachments may be 
privileged and confidential and intended for the exclusive use of the intended 
recipient. If you are not the intended recipient any disclosure, reproduction, 
distribution or other dissemination or use of this communications is strictly 
prohibited. The views expressed in this email are those of the individual and 
not necessarily of SecureData Europe Ltd. Any prices quoted are only valid if 
followed up by a formal written quote.

SecureData Europe Limited. Registered in England & Wales 04365896. Registered 
Address: SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent, 
ME16 9NT