Re: Object not recognized in ExecuteScript

[Mike Thomsen]

Jim,

You need to call session.commit() periodically if you want to progressively
push flowfiles out. Though you need to think about the failure scenarios
your script has and whether you really want to sacrifice the ability to do
a rollback in the event things go wrong.


On Thu, Feb 15, 2018 at 6:27 AM, James McMahon  wrote:

> Matt, I have noticed another behavior pattern I hope to correct, but I
> don't fully understand it and so hope that you can provide some insights.
>
> I have embedded the flowFile related commands to create the flowFiles
> within my file processing loop. My expectation was that I would see the
> output queue from ExecuteScript increment as each file was created. But I
> don't see that behavior. instead, I see a zero in the output queue for
> several minutes, and then suddenly the count jumps to the total set of
> flowFiles created.
>
> Because working back through the directory structure in time can be a time
> consuming process, I hope to output to flow incremental results as files
> are found. How can I make that happen? Is there a means to periodically
> flush what is in the output buffer every N files, perhaps?
>
> Thanks again for your insights.  -Jim
>
> On Mon, Feb 12, 2018 at 12:24 PM, James McMahon 
> wrote:
>
>> I see. Thank you Matt. -Jim
>>
>> On Mon, Feb 12, 2018 at 12:10 PM, Matt Burgess 
>> wrote:
>>
>>> Jim,
>>>
>>> In this case I don't think it's as much that the modules aren't being
>>> found, rather that the datetime module in Jython returns
>>> java.sql.Timestamp (Java) objects, rather than Jython/Python datetime
>>> objects, and the former do not support the methods/attributes of the
>>> latter, including timetuple(). Apparently [1] this change was made
>>> around Jython 2.5, and NiFi uses 2.7.1.
>>>
>>> Looks like you'll need to write your own timetuple() function, using
>>> the java.sql.Timestamp [2] and related APIs.
>>>
>>> Regards,
>>> Matt
>>>
>>> [1] http://www.jython.org/javadoc/com/ziclix/python/sql/Jython22
>>> DataHandler.html
>>> [2] https://docs.oracle.com/javase/8/docs/api/java/sql/Timestamp.html
>>>
>>> On Mon, Feb 12, 2018 at 11:58 AM, James McMahon 
>>> wrote:
>>> > Good afternoon. I havd a python script that I can execute from the
>>> command
>>> > line via my python interpreter. In it, I do this
>>> >
>>> > myTime = time.mktime(myDateTime.timetuple())
>>> >
>>> > When I try to run from my ExecuteScript processor in NiFi, this is not
>>> > recognized. This error gets thrown:
>>> >
>>> > 'java.sql.Timestamp' object has no attribute 'timetuple' in 

Re: Object not recognized in ExecuteScript

[James McMahon]

Matt, I have noticed another behavior pattern I hope to correct, but I
don't fully understand it and so hope that you can provide some insights.

I have embedded the flowFile related commands to create the flowFiles
within my file processing loop. My expectation was that I would see the
output queue from ExecuteScript increment as each file was created. But I
don't see that behavior. instead, I see a zero in the output queue for
several minutes, and then suddenly the count jumps to the total set of
flowFiles created.

Because working back through the directory structure in time can be a time
consuming process, I hope to output to flow incremental results as files
are found. How can I make that happen? Is there a means to periodically
flush what is in the output buffer every N files, perhaps?

Thanks again for your insights.  -Jim

On Mon, Feb 12, 2018 at 12:24 PM, James McMahon 
wrote:

> I see. Thank you Matt. -Jim
>
> On Mon, Feb 12, 2018 at 12:10 PM, Matt Burgess 
> wrote:
>
>> Jim,
>>
>> In this case I don't think it's as much that the modules aren't being
>> found, rather that the datetime module in Jython returns
>> java.sql.Timestamp (Java) objects, rather than Jython/Python datetime
>> objects, and the former do not support the methods/attributes of the
>> latter, including timetuple(). Apparently [1] this change was made
>> around Jython 2.5, and NiFi uses 2.7.1.
>>
>> Looks like you'll need to write your own timetuple() function, using
>> the java.sql.Timestamp [2] and related APIs.
>>
>> Regards,
>> Matt
>>
>> [1] http://www.jython.org/javadoc/com/ziclix/python/sql/Jython22
>> DataHandler.html
>> [2] https://docs.oracle.com/javase/8/docs/api/java/sql/Timestamp.html
>>
>> On Mon, Feb 12, 2018 at 11:58 AM, James McMahon 
>> wrote:
>> > Good afternoon. I havd a python script that I can execute from the
>> command
>> > line via my python interpreter. In it, I do this
>> >
>> > myTime = time.mktime(myDateTime.timetuple())
>> >
>> > When I try to run from my ExecuteScript processor in NiFi, this is not
>> > recognized. This error gets thrown:
>> >
>> > 'java.sql.Timestamp' object has no attribute 'timetuple' in 

Re: Object not recognized in ExecuteScript

[James McMahon]

I see. Thank you Matt. -Jim

On Mon, Feb 12, 2018 at 12:10 PM, Matt Burgess  wrote:

> Jim,
>
> In this case I don't think it's as much that the modules aren't being
> found, rather that the datetime module in Jython returns
> java.sql.Timestamp (Java) objects, rather than Jython/Python datetime
> objects, and the former do not support the methods/attributes of the
> latter, including timetuple(). Apparently [1] this change was made
> around Jython 2.5, and NiFi uses 2.7.1.
>
> Looks like you'll need to write your own timetuple() function, using
> the java.sql.Timestamp [2] and related APIs.
>
> Regards,
> Matt
>
> [1] http://www.jython.org/javadoc/com/ziclix/python/sql/
> Jython22DataHandler.html
> [2] https://docs.oracle.com/javase/8/docs/api/java/sql/Timestamp.html
>
> On Mon, Feb 12, 2018 at 11:58 AM, James McMahon 
> wrote:
> > Good afternoon. I havd a python script that I can execute from the
> command
> > line via my python interpreter. In it, I do this
> >
> > myTime = time.mktime(myDateTime.timetuple())
> >
> > When I try to run from my ExecuteScript processor in NiFi, this is not
> > recognized. This error gets thrown:
> >
> > 'java.sql.Timestamp' object has no attribute 'timetuple' in 

Re: Object not recognized in ExecuteScript

[Matt Burgess]

Jim,

In this case I don't think it's as much that the modules aren't being
found, rather that the datetime module in Jython returns
java.sql.Timestamp (Java) objects, rather than Jython/Python datetime
objects, and the former do not support the methods/attributes of the
latter, including timetuple(). Apparently [1] this change was made
around Jython 2.5, and NiFi uses 2.7.1.

Looks like you'll need to write your own timetuple() function, using
the java.sql.Timestamp [2] and related APIs.

Regards,
Matt

[1] http://www.jython.org/javadoc/com/ziclix/python/sql/Jython22DataHandler.html
[2] https://docs.oracle.com/javase/8/docs/api/java/sql/Timestamp.html

On Mon, Feb 12, 2018 at 11:58 AM, James McMahon  wrote:
> Good afternoon. I havd a python script that I can execute from the command
> line via my python interpreter. In it, I do this
>
> myTime = time.mktime(myDateTime.timetuple())
>
> When I try to run from my ExecuteScript processor in NiFi, this is not
> recognized. This error gets thrown:
>
> 'java.sql.Timestamp' object has no attribute 'timetuple' in 

Object not recognized in ExecuteScript

[James McMahon]

Good afternoon. I havd a python script that I can execute from the command
line via my python interpreter. In it, I do this

myTime = time.mktime(myDateTime.timetuple())

When I try to run from my ExecuteScript processor in NiFi, this is not
recognized. This error gets thrown:

'java.sql.Timestamp' object has no attribute 'timetuple' in