Guessing - if it's a new container on the first invocation, maybe the JVM
is generating the binary class cache? This operation is performed only once
ever, but with a clean environment every time I can see it being invoked
again and again.
Other than that, you'd need to connect a profiler and see
Not sure I understand what you mean. Are you using the flow file to
trigger ExecuteStreamCommand to schedule a cron job? Or do you mean
scheduling a processor to run in NiFi? Or something else?
On Mon, Oct 29, 2018 at 3:58 PM l vic wrote:
>
> QDT works, eg it can detect change in MaximumValue
QDT works, eg it can detect change in MaximumValue column but how can I use
it to schedule cron job? I know it's possible to schedule cron from UI but
how can i do it based on the value of attribute?
Thank you again,
V.
On Mon, Oct 29, 2018 at 12:39 PM Matt Burgess wrote:
> Victor,
>
> Yes,
Hi Andrew - yes its a container environment. Everything I work with is in
effect 'dockerized'. I created the container environment with 128min memory
and 256 max memory and 500m cpu which is half a core (2.40GHz per core). I
think that should be fine for the CLI but you tell me otherwise.
Hi Andy
Hey Victor,
If you already pulled the record and know new value - that won't really
help you to determine a change in a schedule.
In my opinion, the schedule determined by the acceptable data latency for
given application, in other words, how soon you want your changed data be
captured.
The
I started to test my container locally and saw the speed was drastically
faster.
So I limited my docker cpu on my laptop to half a core instead of the 4 or 8
my mac has.. and the script went from running in 13 seconds to 1+ minute.
On the cluster i changed half core to be using 1 whole core,
how many connections does your sftp server allow at once and how many
threads is nifi given for communication?
On Mon, Oct 29, 2018, 7:10 AM Alfredo De Luca
wrote:
> Hi all. We have some issue with our nifi (v. 1.6) where occasionally we
> have an error saying ...
> Auth fail.
>
> On the
nifi.swap.in.threads=1
nifi.swap.out.threads=4
nifi.cluster.node.protocol.threads=25
nifi.cluster.node.protocol.max.threads=50
On Mon, Oct 29, 2018 at 12:28 PM Alfredo De Luca
wrote:
> anyway...this is the error
> 2018-10-29 11:21:06,768 ERROR [Timer-Driven Process Thread-13]
>
Hi all. We have some issue with our nifi (v. 1.6) where occasionally we
have an error saying ...
Auth fail.
On the sftp server side we get
Oct 29 11:48:06 sftp sshd[13845]: pam_sss(sshd:account): Access denied for
user <*sftp user*>: 4 (System error)
Any idea?
--
*Alfredo*
anyway...this is the error
2018-10-29 11:21:06,768 ERROR [Timer-Driven Process Thread-13]
SimpleProcessLogger.java:254
ListSFTP[id=766ac418-27ce-335a-97c9-8823f2cf5a96] Failed to perform listing
on remote host due to java.io.IOException: Failed to obtain connection to
remote host due to
Hi Joe.
Sftp server allows 200 connections... not sure about the nifi. I ll check
On Mon, Oct 29, 2018 at 12:21 PM Joe Witt wrote:
> how many connections does your sftp server allow at once and how many
> threads is nifi given for communication?
>
>
>
> On Mon, Oct 29, 2018, 7:10 AM Alfredo De
You can use QueryDatabaseTable (QDT) for this, you'd set your
"event_time" column as the "Maximum Value Column(s)" property in the
processor. The first time QDT executes, it will fetch all the rows
(since it has not seen event_time before), then it will keep track of
the largest value of
hi all. any idea? I am trying everything..but not sure what cause this
error.
Cheers
On Mon, Oct 29, 2018 at 12:34 PM Alfredo De Luca
wrote:
> nifi.swap.in.threads=1
> nifi.swap.out.threads=4
> nifi.cluster.node.protocol.threads=25
> nifi.cluster.node.protocol.max.threads=50
>
>
>
>
> On Mon,
Hi,
i have "event_time" field in SQLite database that means epoch time for
triggering of external event. What processor(s) can i use to implement
schedule monitoring/ execution based on change in "event_time" value?
Thanks,
Hi Matt,
Nifi does handle other parts of it, just different process group.
Regards,
Victor
On Mon, Oct 29, 2018 at 12:39 PM Matt Burgess wrote:
> Victor,
>
> Yes, both QDT and GTF would generate something like "SELECT * from
> myTable where event_time > X", and QDT will execute it and update X.
Alfredo
There are quite a few details that will be needed to help folks guess
at possible issues.
Configuration of the machine running nifi
how many cores?, etc..
Configuration of nifi
you shared common properties from nifi.properties which is good.
how many threads does the flow
What if have only one row and update the values in it? Will QDT fetch
updates?
Thank you,
Victor
On Mon, Oct 29, 2018 at 11:54 AM Matt Burgess wrote:
> You can use QueryDatabaseTable (QDT) for this, you'd set your
> "event_time" column as the "Maximum Value Column(s)" property in the
>
Victor,
Yes, both QDT and GTF would generate something like "SELECT * from
myTable where event_time > X", and QDT will execute it and update X.
So if event_time is always increasing, it will continue to pick up the
same row(s).
That's a curious use case, maybe NiFi could handle other parts of it
Thanks Joe. Appreciated. I will get back to you asap with the info.
Cheers
On Mon, Oct 29, 2018 at 5:26 PM Joe Witt wrote:
> Alfredo
>
> There are quite a few details that will be needed to help folks guess
> at possible issues.
>
> Configuration of the machine running nifi
> how many
19 matches
Mail list logo
