Re: Scheduling using CRON driven on Windows OS

2016-06-16 Thread Keith Lim
, at 5:39 PM, Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>> wrote: Just found out from my coworker the NiFi is running on a cluster of 20 nodes, hence the 20 flowfiles generated. Also found out that when the NiFi is hosted in the clustered environment, there is an

Re: Scheduling using CRON driven on Windows OS

2016-06-16 Thread Keith Lim
nds. In your case, I believe you'd like something like 00 02 10 * * ? Hope this helps. 2016-06-16 19:13 GMT+02:00 Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>>: My GenerateFlowFile processor is enabled and started with Scheduling Strategy: Cron Driven Run S

Re: Scheduling using CRON driven on Windows OS

2016-06-16 Thread Keith Lim
2016 4:10 PM To: users@nifi.apache.org Subject: Re: Scheduling using CRON driven on Windows OS Keith, Was your processor running at all times? It has to be started and enabled. I guess sharing the cron expression and maybe a quick screenshot will help next. Andrew On Wed, Jun 15, 2016, 6:58 PM

Scheduling using CRON driven on Windows OS

2016-06-15 Thread Keith Lim
I tried setting Scheduling Strategy property to CRON Driven but does not seem to work. Sometimes it would fire when not expected to and others not fire when expected to. This is on Windows OS and the processor I tried was GenerateFlowFile. Is CRON Driven setting not designed to work on

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-09 Thread Keith Lim
d from a template from another NiFi version. Do you know how recently your 0.7.0-SNAPSHOT was built from source? Have you tried the latest code in the 0.x branch? Thanks, James On Wed, Jun 8, 2016 at 5:43 PM, Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>> wrote: M

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-08 Thread Keith Lim
th, Would you be able to share your sample template with us as an attachment, a GitHub Gist, or something similar? Thanks, James On Wed, Jun 8, 2016 at 3:36 PM, Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>> wrote: Hi Joe, I created a simple workflow with Ev

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-08 Thread Keith Lim
, 2016 at 3:36 PM, Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>> wrote: Hi Joe, I created a simple workflow with EvaluateXPath and managed to repro the issue. Here is the snapshot of the screen. This is on Windows with 0.7.0 snapshot. There is no issue on 0.6.1 on

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-08 Thread Keith Lim
in 0.7.0 Thanks, Keith [cid:36ee5712-5a4f-45cb-847e-3a9234eb4887] From: Keith Lim <keith@ds-iq.com> Sent: Wednesday, June 08, 2016 10:28 AM To: users@nifi.apache.org Subject: Re: Failure when running a workflow created from a template from anothe

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-08 Thread Keith Lim
s from the nifi-app.log? Not in valid state should be associated with some exceptions and such that ideally will be in the logs. Thanks Joe On Wed, Jun 8, 2016 at 1:00 PM, Keith Lim <keith@ds-iq.com> wrote: > Any help on this before while I am upgrading m

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-08 Thread Keith Lim
, Keith From: Keith Lim <keith@ds-iq.com> Sent: Tuesday, June 07, 2016 5:07 PM To: users@nifi.apache.org Subject: Failure when running a workflow created from a template from another NiFi version. I am getting a weird error when I ran a workflow create

Failure when running a workflow created from a template from another NiFi version.

2016-06-07 Thread Keith Lim
I am getting a weird error when I ran a workflow created from a template exported from another NiFi environment. The source template was developed in NiFi 0.6.1 and I ran it on 0.7.0. The error is Node is unable to fulfill this request due to:

Re: Which processor to use to cleanly convert xml to json?

2016-06-01 Thread Keith Lim
Thanks for pointing out. The assumptions stated work for my data, luckily as long as the data is captured, I don't need to differentiate if they are attribute or element and ordering is not required. Those ignored are non pertinent data in my context. Thanks, Keith

Re: Which processor to use to cleanly convert xml to json?

2016-06-01 Thread Keith Lim
= XML.toJSONObject(xml) xmlJSONObj.toString(textIndent) Thanks, Keith From: Keith Lim Sent: Wednesday, June 01, 2016 9:44:11 AM To: users@nifi.apache.org Subject: Re: Which processor to use to cleanly convert xml to json? Thanks Bryan and Thad for the quick response, I like

Re: Which processor to use to cleanly convert xml to json?

2016-06-01 Thread Keith Lim
Thanks Bryan and Thad for the quick response, I like these more established libraries. I will go with the Groovy example. Thanks, Keith From: Thad Guidry Sent: Wednesday, June 01, 2016 9:35 AM To: users@nifi.apache.org Subject: Re:

Re: Which processor to use to cleanly convert xml to json?

2016-06-01 Thread Keith Lim
] } } } I am wondering how much more is missing from the stylesheet to enable that, I guess I have to do some reading now to familiarize myself with that. Thanks, Keith From: Keith Lim <keith@ds-iq.com> Sent: Tuesday, May 31, 2016 4:07 PM To: users@ni

RE: Which processor to use to cleanly convert xml to json?

2016-06-01 Thread Keith Lim
Any help guidance much appreciated. Thanks, Keith From: Keith Lim<mailto:keith@ds-iq.com> Sent: ‎5/‎31/‎2016 4:07 PM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Which processor to use to cleanly convert xml to json? Whi

Which processor to use to cleanly convert xml to json?

2016-05-31 Thread Keith Lim
Which processor should I use to cleanly convert from xml to json? This article illustrates using TransformXML with a stylesheet to convert xml to json. https://community.hortonworks.com/articles/29474/nifi-converting-xml-to-json.html However, I am seeing that it does not convert values with

Re: Starting Nifi encounters "unexpected error"

2016-05-16 Thread Keith Lim
sto.apa...@gmail.com> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On May 16, 2016, at 2:31 PM, Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>> wrote: My Windows server was restarted and when I started Nifi again, I can longer browse to it :

Re: How to extract scalar info of json array using EvaluateJsonPath processor?

2016-05-13 Thread Keith Lim
.apache.org/jira/browse/NIFI-1875 On Thu, May 12, 2016 at 2:52 PM, Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>> wrote: Hi Aldrin, I am still not able to get this to work. I see that the expression language guide (http://nifi.apache.org/docs/nifi-docs/html/expre

How to extract scalar info of json array using EvaluateJsonPath processor?

2016-05-12 Thread Keith Lim
I have a flow file with json array and would like to use EvaluateJsonPath processor the extract the item count of that array. Does the Nifi Expression Language in combination of json path feature supports this without writing script? What is the syntax? Thanks, Keith

How does Nifi support calling a stored proc that returns a tabular result?

2016-05-05 Thread Keith Lim
Does any Nifi processor support executing a stored proc that returns results via a select statement? E.g.: Create PROCEDURE dbo.MyStoredProc AS BEGIN ...—do something SELECT … END I tried using a Nifi ExecuteSql processor and in lieu a select statement, I entered “exec

Windows Nifi ExecuteSql querying Azure Sql Server using JDBC failed to load database driver.

2016-05-04 Thread Keith Lim
I have a simple ExecuteSql processor that uses jdbc to connect to Microsoft Azure Sql Server. My DBCPConnectionPool is set as below: Database Connection URL: jdbc:sqlserver://dsiq-dev.database.windows.net:1433;database=CDBStaging (I have tried with the full connection as extracted from Azure

RE: Starting Nifi encounters "unexpected error"

2016-05-04 Thread Keith Lim
e, can you trying deleting the users database from database_repository directory and restart again. The database will be created during startup so if it is in a bad state it should be re-created. The database is simply a cache of users for the system. Matt On Wed, May 4, 2016 at 4:51 PM, Keith L

RE: Starting Nifi encounters "unexpected error"

2016-05-04 Thread Keith Lim
ymous user. This user is created during startup, stored in the users database, and should always be available. Any additional errors or details would be helpful. Thanks. Matt On Wed, May 4, 2016 at 4:05 PM, Keith Lim <keith@ds-iq.com<mailto:keith@ds-iq.com>> wrote: My windows

Starting Nifi encounters "unexpected error"

2016-05-04 Thread Keith Lim
My windows machine blue-d screen and after reboot I restarted nifi only to get the following error when I browse to nifi (http://localhost:8080/nifi/) I can do the long route or cleaning up the nifi folder and rebuilding it, but want to know if there is a surgical way to remove this issue.

Where can I find documentation of implementing custom processor?

2016-04-20 Thread Keith Lim
Is there any good example of implementing a custom Processor? Thanks, Keith