RE: Exception: java.lang.NoClassDefFoundError: retrofit2/BaseUrl

2016-06-09 Thread Kumiko Yada
I found out that I didn’t have specify this dependency in pom file that it would be packaged as part of below by finding the jar file in …\nifi-azure-dlstore-nar\target\classes\META-INF\bundled-dependencies. I verified that it is packaged with a new Nar, and I even copied this jar file in

Re: Funnel

2016-06-09 Thread Igor Kravzov
Ok. Got it. Thanks guys. On Thu, Jun 9, 2016 at 7:37 PM James Wing wrote: > The funnel merges multiple queues into a single queue. You then have one > queue to configure prioritization, backpressure, etc. And one queue to > monitor and administer. It can also be helpful

RE: Exception: java.lang.NoClassDefFoundError: retrofit2/BaseUrl

2016-06-09 Thread Kumiko Yada
Does this mean that I have to include this dependency in nar pom file now? com.squareup.retrofit2 retrofit 2.0.4

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-09 Thread James Wing
Thanks for reporting back, Keith. It sounds like a bug got fixed, and that's always great news even if we don't exactly where :). James On Thu, Jun 9, 2016 at 4:10 PM, Keith Lim wrote: > Thanks James, I built from the latest 0.x branch and the issue does not > show up. >

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-09 Thread Keith Lim
Thanks James, I built from the latest 0.x branch and the issue does not show up. Thanks, Keith From: James Wing Sent: Wednesday, June 08, 2016 9:23:22 PM To: users@nifi.apache.org Subject: Re: Failure when running a workflow created from a

Exception: java.lang.NoClassDefFoundError: retrofit2/BaseUrl

2016-06-09 Thread Kumiko Yada
Hello, My custom processor is throwing the run-time exception, so I reverted the changes that I made and I'm still getting this below error. Does this mean that there is some changes in Microsoft Azure SDK? My custom processor nar file which I compiled yesterday is working fine. Note: I'm

Funnel

2016-06-09 Thread Igor Kravzov
Guys, what is the actual role of funnel?

RE: Custom processor is failing for concurrency

2016-06-09 Thread Kumiko Yada
Here is the code: https://github.com/kyada1/PutFileAzureDLStore/blob/master/nifi-azure-dlstore-bundle/nifi-azure-dlstore-processors/src/main/java/nifi/azure/dlstore/processors/PutFileAzureDLStore.java The problems are : Line 209 - SetupClients(creds); Line 217 - CreateFile(_path + _filename,

Re: Not a Valid JSON content

2016-06-09 Thread Pierre Villard
Hi Anuj, This method supposes your XML input file is not too complex. Have a look on this thread, this should answer your questions: https://mail-archives.apache.org/mod_mbox/nifi-users/201606.mbox/%3CCALo_M1_erYWJkWB=D7kJ17gvpmnyeVGmYiQH9=mqqktb-nh...@mail.gmail.com%3E Hope this helps.

Re: Best way to compare dates

2016-06-09 Thread Igor Kravzov
Thanks James. That's exactly what I ended up with :) On Thu, Jun 9, 2016 at 12:48 PM, James Wing <jvw...@gmail.com> wrote: > Igor, > > One way would be to format both dates as strings (like "20160609" in your > HDFS paths) first, then compare

Not a Valid JSON content

2016-06-09 Thread Anuj Handa
Hi , I am getting the below error of not a valid JSON content a text file containing XML was sconverted to JSON. 2016-06-09 16:10:11,765 ERROR [Timer-Driven Process Thread-10] o.a.n.p.standard.EvaluateJsonPath EvaluateJsonPath[id=475d43ec-549b-4378-81d5-778041558781] FlowFile

Re: Custom processor is failing for concurrency

2016-06-09 Thread Matt Foley
​If there are multiple SDK calls that share the same problematic code (that is, multiple SDK methods that would interact with each other in a non-thread-safe way), then one must synchronize their calls to a shared lock object, which only requires a couple more lines of code.

Re: Custom processor is failing for concurrency

2016-06-09 Thread Oleg Zhurakousky
+1, was just responding with the same. On Jun 9, 2016, at 3:11 PM, Matt Foley > wrote: Kumiko, would it be sufficient to just wrap your call to the non-thread-safe MS SDK routine, in a 'synchronized' method? You could then use the standard

Re: Custom processor is failing for concurrency

2016-06-09 Thread Matt Foley
Kumiko, would it be sufficient to just wrap your call to the non-thread-safe MS SDK routine, in a 'synchronized' method? You could then use the standard NiFi thread management and avoid a lot of complexity. And the result should be >= efficiency of having a dedicated thread to manage the

Re: Custom processor is failing for concurrency

2016-06-09 Thread Bryan Bende
Yes I think it ends up being the same thing. If you create multiple threads that all use the same DataLakeStoreFileSystemManagementClient, or if you increase concurrent threads > 1 in the UI, both will potentially run into the problem in the Microsoft SDK. On Thu, Jun 9, 2016 at 2:49 PM, Kumiko

Re: Custom processor is failing for concurrency

2016-06-09 Thread Kumiko Yada
Hi Bryan, Does this mean that even I create the multiple threads in onTriger, I will still hit the Microsoft SDK issue where it's not a thread safe? Sounds like basically what I am trying to do and creating the multiple threads via UI might be the same thing. Thanks Kumiko

Re: Custom processor is failing for concurrency

2016-06-09 Thread Bryan Bende
Kumiko, In general you shouldn't have to create threads in your processors, with the exception of some special cases. The framework has a thread pool and it takes one of those threads and calls the onTrigger method of your processor. If you want multiple threads to call onTrigger, then each

RE: Custom processor is failing for concurrency

2016-06-09 Thread Kumiko Yada
Microsoft found this is an issue with the SDK, they are working on a fix, they do not have the ETA for the fix. To workaround this issue, I’m trying to create the multiple threads in using AbstractSessionFactoryProcessor and handle the Create a file in a single thread. I’m having a problem

Re: Kerberos / NiFi 0.5.1 / ListHDFS

2016-06-09 Thread Michael Dyer
Brian, Thank you - that was the problem. I updated my xml files from the cluster in question and all is working fine now. Michael

Re: Best way to compare dates

2016-06-09 Thread James Wing
Igor, One way would be to format both dates as strings (like "20160609" in your HDFS paths) first, then compare the two strings for equality. In a RouteOnAttribute expression: ${now():format("MMdd"):equals(${entryDate:format("MMdd")})} If your goal is to

Best way to compare dates

2016-06-09 Thread Igor Kravzov
Hi, In my workflow data is coming form Kafka Topic in JSON format and saved ot HDFS directory by day. HDFS directory path constructed as /mypath/${now():format("MMdd")} JSON file contains attribute {... "adddate":"2016-06-07 17:06:17" } I want to avoid over-spill of a previous date files