RE: Upgrade 0.7.0 to 1.0.0

From: Kumiko Yada [mailto:kumiko.y...@ds-iq.com] Sent: Friday, September 9, 2016 10:06 AM To: users@nifi.apache.org Cc: Chien Le <chien...@ds-iq.com>; Kevin Verhoeven <kevin.verhoe...@ds-iq.com>; Wei Zhang <wei.zh...@ds-iq.com> Subject: Upgrade 0.7.0 to 1.0.0 Hello, We upgra

RE: Issue writing file (~50mb) to azure data lake with Nifi

From: Tony Kurc [mailto:trk...@gmail.com] Sent: Tuesday, September 6, 2016 3:02 PM To: Kumiko Yada <kumiko.y...@ds-iq.com> Cc: users@nifi.apache.org; Joe Witt <joe.w...@gmail.com>; #Operations Automation and Tools <#opsto...@ds-iq.com>; Kevin Verhoeven <kevin.verhoe...@d

RE: Issue writing file (~50mb) to azure data lake with Nifi

g TENANT_ID = ""; final static String SUBSCRIPTION_ID = ""; final static String CLIENT_ID = ""; final static String CLIENT_SECRET = ""; / Thanks Kumiko From: Tony Kurc [mailto:trk...@gmail.com] Sent: Tuesday, September 6, 2016 2:41 PM To: Kumiko Yada <kumiko

RE: IDE-specific setup

that I think all of the defaults should work. > > -Bryan > > > On Tue, Jun 21, 2016 at 4:47 PM, Kumiko Yada <kumiko.y...@ds-iq.com> wrote: >> >> I downloaded the source code by git clone >> http://git-wip-us.apache.org/repos/asf/nifi.git. Is it possible

RE: IDE-specific setup

<mailto:alopre...@apache.org> alopresto.apa...@gmail.com<mailto:alopresto.apa...@gmail.com> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Jun 21, 2016, at 10:06 AM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Hi Andy, I

RE: IDE-specific setup

+Guide#ContributorGuide-CodeStyle Andy LoPresto alopre...@apache.org<mailto:alopre...@apache.org> alopresto.apa...@gmail.com<mailto:alopresto.apa...@gmail.com> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Jun 21, 2016, at 9:57 AM, Kumiko Yada <kumiko

IDE-specific setup

Hello, Is there any documentations on how to setup the Nifi with IDE-specific setup? Thanks Kumiko

RE: Custom Controller Service

s helps! Thanks -Mark On Jun 17, 2016, at 3:00 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Hello, I wrote the Custom Controller Service to authenticate the Azure Datalake Store, and I’d like to refresh a token every 24 hours. What is the best w

Custom Controller Service

Hello, I wrote the Custom Controller Service to authenticate the Azure Datalake Store, and I'd like to refresh a token every 24 hours. What is the best way to authenticate every 24 hours? Should I loop the authenticate code every 24 hours in OnEnabled? Thanks Kumiko

RE: Exception: java.lang.NoClassDefFoundError: retrofit2/BaseUrl

to your code otherwise you will continue to get ClassNotFoundException Oleg On Jun 9, 2016, at 8:06 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Does this mean that I have to include this dependency in nar

RE: Exception: java.lang.NoClassDefFoundError: retrofit2/BaseUrl

class path (see class not found at the bottom of the trace). Is it packaged with your new Nar? Sent from my iPhone On Jun 9, 2016, at 19:01, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Hello, My custom processor is throwing the run-time exception, s

Exception: java.lang.NoClassDefFoundError: retrofit2/BaseUrl

Hello, My custom processor is throwing the run-time exception, so I reverted the changes that I made and I'm still getting this below error. Does this mean that there is some changes in Microsoft Azure SDK? My custom processor nar file which I compiled yesterday is working fine. Note: I'm

RE: Custom processor is failing for concurrency

all to the non-thread-safe MS SDK routine, in a 'synchronized' method? You could then use the standard NiFi thread management and avoid a lot of complexity. And the result should be >= efficiency of having a dedicated thread to manage the problem action. --Matt F ____

Re: Custom processor is failing for concurrency

6 at 2:11 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Microsoft found this is an issue with the SDK, they are working on a fix, they do not have the ETA for the fix. To workaround this issue, I’m trying to create the multipl

RE: Custom processor is failing for concurrency

Kumiko From: Kumiko Yada [mailto:kumiko.y...@ds-iq.com] Sent: Sunday, June 5, 2016 6:28 PM To: users@nifi.apache.org Cc: Ki Kang <ki.k...@ds-iq.com>; Kevin Verhoeven <kevin.verhoe...@ds-iq.com> Subject: RE: Custom processor is failing for concurrency Thank you, Bryan. I’m working wi

RE: Custom processor is failing for concurrency

ave you looked at the source to see what’s going on there? If its open please link and we can tale a look. Cheers Oleg On Jun 3, 2016, at 4:58 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Here is the code, https://github.com/kyada1/PutFileAzureDLStor

RE: Custom processor is failing for concurrency

? Have you looked at the source to see what’s going on there? If its open please link and we can tale a look. Cheers Oleg On Jun 3, 2016, at 4:58 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Here is the code, https://github.com/kyada1/PutFileAzureDLSt

Re: How to access the counter and provenance Info

Is there a way to query all the rows in provenance? This does't have to be in processors. Thanks Kumiko From: Kumiko Yada <kumiko.y...@ds-iq.com> Sent: Thursday, June 2, 2016 9:27:46 AM To: users@nifi.apache.org Subject: Re: How to access the c

Re: How to access the counter and provenance Info

d have an AtomicInteger counter that increments on every onTrigger and reset when 24 hours has passed? Thanks, Bryan On Thu, Jun 2, 2016 at 10:50 AM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Could anyone help me on this? Thanks Kumiko ___

Re: How to access the counter and provenance Info

/org/apache/nifi/processor/ProcessSession.java#L123 Thanks On Thu, Jun 2, 2016 at 7:50 AM, Kumiko Yada <kumiko.y...@ds-iq.com> wrote: > Could anyone help me on this? > > Thanks > Kumiko > ____ > From: Kumiko Yada <kumiko.y...@ds-iq.com&

Re: How to access the counter and provenance Info

Could anyone help me on this? Thanks Kumiko From: Kumiko Yada <kumiko.y...@ds-iq.com> Sent: Wednesday, June 1, 2016 4:22:25 PM To: users@nifi.apache.org Subject: RE: How to access the counter and provenance Info When I clicked the Data Provenance from c

RE: How to access the counter and provenance Info

When I clicked the Data Provenance from context menu of processor, the item history are showed the NiFi Flow Data Provenance UI. How can I get this item list from the custom processor? Thanks Kumiko From: Kumiko Yada [mailto:kumiko.y...@ds-iq.com] Sent: Wednesday, June 1, 2016 12:56 PM

Best way to process the process the requests in batch

Hello, We implemented the custom process that are similar to the InvokeHTTP that the part of URL can be replace with the Context Data List, then write the weather to the flowfile. For example, URL to get the weather feed have to include the zip code in URL, and the ZIP code is {0} in the URL

RE: Dependency for SSL

and its dependencies. Regards, Matt On May 25, 2016, at 5:38 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Thank you Bryan and Matt. I added the dependency in the pom of processor jar & nar project, and I’m getting the ClassN

Dependencies for StandardValidators & SoftLimitBoundedByteArrayOutputStream

Hello, I'd like to use the followings in my custom processor, and I'm still getting the build errors that they do not exist. Am I able to use them by additional dependencies, if so what are the missing dependencies or do I need to port the codes? import

Dependency for SSL

Hello, I'd like to use the following in the custom processor in Nifi 0.6.1, and I added the dependency. However, I'm getting the following errors. Do I need to add something else to correctly handle this dependency? import org.apache.nifi.ssl.SSLContextService; import

RE: Read the custom attributes

/attributes/UpdateAttribute.java#L194 On Tue, May 24, 2016 at 3:43 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Hello, When the Nifi expression language is used in the custom processor property, it’s working. However, I’m not able get from express

Read the custom attributes

Hello, When the Nifi expression language is used in the custom processor property, it's working. However, I'm not able get from expression language for the custom attributes. How can I enable to read the custom attributes? Thanks Kumiko

RE: Exception: .java.sql.SQLException: No suitable driver found for jdbc:sqlserver:

if it helps. Pierre 2016-05-24 18:18 GMT+02:00 Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>>: Hello, I’m writing the custom processor to execute the store procedure against Azure SQL Data Warehouse, and I wrote the test code and it’s working. However, when I execut

Exception: .java.sql.SQLException: No suitable driver found for jdbc:sqlserver:

Hello, I’m writing the custom processor to execute the store procedure against Azure SQL Data Warehouse, and I wrote the test code and it’s working. However, when I executed this code as the custom processor, I’m getting an exception that java.sql.SQLException: No suitable driver found for

RE: Build error for the custom processor against 0.7.0-SNAPSHOT

, 2016 at 2:48 PM, Kumiko Yada <kumiko.y...@ds-iq.com<mailto:kumiko.y...@ds-iq.com>> wrote: Hello, I’m getting the following build error against 0.7.0-SNAPSHOT. I also tried to create a new custom processor again for 0.7.0-SNAPSHOT and getting the same error. Is this a known i

Build error for the custom processor against 0.7.0-SNAPSHOT

Hello, I'm getting the following build error against 0.7.0-SNAPSHOT. I also tried to create a new custom processor again for 0.7.0-SNAPSHOT and getting the same error. Is this a known issue? Build error: [INFO] Scanning for projects... [ERROR] [ERROR] Some problems were encountered while

RE: Cannot Authenticate the Azure Datalake Store

needs to be included in your NAR. Can you share what dependencies you have in the pom.xml of your processors project? You can check under NIFI_HOME/work/nar/extensions//META-INF/bundled-dependencies/ to see the JARs that are being included in your NAR. -Bryan On Fri, May 6, 2016 at 12:52 AM,

Cannot Authenticate the Azure Datalake Store

Hello, I'm writing the custom processor to create a file in the Azure Datalake Store. I wrote the sample java program to do using this sample code, https://azure.microsoft.com/en-us/documentation/articles/data-lake-store-get-started-java-sdk/. This is working fine. However, when I wrote the

RE: Apache NiFi - WebHDFS

gt; wrote: > > What about Knox Gateway? > >> On Apr 21, 2016, at 3:21 PM, Kumiko Yada <kumiko.y...@ds-iq.com> wrote: >> >> Will do. >> >> Thanks >> Kumiko >> >> -Original Message- >> From: Joe Witt [mailto:joe.w...@gmail

RE: Apache NiFi - WebHDFS

please let us know. Thanks Joe On Thu, Apr 21, 2016 at 3:43 PM, Kumiko Yada <kumiko.y...@ds-iq.com> wrote: > Hello, > > > > Has anyone written the custom process for WebHDFS? > > > > Thanks > > Kumiko

Apache NiFi - WebHDFS

Hello, Has anyone written the custom process for WebHDFS? Thanks Kumiko