RE: Getting org.springframework.web.client.HttpClientErrorException: 409 Conflict on DELETE ProcessGroup

[Nirmal Kumar]

Each time I fire the DELETE request I first GET the ProcessGroup details and 
then fetch the *version*.
So I am passing the correct version in DELETE request.

Thanks,
-Nirmal

From: Matt Gilman [mailto:matt.c.gil...@gmail.com]
Sent: Wednesday, December 7, 2016 12:37 AM
To: users@nifi.apache.org
Subject: Re: Getting org.springframework.web.client.HttpClientErrorException: 
409 Conflict on DELETE ProcessGroup

I'm not familiar with Spring RestTemplate but I'm sure they offer the ability 
to get at the response body. Maybe it's possible to implement a custom error 
handler or something.

Without knowing more, I could venture a guess that your requests are failing 
intermittently because you not passing along the correct revision. NiFi employs 
an optimistic locking model whereby clients pass along a revision version to 
indicate that currently hold the most update to date version of that component.

Here a blog that details this and provides a nice sequence diagram to help 
visualize it [1]. The relevant details are towards the bottom.

Matt

[1] 
https://community.hortonworks.com/content/kbentry/3160/update-nifi-flow-on-the-fly-via-api.html

On Tue, Dec 6, 2016 at 1:53 PM, Nirmal Kumar 
mailto:nirmal.ku...@impetus.co.in>> wrote:
Thanks but as I calling it programmatically using Spring RestTemplate I just 
get the stack trace as mentioned in my below mail.

-Nirmal

From: Matt Gilman 
[mailto:matt.c.gil...@gmail.com]
Sent: Tuesday, December 6, 2016 6:58 PM
To: users@nifi.apache.org
Subject: Re: Getting org.springframework.web.client.HttpClientErrorException: 
409 Conflict on DELETE ProcessGroup

Nirmal,

If you consume the body of the response it should indicate the reason for the 
409. On failed responses, the response body should be text/plain. Let me know 
if that helps.

Thanks

Matt


On Mon, Dec 5, 2016 at 11:29 PM, Nirmal Kumar 
mailto:nirmal.ku...@impetus.co.in>> wrote:
Hi All,

I am getting the following exception *intermittently* when firing a DELETE 
request for deleting the Process group:
http://xxx.xxx.xxx.xxx:9091/nifi-api/process-groups/103111c4-123b-14f9-a794-14795038dbdf?version=0

org.springframework.web.client.HttpClientErrorException: 409 Conflict
at 
org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
 ~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.handleResponseError(RestTemplate.java:615)
 ~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:573) 
~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:529) 
~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.delete(RestTemplate.java:401) 
~[nifi-util-1.0.0.jar:na]

I am using Nifi 1.0.0 version.

Not sure if setting these timeout will help here:
  public RestTemplate restTemplate() {
return new RestTemplate(clientHttpRequestFactory());
}

private ClientHttpRequestFactory clientHttpRequestFactory() {
HttpComponentsClientHttpRequestFactory factory = new 
HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(2000);
factory.setConnectTimeout(2000);
return factory;
}

Any pointers here would be great.

Thanks,
-Nirmal









NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.









NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.









NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
vi

Re: How to integrate a custom protocol with Apache Nifi

[kant kodali]

Thanks a ton guys! Didn't expect Nifi community to be so good! (Another
convincing reason!)

Coming back to the problem, We use NSQ a lot (although not my favorite) and
want to be able integrate Nifi with NSQ to other systems such as Kafka,
Spark, Cassandra, ElasticSearch, some micro services that uses HTTP2 and
 some micro service that uses Websockets

(which brings in other question if Nifi has HTTP2 and Websocket processors?)

Also below is NSQ protocol spec. They have Java client library as well.

http://nsq.io/clients/tcp_protocol_spec.html


On Tue, Dec 6, 2016 at 5:14 PM, Oleg Zhurakousky <
ozhurakou...@hortonworks.com> wrote:

> Hi Kant
>
> What you’re trying to accomplish is definitely possible, however more
> information may be needed from you.
> For example, the way I understand your statement about “integration with
> many systems” is something like JMS, Kafka, TCP, FTP etc…. If that is the
> case such integration is definitely possible with your “custom system” by
> developing custom Processor and/or ControllerService.
> Processors and ControllerServices are the two main integration points
> within NiFi
> You can definitely find may examples by looking at some of the processors
> (i.e., PublishKafka or ConsumeKafka, PublishJMS or ConsumeJMS etc.)
>
> Let us know if you need more help to guide you through the process.
>
> Cheers
> Oleg
>
> > On Dec 6, 2016, at 7:46 PM, kant kodali  wrote:
> >
> > HI All,
> >
> > I understand that Apache Nifi has integration with many systems but what
> If I have an application that talks a custom protocol ? How do I integrate
> Apache Nifi with the custom protocol?
> >
> > Thanks,
> > kant
>
>

Re: How to integrate a custom protocol with Apache Nifi

[Oleg Zhurakousky]

Hi Kant

What you’re trying to accomplish is definitely possible, however more 
information may be needed from you.
For example, the way I understand your statement about “integration with many 
systems” is something like JMS, Kafka, TCP, FTP etc…. If that is the case such 
integration is definitely possible with your “custom system” by developing 
custom Processor and/or ControllerService.
Processors and ControllerServices are the two main integration points within 
NiFi 
You can definitely find may examples by looking at some of the processors 
(i.e., PublishKafka or ConsumeKafka, PublishJMS or ConsumeJMS etc.)

Let us know if you need more help to guide you through the process.

Cheers
Oleg

> On Dec 6, 2016, at 7:46 PM, kant kodali  wrote:
> 
> HI All,
> 
> I understand that Apache Nifi has integration with many systems but what If I 
> have an application that talks a custom protocol ? How do I integrate Apache 
> Nifi with the custom protocol?
> 
> Thanks,
> kant

Re: How to integrate a custom protocol with Apache Nifi

[Joe Witt]

Kant

Typically this is as simple as creating your processor to utilize that protocol.

Take a look at the developer guide [1] and of course you can always
look at the many example processors and components that come in NiFi
out of the box [2].

[1] https://nifi.apache.org/docs.html
[2] https://git-wip-us.apache.org/repos/asf/nifi.git

Thanks
Joe

On Tue, Dec 6, 2016 at 6:46 PM, kant kodali  wrote:
> HI All,
>
> I understand that Apache Nifi has integration with many systems but what If
> I have an application that talks a custom protocol ? How do I integrate
> Apache Nifi with the custom protocol?
>
> Thanks,
> kant

Re: How to integrate a custom protocol with Apache Nifi

[Andrew Grande]

Kant,

Look into a custom processor. You have a choice of either implementing e.g.
a parser for your data, or, if the protocol is more involved, implement a
receiver in the processor as well which would spit out meaningful data
messages next.

Andrew

On Tue, Dec 6, 2016, 7:47 PM kant kodali  wrote:

> HI All,
>
> I understand that Apache Nifi has integration with many systems but what
> If I have an application that talks a custom protocol ? How do I integrate
> Apache Nifi with the custom protocol?
>
> Thanks,
> kant
>

How to integrate a custom protocol with Apache Nifi

[kant kodali]

HI All,

I understand that Apache Nifi has integration with many systems but what If
I have an application that talks a custom protocol ? How do I integrate
Apache Nifi with the custom protocol?

Thanks,
kant

Large scale Secure MiNiFi deployments

[Andre]

All,

Is anyone else using a large number of MiNiFi agents together with secure
Site to Site?

How are you managing the authorization policy so that MiNiFi agents can
connect to the remote process groups ports?

Cheers

Re: How do I use Cassandra CQL V3 protocol with Nifi?

[kant kodali]

Hi Matt,


Thanks for the response! I am very new to Nifi so I will use the latest
version. I am still trying to understand its capabilities. It sounds
awesome to me .

This talk https://www.youtube.com/watch?v=sQCgtCoZyFQ is very good. I just
want see to little bit more concrete examples on

1) how to deal with message format changes
2) how to integrate systems with different protocols
3) authorized access

Any pointers?



On Tue, Dec 6, 2016 at 12:27 PM, Matt Burgess  wrote:

> What version of NiFi are you using? In NiFi 0.7.0 and 1.0.0 (with the
> inclusion of NIFI-1998 [1] ), they use the Cassandra 3.0.x driver
> which supports the V3 protocol [2] (actually v1-v4 protocols).
>
> Regards,
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-1998
> [2] http://docs.datastax.com/en/developer/java-driver/3.1/
> manual/native_protocol/
>
> On Tue, Dec 6, 2016 at 3:10 PM, kant kodali  wrote:
> > How do I use Cassandra CQL V3 protocol with Nifi?
>

Re: How do I use Cassandra CQL V3 protocol with Nifi?

[Matt Burgess]

What version of NiFi are you using? In NiFi 0.7.0 and 1.0.0 (with the
inclusion of NIFI-1998 [1] ), they use the Cassandra 3.0.x driver
which supports the V3 protocol [2] (actually v1-v4 protocols).

Regards,
Matt

[1] https://issues.apache.org/jira/browse/NIFI-1998
[2] 
http://docs.datastax.com/en/developer/java-driver/3.1/manual/native_protocol/

On Tue, Dec 6, 2016 at 3:10 PM, kant kodali  wrote:
> How do I use Cassandra CQL V3 protocol with Nifi?

How do I use Cassandra CQL V3 protocol with Nifi?

[kant kodali]

How do I use Cassandra CQL V3 protocol with Nifi?

Re: Data SLA

[Jeremy Farbota]

Richard,

Piggy backing on Andy's comment, you could also use getHDFS to seek files
based on the date/time or whatever parameters then use the failure
relationship to post to Slack or send an email.

On Tue, Dec 6, 2016 at 10:23 AM, Andy LoPresto  wrote:

> Richard,
>
> In addition to the flow that loads those files and performs whatever
> follow-on transformation and routing, you can have a parallel flow which
> monitors the directory and uses cron scheduling to run at a specific time
> (0900 daily) and is configured to expect a threshold number of files and in
> a specific naming pattern. This separate flow can route to email
> processors, etc. to alert relevant teams. If ListHDFS does not meet your
> requirements for monitoring, ExecuteScript is very versatile in performing
> non-standard actions within NiFi.
>
> You should also take a look at the monitoring capabilities of the NiFi
> UI/API such as data provenance, queue interaction, and
> statistics/monitoring. These features provide live monitoring for your
> administrators/data flow managers. You can read from these
> resources/queries via external monitoring tools if necessary as well.
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Dec 6, 2016, at 9:29 AM, Richard Duarte 
> wrote:
>
> Hello,
>
> I'm working on a project that I'd like to use Nifi for, however I'm trying
> to figure out if there's a way to alert on the availability of data being
> provided correctly to an agreement.
>
> Example:
> 1. Team X has agreed to provide files in directory Y in HDFS
> 2. Team X has agreed to provide those files once per day by 9 AM PST
> 3. Team X has agreed to name those files following naming convention
> [x]-[date]-[abc]
>
>
> Is there any way for Nifi to alert on Team X violating any of those
> agreements?  I.E. can we get Nifi to alert on the files not being there,
> the files not being there in time, and/or the files not following the
> correct naming conventions?
>
> If not, are there any other tools that tie into Nifi to provide help on
> the Data SLA?
>
> Thanks,
> Richard
>
>
>


-- 

[image: Payoff, Inc.] 

Jeremy Farbota
Software Engineer, Data
jfarb...@payoff.com  • (217) 898-8110 <(949)+430-0630>

I'm a Storyteller. Discover your Financial Personality!


[image: Facebook]   [image: Twitter]
 [image: Linkedin]

Re: Getting org.springframework.web.client.HttpClientErrorException: 409 Conflict on DELETE ProcessGroup

[Matt Gilman]

I'm not familiar with Spring RestTemplate but I'm sure they offer the
ability to get at the response body. Maybe it's possible to implement a
custom error handler or something.

Without knowing more, I could venture a guess that your requests are
failing intermittently because you not passing along the correct revision.
NiFi employs an optimistic locking model whereby clients pass along a
revision version to indicate that currently hold the most update to date
version of that component.

Here a blog that details this and provides a nice sequence diagram to help
visualize it [1]. The relevant details are towards the bottom.

Matt

[1]
https://community.hortonworks.com/content/kbentry/3160/update-nifi-flow-on-the-fly-via-api.html

On Tue, Dec 6, 2016 at 1:53 PM, Nirmal Kumar 
wrote:

> Thanks but as I calling it programmatically using Spring RestTemplate I
> just get the stack trace as mentioned in my below mail.
>
>
>
> -Nirmal
>
>
>
> *From:* Matt Gilman [mailto:matt.c.gil...@gmail.com]
> *Sent:* Tuesday, December 6, 2016 6:58 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: Getting 
> org.springframework.web.client.HttpClientErrorException:
> 409 Conflict on DELETE ProcessGroup
>
>
>
> Nirmal,
>
>
>
> If you consume the body of the response it should indicate the reason for
> the 409. On failed responses, the response body should be text/plain. Let
> me know if that helps.
>
>
>
> Thanks
>
>
>
> Matt
>
>
>
>
>
> On Mon, Dec 5, 2016 at 11:29 PM, Nirmal Kumar 
> wrote:
>
> Hi All,
>
>
>
> I am getting the following exception **intermittently** when firing a
> DELETE request for deleting the Process group:
>
> http://xxx.xxx.xxx.xxx:9091/nifi-api/process-groups/
> 103111c4-123b-14f9-a794-14795038dbdf?version=0
>
>
>
> org.springframework.web.client.HttpClientErrorException: 409 Conflict
>
> at org.springframework.web.client.DefaultResponseErrorHandler.
> handleError(DefaultResponseErrorHandler.java:91) ~[nifi-util-1.0.0.jar:na]
>
> at org.springframework.web.client.RestTemplate.
> handleResponseError(RestTemplate.java:615) ~[nifi-util-1.0.0.jar:na]
>
> at 
> org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:573)
> ~[nifi-util-1.0.0.jar:na]
>
> at 
> org.springframework.web.client.RestTemplate.execute(RestTemplate.java:529)
> ~[nifi-util-1.0.0.jar:na]
>
> at 
> org.springframework.web.client.RestTemplate.delete(RestTemplate.java:401)
> ~[nifi-util-1.0.0.jar:na]
>
>
>
> I am using Nifi 1.0.0 version.
>
>
>
> Not sure if setting these timeout will help here:
>
>   *public* RestTemplate restTemplate() {
>
> *return* *new* RestTemplate(*clientHttpRequestFactory*());
>
> }
>
>
>
> *private* *ClientHttpRequestFactory* clientHttpRequestFactory() {
>
> *HttpComponentsClientHttpRequestFactory* factory = *new*
> *HttpComponentsClientHttpRequestFactory*();
>
> factory.setReadTimeout(2000);
>
> factory.setConnectTimeout(2000);
>
> *return* factory;
>
> }
>
>
>
> Any pointers here would be great.
>
>
>
> Thanks,
>
> -Nirmal
>
>
>
>
> --
>
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
>
>
> --
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>

RE: Getting org.springframework.web.client.HttpClientErrorException: 409 Conflict on DELETE ProcessGroup

[Nirmal Kumar]

Thanks but as I calling it programmatically using Spring RestTemplate I just 
get the stack trace as mentioned in my below mail.

-Nirmal

From: Matt Gilman [mailto:matt.c.gil...@gmail.com]
Sent: Tuesday, December 6, 2016 6:58 PM
To: users@nifi.apache.org
Subject: Re: Getting org.springframework.web.client.HttpClientErrorException: 
409 Conflict on DELETE ProcessGroup

Nirmal,

If you consume the body of the response it should indicate the reason for the 
409. On failed responses, the response body should be text/plain. Let me know 
if that helps.

Thanks

Matt


On Mon, Dec 5, 2016 at 11:29 PM, Nirmal Kumar 
mailto:nirmal.ku...@impetus.co.in>> wrote:
Hi All,

I am getting the following exception *intermittently* when firing a DELETE 
request for deleting the Process group:
http://xxx.xxx.xxx.xxx:9091/nifi-api/process-groups/103111c4-123b-14f9-a794-14795038dbdf?version=0

org.springframework.web.client.HttpClientErrorException: 409 Conflict
at 
org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
 ~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.handleResponseError(RestTemplate.java:615)
 ~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:573) 
~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:529) 
~[nifi-util-1.0.0.jar:na]
at 
org.springframework.web.client.RestTemplate.delete(RestTemplate.java:401) 
~[nifi-util-1.0.0.jar:na]

I am using Nifi 1.0.0 version.

Not sure if setting these timeout will help here:
  public RestTemplate restTemplate() {
return new RestTemplate(clientHttpRequestFactory());
}

private ClientHttpRequestFactory clientHttpRequestFactory() {
HttpComponentsClientHttpRequestFactory factory = new 
HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(2000);
factory.setConnectTimeout(2000);
return factory;
}

Any pointers here would be great.

Thanks,
-Nirmal









NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.









NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.

Re: Data SLA

[Andy LoPresto]

Richard,

In addition to the flow that loads those files and performs whatever follow-on 
transformation and routing, you can have a parallel flow which monitors the 
directory and uses cron scheduling to run at a specific time (0900 daily) and 
is configured to expect a threshold number of files and in a specific naming 
pattern. This separate flow can route to email processors, etc. to alert 
relevant teams. If ListHDFS does not meet your requirements for monitoring, 
ExecuteScript is very versatile in performing non-standard actions within NiFi.

You should also take a look at the monitoring capabilities of the NiFi UI/API 
such as data provenance, queue interaction, and statistics/monitoring. These 
features provide live monitoring for your administrators/data flow managers. 
You can read from these resources/queries via external monitoring tools if 
necessary as well.

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Dec 6, 2016, at 9:29 AM, Richard Duarte  wrote:
> 
> Hello,
> 
> I'm working on a project that I'd like to use Nifi for, however I'm trying to 
> figure out if there's a way to alert on the availability of data being 
> provided correctly to an agreement.
> 
> Example:
> 1. Team X has agreed to provide files in directory Y in HDFS
> 2. Team X has agreed to provide those files once per day by 9 AM PST
> 3. Team X has agreed to name those files following naming convention 
> [x]-[date]-[abc]
> 
> 
> Is there any way for Nifi to alert on Team X violating any of those 
> agreements?  I.E. can we get Nifi to alert on the files not being there, the 
> files not being there in time, and/or the files not following the correct 
> naming conventions?
> 
> If not, are there any other tools that tie into Nifi to provide help on the 
> Data SLA?
> 
> Thanks,
> Richard



signature.asc
Description: Message signed with OpenPGP using GPGMail

Data SLA

[Richard Duarte]

Hello,

I'm working on a project that I'd like to use Nifi for, however I'm trying
to figure out if there's a way to alert on the availability of data being
provided correctly to an agreement.

Example:
1. Team X has agreed to provide files in directory Y in HDFS
2. Team X has agreed to provide those files once per day by 9 AM PST
3. Team X has agreed to name those files following naming convention
[x]-[date]-[abc]


Is there any way for Nifi to alert on Team X violating any of those
agreements?  I.E. can we get Nifi to alert on the files not being there,
the files not being there in time, and/or the files not following the
correct naming conventions?

If not, are there any other tools that tie into Nifi to provide help on the
Data SLA?

Thanks,
Richard

Re: Flow File Stuck for no reason

[Joe Witt]

Hello

This part of the stack dump is the only thing that appears out of
place to me.  it appears to be attempting to read from the socket or
stuck there.  Perhaps you can use a more recent client library for
that custom HD processor?

"Timer-Driven Process Thread-8" Id=78 RUNNABLE  (in native code)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
- waiting on java.io.BufferedInputStream@7b23ac8
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
- waiting on sun.net.www.protocol.http.HttpURLConnection@3d1cc911
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
- waiting on sun.net.www.protocol.http.HttpURLConnection@3d1cc911
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at 
com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:124)
at 
com.microsoft.azure.storage.core.LazySegmentedIterator.hasNext(LazySegmentedIterator.java:109)
at 
org.apache.hadoop.fs.azure.StorageInterfaceImpl$WrappingIterator.hasNext(StorageInterfaceImpl.java:128)
at 
org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.retrieveMetadata(AzureNativeFileSystemStore.java:1909)
at 
org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatus(NativeAzureFileSystem.java:1592)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at 
org.apache.hadoop.fs.azure.NativeAzureFileSystem.conditionalRedoFolderRename(NativeAzureFileSystem.java:1638)
at 
org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatus(NativeAzureFileSystem.java:1603)
at 
custom.nifi.processor.hdfs.PutHDFSHDCustom.onTrigger(PutHDFSHDCustom.java:247)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
at 
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
Source)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Number of Locked Synchronizers: 1
- java.util.concurrent.ThreadPoolExecutor$Worker@3b501d78

Thanks
Joe

On Tue, Dec 6, 2016 at 9:05 AM, Manish G  wrote:
> Hi,
>
> The issue happened again today. I restarted the NiFi and it started working
> again. Below are the rows from bootstrap.log after nifi.sh dump.
> Could this be because of some other process like diagnostic monitor on Azure
> storage blob?
>
> Thanks,
> Manish
>
>
> 2016-12-06 08:41:47,079 INFO [main] org.apache.nifi.bootstrap.RunNiFi "
> "Attach Listener" Id=6 RUNNABLE
>
> "AzureNativeFilesystemStore-UploadBandwidthUpdater" Id=94 TIMED_WAITING  on
> null
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.fs.azure.metrics.BandwidthGaugeUpdater$UploadBandwidthUpdater.run(BandwidthGaugeUpdater.java:266)
> at java.lang.Thread.run(Unknown Source)
>
> "Cleanup Archive for default" Id=54 RUNNABLE
> at sun.nio.fs.WindowsNativeDispatcher.FindClose(Native Method)
> at sun.nio.fs.WindowsDirectoryStream.close(Unknown Source)
> at sun.nio.fs.WindowsFileSystemProvider.checkReadAccess(Unknown Source)
> at sun.nio.fs.WindowsFileSystemProvider.checkAccess(Unknown Source)
> at java.nio.file.Files.exists(Unknown Source)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.destroyExpiredArchives(FileSystemRepository.java:1307)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.access$1800(FileSystemRepository.java:84)
> at
> org.apache.nifi.controller.repository.FileSystemRepository$DestroyExpiredArchiveClaims.run(FileSystemRepository.java:1546)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> Source)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
> at java.util.co

Re: Getting org.springframework.web.client.HttpClientErrorException: 409 Conflict on DELETE ProcessGroup

[Matt Gilman]

Nirmal,

If you consume the body of the response it should indicate the reason for
the 409. On failed responses, the response body should be text/plain. Let
me know if that helps.

Thanks

Matt


On Mon, Dec 5, 2016 at 11:29 PM, Nirmal Kumar 
wrote:

> Hi All,
>
>
>
> I am getting the following exception **intermittently** when firing a
> DELETE request for deleting the Process group:
>
> http://xxx.xxx.xxx.xxx:9091/nifi-api/process-groups/
> 103111c4-123b-14f9-a794-14795038dbdf?version=0
>
>
>
> org.springframework.web.client.HttpClientErrorException: 409 Conflict
>
> at org.springframework.web.client.DefaultResponseErrorHandler.
> handleError(DefaultResponseErrorHandler.java:91) ~[nifi-util-1.0.0.jar:na]
>
> at org.springframework.web.client.RestTemplate.
> handleResponseError(RestTemplate.java:615) ~[nifi-util-1.0.0.jar:na]
>
> at 
> org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:573)
> ~[nifi-util-1.0.0.jar:na]
>
> at 
> org.springframework.web.client.RestTemplate.execute(RestTemplate.java:529)
> ~[nifi-util-1.0.0.jar:na]
>
> at 
> org.springframework.web.client.RestTemplate.delete(RestTemplate.java:401)
> ~[nifi-util-1.0.0.jar:na]
>
>
>
> I am using Nifi 1.0.0 version.
>
>
>
> Not sure if setting these timeout will help here:
>
>   *public* RestTemplate restTemplate() {
>
> *return* *new* RestTemplate(*clientHttpRequestFactory*());
>
> }
>
>
>
> *private* *ClientHttpRequestFactory* clientHttpRequestFactory() {
>
> *HttpComponentsClientHttpRequestFactory* factory = *new*
> *HttpComponentsClientHttpRequestFactory*();
>
> factory.setReadTimeout(2000);
>
> factory.setConnectTimeout(2000);
>
> *return* factory;
>
> }
>
>
>
> Any pointers here would be great.
>
>
>
> Thanks,
>
> -Nirmal
>
>
>
> --
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>