RE: Loaded template cache

[William Gosse]

Can be safely removed before startup?  Will it get regenerated?

From: Bryan Bende 
Sent: Thursday, April 16, 2020 5:03 PM
To: users@nifi.apache.org
Subject: Re: Loaded template cache

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

Hello,

Templates belong to a process group so that they can be protected with the same 
access policies as the group they belong to.

As a result, they are stored in flow.xml.gz inside their containing process 
group.

Thanks,

Bryan

On Thu, Apr 16, 2020 at 4:52 PM William Gosse 
mailto:william.go...@aifoundry.com>> wrote:
Before  I start up Nifi I removed all the _repository folders in order to 
remove any remnant data such as flow files in queues. One thing I notice that 
doesn’t get cleared up are old copies of the loaded templates. I have to use 
nifi’s api to do that.

Is there place where the loaded templates data is being cached on the file 
system that could be cleaned up before start up?

Loaded template cache

[William Gosse]

Before  I start up Nifi I removed all the _repository folders in order to 
remove any remnant data such as flow files in queues. One thing I notice that 
doesn’t get cleared up are old copies of the loaded templates. I have to use 
nifi’s api to do that.

Is there place where the loaded templates data is being cached on the file 
system that could be cleaned up before start up?

RE: DistributedMapCacheService - automated creation

[William Gosse]

That was a very helpful suggestion. Here's what I came up with which seems to 
work:
#!/bin/bash
set -x

NIFI_ENCOMPASS_URL=http://xxx.xxx.xx.xxx:16080/nifi-api


ROOT_PROCESS=$(curl $NIFI_ENCOMPASS_URL/flow/process-groups/root)
ROOT_ID=`echo $ROOT_PROCESS | jq --raw-output '.processGroupFlow.id'`
echo "Nifi Encompass Connector Root ID = $ROOT_ID"

response=$(curl 
$NIFI_ENCOMPASS_URL/flow/process-groups/$ROOT_ID/controller-services)
controllerServices=`echo $response | jq --raw-output '.controllerServices'`
if [[ $controllerServices == '[]' ]]; then
echo "No existing controllerServices"
createService=true
else
if [[ $controllerServices == *"DistributedMapCacheServer"* ]]; then
echo "DistributedMapCacheServer exists"
else
echo "DistributedMapCacheServer does not exists"
createService=true
fi
fi

if [ "$createService" = true ]; then
clientId=$(curl $NIFI_ENCOMPASS_URL/flow/client-id)
curl -d 
'{"revision":{"clientId":"{$clientId}","version":0},"disconnectedNodeAcknowledged":false,"component":{"type":"org.apache.nifi.distributed.cache.server.map.DistributedMapCacheServer","bundle":{"group":"org.apache.nifi","artifact":"nifi-distributed-cache-services-nar","version":"1.10.0"}}}'
 -H "Content-Type: application/json" -X POST 
$NIFI_ENCOMPASS_URL/process-groups/$ROOT_ID/controller-services
fi

-Original Message-
From: Bryan Bende  
Sent: Wednesday, January 15, 2020 9:34 AM
To: users@nifi.apache.org
Subject: Re: DistributedMapCacheService - automated creation

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

Anything you can do from the UI goes through the REST API. You can open Chrome 
Dev Tools and see what calls are made to the server when you create the 
controller service from the UI.

On Wed, Jan 15, 2020 at 9:25 AM William Gosse  
wrote:
>
> Well we will be deploy Nifi in a docker container and need to configure 
> everything at the time of the container’s creation and startup.
>
>
>
> To be specific I’m using DistributedMapCacheService with a DetectDuplicate 
> process to track whether a flowfile with a same attribute value had been 
> already created in the last 10 minutes. If so abandon the new flowfile.
>
>
>
> I guess the thin I liked about the DistributedMapCacheService is that it 
> already exists in NiFi, no additional installation required.
>
>
>
>
>
> From: Mike Thomsen 
> Sent: Tuesday, January 14, 2020 7:42 PM
> To: users@nifi.apache.org
> Subject: Re: DistributedMapCacheService - automated creation
>
>
>
> [CAUTION: This email originated from outside of Kodak Alaris. Do not 
> click links or open attachments unless you recognize the sender and 
> know the content is safe.]
>
> ____
>
> What is your use case? Because this sounds vaguely like something that you 
> could implement implicitly with the HBase version by just creating one big 
> cache table with an eye for not overlapping column qualifiers and possibly 
> liberal use of HBase TTLs.
>
>
>
> On Tue, Jan 14, 2020 at 6:04 PM William Gosse  
> wrote:
>
> Is there any way to automate the creation of a 
> DistributedMapCacheService
>
>
>
> I was hoping I could do this via a template but it doesn’t to get exported 
> the way the  DistributedMapCacheClientService does.
>
>
>
> The client got created just fine but there’s no service.
>
>
>
> I don’t see away to do this with restful api either.
>
>
>
> Any help would be greatly appreciated.
>
>
>
>

RE: DistributedMapCacheService - automated creation

[William Gosse]

Well we will be deploy Nifi in a docker container and need to configure 
everything at the time of the container’s creation and startup.

To be specific I’m using DistributedMapCacheService with a DetectDuplicate 
process to track whether a flowfile with a same attribute value had been 
already created in the last 10 minutes. If so abandon the new flowfile.

I guess the thin I liked about the DistributedMapCacheService is that it 
already exists in NiFi, no additional installation required.


From: Mike Thomsen 
Sent: Tuesday, January 14, 2020 7:42 PM
To: users@nifi.apache.org
Subject: Re: DistributedMapCacheService - automated creation

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

What is your use case? Because this sounds vaguely like something that you 
could implement implicitly with the HBase version by just creating one big 
cache table with an eye for not overlapping column qualifiers and possibly 
liberal use of HBase TTLs.

On Tue, Jan 14, 2020 at 6:04 PM William Gosse 
mailto:william.go...@aifoundry.com>> wrote:
Is there any way to automate the creation of a DistributedMapCacheService

I was hoping I could do this via a template but it doesn’t to get exported the 
way the  DistributedMapCacheClientService does.

The client got created just fine but there’s no service.

I don’t see away to do this with restful api either.

Any help would be greatly appreciated.

DistributedMapCacheService - automated creation

[William Gosse]

Is there any way to automate the creation of a DistributedMapCacheService

I was hoping I could do this via a template but it doesn’t to get exported the 
way the  DistributedMapCacheClientService does.

The client got created just fine but there’s no service.

I don’t see away to do this with restful api either.

Any help would be greatly appreciated.

Check if a Flowfile is already running

[William Gosse]

Is there a way to check if there is any existing flowfile with a specific 
attribute value already running.

I’m running a schedule task that starts the processing of a flowfile. If a 
flowfile with a certain value I want to terminate processing of the new file.

RE: Need help with DetectDuplicate

[William Gosse]

Will the DetectDuplicate with DistributedMapCacheClientService only work in a 
cluster?  I’m trying to use it in a standalone NiFi.

From: Emanuel Oliveira 
Sent: Tuesday, December 24, 2019 4:29 PM
To: users@nifi.apache.org
Subject: Re: Need help with DetectDuplicate

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

Hi,

Depending on how your cluster setup you may need to add/setup ssl controller 
service?

Emanuel

On Tue 24 Dec 2019, 18:16 William Gosse, 
mailto:william.go...@aifoundry.com>> wrote:
I’m trying to use DetectDuplicate processor but not having much luck. Here the 
config:
Cache Entry Identifier
${resourceId}
FlowFile Description
Ingestion
Age Off Duration
60 sec
Cache The Entry Identifier
true
Distributed Cache Service
DistributedMapCacheClientService

I created abd enabled a DistributedMapCacheClientService. Here’s its config:
Server Hostname
localhost
Server Port
4557
SSL Context ServiceNo value setCommunications Timeout
30 secs

When I run it I get the following error:
2019-12-24 13:14:05,355 ERROR [Timer-Driven Process Thread-9] 
o.a.n.p.standard.DetectDuplicate 
DetectDuplicate[id=38cb8a64-016f-1000-b55b-f6c4e0f69f61] Unable to communicate 
with cache when processing 
StandardFlowFileRecord[uuid=7f049d8e-1d04-4fee-9f04-320d6980bc55,claim=StandardContentClaim
 [resourceClaim=StandardResourceClaim[id=1577202782598-43, container=default, 
section=43], offset=74111, 
length=4528],offset=0,name=84068ffb-69b1-4471-abbd-29243d3be39e,size=4528] due 
to java.net.ConnectException: Connection refused: no further information: 
java.net.ConnectException: Connection refused: no further information
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
at 
org.apache.nifi.distributed.cache.client.StandardCommsSession.(StandardCommsSession.java:52)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.createCommsSession(DistributedMapCacheClientService.java:410)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.leaseCommsSession(DistributedMapCacheClientService.java:425)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.withCommsSession(DistributedMapCacheClientService.java:491)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.getAndPutIfAbsent(DistributedMapCacheClientService.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy142.getAndPutIfAbsent(Unknown Source)
at 
org.apache.nifi.processors.standard.DetectDuplicate.onTrigger(DetectDuplicate.java:183)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Not sure whats missing?

Need help with DetectDuplicate

[William Gosse]

I’m trying to use DetectDuplicate processor but not having much luck. Here the 
config:
Cache Entry Identifier
${resourceId}
FlowFile Description
Ingestion
Age Off Duration
60 sec
Cache The Entry Identifier
true
Distributed Cache Service
DistributedMapCacheClientService

I created abd enabled a DistributedMapCacheClientService. Here’s its config:
Server Hostname
localhost
Server Port
4557
SSL Context ServiceNo value setCommunications Timeout
30 secs

When I run it I get the following error:
2019-12-24 13:14:05,355 ERROR [Timer-Driven Process Thread-9] 
o.a.n.p.standard.DetectDuplicate 
DetectDuplicate[id=38cb8a64-016f-1000-b55b-f6c4e0f69f61] Unable to communicate 
with cache when processing 
StandardFlowFileRecord[uuid=7f049d8e-1d04-4fee-9f04-320d6980bc55,claim=StandardContentClaim
 [resourceClaim=StandardResourceClaim[id=1577202782598-43, container=default, 
section=43], offset=74111, 
length=4528],offset=0,name=84068ffb-69b1-4471-abbd-29243d3be39e,size=4528] due 
to java.net.ConnectException: Connection refused: no further information: 
java.net.ConnectException: Connection refused: no further information
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
at 
org.apache.nifi.distributed.cache.client.StandardCommsSession.(StandardCommsSession.java:52)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.createCommsSession(DistributedMapCacheClientService.java:410)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.leaseCommsSession(DistributedMapCacheClientService.java:425)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.withCommsSession(DistributedMapCacheClientService.java:491)
at 
org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService.getAndPutIfAbsent(DistributedMapCacheClientService.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy142.getAndPutIfAbsent(Unknown Source)
at 
org.apache.nifi.processors.standard.DetectDuplicate.onTrigger(DetectDuplicate.java:183)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Not sure whats missing?

RE: HandleHTTPRequest 503 error issue

[William Gosse]

I’ve seen the same issue in 1.92 as well and opened the following ticket for it:
https://issues.apache.org/jira/browse/NIFI-6661

From: Wyllys Ingersoll 
Sent: Friday, October 4, 2019 12:28 PM
To: users@nifi.apache.org
Subject: HandleHTTPRequest 503 error issue

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]


I believe this issue: 
https://issues.apache.org/jira/browse/NIFI-5522
  is a regression in Nifi 1.9.2, it was marked as fixed in 1.8.0, but Im seeing 
it a lot in 1.9.2

I can recreate it fairly regularly when I am hitting the endpoint pretty 
heavily and an error is encountered.  The process return 503 error (or nothing 
at all) and cannot be stopped or restarted without rebooting the entire node.

-Wyllys Ingersoll

RE: HandleHttpRequest - Failed to receive content from HTTP Request

[William Gosse]

Looking for similar issues in google these ones seem to be close:
https://community.cloudera.com/t5/Support-Questions/Apache-Nifi-handlehttprequest-response-issue/m-p/199086
http://apache-nifi-users-list.2361937.n4.nabble.com/HandleHttpRequest-failing-td1062.html
I'm pretty sure I'm terminating all my flows with a proper HandleHtmlResponse. 
I'm kind of concerned that Nifi could get a stuck thread that causes the 
request handler to be non-responsive. Can't anything be done about this without 
having to totally restarting Nif?

This is a rather serious issue for us. We can't have Nifi stop working for any 
reason if possible especially for something like a client dropping its 
connection.


From: William Gosse 
Sent: Friday, September 13, 2019 8:33 AM
To: users@nifi.apache.org
Subject: RE: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2uyteg1gsj-1 
2uyteg1gsj
I created the following Jira issue: 
https://issues.apache.org/jira/browse/NIFI-6661<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__issues.apache.org_jira_browse_NIFI-2D6661%26d%3DDwMGaQ%26c%3DskUQKLuTFULvQUMqV0uyBaBvPxsA7Wk8MKjvMTLjY3w%26r%3DWm89MSTpkAcFFP1M0uvGefI8slyO_VIaIza2IJwAlsU%26m%3DFeafQec1Vx7cJvI_Sg6wnvCs3r2AtNh80BPq_a7kNQA%26s%3DqRXhv1kd6YT6BhMH9xUQF6pnXxIKnhiOJehD-rjqY6c%26e%3D=02%7C01%7Cwilliam.gosse%40aifoundry.com%7C84a5d6a52d9a498db26c08d738468778%7Cd29b7a9b6edb472099a83c5c6c3eeeb0%7C0%7C0%7C637039747880123423=qB4M9PW2DXcimg6tFiWfcZu1UkOFfYjVfeqTU2nohj8%3D=0>

Here’s some more info on this issue:
Also I only see this happening with one of my user who may be ealing with 
network latency.

I have not been able to recreate the issue myself with this exact excetion 
message. However the fact the it causes HandleHttpRequest processor to stop 
functioning seems serious to me.
The only time that ever happened to me is when it to long to get to the 
HandleHttpResponse. I currently have the Request Expiration property for my 
StandardHttpContextMap set to 10 minutes. If the this value is exceeded the 
HandleHttpRequest hangs up. In this specific issue that time out also was 
reached but it seems that the flow never got beyond the HandleHttpRequest.

From: William Gosse 
mailto:william.go...@aifoundry.com>>
Sent: Tuesday, September 10, 2019 5:14 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: RE: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2uv82r5u4t-1 
2uv82r5u4t
I’m only seeing it with one user and I’m not sure how they are doing it.

From: Pierre Villard 
mailto:pierre.villard...@gmail.com>>
Sent: Tuesday, September 10, 2019 8:32 AM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

I'm AFK for the moment but could you file a JIRA with some steps to reproduce 
the issue? Happy to have a look once I get back at a computer.

Le mar. 10 sept. 2019 à 08:09, William Gosse 
mailto:william.go...@aifoundry.com>> a écrit :
It’s 1.9.2

From: Pierre Villard 
mailto:pierre.villard...@gmail.com>>
Sent: Monday, September 9, 2019 7:43 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click

RE: HandleHttpRequest - Failed to receive content from HTTP Request

[William Gosse]

I created the following Jira issue: 
https://issues.apache.org/jira/browse/NIFI-6661

Here’s some more info on this issue:
Also I only see this happening with one of my user who may be ealing with 
network latency.

I have not been able to recreate the issue myself with this exact excetion 
message. However the fact the it causes HandleHttpRequest processor to stop 
functioning seems serious to me.
The only time that ever happened to me is when it to long to get to the 
HandleHttpResponse. I currently have the Request Expiration property for my 
StandardHttpContextMap set to 10 minutes. If the this value is exceeded the 
HandleHttpRequest hangs up. In this specific issue that time out also was 
reached but it seems that the flow never got beyond the HandleHttpRequest.

From: William Gosse 
Sent: Tuesday, September 10, 2019 5:14 PM
To: users@nifi.apache.org
Subject: RE: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2uv82r5u4t-1 
2uv82r5u4t
I’m only seeing it with one user and I’m not sure how they are doing it.

From: Pierre Villard 
mailto:pierre.villard...@gmail.com>>
Sent: Tuesday, September 10, 2019 8:32 AM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

I'm AFK for the moment but could you file a JIRA with some steps to reproduce 
the issue? Happy to have a look once I get back at a computer.

Le mar. 10 sept. 2019 à 08:09, William Gosse 
mailto:william.go...@aifoundry.com>> a écrit :
It’s 1.9.2

From: Pierre Villard 
mailto:pierre.villard...@gmail.com>>
Sent: Monday, September 9, 2019 7:43 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

Hey William,

Which version of NiFi are you using?

Pierre

Le lun. 9 sept. 2019 à 16:20, William Gosse 
mailto:william.go...@aifoundry.com>> a écrit :
I have had couple occurrence of the following exception occurring on an 
HandleHttpRequest pprocessor:
nifi-app_2019-09-06_12.0.log:2019-09-06 12:24:48,132 ERROR [Timer-Driven 
Process Thread-3] o.a.n.p.standard.HandleHttpRequest 
HandleHttpRequest[id=6ceef915-4430-30fa-09d2-b12bb2142172] Failed to receive 
content from HTTP Request from 108.26.163.22 due to java.io.IOException: 
java.util.concurrent.TimeoutException: Idle timeout expired: 61/60 ms: 
java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
expired: 61/60 ms

When it occurs the HandleHttpRequest processor stops excepting requests and I 
have to restart nifi in order to recover.

Is there anything I can do to better handle this exception?

RE: HandleHttpRequest - Failed to receive content from HTTP Request

[William Gosse]

I’m only seeing it with one user and I’m not sure how they are doing it.

From: Pierre Villard 
Sent: Tuesday, September 10, 2019 8:32 AM
To: users@nifi.apache.org
Subject: Re: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

I'm AFK for the moment but could you file a JIRA with some steps to reproduce 
the issue? Happy to have a look once I get back at a computer.

Le mar. 10 sept. 2019 à 08:09, William Gosse 
mailto:william.go...@aifoundry.com>> a écrit :
It’s 1.9.2

From: Pierre Villard 
mailto:pierre.villard...@gmail.com>>
Sent: Monday, September 9, 2019 7:43 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

Hey William,

Which version of NiFi are you using?

Pierre

Le lun. 9 sept. 2019 à 16:20, William Gosse 
mailto:william.go...@aifoundry.com>> a écrit :
I have had couple occurrence of the following exception occurring on an 
HandleHttpRequest pprocessor:
nifi-app_2019-09-06_12.0.log:2019-09-06 12:24:48,132 ERROR [Timer-Driven 
Process Thread-3] o.a.n.p.standard.HandleHttpRequest 
HandleHttpRequest[id=6ceef915-4430-30fa-09d2-b12bb2142172] Failed to receive 
content from HTTP Request from 108.26.163.22 due to java.io.IOException: 
java.util.concurrent.TimeoutException: Idle timeout expired: 61/60 ms: 
java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
expired: 61/60 ms

When it occurs the HandleHttpRequest processor stops excepting requests and I 
have to restart nifi in order to recover.

Is there anything I can do to better handle this exception?

RE: HandleHttpRequest - Failed to receive content from HTTP Request

[William Gosse]

It’s 1.9.2

From: Pierre Villard 
Sent: Monday, September 9, 2019 7:43 PM
To: users@nifi.apache.org
Subject: Re: HandleHttpRequest - Failed to receive content from HTTP Request

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

Hey William,

Which version of NiFi are you using?

Pierre

Le lun. 9 sept. 2019 à 16:20, William Gosse 
mailto:william.go...@aifoundry.com>> a écrit :
I have had couple occurrence of the following exception occurring on an 
HandleHttpRequest pprocessor:
nifi-app_2019-09-06_12.0.log:2019-09-06 12:24:48,132 ERROR [Timer-Driven 
Process Thread-3] o.a.n.p.standard.HandleHttpRequest 
HandleHttpRequest[id=6ceef915-4430-30fa-09d2-b12bb2142172] Failed to receive 
content from HTTP Request from 108.26.163.22 due to java.io.IOException: 
java.util.concurrent.TimeoutException: Idle timeout expired: 61/60 ms: 
java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
expired: 61/60 ms

When it occurs the HandleHttpRequest processor stops excepting requests and I 
have to restart nifi in order to recover.

Is there anything I can do to better handle this exception?

HandleHttpRequest - Failed to receive content from HTTP Request

[William Gosse]

I have had couple occurrence of the following exception occurring on an 
HandleHttpRequest pprocessor:
nifi-app_2019-09-06_12.0.log:2019-09-06 12:24:48,132 ERROR [Timer-Driven 
Process Thread-3] o.a.n.p.standard.HandleHttpRequest 
HandleHttpRequest[id=6ceef915-4430-30fa-09d2-b12bb2142172] Failed to receive 
content from HTTP Request from 108.26.163.22 due to java.io.IOException: 
java.util.concurrent.TimeoutException: Idle timeout expired: 61/60 ms: 
java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
expired: 61/60 ms

When it occurs the HandleHttpRequest processor stops excepting requests and I 
have to restart nifi in order to recover.

Is there anything I can do to better handle this exception?

How keep the from losing original content when a replacetext is performed for an invokehttp

[William Gosse]

I have a situation where I need to do a post using an invokehttp for performing 
a lock operation on another web server. This requires me to do a replacetext in 
order to add the required json needed for the post. I wanted to do this call 
before I performed a putfile call on my system but that blows away the file 
content I needed to put. I really want to do these to calls in this order so if 
the invokehttp to get the lock fails I won’t do my put. I did try using the a 
mergecontent before the put in order the merge the original file content with 
the invokehttp calls content and that seem to work with some other gymnastics. 
I am looking for a better if one exists.

SOLVED: NIFI 1.9.2 PutMongo Update An Existing Filed

[William Gosse]

This turned out to be pilot error because I had the wrong variable  int the 
mongo collection field

From: William Gosse 
Sent: Thursday, August 8, 2019 4:15 PM
To: users@nifi.apache.org
Subject: RE: NIFI 1.9.2 PutMongo Update An Existing Filed

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2u8kp70xa9-1 
2u8kp70xa9
I'm trying to get a PutMongo processor to preform an update on an existing field

It the processor I have the following settings:
Mode: update
Upsert: false
Update Query: {"resourceId" : ${resourceId}}
Update Mode: With operators enabled

Before I go into the PutMongo I do a ReplaceText to put my update operation in 
the content:
Search Value: (?s)(^.*$)
Replacement Value: {$set:{"documentId":"${documentId}"}}
Replacement Strategy: Always Replace
Evaluation Mode: Entire text

Viewing the data providence shows me that this replacement is working correctly

It's the first time I'm trying this so I'm totally open to the fact that I 
could be doing something wrong

RE: NIFI 1.9.2 PutMongo Update An Existing Filed

[William Gosse]

I'm trying to get a PutMongo processor to preform an update on an existing field

It the processor I have the following settings:
Mode: update
Upsert: false
Update Query: {"resourceId" : ${resourceId}}
Update Mode: With operators enabled

Before I go into the PutMongo I do a ReplaceText to put my update operation in 
the content:
Search Value: (?s)(^.*$)
Replacement Value: {$set:{"documentId":"${documentId}"}}
Replacement Strategy: Always Replace
Evaluation Mode: Entire text

Viewing the data providence shows me that this replacement is working correctly

It's the first time I'm trying this so I'm totally open to the fact that I 
could be doing something wrong

NIFI 1.9.2 PutMongo Indexes

[William Gosse]

Is there any way to have Mongo create an index the first time it puts a 
document into a collection. It creates the database and collection just find if 
they don't exit. I like to have add indexes as well.

RESOLVED: NIFI 1.9.2 StandardHttpContextMap Request Expiration value setting

[William Gosse]

Actually I just realized that the settings for the StandardHttpContextMap are 
in my template so I can set the Request Expiration value there.

I have already automated the deployment of my template so the change to value 
happens when I redeploy.

From: William Gosse 
Sent: Wednesday, May 29, 2019 6:32 PM
To: users@nifi.apache.org
Subject: NIFI 1.9.2 StandardHttpContextMap Request Expiration value setting

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2st0gw073s-1 
2st0gw073s
Is there a way to set the StandardHttpContextMap Request Expiration value other 
than through the GUI?

Can it be set through a configuration file or by the restful API?

NIFI 1.9.2 StandardHttpContextMap Request Expiration value setting

[William Gosse]

Is there a way to set the StandardHttpContextMap Request Expiration value other 
than through the GUI?

Can it be set through a configuration file or by the restful API?

RE: 503 Service Unavailable - Jetty Server

[William Gosse]

This sounds a lot like what I'm experiencing:
http://mail-archives.apache.org/mod_mbox/nifi-issues/201808.mbox/%3cjira.13179212.1534379379000.90358.1534379760...@atlassian.jira%3E

I don't quite have the volumes but it's the same symptoms.

From: William Gosse 
Sent: Saturday, May 25, 2019 12:22 PM
To: users@nifi.apache.org
Subject: 503 Service Unavailable - Jetty Server

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2sq17k8g32-1 
2sq17k8g32
It looks like something went wrong with my http jetty server I think after I 
restarted NiFi.

I began having a 503 error response being generated when I made requests 
against my HandleHttpRequest processor.

I also noticed that I could not properly disable and re-enable  the 
StandardHttpContextMap reference that HandleHttpRequest processor.

I've never seen this before but if any one has any thoughts or comments I loved 
to hear them.

RE: 503 Service Unavailable - Jetty Server

[William Gosse]

I should mention that I'm on Nifi 1.9.2

From: William Gosse
Sent: Wednesday, May 29, 2019 4:04 PM
To: users@nifi.apache.org
Subject: RE: 503 Service Unavailable - Jetty Server

This sounds a lot like what I'm experiencing:
http://mail-archives.apache.org/mod_mbox/nifi-issues/201808.mbox/%3cjira.13179212.1534379379000.90358.1534379760...@atlassian.jira%3E

I don't quite have the volumes but it's the same symptoms.

From: William Gosse 
mailto:william.go...@aifoundry.com>>
Sent: Saturday, May 25, 2019 12:22 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: 503 Service Unavailable - Jetty Server

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2sq17k8g32-1 
2sq17k8g32
It looks like something went wrong with my http jetty server I think after I 
restarted NiFi.

I began having a 503 error response being generated when I made requests 
against my HandleHttpRequest processor.

I also noticed that I could not properly disable and re-enable  the 
StandardHttpContextMap reference that HandleHttpRequest processor.

I've never seen this before but if any one has any thoughts or comments I loved 
to hear them.

503 Service Unavailable - Jetty Server

[William Gosse]

It looks like something went wrong with my http jetty server I think after I 
restarted NiFi.

I began having a 503 error response being generated when I made requests 
against my HandleHttpRequest processor.

I also noticed that I could not properly disable and re-enable  the 
StandardHttpContextMap reference that HandleHttpRequest processor.

I've never seen this before but if any one has any thoughts or comments I loved 
to hear them.

RE: java.lang.IllegalStateException: AsyncContext completed and/or Request lifecycle recycled ???

[William Gosse]

Is there any I can do to prevent this exception?

From: Peter Turcsanyi 
Sent: Friday, May 24, 2019 3:16 PM
To: users@nifi.apache.org
Subject: Re: java.lang.IllegalStateException: AsyncContext completed and/or 
Request lifecycle recycled ???

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

The request has a timeout configured on HttpContextMap. If there is some delay 
in the flow and the response is being processed around the expiration time, it 
can happen that the clean-up mechanism in HttpContextMap closes the expired 
request and sends 503 response to the client (while the response thread is 
still processing). You get this error when the response processor tries to 
complete the AsyncContext but it is already completed (AsyncContext is the 
context that wraps the asynchronous request / reponse).
It is a kind of race condition between the response processor and the clean-up 
threads. Recently I was thinking about adding some synchronisation logic for 
these threads, but in the end I decided not to do so because the examples I 
found ([1], [2]) do not handle it. The exception seems to me harmless, it just 
signs that there was a race condition.

[1] 
https://www.logicbig.com/tutorials/java-ee-tutorial/java-servlet/servlet-async.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.logicbig.com_tutorials_java-2Dee-2Dtutorial_java-2Dservlet_servlet-2Dasync.html=DwMFaQ=skUQKLuTFULvQUMqV0uyBaBvPxsA7Wk8MKjvMTLjY3w=Wm89MSTpkAcFFP1M0uvGefI8slyO_VIaIza2IJwAlsU=JvznxQ7jbDiml8t8EUz0c7PdrjHhKPylIuPz8qYe7_8=8VbWuWOFH7BGQXCii5FpZBDSN9EFJHXek2RHSeuP9R8=>
[2] 
https://www.journaldev.com/2008/async-servlet-example<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.journaldev.com_2008_async-2Dservlet-2Dexample=DwMFaQ=skUQKLuTFULvQUMqV0uyBaBvPxsA7Wk8MKjvMTLjY3w=Wm89MSTpkAcFFP1M0uvGefI8slyO_VIaIza2IJwAlsU=JvznxQ7jbDiml8t8EUz0c7PdrjHhKPylIuPz8qYe7_8=RzoNqLyaY6qKaQCAGpp6TKhpyvTKCjajxQcIvD8ZroE=>

On Fri, May 24, 2019 at 5:59 PM William Gosse 
mailto:william.go...@aifoundry.com>> wrote:
Does anyone know what this exception is about:

2019-05-23 19:09:07,102 ERROR [Timer-Driven Process Thread-2] 
o.a.n.p.standard.HandleHttpResponse 
HandleHttpResponse[id=d7b2feb7-4a1e-38c8-07b9-e3d9448bc92b] Failed to complete 
HTTP Transaction for 
StandardFlowFileRecord[uuid=423f4435-95ed-43fa-b03d-d1798a40f21e,claim=StandardContentClaim
 [resourceClaim=StandardResourceClaim[id=1558638515089-4, container=default, 
section=4], offset=127, length=129],offset=0,name=1500212603.pdf,size=129] due 
to java.lang.IllegalStateException: AsyncContext completed and/or Request 
lifecycle recycled: java.lang.IllegalStateException: AsyncContext completed 
and/or Request lifecycle recycled
java.lang.IllegalStateException: AsyncContext completed and/or Request 
lifecycle recycled
at 
org.eclipse.jetty.server.AsyncContextState.state(AsyncContextState.java:54)
at 
org.eclipse.jetty.server.AsyncContextState.complete(AsyncContextState.java:99)
at 
org.apache.nifi.http.StandardHttpContextMap.complete(StandardHttpContextMap.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy87.complete(Unknown Source)
at 
org.apache.nifi.processors.standard.HandleHttpResponse.onTrigger(HandleHttpResponse.java:184)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

java.lang.IllegalStateException: AsyncContext completed and/or Request lifecycle recycled ???

[William Gosse]

Does anyone know what this exception is about:

2019-05-23 19:09:07,102 ERROR [Timer-Driven Process Thread-2] 
o.a.n.p.standard.HandleHttpResponse 
HandleHttpResponse[id=d7b2feb7-4a1e-38c8-07b9-e3d9448bc92b] Failed to complete 
HTTP Transaction for 
StandardFlowFileRecord[uuid=423f4435-95ed-43fa-b03d-d1798a40f21e,claim=StandardContentClaim
 [resourceClaim=StandardResourceClaim[id=1558638515089-4, container=default, 
section=4], offset=127, length=129],offset=0,name=1500212603.pdf,size=129] due 
to java.lang.IllegalStateException: AsyncContext completed and/or Request 
lifecycle recycled: java.lang.IllegalStateException: AsyncContext completed 
and/or Request lifecycle recycled
java.lang.IllegalStateException: AsyncContext completed and/or Request 
lifecycle recycled
at 
org.eclipse.jetty.server.AsyncContextState.state(AsyncContextState.java:54)
at 
org.eclipse.jetty.server.AsyncContextState.complete(AsyncContextState.java:99)
at 
org.apache.nifi.http.StandardHttpContextMap.complete(StandardHttpContextMap.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy87.complete(Unknown Source)
at 
org.apache.nifi.processors.standard.HandleHttpResponse.onTrigger(HandleHttpResponse.java:184)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at 
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

RE: NiFi 1.9.1 release contains a bug causing content repos to fill...

[William Gosse]

"Although I'm not sure why that would be preferred over getting the officially 
released fix in 1.9.2" - I don't prefer applying a patch but I do have some 
systems already setup with 1.9.1. I'll probably bite the bullet and moved to 
1.9.2 as soon as its available.  We are in the process of containerizing nifi 
with docker so hopefully  that will make moving to a newer release not a big 
deal. 

-Original Message-
From: Bryan Bende  
Sent: Tuesday, April 2, 2019 12:25 PM
To: users@nifi.apache.org
Subject: Re: NiFi 1.9.1 release contains a bug causing content repos to fill...

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

The associated JIRA is NIFI-6150 [1] and it leads to the content repo not 
properly cleaning up and eventually filling up disks.

There will be a 1.9.2 release to address this. Of course you are free to take 
the patch for the fix [2], apply it to the 1.9.1 release tag [3], and build 
your own fix, although I'm not sure why that would be preferred over getting 
the officially released fix in 1.9.2.

[1] 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__issues.apache.org_jira_browse_NIFI-2D6150%26d%3DDwIFaQ%26c%3DskUQKLuTFULvQUMqV0uyBaBvPxsA7Wk8MKjvMTLjY3w%26r%3DWm89MSTpkAcFFP1M0uvGefI8slyO_VIaIza2IJwAlsU%26m%3DIcEOLvLqMRwVGHZrj8MeEuGvHHj6WxiOyETFetIDgS8%26s%3D26O4gEjALxwRdlVmLjatxiCTiApEi5UN76no-9D--yQ%26edata=02%7C01%7Cwilliam.gosse%40aifoundry.com%7C65f8067fa3cf41f6eb6e08d6b78b241d%7Cd29b7a9b6edb472099a83c5c6c3eeeb0%7C0%7C0%7C636898205580491866sdata=emHvuWSerL0yVKRSam5metw3KBFYCphFpI%2BlHEpfmOc%3Dreserved=0=
[2] 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__patch-2Ddiff.githubusercontent.com_raw_apache_nifi_pull_3391.patch%26d%3DDwIFaQ%26c%3DskUQKLuTFULvQUMqV0uyBaBvPxsA7Wk8MKjvMTLjY3w%26r%3DWm89MSTpkAcFFP1M0uvGefI8slyO_VIaIza2IJwAlsU%26m%3DIcEOLvLqMRwVGHZrj8MeEuGvHHj6WxiOyETFetIDgS8%26s%3D2_4CR7XulBNmfR2MlPlDcdTgKD6NIiP6S_u8YG17qO0%26edata=02%7C01%7Cwilliam.gosse%40aifoundry.com%7C65f8067fa3cf41f6eb6e08d6b78b241d%7Cd29b7a9b6edb472099a83c5c6c3eeeb0%7C0%7C0%7C636898205580491866sdata=dFHMeTTGT%2FwzbfoPBizqRUJ1dMXhoGS2Q1d%2F3v%2FOBx8%3Dreserved=0=
[3] 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__github.com_apache_nifi_tree_rel_nifi-2D1.9.1%26d%3DDwIFaQ%26c%3DskUQKLuTFULvQUMqV0uyBaBvPxsA7Wk8MKjvMTLjY3w%26r%3DWm89MSTpkAcFFP1M0uvGefI8slyO_VIaIza2IJwAlsU%26m%3DIcEOLvLqMRwVGHZrj8MeEuGvHHj6WxiOyETFetIDgS8%26s%3DVP_sGtxIOZIh_pm1wipeONZ6l1i3G5epzYUT_J_K2dk%26edata=02%7C01%7Cwilliam.gosse%40aifoundry.com%7C65f8067fa3cf41f6eb6e08d6b78b241d%7Cd29b7a9b6edb472099a83c5c6c3eeeb0%7C0%7C0%7C636898205580491866sdata=9B2wS%2FXeHsS9U02A72CKE4MWcr9mQXXY2LIGnFD8ME0%3Dreserved=0=

On Tue, Apr 2, 2019 at 12:17 PM William Gosse  
wrote:
>
> Also can someone explain more what this issue is and how serious it is.
>
> Can a patch be made available to fix this issue for  existing 1.9.1 
> installations?
>
>
>
> From: Joe Witt 
> Sent: Saturday, March 30, 2019 12:32 PM
> To: users@nifi.apache.org
> Subject: Re: NiFi 1.9.1 release contains a bug causing content repos to 
> fill...
>
>
>
> [CAUTION: This email originated from outside of Kodak Alaris. Do not 
> click links or open attachments unless you recognize the sender and 
> know the content is safe.]
>
> 
>
> mathieu
>
>
>
> just a matter of finding the chunk of time to knock it out.  perhaps this 
> evening.
>
>
>
> thanks
>
> On Sat, Mar 30, 2019, 11:27 AM Louis-Ferdinand, Mathieu 
>  wrote:
>
> Hi Joe,
>
>
>
> Do you have an idea of the timeline for 1.9.2 RC?
>
>
>
> Cheers,
>
> Mathieu.
>
>
>
> From: Joe Witt [mailto:joe.w...@gmail.com]
> Sent: Wednesday, March 27, 2019 11:00 AM
> To: users@nifi.apache.org
> Subject: NiFi 1.9.1 release contains a bug causing content repos to fill...
>
>
>
> All,
>
>
>
> We will work to promptly produce an Apache NiFi 1.9.2 release which will 
> correct a regression introduced in the 1.9.1 release.  The issue, NIFI-6150 
> has been identified and fixed on master and once further testing confirms 
> we're good we'll get the RC going.
>
>
>
> I've updated the release guide to discourage use of 1.9.1 until the fix is 
> out.
>
>
>
> Thanks
>
> Joe
>
>
>
> 
>
> NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions 
> or views contained herein are not intended to be, and do not constitute, 
> advice within the meaning 

RE: NiFi 1.9.1 release contains a bug causing content repos to fill...

[William Gosse]

Also can someone explain more what this issue is and how serious it is.
Can a patch be made available to fix this issue for  existing 1.9.1 
installations?

From: Joe Witt mailto:joe.w...@gmail.com>>
Sent: Saturday, March 30, 2019 12:32 PM
To: users@nifi.apache.org
Subject: Re: NiFi 1.9.1 release contains a bug causing content repos to fill...

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

mathieu

just a matter of finding the chunk of time to knock it out.  perhaps this 
evening.

thanks
On Sat, Mar 30, 2019, 11:27 AM Louis-Ferdinand, Mathieu 
mailto:mathieu.louis-ferdin...@morganstanley.com>>
 wrote:
Hi Joe,

Do you have an idea of the timeline for 1.9.2 RC?

Cheers,
Mathieu.

From: Joe Witt [mailto:joe.w...@gmail.com]
Sent: Wednesday, March 27, 2019 11:00 AM
To: users@nifi.apache.org
Subject: NiFi 1.9.1 release contains a bug causing content repos to fill...

All,

We will work to promptly produce an Apache NiFi 1.9.2 release which will 
correct a regression introduced in the 1.9.1 release.  The issue, NIFI-6150 has 
been identified and fixed on master and once further testing confirms we're 
good we'll get the RC going.

I've updated the release guide to discourage use of 1.9.1 until the fix is out.

Thanks
Joe


NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or 
views contained herein are not intended to be, and do not constitute, advice 
within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and 
Consumer Protection Act. If you have received this communication in error, 
please destroy all electronic and paper copies and notify the sender 
immediately. Mistransmission is not intended to waive confidentiality or 
privilege. Morgan Stanley reserves the right, to the extent required and/or 
permitted under applicable law, to monitor electronic communications, including 
telephone calls with Morgan Stanley personnel. This message is subject to the 
Morgan Stanley General Disclaimers available at the following link: 
http://www.morganstanley.com/disclaimers.
  If you cannot access the links, please notify us by reply message and we will 
send the contents to you. By communicating with Morgan Stanley you acknowledge 
that you have read, understand and consent, (where applicable), to the 
foregoing and the Morgan Stanley General Disclaimers.

RE: NiFi 1.9.1 release contains a bug causing content repos to fill...

[William Gosse]

Can a patch be made available to fix this issue for  existing 1.9.1 
installations?

From: Joe Witt 
Sent: Saturday, March 30, 2019 12:32 PM
To: users@nifi.apache.org
Subject: Re: NiFi 1.9.1 release contains a bug causing content repos to fill...

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

mathieu

just a matter of finding the chunk of time to knock it out.  perhaps this 
evening.

thanks
On Sat, Mar 30, 2019, 11:27 AM Louis-Ferdinand, Mathieu 
mailto:mathieu.louis-ferdin...@morganstanley.com>>
 wrote:
Hi Joe,

Do you have an idea of the timeline for 1.9.2 RC?

Cheers,
Mathieu.

From: Joe Witt [mailto:joe.w...@gmail.com]
Sent: Wednesday, March 27, 2019 11:00 AM
To: users@nifi.apache.org
Subject: NiFi 1.9.1 release contains a bug causing content repos to fill...

All,

We will work to promptly produce an Apache NiFi 1.9.2 release which will 
correct a regression introduced in the 1.9.1 release.  The issue, NIFI-6150 has 
been identified and fixed on master and once further testing confirms we're 
good we'll get the RC going.

I've updated the release guide to discourage use of 1.9.1 until the fix is out.

Thanks
Joe


NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or 
views contained herein are not intended to be, and do not constitute, advice 
within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and 
Consumer Protection Act. If you have received this communication in error, 
please destroy all electronic and paper copies and notify the sender 
immediately. Mistransmission is not intended to waive confidentiality or 
privilege. Morgan Stanley reserves the right, to the extent required and/or 
permitted under applicable law, to monitor electronic communications, including 
telephone calls with Morgan Stanley personnel. This message is subject to the 
Morgan Stanley General Disclaimers available at the following link: 
http://www.morganstanley.com/disclaimers.
  If you cannot access the links, please notify us by reply message and we will 
send the contents to you. By communicating with Morgan Stanley you acknowledge 
that you have read, understand and consent, (where applicable), to the 
foregoing and the Morgan Stanley General Disclaimers.

RE: Merge? Notify? Wait?

[William Gosse]

That is an awesome idea!  I’m definitely going to try that. Thank you.

From: Mark Payne 
Sent: Wednesday, March 27, 2019 2:42 PM
To: users@nifi.apache.org
Subject: Re: Merge? Notify? Wait?

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

William,

You should be able to perform this using the Wait/Notify pattern. But I think a 
simpler (and likely better performing) alternative would be
to avoid splitting out into two directions at all and instead use a linear flow:

ListFile -> AttributesToJson -> ReplaceText -> InvokeHttp -> FetchFile -> 
UpdateAttribute (to rename file) -> PutFile

Because the first direction does not appear to need the contents of the file, 
since it is using attributes to create the JSON payload,
there's no need to fetch the contents of the file until after you've already 
made the web request and retrieved the client id. Once you
have that, you can fetch the contents and then use PutFile.

Thanks
-Mark



On Mar 27, 2019, at 2:25 PM, William Gosse 
mailto:william.go...@aifoundry.com>> wrote:

I’ve been trying to solve this problem in Nifi for the past couple of days and 
haven’t been able to find the secret sauce yet.

I goes something like this:
List/Fetch a file
Extract an client name from part of the filename using an UpdateAttribute

At this I send the flow in two different directions:
Direction 1 does this:
Take the original flow and create a new JSON content out of it 
using an AttributesToJson and ReplaceText
Post my new JSON content to a web service using an InvokeHttp 
to create a new client with the client name
Extract a client new client id from the web service response.

Direction 2 does this
Wait for the new client id return from the webservice  response 
(this is the part I’m not getting)
Use client id to rename original file
Put file to specified folder.

I’ll admit I could be doing this all wrong but what I’m having trouble with is 
bringing together my original flowfile with the result of my web service call.

I thought I could use a mergecontent somehow or a notify/wait but neither 
approach seems to work.

Any help will be greatly appreciated.

Merge? Notify? Wait?

[William Gosse]

I’ve been trying to solve this problem in Nifi for the past couple of days and 
haven’t been able to find the secret sauce yet.

I goes something like this:
List/Fetch a file
Extract an client name from part of the filename using an UpdateAttribute

At this I send the flow in two different directions:
Direction 1 does this:
Take the original flow and create a new JSON content out of it 
using an AttributesToJson and ReplaceText
Post my new JSON content to a web service using an InvokeHttp 
to create a new client with the client name
Extract a client new client id from the web service response.

Direction 2 does this
Wait for the new client id return from the webservice  response 
(this is the part I’m not getting)
Use client id to rename original file
Put file to specified folder.

I’ll admit I could be doing this all wrong but what I’m having trouble with is 
bringing together my original flowfile with the result of my web service call.

I thought I could use a mergecontent somehow or a notify/wait but neither 
approach seems to work.

Any help will be greatly appreciated.

RE: Weird ListFile Issue

[William Gosse]

Yep exactly.  It probably shouldn’t do anything if its null or empty.

From: Andrew Grande 
Sent: Friday, March 22, 2019 5:04 PM
To: users@nifi.apache.org
Subject: Re: Weird ListFile Issue

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

Looks like the processor started listing CWD
On Fri, Mar 22, 2019, 2:00 PM William Gosse 
mailto:william.go...@aifoundry.com>> wrote:
The real var name was aimuploaddir and the error was aimduploaddir.  Think I 
have a work around to prevent the tragedy was wiping out Nifi. I did this:
${aimuploaddir:replaceEmpty('')}


From: Joe Witt mailto:joe.w...@gmail.com>>
Sent: Friday, March 22, 2019 4:06 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: Weird ListFile Issue

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

William

What was the real dir name vs the erroneous name?   What might happens depends 
on many factors such as os behaviors.

thanks

On Fri, Mar 22, 2019, 12:47 PM William Gosse 
mailto:william.go...@aifoundry.com>> wrote:
I ran into kind of a weird issue with the ListFile processor.   I was 
referencing a variable for my input directory and had enter the variable name 
incorrectly.

So I assumed that the with my incorrect variable name the value of the Input 
Directory would be null or an empty string.

When I started the InputFile it started listing all the files inside of my Nifi 
install directory.  Things got  bad when the my FetchFile started to delete 
them.

I'm on 1.9.1 but I was wondering if this was the normal behavior or some kind 
of bug.

If anyone has seen this behavior and has some kind of viable workaround please 
let me know.

RE: Weird ListFile Issue

[William Gosse]

The real var name was aimuploaddir and the error was aimduploaddir.  Think I 
have a work around to prevent the tragedy was wiping out Nifi. I did this:
${aimuploaddir:replaceEmpty('')}


From: Joe Witt 
Sent: Friday, March 22, 2019 4:06 PM
To: users@nifi.apache.org
Subject: Re: Weird ListFile Issue

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.]

William

What was the real dir name vs the erroneous name?   What might happens depends 
on many factors such as os behaviors.

thanks

On Fri, Mar 22, 2019, 12:47 PM William Gosse 
mailto:william.go...@aifoundry.com>> wrote:
I ran into kind of a weird issue with the ListFile processor.   I was 
referencing a variable for my input directory and had enter the variable name 
incorrectly.

So I assumed that the with my incorrect variable name the value of the Input 
Directory would be null or an empty string.

When I started the InputFile it started listing all the files inside of my Nifi 
install directory.  Things got  bad when the my FetchFile started to delete 
them.

I'm on 1.9.1 but I was wondering if this was the normal behavior or some kind 
of bug.

If anyone has seen this behavior and has some kind of viable workaround please 
let me know.

Weird ListFile Issue

[William Gosse]

I ran into kind of a weird issue with the ListFile processor.   I was 
referencing a variable for my input directory and had enter the variable name 
incorrectly.

So I assumed that the with my incorrect variable name the value of the Input 
Directory would be null or an empty string.

When I started the InputFile it started listing all the files inside of my Nifi 
install directory.  Things got  bad when the my FetchFile started to delete 
them.

I'm on 1.9.1 but I was wondering if this was the normal behavior or some kind 
of bug.

If anyone has seen this behavior and has some kind of viable workaround please 
let me know.

RE: Automate the Enablement of a StandardHttpContextMap

[William Gosse]

Ok I got this working by configuring each Http component's 
StandardHttpContextMap  individual instead having them all ganged all onto one.

Each of the StandardHttpContextMap  is stills coped for the process group that 
all the components are in.

I'll admit I'm still a little shaky on controller services but it seem to work 
now.  

I'm able to enable all the controller services form the root group with 
.\conf\cli.properties
PS C:\Dev\aim\nifi-toolkit-1.9.1\bin> .\cli.bat nifi pg-enable-services -pgid 
911b83b1-0169-1000-17bb-823bb530cf8c -p ..\conf\cli.properties

Thanks for the help it is much approceiated.

-Original Message-
From: William Gosse  
Sent: Thursday, March 21, 2019 2:55 PM
To: users@nifi.apache.org
Subject: RE: Automate the Enablement of a StandardHttpContextMap

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

[WARNING: Potential spoofed mail]
The sender of this email used an @kodakalaris.com email address, but the email 
did not come from a known Kodak Alaris email system. This email might not be 
legitimate. Delete this email unless it was sent from a known, trusted source.
To avoid this notice if this email was sent from a known trusted source please 
submit an IT_KAre request to have this sender added as a valid email sender.

Proofpoint rule module.access.rule.anti_spoofing_rule Message ID 2rceu6r68c-1 
2rceu6r68c

I'm having a problem with the command you showed below. When I run it for my 
stuff I get the following:
PS C:\Dev\aim\nifi-toolkit-1.9.1\bin> .\cli.bat nifi pg-enable-services -pgid 
aee943fb-48ac-351e-aef7-29db27d33b0f -p ..
\conf\cli.properties

ERROR: Error executing command 'pg-enable-services' : One or more services 
could not be enabled, run command with -verbose to obtain more details

The are for components referencing the StandardHttpContextMap, one 
HandleHttpRequest component and three  HandleHttpResponse components.

Any suggestion on how to handle this error?

-Original Message-
From: Bryan Bende 
Sent: Thursday, March 21, 2019 12:27 PM
To: users@nifi.apache.org
Subject: Re: Automate the Enablement of a StandardHttpContextMap

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

When a template, or a versioned flow, is instantiated, all the processors are 
initially stopped and the services are disabled. This is done on purpose since 
starting and enabling components may cause interactions with external systems, 
and it may not be clear or desired that just by instantiating the template that 
all of these interactions are automatically going to take place. So it requires 
an extra step where the user (or your script) must explicitly enabled and start 
the components.

The NIFI CLI has a command "pg-enable-services" that can be used to enable all 
the services in a given process group. You could call this from your script 
like "./bin/cli.sh nifi pg-enable-service -pgid someid".

On Thu, Mar 21, 2019 at 11:29 AM William Gosse  
wrote:
>
> I’m trying to automate the deployment of a template that contains a 
> HandleHttpRequest processor.
>
>
>
> My issue is that this always comes up with an Invalid component marker after 
> a fresh deployment of the template.
>
> The cause of this marker is that the StandardHttpContextMap in the 
> HandleHttpRequest processor needs to be manually enabled in Controller 
> Services.
>
>
>
> Is there any way to avoid this manual enablement?
>
>
>
>

RE: Automate the Enablement of a StandardHttpContextMap

[William Gosse]

I'm having a problem with the command you showed below. When I run it for my 
stuff I get the following:
PS C:\Dev\aim\nifi-toolkit-1.9.1\bin> .\cli.bat nifi pg-enable-services -pgid 
aee943fb-48ac-351e-aef7-29db27d33b0f -p ..
\conf\cli.properties

ERROR: Error executing command 'pg-enable-services' : One or more services 
could not be enabled, run command with -verbose to obtain more details

The are for components referencing the StandardHttpContextMap, one 
HandleHttpRequest component and three  HandleHttpResponse components.

Any suggestion on how to handle this error?

-Original Message-
From: Bryan Bende  
Sent: Thursday, March 21, 2019 12:27 PM
To: users@nifi.apache.org
Subject: Re: Automate the Enablement of a StandardHttpContextMap

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

When a template, or a versioned flow, is instantiated, all the processors are 
initially stopped and the services are disabled. This is done on purpose since 
starting and enabling components may cause interactions with external systems, 
and it may not be clear or desired that just by instantiating the template that 
all of these interactions are automatically going to take place. So it requires 
an extra step where the user (or your script) must explicitly enabled and start 
the components.

The NIFI CLI has a command "pg-enable-services" that can be used to enable all 
the services in a given process group. You could call this from your script 
like "./bin/cli.sh nifi pg-enable-service -pgid someid".

On Thu, Mar 21, 2019 at 11:29 AM William Gosse  
wrote:
>
> I’m trying to automate the deployment of a template that contains a 
> HandleHttpRequest processor.
>
>
>
> My issue is that this always comes up with an Invalid component marker after 
> a fresh deployment of the template.
>
> The cause of this marker is that the StandardHttpContextMap in the 
> HandleHttpRequest processor needs to be manually enabled in Controller 
> Services.
>
>
>
> Is there any way to avoid this manual enablement?
>
>
>
>

Automate the Enablement of a StandardHttpContextMap

[William Gosse]

I'm trying to automate the deployment of a template that contains a 
HandleHttpRequest processor.

My issue is that this always comes up with an Invalid component marker after a 
fresh deployment of the template.
The cause of this marker is that the StandardHttpContextMap in the 
HandleHttpRequest processor needs to be manually enabled in Controller Services.

Is there any way to avoid this manual enablement?

RE: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted

[William Gosse]

"My workaround is to repeatedly call GET on the RPG until the port becomes 
available."

Can you explain a little more by what you mean by this. Or if you have an 
example please send it.

-Original Message-
From: christophe.mon...@post.ch  
Sent: Friday, November 16, 2018 12:46 AM
To: users@nifi.apache.org
Subject: RE: NiFi 1.7.1 remote group not connecting when added through restful 
api until nifi restarted

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

Hi William

My experience with using the REST API to create RPG is that you must wait “some 
time” before the port becomes available. A restart is not needed though.
My workaround is to repeatedly call GET on the RPG until the port becomes 
available. It typically takes more than 10 seconds on a 5 nodes cluster.

Chris

-Original Message-----
From: William Gosse 
Sent: Donnerstag, 15 November 2018 18:03
To: users@nifi.apache.org
Subject: RE: NiFi 1.7.1 remote group not connecting when added through restful 
api until nifi restarted

I don't believe I see any error messages on the initial remote process group 
that gets created.

I do see the error message when I do a restart and there are existing remote 
process groups but they do seem to connect eventually

It's only the initial create of  the first remote process group that seems to 
be acting weird.

-Original Message-
From: Koji Kawamura 
Sent: Thursday, November 15, 2018 1:05 AM
To: users@nifi.apache.org
Subject: Re: NiFi 1.7.1 remote group not connecting when added through restful 
api until nifi restarted

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

Hello William,

> fails to connect to the existing input port until I do a restart of 
> NiFi
Is there any error message when it fails? Connection refused?
It should not require a NiFi restart to establish connection.

Thanks,
Koji



On Thu, Nov 15, 2018 at 1:38 AM William Gosse  
wrote:
>
> I'm using NiFi's restful api to create a process group from a template 
> that contains a file folder listener and a remote group. I also have 
> an existing input port going to a process group that has the file 
> fetch and everything else for processing the file that was fetched.
> This is not added by a restful api but is loaded from a template after 
> the initial startup of Nifi. What I wind up with is a list fetch 
> pattern that should cluster ready/
>
> My problem is that the first time I use the rest api to add the listener 
> process group the remote group that's inside of it fails to connect to the 
> existing input port until I do a restart of NiFi. At that point all 
> subsequent listeners I add seem to connect just fine.
>
> Is there way that I can cause the initial connection to occur without having 
> to perform a restart?
>
>

RE: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted

[William Gosse]

I don't believe I see any error messages on the initial remote process group 
that gets created.

I do see the error message when I do a restart and there are existing remote 
process groups but they do seem to connect eventually

It's only the initial create of  the first remote process group that seems to 
be acting weird.

-Original Message-
From: Koji Kawamura  
Sent: Thursday, November 15, 2018 1:05 AM
To: users@nifi.apache.org
Subject: Re: NiFi 1.7.1 remote group not connecting when added through restful 
api until nifi restarted

[CAUTION: This email originated from outside of Kodak Alaris. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.] 

Hello William,

> fails to connect to the existing input port until I do a restart of 
> NiFi
Is there any error message when it fails? Connection refused?
It should not require a NiFi restart to establish connection.

Thanks,
Koji



On Thu, Nov 15, 2018 at 1:38 AM William Gosse  
wrote:
>
> I'm using NiFi's restful api to create a process group from a template 
> that contains a file folder listener and a remote group. I also have 
> an existing input port going to a process group that has the file 
> fetch and everything else for processing the file that was fetched. 
> This is not added by a restful api but is loaded from a template after 
> the initial startup of Nifi. What I wind up with is a list fetch 
> pattern that should cluster ready/
>
> My problem is that the first time I use the rest api to add the listener 
> process group the remote group that's inside of it fails to connect to the 
> existing input port until I do a restart of NiFi. At that point all 
> subsequent listeners I add seem to connect just fine.
>
> Is there way that I can cause the initial connection to occur without having 
> to perform a restart?
>
>

NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted

[William Gosse]

I'm using NiFi's restful api to create a process group from a template that 
contains a file folder listener and a remote group. I also have an existing 
input port going to a process group that has the file fetch and everything else 
for processing the file that was fetched. This is not added by a restful api 
but is loaded from a template after the initial startup of Nifi. What I wind up 
with is a list fetch pattern that should cluster ready/

My problem is that the first time I use the rest api to add the listener 
process group the remote group that's inside of it fails to connect to the 
existing input port until I do a restart of NiFi. At that point all subsequent 
listeners I add seem to connect just fine.

Is there way that I can cause the initial connection to occur without having to 
perform a restart?

Problem with remote groups reconnecting to input port after restart.

[William Gosse]

Hi,

I'm trying to implement a Listfile>FetchFile pattern similar to the one 
described on the following link:
https://pierrevillard.com/2017/02/23/listfetch-pattern-and-remote-process-group-in-apache-nifi/

I have multiple Listfiles process groups each containing a remote process group 
that point to a common input port.

The input port itself points to a common fetchfile process group.

My issue I'm having is that after I do a Nifi restart not all my remote process 
groups successfully connect to the input port.

Are there any setting I should be aware to help eliminate this problem? Should 
use RAW vs HTTP, I'm currently using HTTP.

Any help be greatly appreciated.

Regards,
Bill