Re: Reverse Proxy Template Import Error

[Matt Gilman]

Awesome thanks! We'll get this reviewed and then it'll get merged and included 
in 1.2.0. Thanks for the verification assist!

Matt

Sent from my iPhone

> On Dec 15, 2016, at 8:45 PM, Richard St. John  wrote:
> 
> Matt,
> 
> Great news!  I was able to successfully test the changes you made to
> address the import template issue.  I do not know what other verification
> is needed for your PR to be approved, but after building NIFI
> 1.2.0-SNAPSHOT with your changes from NIFI-3207, the issue is resolved.
> 
> Thanks.
> 
> Rick.
> 
>> On Thu, Dec 15, 2016 at 1:43 PM Matt Gilman  wrote:
>> 
>> Rick,
>> 
>> Here's my proposed solution for the template upload issue [1]. Please give
>> it a shot behind your proxy.
>> 
>> Thanks
>> 
>> Matt
>> 
>> [1] https://github.com/apache/nifi/pull/1334
>> 
>> On Thu, Dec 15, 2016 at 12:58 PM, Matt Gilman 
>> wrote:
>> 
>>> Awesome. Here's the JIRA [1]. I should have the PR posted shortly.
>>> 
>>> Matt
>>> 
>>> [1] https://issues.apache.org/jira/browse/NIFI-3207
>>> 
>>> On Thu, Dec 15, 2016 at 12:53 PM, Richard St. John 
>>> wrote:
>>> 
 I should be able to test tonight.
 
 On Thu, Dec 15, 2016 at 12:45 PM Matt Gilman 
 wrote:
 
> Rick,
> 
> I think I see what the issue is however, I have don't have the
 environment
> available for verifying it. If I posted a PR would you be in a
>> position
 to
> try it out?
> 
> Matt
> 
> On Wed, Dec 14, 2016 at 9:00 PM, Richard St. John <
>> rstjoh...@gmail.com>
> wrote:
> 
>> Matt,
>> 
>> This is where it gets interesting.  The issue only arises when nginx
 is
>> used as a reverse proxy for a *clustered* NiFi (within a trusted
>> environment using http).  In fact, it even occurs when the reverse
 proxy
>> upstream points to only one of the nifi nodes in the cluster.
 Moreover,
>> the problem is limited to importing/uploading a template.  Deleting,
>> downloading, and instantiating existing templates works fine.
>> 
>> There is another strange pattern that I discovered.  I can get
 templates
> to
>> import if I hard-code the scheme header as follows:
>> *proxy_set_header
>> X-ProxyScheme http;*  However, when I hard-code the proxy scheme to
> http, I
>> cannot delete a template after it is uploaded.
>> 
>> Below are several of the logs (IPs and DNS were changed for security
>> reasons).
>> 
>> *nginx.err*
>> 2016/12/15 01:19:42 [warn] 29377#29377: *16 a client request body is
>> buffered to a temporary file
>> /var/cache/nginx/client_temp/05,
>> client: X.X.X.X, server: nifi-rix.somedomain.com, request: "POST
>> /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
>> 8a86805ff8bb/templates/upload
>> HTTP/1.1", host: "nifi-rix.somedomain.com", referrer: "
>> https://nifi-rix.somedomain.com/nifi/;
>> 
>> *nginx access log*
>> [15/Dec/2016:01:22:17 +] - (X.X.X.X - rstjohn) "POST
>> /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
>> 8a86805ff8bb/templates/upload
>> HTTP/1.1" 500 0 "https://nifi-rix.somedomain.com/nifi/;
>> "Mozilla/5.0
>> (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like
> Gecko)
>> Chrome/55.0.2883.95 Safari/537.36" "-"
>> 
>> 
>> *NiFi Log: nifi-app.log*
>> 2016-12-15 01:25:00,362 WARN [Replicate Request Thread-4]
>> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate
>> request
> POST
>> /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
>> 8a86805ff8bb/templates/import
>> to ip-X-X-X-X.ec2.internal:8080 due to {}
>> com.sun.jersey.api.client.ClientHandlerException:
>> javax.ws.rs.WebApplicationException:
>> javax.xml.bind.MarshalException
>> - with linked exception:
>> [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
>> connection?]
>> at
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandl
 er.handle(
>> URLConnectionClientHandler.java:155)
>> ~[jersey-client-1.19.jar:1.19]
>> at com.sun.jersey.api.client.Client.handle(Client.java:652)
>> ~[jersey-client-1.19.jar:1.19]
>> at
>> com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
>> ~[jersey-client-1.19.jar:1.19]
>> at com.sun.jersey.api.client.WebResource.access$200(WebResource
 .java:74)
>> ~[jersey-client-1.19.jar:1.19]
>> at com.sun.jersey.api.client.WebResource$Builder.post(
>> WebResource.java:560)
>> ~[jersey-client-1.19.jar:1.19]
>> at
>> org.apache.nifi.cluster.coordination.http.replication.
>> ThreadPoolRequestReplicator.replicateRequest(ThreadPoolReque
 stReplicator.
>> java:587)
>> ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
>> at
>> 

Re: Reverse Proxy Template Import Error

[Richard St. John]

Matt,

Great news!  I was able to successfully test the changes you made to
address the import template issue.  I do not know what other verification
is needed for your PR to be approved, but after building NIFI
1.2.0-SNAPSHOT with your changes from NIFI-3207, the issue is resolved.

Thanks.

Rick.

On Thu, Dec 15, 2016 at 1:43 PM Matt Gilman  wrote:

> Rick,
>
> Here's my proposed solution for the template upload issue [1]. Please give
> it a shot behind your proxy.
>
> Thanks
>
> Matt
>
> [1] https://github.com/apache/nifi/pull/1334
>
> On Thu, Dec 15, 2016 at 12:58 PM, Matt Gilman 
> wrote:
>
> > Awesome. Here's the JIRA [1]. I should have the PR posted shortly.
> >
> > Matt
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-3207
> >
> > On Thu, Dec 15, 2016 at 12:53 PM, Richard St. John 
> > wrote:
> >
> >> I should be able to test tonight.
> >>
> >> On Thu, Dec 15, 2016 at 12:45 PM Matt Gilman 
> >> wrote:
> >>
> >> > Rick,
> >> >
> >> > I think I see what the issue is however, I have don't have the
> >> environment
> >> > available for verifying it. If I posted a PR would you be in a
> position
> >> to
> >> > try it out?
> >> >
> >> > Matt
> >> >
> >> > On Wed, Dec 14, 2016 at 9:00 PM, Richard St. John <
> rstjoh...@gmail.com>
> >> > wrote:
> >> >
> >> > > Matt,
> >> > >
> >> > > This is where it gets interesting.  The issue only arises when nginx
> >> is
> >> > > used as a reverse proxy for a *clustered* NiFi (within a trusted
> >> > > environment using http).  In fact, it even occurs when the reverse
> >> proxy
> >> > > upstream points to only one of the nifi nodes in the cluster.
> >> Moreover,
> >> > > the problem is limited to importing/uploading a template.  Deleting,
> >> > > downloading, and instantiating existing templates works fine.
> >> > >
> >> > > There is another strange pattern that I discovered.  I can get
> >> templates
> >> > to
> >> > > import if I hard-code the scheme header as follows:
> *proxy_set_header
> >> > > X-ProxyScheme http;*  However, when I hard-code the proxy scheme to
> >> > http, I
> >> > > cannot delete a template after it is uploaded.
> >> > >
> >> > > Below are several of the logs (IPs and DNS were changed for security
> >> > > reasons).
> >> > >
> >> > > *nginx.err*
> >> > > 2016/12/15 01:19:42 [warn] 29377#29377: *16 a client request body is
> >> > > buffered to a temporary file
> /var/cache/nginx/client_temp/05,
> >> > > client: X.X.X.X, server: nifi-rix.somedomain.com, request: "POST
> >> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> >> > > 8a86805ff8bb/templates/upload
> >> > > HTTP/1.1", host: "nifi-rix.somedomain.com", referrer: "
> >> > > https://nifi-rix.somedomain.com/nifi/;
> >> > >
> >> > > *nginx access log*
> >> > > [15/Dec/2016:01:22:17 +] - (X.X.X.X - rstjohn) "POST
> >> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> >> > > 8a86805ff8bb/templates/upload
> >> > > HTTP/1.1" 500 0 "https://nifi-rix.somedomain.com/nifi/;
> "Mozilla/5.0
> >> > > (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like
> >> > Gecko)
> >> > > Chrome/55.0.2883.95 Safari/537.36" "-"
> >> > >
> >> > >
> >> > > *NiFi Log: nifi-app.log*
> >> > > 2016-12-15 01:25:00,362 WARN [Replicate Request Thread-4]
> >> > > o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate
> request
> >> > POST
> >> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> >> > > 8a86805ff8bb/templates/import
> >> > > to ip-X-X-X-X.ec2.internal:8080 due to {}
> >> > > com.sun.jersey.api.client.ClientHandlerException:
> >> > > javax.ws.rs.WebApplicationException:
> javax.xml.bind.MarshalException
> >> > >  - with linked exception:
> >> > > [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
> >> > > connection?]
> >> > > at
> >> > > com.sun.jersey.client.urlconnection.URLConnectionClientHandl
> >> er.handle(
> >> > > URLConnectionClientHandler.java:155)
> >> > > ~[jersey-client-1.19.jar:1.19]
> >> > > at com.sun.jersey.api.client.Client.handle(Client.java:652)
> >> > > ~[jersey-client-1.19.jar:1.19]
> >> > > at
> com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> >> > > ~[jersey-client-1.19.jar:1.19]
> >> > > at com.sun.jersey.api.client.WebResource.access$200(WebResource
> >> .java:74)
> >> > > ~[jersey-client-1.19.jar:1.19]
> >> > > at com.sun.jersey.api.client.WebResource$Builder.post(
> >> > > WebResource.java:560)
> >> > > ~[jersey-client-1.19.jar:1.19]
> >> > > at
> >> > > org.apache.nifi.cluster.coordination.http.replication.
> >> > > ThreadPoolRequestReplicator.replicateRequest(ThreadPoolReque
> >> stReplicator.
> >> > > java:587)
> >> > > ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
> >> > > at
> >> > > org.apache.nifi.cluster.coordination.http.replication.
> >> > > ThreadPoolRequestReplicator$NodeHttpRequest.run(
> >> > > ThreadPoolRequestReplicator.java:770)
> >> > > 

Re: Reverse Proxy Template Import Error

[Matt Gilman]

Rick,

Here's my proposed solution for the template upload issue [1]. Please give
it a shot behind your proxy.

Thanks

Matt

[1] https://github.com/apache/nifi/pull/1334

On Thu, Dec 15, 2016 at 12:58 PM, Matt Gilman 
wrote:

> Awesome. Here's the JIRA [1]. I should have the PR posted shortly.
>
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-3207
>
> On Thu, Dec 15, 2016 at 12:53 PM, Richard St. John 
> wrote:
>
>> I should be able to test tonight.
>>
>> On Thu, Dec 15, 2016 at 12:45 PM Matt Gilman 
>> wrote:
>>
>> > Rick,
>> >
>> > I think I see what the issue is however, I have don't have the
>> environment
>> > available for verifying it. If I posted a PR would you be in a position
>> to
>> > try it out?
>> >
>> > Matt
>> >
>> > On Wed, Dec 14, 2016 at 9:00 PM, Richard St. John 
>> > wrote:
>> >
>> > > Matt,
>> > >
>> > > This is where it gets interesting.  The issue only arises when nginx
>> is
>> > > used as a reverse proxy for a *clustered* NiFi (within a trusted
>> > > environment using http).  In fact, it even occurs when the reverse
>> proxy
>> > > upstream points to only one of the nifi nodes in the cluster.
>> Moreover,
>> > > the problem is limited to importing/uploading a template.  Deleting,
>> > > downloading, and instantiating existing templates works fine.
>> > >
>> > > There is another strange pattern that I discovered.  I can get
>> templates
>> > to
>> > > import if I hard-code the scheme header as follows: *proxy_set_header
>> > > X-ProxyScheme http;*  However, when I hard-code the proxy scheme to
>> > http, I
>> > > cannot delete a template after it is uploaded.
>> > >
>> > > Below are several of the logs (IPs and DNS were changed for security
>> > > reasons).
>> > >
>> > > *nginx.err*
>> > > 2016/12/15 01:19:42 [warn] 29377#29377: *16 a client request body is
>> > > buffered to a temporary file /var/cache/nginx/client_temp/05,
>> > > client: X.X.X.X, server: nifi-rix.somedomain.com, request: "POST
>> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
>> > > 8a86805ff8bb/templates/upload
>> > > HTTP/1.1", host: "nifi-rix.somedomain.com", referrer: "
>> > > https://nifi-rix.somedomain.com/nifi/;
>> > >
>> > > *nginx access log*
>> > > [15/Dec/2016:01:22:17 +] - (X.X.X.X - rstjohn) "POST
>> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
>> > > 8a86805ff8bb/templates/upload
>> > > HTTP/1.1" 500 0 "https://nifi-rix.somedomain.com/nifi/; "Mozilla/5.0
>> > > (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like
>> > Gecko)
>> > > Chrome/55.0.2883.95 Safari/537.36" "-"
>> > >
>> > >
>> > > *NiFi Log: nifi-app.log*
>> > > 2016-12-15 01:25:00,362 WARN [Replicate Request Thread-4]
>> > > o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request
>> > POST
>> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
>> > > 8a86805ff8bb/templates/import
>> > > to ip-X-X-X-X.ec2.internal:8080 due to {}
>> > > com.sun.jersey.api.client.ClientHandlerException:
>> > > javax.ws.rs.WebApplicationException: javax.xml.bind.MarshalException
>> > >  - with linked exception:
>> > > [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
>> > > connection?]
>> > > at
>> > > com.sun.jersey.client.urlconnection.URLConnectionClientHandl
>> er.handle(
>> > > URLConnectionClientHandler.java:155)
>> > > ~[jersey-client-1.19.jar:1.19]
>> > > at com.sun.jersey.api.client.Client.handle(Client.java:652)
>> > > ~[jersey-client-1.19.jar:1.19]
>> > > at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
>> > > ~[jersey-client-1.19.jar:1.19]
>> > > at com.sun.jersey.api.client.WebResource.access$200(WebResource
>> .java:74)
>> > > ~[jersey-client-1.19.jar:1.19]
>> > > at com.sun.jersey.api.client.WebResource$Builder.post(
>> > > WebResource.java:560)
>> > > ~[jersey-client-1.19.jar:1.19]
>> > > at
>> > > org.apache.nifi.cluster.coordination.http.replication.
>> > > ThreadPoolRequestReplicator.replicateRequest(ThreadPoolReque
>> stReplicator.
>> > > java:587)
>> > > ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
>> > > at
>> > > org.apache.nifi.cluster.coordination.http.replication.
>> > > ThreadPoolRequestReplicator$NodeHttpRequest.run(
>> > > ThreadPoolRequestReplicator.java:770)
>> > > ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
>> > > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > > [na:1.8.0_112]
>> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > [na:1.8.0_112]
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor.runWorker(
>> > > ThreadPoolExecutor.java:1142)
>> > > [na:1.8.0_112]
>> > > at
>> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> > > ThreadPoolExecutor.java:617)
>> > > [na:1.8.0_112]
>> > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
>> > > Caused by: javax.ws.rs.WebApplicationException:
>> > > 

Re: Adding JVM arguments when running NiFi...

[Russell Bateman]

Joe,

I have more detail and success to report...

There is no problem in NiFi /per se/. It was not really necessary to dig 
down into the code really because there's no problem there. It's just 
how java.arg./n/is consumed that creates trouble. One must play around 
with the value of /n/ when argument ordering is crucial. In the present 
case, I was able to turn on JFR using the following (in bold) in 
/conf/bootstrap.conf/:


   #Set headless mode by default
   java.arg.14=-Djava.awt.headless=true

   *# Light up the Java Flight Recorder...**
   **java.arg.32=-XX:+UnlockCommercialFeatures**
   **java.arg.31=-XX:+FlightRecorder**
   **java.arg.42=-XX:StartFlightRecording=duration=120m,filename=recording.jfr*

   # Master key in hexadecimal format for encrypted sensitive
   configuration values
   nifi.bootstrap.sensitive.key=

The point was that -XX:+UnlockCommercialFeatures must absolutely precede 
the other two options. Because of these arguments passing through 
HashMap, some values of /n/ will fail to sort an argument before others 
and the addition of other arguments might also come along and disturb 
this order. What I did here was by trial and error. I guess NiFi code 
could man-handle /n/ in every case such that the order be respected. I'm 
guessing that my case is the only or one of the very rare times options 
are order-dependent. I wouldn't personally up-vote a JIRA to fix this.


Hoping this forum has an adequate profile out there in Googleland, this 
very discussion will be enough to help the next guy.


Russ

P.S. My trial and error went pretty fast using this command line between 
reassigning instances of /n/ for these arguments. It took me maybe 6 
tries. (Script /bounce-nifi.sh/ does little more than shut down, then 
start up NiFi, something I do a hundred times per day across at least 
two versions.)


   ~/dev/nifi/nifi-1.1.0/logs $ rm *.log ; bounce-nifi.sh 1.1.0 ; tail
   -f nifi-bootstrap.log




On 12/14/2016 07:41 PM, Joe Witt wrote:

The way you set it up initially is what I'd thought would have done
the trick.  Perhaps we're not ordering the arguments in the same
manner supplied.  Will need to look into that.

Thanks
Joe

On Wed, Dec 14, 2016 at 9:00 PM, Russell Bateman  wrote:

Of course, as many have done, I've run Java applications with JFR enabled
using these options against this very JVM (jdk1.8.0_112). So, it isn't a
problem for the JVM I'm using. I haven't finished digging down into
ProcessBuilder, or deeper, to figure out why these options are not getting
love. I'll get back at it tomorrow and report back.


On 12/14/2016 11:05 AM, Russell Bateman wrote:


I've doctored /conf/bootstrap.conf/ to contain these additional lines:

 java.arg.15=-XX:+UnlockCommercialFeatures
 java.arg.16=-XX:+FlightRecorder

java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr


In the end, NiFi's grumpy about this and won't start (from
/logs/nifi-bootstrap.log/):

2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock using
-XX:+UnlockCommercialFeatures.*
2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will
exit.
2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi
never started. Will not restart NiFi

I tried using all options as one (in case the order is disturbed, which it
was):


java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr


and then I get:

*2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Unrecognized VM option 'UnlockCommercialFeatures
-XX:+FlightRecorder
-XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will
exit.
2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi
never started. Will not restart NiFi

Here's the second command line from /logs/nifi-bootstrap.log/, which I've
wrapped for convenience in reading (see JVM version here too):

2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
Working Directory: /home/russ/dev/nifi/nifi-1.1.0
2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
Command: */home/russ/dev/jdk1.8.0_112/bin/java*
-classpath

Re: Reverse Proxy Template Import Error

[Matt Gilman]

Awesome. Here's the JIRA [1]. I should have the PR posted shortly.

Matt

[1] https://issues.apache.org/jira/browse/NIFI-3207

On Thu, Dec 15, 2016 at 12:53 PM, Richard St. John 
wrote:

> I should be able to test tonight.
>
> On Thu, Dec 15, 2016 at 12:45 PM Matt Gilman 
> wrote:
>
> > Rick,
> >
> > I think I see what the issue is however, I have don't have the
> environment
> > available for verifying it. If I posted a PR would you be in a position
> to
> > try it out?
> >
> > Matt
> >
> > On Wed, Dec 14, 2016 at 9:00 PM, Richard St. John 
> > wrote:
> >
> > > Matt,
> > >
> > > This is where it gets interesting.  The issue only arises when nginx is
> > > used as a reverse proxy for a *clustered* NiFi (within a trusted
> > > environment using http).  In fact, it even occurs when the reverse
> proxy
> > > upstream points to only one of the nifi nodes in the cluster.
> Moreover,
> > > the problem is limited to importing/uploading a template.  Deleting,
> > > downloading, and instantiating existing templates works fine.
> > >
> > > There is another strange pattern that I discovered.  I can get
> templates
> > to
> > > import if I hard-code the scheme header as follows: *proxy_set_header
> > > X-ProxyScheme http;*  However, when I hard-code the proxy scheme to
> > http, I
> > > cannot delete a template after it is uploaded.
> > >
> > > Below are several of the logs (IPs and DNS were changed for security
> > > reasons).
> > >
> > > *nginx.err*
> > > 2016/12/15 01:19:42 [warn] 29377#29377: *16 a client request body is
> > > buffered to a temporary file /var/cache/nginx/client_temp/05,
> > > client: X.X.X.X, server: nifi-rix.somedomain.com, request: "POST
> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> > > 8a86805ff8bb/templates/upload
> > > HTTP/1.1", host: "nifi-rix.somedomain.com", referrer: "
> > > https://nifi-rix.somedomain.com/nifi/;
> > >
> > > *nginx access log*
> > > [15/Dec/2016:01:22:17 +] - (X.X.X.X - rstjohn) "POST
> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> > > 8a86805ff8bb/templates/upload
> > > HTTP/1.1" 500 0 "https://nifi-rix.somedomain.com/nifi/; "Mozilla/5.0
> > > (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like
> > Gecko)
> > > Chrome/55.0.2883.95 Safari/537.36" "-"
> > >
> > >
> > > *NiFi Log: nifi-app.log*
> > > 2016-12-15 01:25:00,362 WARN [Replicate Request Thread-4]
> > > o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request
> > POST
> > > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> > > 8a86805ff8bb/templates/import
> > > to ip-X-X-X-X.ec2.internal:8080 due to {}
> > > com.sun.jersey.api.client.ClientHandlerException:
> > > javax.ws.rs.WebApplicationException: javax.xml.bind.MarshalException
> > >  - with linked exception:
> > > [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
> > > connection?]
> > > at
> > > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> > > URLConnectionClientHandler.java:155)
> > > ~[jersey-client-1.19.jar:1.19]
> > > at com.sun.jersey.api.client.Client.handle(Client.java:652)
> > > ~[jersey-client-1.19.jar:1.19]
> > > at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> > > ~[jersey-client-1.19.jar:1.19]
> > > at com.sun.jersey.api.client.WebResource.access$200(
> WebResource.java:74)
> > > ~[jersey-client-1.19.jar:1.19]
> > > at com.sun.jersey.api.client.WebResource$Builder.post(
> > > WebResource.java:560)
> > > ~[jersey-client-1.19.jar:1.19]
> > > at
> > > org.apache.nifi.cluster.coordination.http.replication.
> > > ThreadPoolRequestReplicator.replicateRequest(
> ThreadPoolRequestReplicator.
> > > java:587)
> > > ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
> > > at
> > > org.apache.nifi.cluster.coordination.http.replication.
> > > ThreadPoolRequestReplicator$NodeHttpRequest.run(
> > > ThreadPoolRequestReplicator.java:770)
> > > ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
> > > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > [na:1.8.0_112]
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > [na:1.8.0_112]
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > ThreadPoolExecutor.java:1142)
> > > [na:1.8.0_112]
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > ThreadPoolExecutor.java:617)
> > > [na:1.8.0_112]
> > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
> > > Caused by: javax.ws.rs.WebApplicationException:
> > > javax.xml.bind.MarshalException
> > >  - with linked exception:
> > > [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
> > > connection?]
> > > at
> > > com.sun.jersey.core.provider.jaxb.AbstractRootElementProvider.writeTo(
> > > AbstractRootElementProvider.java:159)
> > > ~[jersey-core-1.19.jar:1.19]
> > > at
> > > com.sun.jersey.api.client.RequestWriter.writeRequestEntity(
> > > 

Re: Reverse Proxy Template Import Error

[Richard St. John]

I should be able to test tonight.

On Thu, Dec 15, 2016 at 12:45 PM Matt Gilman 
wrote:

> Rick,
>
> I think I see what the issue is however, I have don't have the environment
> available for verifying it. If I posted a PR would you be in a position to
> try it out?
>
> Matt
>
> On Wed, Dec 14, 2016 at 9:00 PM, Richard St. John 
> wrote:
>
> > Matt,
> >
> > This is where it gets interesting.  The issue only arises when nginx is
> > used as a reverse proxy for a *clustered* NiFi (within a trusted
> > environment using http).  In fact, it even occurs when the reverse proxy
> > upstream points to only one of the nifi nodes in the cluster.  Moreover,
> > the problem is limited to importing/uploading a template.  Deleting,
> > downloading, and instantiating existing templates works fine.
> >
> > There is another strange pattern that I discovered.  I can get templates
> to
> > import if I hard-code the scheme header as follows: *proxy_set_header
> > X-ProxyScheme http;*  However, when I hard-code the proxy scheme to
> http, I
> > cannot delete a template after it is uploaded.
> >
> > Below are several of the logs (IPs and DNS were changed for security
> > reasons).
> >
> > *nginx.err*
> > 2016/12/15 01:19:42 [warn] 29377#29377: *16 a client request body is
> > buffered to a temporary file /var/cache/nginx/client_temp/05,
> > client: X.X.X.X, server: nifi-rix.somedomain.com, request: "POST
> > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> > 8a86805ff8bb/templates/upload
> > HTTP/1.1", host: "nifi-rix.somedomain.com", referrer: "
> > https://nifi-rix.somedomain.com/nifi/;
> >
> > *nginx access log*
> > [15/Dec/2016:01:22:17 +] - (X.X.X.X - rstjohn) "POST
> > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> > 8a86805ff8bb/templates/upload
> > HTTP/1.1" 500 0 "https://nifi-rix.somedomain.com/nifi/; "Mozilla/5.0
> > (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like
> Gecko)
> > Chrome/55.0.2883.95 Safari/537.36" "-"
> >
> >
> > *NiFi Log: nifi-app.log*
> > 2016-12-15 01:25:00,362 WARN [Replicate Request Thread-4]
> > o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request
> POST
> > /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> > 8a86805ff8bb/templates/import
> > to ip-X-X-X-X.ec2.internal:8080 due to {}
> > com.sun.jersey.api.client.ClientHandlerException:
> > javax.ws.rs.WebApplicationException: javax.xml.bind.MarshalException
> >  - with linked exception:
> > [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
> > connection?]
> > at
> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> > URLConnectionClientHandler.java:155)
> > ~[jersey-client-1.19.jar:1.19]
> > at com.sun.jersey.api.client.Client.handle(Client.java:652)
> > ~[jersey-client-1.19.jar:1.19]
> > at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> > ~[jersey-client-1.19.jar:1.19]
> > at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> > ~[jersey-client-1.19.jar:1.19]
> > at com.sun.jersey.api.client.WebResource$Builder.post(
> > WebResource.java:560)
> > ~[jersey-client-1.19.jar:1.19]
> > at
> > org.apache.nifi.cluster.coordination.http.replication.
> > ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.
> > java:587)
> > ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
> > at
> > org.apache.nifi.cluster.coordination.http.replication.
> > ThreadPoolRequestReplicator$NodeHttpRequest.run(
> > ThreadPoolRequestReplicator.java:770)
> > ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > [na:1.8.0_112]
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_112]
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> > [na:1.8.0_112]
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> > [na:1.8.0_112]
> > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
> > Caused by: javax.ws.rs.WebApplicationException:
> > javax.xml.bind.MarshalException
> >  - with linked exception:
> > [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
> > connection?]
> > at
> > com.sun.jersey.core.provider.jaxb.AbstractRootElementProvider.writeTo(
> > AbstractRootElementProvider.java:159)
> > ~[jersey-core-1.19.jar:1.19]
> > at
> > com.sun.jersey.api.client.RequestWriter.writeRequestEntity(
> > RequestWriter.java:300)
> > ~[jersey-client-1.19.jar:1.19]
> > at
> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(
> > URLConnectionClientHandler.java:217)
> > ~[jersey-client-1.19.jar:1.19]
> > at
> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> > URLConnectionClientHandler.java:153)
> > ~[jersey-client-1.19.jar:1.19]
> > ... 11 common frames omitted
> > Caused by: 

Re: Reverse Proxy Template Import Error

[Matt Gilman]

Rick,

I think I see what the issue is however, I have don't have the environment
available for verifying it. If I posted a PR would you be in a position to
try it out?

Matt

On Wed, Dec 14, 2016 at 9:00 PM, Richard St. John 
wrote:

> Matt,
>
> This is where it gets interesting.  The issue only arises when nginx is
> used as a reverse proxy for a *clustered* NiFi (within a trusted
> environment using http).  In fact, it even occurs when the reverse proxy
> upstream points to only one of the nifi nodes in the cluster.  Moreover,
> the problem is limited to importing/uploading a template.  Deleting,
> downloading, and instantiating existing templates works fine.
>
> There is another strange pattern that I discovered.  I can get templates to
> import if I hard-code the scheme header as follows: *proxy_set_header
> X-ProxyScheme http;*  However, when I hard-code the proxy scheme to http, I
> cannot delete a template after it is uploaded.
>
> Below are several of the logs (IPs and DNS were changed for security
> reasons).
>
> *nginx.err*
> 2016/12/15 01:19:42 [warn] 29377#29377: *16 a client request body is
> buffered to a temporary file /var/cache/nginx/client_temp/05,
> client: X.X.X.X, server: nifi-rix.somedomain.com, request: "POST
> /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> 8a86805ff8bb/templates/upload
> HTTP/1.1", host: "nifi-rix.somedomain.com", referrer: "
> https://nifi-rix.somedomain.com/nifi/;
>
> *nginx access log*
> [15/Dec/2016:01:22:17 +] - (X.X.X.X - rstjohn) "POST
> /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> 8a86805ff8bb/templates/upload
> HTTP/1.1" 500 0 "https://nifi-rix.somedomain.com/nifi/; "Mozilla/5.0
> (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/55.0.2883.95 Safari/537.36" "-"
>
>
> *NiFi Log: nifi-app.log*
> 2016-12-15 01:25:00,362 WARN [Replicate Request Thread-4]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request POST
> /nifi-api/process-groups/fe2e65ad-0158-1000-5fa5-
> 8a86805ff8bb/templates/import
> to ip-X-X-X-X.ec2.internal:8080 due to {}
> com.sun.jersey.api.client.ClientHandlerException:
> javax.ws.rs.WebApplicationException: javax.xml.bind.MarshalException
>  - with linked exception:
> [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
> connection?]
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> URLConnectionClientHandler.java:155)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource$Builder.post(
> WebResource.java:560)
> ~[jersey-client-1.19.jar:1.19]
> at
> org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.
> java:587)
> ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
> at
> org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator$NodeHttpRequest.run(
> ThreadPoolRequestReplicator.java:770)
> ~[nifi-framework-cluster-1.1.0-A1-SNAPSHOT.jar:1.1.0-A1-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_112]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> [na:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> [na:1.8.0_112]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
> Caused by: javax.ws.rs.WebApplicationException:
> javax.xml.bind.MarshalException
>  - with linked exception:
> [javax.net.ssl.SSLException: Unrecognized SSL message, plaintext
> connection?]
> at
> com.sun.jersey.core.provider.jaxb.AbstractRootElementProvider.writeTo(
> AbstractRootElementProvider.java:159)
> ~[jersey-core-1.19.jar:1.19]
> at
> com.sun.jersey.api.client.RequestWriter.writeRequestEntity(
> RequestWriter.java:300)
> ~[jersey-client-1.19.jar:1.19]
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(
> URLConnectionClientHandler.java:217)
> ~[jersey-client-1.19.jar:1.19]
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> URLConnectionClientHandler.java:153)
> ~[jersey-client-1.19.jar:1.19]
> ... 11 common frames omitted
> Caused by: javax.xml.bind.MarshalException: null
> at
> com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:325)
> ~[jaxb-impl-2.2.3-1.jar:2.2.3]
> at
> com.sun.xml.bind.v2.runtime.MarshallerImpl.marshal(
> MarshallerImpl.java:249)
> ~[jaxb-impl-2.2.3-1.jar:2.2.3]
> at
> javax.xml.bind.helpers.AbstractMarshallerImpl.marshal(
> AbstractMarshallerImpl.java:95)
> 

Re: Content Repository Cleanup

[Alan Jackoway]

I'm going to move future conversation to that jira, but I've tried a few
things now and all of them seem to strand files in the content repository.
My question, reworded is:

How do you clean up new flow files that a processor created when the
session rolls back / penalizes?

On Thu, Dec 15, 2016 at 11:24 AM, Alan Jackoway  wrote:

> Update: session.remove(newFiles) does not work. I filed
> https://issues.apache.org/jira/browse/NIFI-3205
>
> On Thu, Dec 15, 2016 at 11:05 AM, Alan Jackoway 
> wrote:
>
>> I am getting the successfully checkpointed message.
>>
>> I think I figured this out. Now we have to decide whether it's an issue
>> in nifi or an issue in our code.
>>
>> This flow has a process that takes large zip files, unzips them, and does
>> some processing of the files. I noticed that the disk space thing seems to
>> go up fastest when there is a large file that is failing in the middle of
>> one of these steps. I then suspected that something about the way we were
>> creating new flow files out of the zips was the problem.
>>
>> I simulated what we were doing with the following processor in a new nifi
>> 1.1. The processor takes an input file, copies it 5 times (to simulate
>> unzip / process), then throws a runtime exception. I then wired a
>> GenerateFlowFile of 100KB to it. I noticed the following characteristics:
>> * Each time it ran, the size of the content repository went up exactly
>> 500KB.
>> * When I restarted the nifi, I got the messages about unknown files in
>> the FileSystemRepository.
>>
>> So basically what this boils down to is: who is responsible to remove
>> files from a session when a failure occurs? Should we be doing that (I will
>> test next that calling session.remove before the error fixes the problem)
>> or should the session keep track of new flow files that it created. We
>> assumed the session would do so because the session yells at us if we fail
>> to give a transport relationship for one of the files.
>>
>> Thanks for all the help with this. I think we are closing in on the point
>> where I have either a fix or a bug filed or both.
>>
>> Test processor I used:
>> // Copyright 2016 (c) Cloudera
>> package com.cloudera.edh.nifi.processors.bundles;
>>
>> import com.google.common.collect.Lists;
>>
>> import java.io.IOException;
>> import java.io.InputStream;
>> import java.io.OutputStream;
>> import java.util.List;
>>
>> import org.apache.nifi.annotation.behavior.InputRequirement;
>> import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
>> import org.apache.nifi.flowfile.FlowFile;
>> import org.apache.nifi.processor.AbstractProcessor;
>> import org.apache.nifi.processor.ProcessContext;
>> import org.apache.nifi.processor.ProcessSession;
>> import org.apache.nifi.processor.exception.ProcessException;
>> import org.apache.nifi.processor.io.InputStreamCallback;
>> import org.apache.nifi.processor.io.OutputStreamCallback;
>> import org.apache.nifi.stream.io.StreamUtils;
>>
>> /**
>>  * Makes 5 copies of an incoming file, then fails and rolls back.
>>  */
>> @InputRequirement(value = Requirement.INPUT_REQUIRED)
>> public class CopyAndFail extends AbstractProcessor {
>>   @Override
>>   public void onTrigger(ProcessContext context, ProcessSession session)
>>   throws ProcessException {
>> FlowFile inputFile = session.get();
>> if (inputFile == null) {
>>   context.yield();
>>   return;
>> }
>> final List newFiles = Lists.newArrayList();
>>
>> // Copy the file 5 times (simulates us opening a zip file and
>> unpacking its contents)
>> for (int i = 0; i < 5; i++) {
>>   session.read(inputFile, new InputStreamCallback() {
>> @Override
>> public void process(InputStream inputStream) throws IOException {
>>   FlowFile ff = session.create(inputFile);
>>   ff = session.write(ff, new OutputStreamCallback() {
>> @Override
>> public void process(final OutputStream out) throws
>> IOException {
>>   StreamUtils.copy(inputStream, out);
>> }
>>   });
>>   newFiles.add(ff);
>> }
>>   });
>> }
>>
>> // THIS IS WHERE I WILL PUT session.remove TO VERIFY THAT WORKS
>>
>> // Simulate an error handling some file in the zip after unpacking
>> the rest
>> throw new RuntimeException();
>>   }
>> }
>>
>>
>> On Wed, Dec 14, 2016 at 9:23 PM, Mark Payne  wrote:
>>
>>> I'd be very curious to see if changing the limits addresses the issue.
>>> The OOME can certainly be an issue, as well. Once that gets thrown
>>> anywhere
>>> in the JVM, it's hard to vouch for the stability of the JVM at all.
>>>
>>> Seeing the claimant count drop to 0 then back up to 1, 2, and down to 1,
>>> 0 again
>>> is pretty common. The fact that you didn't see it marked as destructible
>>> is interesting.
>>> Around that same time, are you seeing log messages indicating that the
>>> FlowFile 

Re: Content Repository Cleanup

[Alan Jackoway]

Update: session.remove(newFiles) does not work. I filed
https://issues.apache.org/jira/browse/NIFI-3205

On Thu, Dec 15, 2016 at 11:05 AM, Alan Jackoway  wrote:

> I am getting the successfully checkpointed message.
>
> I think I figured this out. Now we have to decide whether it's an issue in
> nifi or an issue in our code.
>
> This flow has a process that takes large zip files, unzips them, and does
> some processing of the files. I noticed that the disk space thing seems to
> go up fastest when there is a large file that is failing in the middle of
> one of these steps. I then suspected that something about the way we were
> creating new flow files out of the zips was the problem.
>
> I simulated what we were doing with the following processor in a new nifi
> 1.1. The processor takes an input file, copies it 5 times (to simulate
> unzip / process), then throws a runtime exception. I then wired a
> GenerateFlowFile of 100KB to it. I noticed the following characteristics:
> * Each time it ran, the size of the content repository went up exactly
> 500KB.
> * When I restarted the nifi, I got the messages about unknown files in the
> FileSystemRepository.
>
> So basically what this boils down to is: who is responsible to remove
> files from a session when a failure occurs? Should we be doing that (I will
> test next that calling session.remove before the error fixes the problem)
> or should the session keep track of new flow files that it created. We
> assumed the session would do so because the session yells at us if we fail
> to give a transport relationship for one of the files.
>
> Thanks for all the help with this. I think we are closing in on the point
> where I have either a fix or a bug filed or both.
>
> Test processor I used:
> // Copyright 2016 (c) Cloudera
> package com.cloudera.edh.nifi.processors.bundles;
>
> import com.google.common.collect.Lists;
>
> import java.io.IOException;
> import java.io.InputStream;
> import java.io.OutputStream;
> import java.util.List;
>
> import org.apache.nifi.annotation.behavior.InputRequirement;
> import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
> import org.apache.nifi.flowfile.FlowFile;
> import org.apache.nifi.processor.AbstractProcessor;
> import org.apache.nifi.processor.ProcessContext;
> import org.apache.nifi.processor.ProcessSession;
> import org.apache.nifi.processor.exception.ProcessException;
> import org.apache.nifi.processor.io.InputStreamCallback;
> import org.apache.nifi.processor.io.OutputStreamCallback;
> import org.apache.nifi.stream.io.StreamUtils;
>
> /**
>  * Makes 5 copies of an incoming file, then fails and rolls back.
>  */
> @InputRequirement(value = Requirement.INPUT_REQUIRED)
> public class CopyAndFail extends AbstractProcessor {
>   @Override
>   public void onTrigger(ProcessContext context, ProcessSession session)
>   throws ProcessException {
> FlowFile inputFile = session.get();
> if (inputFile == null) {
>   context.yield();
>   return;
> }
> final List newFiles = Lists.newArrayList();
>
> // Copy the file 5 times (simulates us opening a zip file and
> unpacking its contents)
> for (int i = 0; i < 5; i++) {
>   session.read(inputFile, new InputStreamCallback() {
> @Override
> public void process(InputStream inputStream) throws IOException {
>   FlowFile ff = session.create(inputFile);
>   ff = session.write(ff, new OutputStreamCallback() {
> @Override
> public void process(final OutputStream out) throws IOException
> {
>   StreamUtils.copy(inputStream, out);
> }
>   });
>   newFiles.add(ff);
> }
>   });
> }
>
> // THIS IS WHERE I WILL PUT session.remove TO VERIFY THAT WORKS
>
> // Simulate an error handling some file in the zip after unpacking the
> rest
> throw new RuntimeException();
>   }
> }
>
>
> On Wed, Dec 14, 2016 at 9:23 PM, Mark Payne  wrote:
>
>> I'd be very curious to see if changing the limits addresses the issue.
>> The OOME can certainly be an issue, as well. Once that gets thrown
>> anywhere
>> in the JVM, it's hard to vouch for the stability of the JVM at all.
>>
>> Seeing the claimant count drop to 0 then back up to 1, 2, and down to 1,
>> 0 again
>> is pretty common. The fact that you didn't see it marked as destructible
>> is interesting.
>> Around that same time, are you seeing log messages indicating that the
>> FlowFile repo
>> is checkpointing? Would have the words "Successfully checkpointed
>> FlowFile Repository"
>> That should happen every 2 minutes, approximately.
>>
>>
>> On Dec 14, 2016, at 8:56 PM, Alan Jackoway > n...@cloudera.com>> wrote:
>>
>> I agree the limits sound low and will address that tomorrow.
>>
>> I'm not seeing FileNotFound or NoSuchFile.
>>
>> Here's an example file:
>> grep 1481763927251 logs/nifi-app.log
>> 2016-12-14 

Re: Content Repository Cleanup

[Alan Jackoway]

I am getting the successfully checkpointed message.

I think I figured this out. Now we have to decide whether it's an issue in
nifi or an issue in our code.

This flow has a process that takes large zip files, unzips them, and does
some processing of the files. I noticed that the disk space thing seems to
go up fastest when there is a large file that is failing in the middle of
one of these steps. I then suspected that something about the way we were
creating new flow files out of the zips was the problem.

I simulated what we were doing with the following processor in a new nifi
1.1. The processor takes an input file, copies it 5 times (to simulate
unzip / process), then throws a runtime exception. I then wired a
GenerateFlowFile of 100KB to it. I noticed the following characteristics:
* Each time it ran, the size of the content repository went up exactly
500KB.
* When I restarted the nifi, I got the messages about unknown files in the
FileSystemRepository.

So basically what this boils down to is: who is responsible to remove files
from a session when a failure occurs? Should we be doing that (I will test
next that calling session.remove before the error fixes the problem) or
should the session keep track of new flow files that it created. We assumed
the session would do so because the session yells at us if we fail to give
a transport relationship for one of the files.

Thanks for all the help with this. I think we are closing in on the point
where I have either a fix or a bug filed or both.

Test processor I used:
// Copyright 2016 (c) Cloudera
package com.cloudera.edh.nifi.processors.bundles;

import com.google.common.collect.Lists;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.List;

import org.apache.nifi.annotation.behavior.InputRequirement;
import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.io.InputStreamCallback;
import org.apache.nifi.processor.io.OutputStreamCallback;
import org.apache.nifi.stream.io.StreamUtils;

/**
 * Makes 5 copies of an incoming file, then fails and rolls back.
 */
@InputRequirement(value = Requirement.INPUT_REQUIRED)
public class CopyAndFail extends AbstractProcessor {
  @Override
  public void onTrigger(ProcessContext context, ProcessSession session)
  throws ProcessException {
FlowFile inputFile = session.get();
if (inputFile == null) {
  context.yield();
  return;
}
final List newFiles = Lists.newArrayList();

// Copy the file 5 times (simulates us opening a zip file and unpacking
its contents)
for (int i = 0; i < 5; i++) {
  session.read(inputFile, new InputStreamCallback() {
@Override
public void process(InputStream inputStream) throws IOException {
  FlowFile ff = session.create(inputFile);
  ff = session.write(ff, new OutputStreamCallback() {
@Override
public void process(final OutputStream out) throws IOException {
  StreamUtils.copy(inputStream, out);
}
  });
  newFiles.add(ff);
}
  });
}

// THIS IS WHERE I WILL PUT session.remove TO VERIFY THAT WORKS

// Simulate an error handling some file in the zip after unpacking the
rest
throw new RuntimeException();
  }
}


On Wed, Dec 14, 2016 at 9:23 PM, Mark Payne  wrote:

> I'd be very curious to see if changing the limits addresses the issue.
> The OOME can certainly be an issue, as well. Once that gets thrown anywhere
> in the JVM, it's hard to vouch for the stability of the JVM at all.
>
> Seeing the claimant count drop to 0 then back up to 1, 2, and down to 1, 0
> again
> is pretty common. The fact that you didn't see it marked as destructible
> is interesting.
> Around that same time, are you seeing log messages indicating that the
> FlowFile repo
> is checkpointing? Would have the words "Successfully checkpointed FlowFile
> Repository"
> That should happen every 2 minutes, approximately.
>
>
> On Dec 14, 2016, at 8:56 PM, Alan Jackoway  n...@cloudera.com>> wrote:
>
> I agree the limits sound low and will address that tomorrow.
>
> I'm not seeing FileNotFound or NoSuchFile.
>
> Here's an example file:
> grep 1481763927251 logs/nifi-app.log
> 2016-12-14 17:05:27,277 DEBUG [Timer-Driven Process Thread-36]
> o.a.n.c.r.c.StandardResourceClaimManager Incrementing claimant count for
> StandardResourceClaim[id=1481763927251-1, container=default, section=1]
> to 1
> 2016-12-14 17:05:27,357 DEBUG [Timer-Driven Process Thread-2]
> o.a.n.c.r.c.StandardResourceClaimManager Incrementing claimant count for
> StandardResourceClaim[id=1481763927251-1, 

Re: NIFI 1.1.0 build error

[Andre]

Martian macnificent is a dependency via ParCEFone (ParseCEF processor).

Will have a look

On 16 Dec 2016 01:56, "Oleg Zhurakousky" 
wrote:

> Actually, i’ll take that back. I believe I never did the ‘purge’ step,
> just ‘clean install’, and that is where I thought you had the
> ‘martiansoftware’ failure.
> Now when I did try the ‘purge’ step I do see the same failure in
> nifi-standard-processors bundle.
>
> Anyway, I am not sure why it is complaining about ‘martiansoftware’ on the
> purge step, but ‘mvn clean install’ works fine. I guess if you really want
> to purge everything you can do it manually for now. Meanwhile you can file
> a bug in JIRA and we’ll take a look.
>
> Cheers
> Oleg
>
> > On Dec 15, 2016, at 8:12 AM, Joe Witt  wrote:
> >
> > Alessio
> >
> > You should not need any sort of special settings.xml file for this to
> > work.  Just grabbing a default maven install and following our dev
> > guide should be sufficient.  If you do have one that is altered it
> > might be changing important details.
> >
> > Thanks
> > Joe
> >
> > On Thu, Dec 15, 2016 at 8:10 AM, Alessio Palma
> >  wrote:
> >> Can I have your settings.xml which is located in ~/.m2 ?
> >>
> >> 
> >> From: Oleg Zhurakousky 
> >> Sent: Thursday, December 15, 2016 1:14:42 PM
> >> To: dev@nifi.apache.org
> >> Subject: Re: NIFI 1.1.0 build error
> >>
> >> Alessio
> >>
> >> Any chance you have some network connectivity issues when you attempt
> too build or may be a proxy that restricts certain sites?
> >> This error is indeed strange and I can’t recall having it and I’ve
> tried to do the same (as you describe) on my mac and all builds fine.
> >>
> >> Also, just to compare the environments:
> >> olegs-mac:~ foo$ mvn --version
> >> Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06;
> 2015-04-22T07:57:37-04:00)
> >> Maven home: /usr/local/Cellar/maven/3.3.3/libexec
> >> Java version: 1.8.0_112, vendor: Oracle Corporation
> >> Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_
> 112.jdk/Contents/Home/jre
> >> Default locale: en_US, platform encoding: UTF-8
> >> OS name: "mac os x", version: "10.11.2", arch: "x86_64", family: "mac"
> >>
> >> Cheers
> >> Oleg
> >>
> >>> On Dec 15, 2016, at 6:49 AM, Alessio Palma <
> alessio.pa...@docomodigital.com> wrote:
> >>>
> >>> Hello all,
> >>>
> >>> I downloaded the source code from
> >>>
> >>>
> >>> http://apache.panu.it/nifi/1.1.0/nifi-1.1.0-source-release.zip
> >>>
> >>>
> >>> then unpacked the archivie and executed:
> >>>
> >>>
> >>> mvn dependency:purge-local-repository
> >>>
> >>> mvn clean install
> >>>
> >>>
> >>> and I got this:
> >>>
> >>>
> >>> [ERROR] Failed to execute goal org.apache.maven.plugins:
> maven-remote-resources-plugin:1.5:process (default) on project
> nifi-standard-processors: Error resolving project artifact: Could not
> transfer artifact com.martiansoftware:macnificent:pom:0.2.0 from/to
> org.apche.nifi (http://https://mvnrepository.com/artifact/org.apache.nifi):
> https: unknown error for project com.martiansoftware:macnificent:jar:0.2.0:
> Unknown host https: unknown error ->
> >>>
> >>>
> >>>
> >>> any pointers how to fix this ?
> >>
> >
>
>

Re: NIFI 1.1.0 build error

[Alessio Palma]

Hello all,
ASAP I'll try again and will keep you update.
Meanwhile thank you all for the support.


From: Oleg Zhurakousky 
Sent: Thursday, December 15, 2016 3:56:36 PM
To: dev@nifi.apache.org
Subject: Re: NIFI 1.1.0 build error

Actually, i’ll take that back. I believe I never did the ‘purge’ step, just 
‘clean install’, and that is where I thought you had the ‘martiansoftware’ 
failure.
Now when I did try the ‘purge’ step I do see the same failure in 
nifi-standard-processors bundle.

Anyway, I am not sure why it is complaining about ‘martiansoftware’ on the 
purge step, but ‘mvn clean install’ works fine. I guess if you really want to 
purge everything you can do it manually for now. Meanwhile you can file a bug 
in JIRA and we’ll take a look.

Cheers
Oleg

> On Dec 15, 2016, at 8:12 AM, Joe Witt  wrote:
>
> Alessio
>
> You should not need any sort of special settings.xml file for this to
> work.  Just grabbing a default maven install and following our dev
> guide should be sufficient.  If you do have one that is altered it
> might be changing important details.
>
> Thanks
> Joe
>
> On Thu, Dec 15, 2016 at 8:10 AM, Alessio Palma
>  wrote:
>> Can I have your settings.xml which is located in ~/.m2 ?
>>
>> 
>> From: Oleg Zhurakousky 
>> Sent: Thursday, December 15, 2016 1:14:42 PM
>> To: dev@nifi.apache.org
>> Subject: Re: NIFI 1.1.0 build error
>>
>> Alessio
>>
>> Any chance you have some network connectivity issues when you attempt too 
>> build or may be a proxy that restricts certain sites?
>> This error is indeed strange and I can’t recall having it and I’ve tried to 
>> do the same (as you describe) on my mac and all builds fine.
>>
>> Also, just to compare the environments:
>> olegs-mac:~ foo$ mvn --version
>> Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 
>> 2015-04-22T07:57:37-04:00)
>> Maven home: /usr/local/Cellar/maven/3.3.3/libexec
>> Java version: 1.8.0_112, vendor: Oracle Corporation
>> Java home: 
>> /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
>> Default locale: en_US, platform encoding: UTF-8
>> OS name: "mac os x", version: "10.11.2", arch: "x86_64", family: "mac"
>>
>> Cheers
>> Oleg
>>
>>> On Dec 15, 2016, at 6:49 AM, Alessio Palma 
>>>  wrote:
>>>
>>> Hello all,
>>>
>>> I downloaded the source code from
>>>
>>>
>>> http://apache.panu.it/nifi/1.1.0/nifi-1.1.0-source-release.zip
>>>
>>>
>>> then unpacked the archivie and executed:
>>>
>>>
>>> mvn dependency:purge-local-repository
>>>
>>> mvn clean install
>>>
>>>
>>> and I got this:
>>>
>>>
>>> [ERROR] Failed to execute goal 
>>> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process 
>>> (default) on project nifi-standard-processors: Error resolving project 
>>> artifact: Could not transfer artifact 
>>> com.martiansoftware:macnificent:pom:0.2.0 from/to org.apche.nifi 
>>> (http://https://mvnrepository.com/artifact/org.apache.nifi): https: unknown 
>>> error for project com.martiansoftware:macnificent:jar:0.2.0: Unknown host 
>>> https: unknown error ->
>>>
>>>
>>>
>>> any pointers how to fix this ?
>>
>

Re: NIFI 1.1.0 build error

[Joe Witt]

Alessio

You should not need any sort of special settings.xml file for this to
work.  Just grabbing a default maven install and following our dev
guide should be sufficient.  If you do have one that is altered it
might be changing important details.

Thanks
Joe

On Thu, Dec 15, 2016 at 8:10 AM, Alessio Palma
 wrote:
> Can I have your settings.xml which is located in ~/.m2 ?
>
> 
> From: Oleg Zhurakousky 
> Sent: Thursday, December 15, 2016 1:14:42 PM
> To: dev@nifi.apache.org
> Subject: Re: NIFI 1.1.0 build error
>
> Alessio
>
> Any chance you have some network connectivity issues when you attempt too 
> build or may be a proxy that restricts certain sites?
> This error is indeed strange and I can’t recall having it and I’ve tried to 
> do the same (as you describe) on my mac and all builds fine.
>
> Also, just to compare the environments:
> olegs-mac:~ foo$ mvn --version
> Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 
> 2015-04-22T07:57:37-04:00)
> Maven home: /usr/local/Cellar/maven/3.3.3/libexec
> Java version: 1.8.0_112, vendor: Oracle Corporation
> Java home: 
> /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.11.2", arch: "x86_64", family: "mac"
>
> Cheers
> Oleg
>
>> On Dec 15, 2016, at 6:49 AM, Alessio Palma  
>> wrote:
>>
>> Hello all,
>>
>> I downloaded the source code from
>>
>>
>> http://apache.panu.it/nifi/1.1.0/nifi-1.1.0-source-release.zip
>>
>>
>> then unpacked the archivie and executed:
>>
>>
>> mvn dependency:purge-local-repository
>>
>> mvn clean install
>>
>>
>> and I got this:
>>
>>
>> [ERROR] Failed to execute goal 
>> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) 
>> on project nifi-standard-processors: Error resolving project artifact: Could 
>> not transfer artifact com.martiansoftware:macnificent:pom:0.2.0 from/to 
>> org.apche.nifi (http://https://mvnrepository.com/artifact/org.apache.nifi): 
>> https: unknown error for project com.martiansoftware:macnificent:jar:0.2.0: 
>> Unknown host https: unknown error ->
>>
>>
>>
>> any pointers how to fix this ?
>

Re: NIFI 1.1.0 build error

[Alessio Palma]

Can I have your settings.xml which is located in ~/.m2 ?


From: Oleg Zhurakousky 
Sent: Thursday, December 15, 2016 1:14:42 PM
To: dev@nifi.apache.org
Subject: Re: NIFI 1.1.0 build error

Alessio

Any chance you have some network connectivity issues when you attempt too build 
or may be a proxy that restricts certain sites?
This error is indeed strange and I can’t recall having it and I’ve tried to do 
the same (as you describe) on my mac and all builds fine.

Also, just to compare the environments:
olegs-mac:~ foo$ mvn --version
Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 
2015-04-22T07:57:37-04:00)
Maven home: /usr/local/Cellar/maven/3.3.3/libexec
Java version: 1.8.0_112, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.2", arch: "x86_64", family: "mac"

Cheers
Oleg

> On Dec 15, 2016, at 6:49 AM, Alessio Palma  
> wrote:
>
> Hello all,
>
> I downloaded the source code from
>
>
> http://apache.panu.it/nifi/1.1.0/nifi-1.1.0-source-release.zip
>
>
> then unpacked the archivie and executed:
>
>
> mvn dependency:purge-local-repository
>
> mvn clean install
>
>
> and I got this:
>
>
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) 
> on project nifi-standard-processors: Error resolving project artifact: Could 
> not transfer artifact com.martiansoftware:macnificent:pom:0.2.0 from/to 
> org.apche.nifi (http://https://mvnrepository.com/artifact/org.apache.nifi): 
> https: unknown error for project com.martiansoftware:macnificent:jar:0.2.0: 
> Unknown host https: unknown error ->
>
>
>
> any pointers how to fix this ?

Re: NIFI 1.1.0 build error

[Oleg Zhurakousky]

Alessio

Any chance you have some network connectivity issues when you attempt too build 
or may be a proxy that restricts certain sites?
This error is indeed strange and I can’t recall having it and I’ve tried to do 
the same (as you describe) on my mac and all builds fine. 

Also, just to compare the environments:
olegs-mac:~ foo$ mvn --version
Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 
2015-04-22T07:57:37-04:00)
Maven home: /usr/local/Cellar/maven/3.3.3/libexec
Java version: 1.8.0_112, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.2", arch: "x86_64", family: "mac"

Cheers
Oleg

> On Dec 15, 2016, at 6:49 AM, Alessio Palma  
> wrote:
> 
> Hello all,
> 
> I downloaded the source code from
> 
> 
> http://apache.panu.it/nifi/1.1.0/nifi-1.1.0-source-release.zip
> 
> 
> then unpacked the archivie and executed:
> 
> 
> mvn dependency:purge-local-repository
> 
> mvn clean install
> 
> 
> and I got this:
> 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) 
> on project nifi-standard-processors: Error resolving project artifact: Could 
> not transfer artifact com.martiansoftware:macnificent:pom:0.2.0 from/to 
> org.apche.nifi (http://https://mvnrepository.com/artifact/org.apache.nifi): 
> https: unknown error for project com.martiansoftware:macnificent:jar:0.2.0: 
> Unknown host https: unknown error ->
> 
> 
> 
> any pointers how to fix this ?

Re: SOAP Service through InvokeHTTP

[Hitendra Kashyap]

Hi Andrew, 

I am unable to use the NiFi-soap with the latest version of nifi (1.1.0).
after build and deploying nar file to lib when I restart nifi it doesn't
start. bootstrap log tail has following: 

2016-12-15 01:25:05,216 INFO [main] org.apache.nifi.bootstrap.Command
Working Directory: /appl/idc/nifi-1.1.0 
2016-12-15 01:25:05,217 INFO [main] org.apache.nifi.bootstrap.Command
Command: java -classpath
/appl/idc/nifi-1.1.0/./conf:/appl/idc/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar
-Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m
-Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true -XX:+UseG1GC
-Djava.protocol.handler.pkgs=sun.net.www.protocol
-Dnifi.properties.file.path=/appl/idc/nifi-1.1.0/./conf/nifi.properties
-Dnifi.bootstrap.listen.port=55731 -Dapp=NiFi
-Dorg.apache.nifi.bootstrap.config.log.dir=/appl/idc/nifi-1.1.0/logs
org.apache.nifi.NiFi 
2016-12-15 01:25:06,323 INFO [NiFi Bootstrap Command Listener]
org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
Bootstrap requests on port 55732 
2016-12-15 01:26:07,036 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected. 
2016-12-15 01:26:07,072 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected. 
2016-12-15 01:26:07,104 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected. 
2016-12-15 01:26:07,126 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected. 
2016-12-15 01:26:07,456 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected. 
2016-12-15 01:26:07,646 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/SOAP-Service-through-InvokeHTTP-tp13129p14238.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: SOAP Service through InvokeHTTP

[Hitendra Kashyap]

Hi Jeff,

I am unable to use the NiFi-soap with the latest version of nifi (1.1.0).
after build and deploying nar file to lib when I restart nifi it doesn't
start. bootstrap log tail has following:

2016-12-15 01:25:05,216 INFO [main] org.apache.nifi.bootstrap.Command
Working Directory: /appl/idc/nifi-1.1.0
2016-12-15 01:25:05,217 INFO [main] org.apache.nifi.bootstrap.Command
Command: java -classpath
/appl/idc/nifi-1.1.0/./conf:/appl/idc/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar
-Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m
-Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true -XX:+UseG1GC
-Djava.protocol.handler.pkgs=sun.net.www.protocol
-Dnifi.properties.file.path=/appl/idc/nifi-1.1.0/./conf/nifi.properties
-Dnifi.bootstrap.listen.port=55731 -Dapp=NiFi
-Dorg.apache.nifi.bootstrap.config.log.dir=/appl/idc/nifi-1.1.0/logs
org.apache.nifi.NiFi
2016-12-15 01:25:06,323 INFO [NiFi Bootstrap Command Listener]
org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
Bootstrap requests on port 55732
2016-12-15 01:26:07,036 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,072 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,104 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,126 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,456 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,646 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/SOAP-Service-through-InvokeHTTP-tp13129p14237.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

NIFI 1.1.0 build error

[Alessio Palma]

Hello all,

I downloaded the source code from


http://apache.panu.it/nifi/1.1.0/nifi-1.1.0-source-release.zip


then unpacked the archivie and executed:


mvn dependency:purge-local-repository

mvn clean install


and I got this:


[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on 
project nifi-standard-processors: Error resolving project artifact: Could not 
transfer artifact com.martiansoftware:macnificent:pom:0.2.0 from/to 
org.apche.nifi (http://https://mvnrepository.com/artifact/org.apache.nifi): 
https: unknown error for project com.martiansoftware:macnificent:jar:0.2.0: 
Unknown host https: unknown error ->



any pointers how to fix this ?