Re: Policy storage in Nifi runner and Kubernetes

ok, it appear my nifi node doesn't start an embedded zookeeper server. Can it be related ? Le 30/09/2019 à 09:59, Nicolas Delsaux a écrit : Hi all I'm running my Nifi node in Kubernetes. For that, the /opt/nifi/nifi-current/conf folder is made writable by an init container prior to starting

Policy storage in Nifi runner and Kubernetes

Hi all I'm running my Nifi node in Kubernetes. For that, the /opt/nifi/nifi-current/conf folder is made writable by an init container prior to starting the runner. All other interesting folders (the *_repository ones) are stored on a read-write-once volume. Each time the pod is restarted, the

Re: can't ush data to bigQuery

Oh well, i've understood my last error : incorrect flow files (with JSOn arrays) were stuck in the queue. I removed them and ... to my delight, data seems to come in BigQuery ! Le 26/09/2019 à 14:45, Nicolas Delsaux a écrit : I didn't knew that command ... i've edited some confidential

Re: can't ush data to bigQuery

;state": "DONE"   },   "user_email": "rabbitmq-inges...@psh-analytics-automation.iam.gserviceaccount.com" } Error message is interesting. If I look in data provenance at the data I'm expected to send to BigQuery, I get [{"ContractualConsent":{

Re: can't ush data to bigQuery

pplication/json' Key: 'path' Value: './' Key: 'record.count' Value: '3' Key: 'uuid' Value: 'e6d604d7-b517-4a87-a398-e4a5df342ce6' 2019-09-26 10:09:39,633 INFO [Timer-Driven Process Thread-4] o.a.n.processors.standard.LogAttribute LogAttribute[id=ce9c171f-0c8f-3cab-e0f2-16156faf15b8] logging for flow f

Re: implementing policies through REST interface

it: https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-api On Tue, Sep 24, 2019 at 3:52 AM Nicolas Delsaux wrote: Hi all i'm deploying my nifi node in containers and, as a consqeunce, i have to periodically rewrite policies to have it working. As it is really painful, i would

implementing policies through REST interface

    Hi all i'm deploying my nifi node in containers and, as a consqeunce, i have to periodically rewrite policies to have it working. As it is really painful, i would like to write a script that will write those policies automatically at first startup. Are there any tutorials about that ? I'm

can't ush data to bigQuery

Hello I'm using PutBigQueryBash and having weird auth issues. I have set the GCP Credentials Controller Service to use Service Account JSON which I have copied from the value given in Google Cloud Console. But when I run my flow, I get the error message "Error while reading data, error

Re: In nifi-registry, why can't I edit other users privileges

On Mon, Sep 2, 2019 at 8:56 AM Nicolas Delsaux wrote: Hi all I'm still trying to connect nifi to registry with both of them using authentication. So far, i've understood that, like in Nifi, I have to set identity-providers.xml and authorizers.xml to have connection to ldap configured

In nifi-registry, why can't I edit other users privileges

   Hi all I'm still trying to connect nifi to registry with both of them using authentication. So far, i've understood that, like in Nifi, I have to set identity-providers.xml and authorizers.xml to have connection to ldap configured. And I can connect to the registry using my ldap, so it

Re: securing nifi-registry

10:30, Nicolas Delsaux a écrit : Hi all I'm trying to secure my nifi registry. So i've created a keystore and a trustore, added to the keystore a private key entry, and configured my nifi-registry docker container to use that keystore/truststore. I can get the key pair in my keystore using

securing nifi-registry

Hi all I'm trying to secure my nifi registry. So i've created a keystore and a trustore, added to the keystore a private key entry, and configured my nifi-registry docker container to use that keystore/truststore. I can get the key pair in my keystore using keytool, both on my machine and in

authenticated nifi agent wih unauthenticated registry

Hi all I have correctly setup my nifi runner to use LDAP auth from my company. I'm now trying to understand why registry no more work. As you may guess from message title, my registry is currently not authenticated. Do I need to have auth enabled on registry when it is enabled on nifi runner

Re: ldap authentication and initial admin identity

a tool tip message in global Policies (when accessed by the hamburger menu) informing users that they might want to go at process group level to have granular policies. Le jeu. 22 août 2019 à 11:55, Nicolas Delsaux mailto:nicolas.dels...@gmx.fr>> a écrit : Well, ok, i've unde

Re: ldap authentication and initial admin identity

l policies (the ones you listed below). https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#access-policy-config-examples Hope this helps. Le jeu. 22 août 2019 à 11:06, Nicolas Delsaux mailto:nicolas.dels...@gmx.fr>> a écrit : Well, I sort of sorted it out. I can inde

Re: ldap authentication and initial admin identity

out! Le mar. 20 août 2019 à 11:30, Nicolas Delsaux mailto:nicolas.dels...@gmx.fr>> a écrit : Wow, I'm really REALLY puzzled. I'm using Nifi through the docker image, and docker-compose. I was used to do docker-compose up/down, and it failed. But this time, I did a docker-c

Re: ldap authentication and initial admin identity

), the files users/authorizations won't be updated with your configuration change... Something you could try: delete authorizations.xml and users.xml files and restart NiFi to be sure it uses the last version of your configuration. Le mar. 20 août 2019 à 10:33, Nicolas Delsaux mailto:nicolas.dels

Re: ldap authentication and initial admin identity

as having a look at the users.xml and authorizations.xml file generated the first time NiFi is starting based on your configuration? Thanks, Pierre Le lun. 19 août 2019 à 11:35, Nicolas Delsaux mailto:nicolas.dels...@gmx.fr>> a écrit : Hello all I now have a nifi instance able to c

ldap authentication and initial admin identity

Hello all I now have a nifi instance able to connect to LDAP server, with valid certificates and so on. But i'm unable to connect to Nifi UI, altough I have set myself as initial admin identity. My ldap full DN is set as initial admin identity file-access-policy-provider

Re: My nifi no more serve admin interface

/jetty/documentation/9.4.19.v20190610/configuring-ssl.html#understanding-certificates-and-keys - see part Layout of keystore and truststore). And this happened because I'm really bad at certificates. Sorry to have consumed some of your time, you all. Le 13/08/2019 à 16:21, Nicolas Delsaux a écrit

Re: My nifi no more serve admin interface

s the standard debug one) Le 13/08/2019 à 16:10, Pierre Villard a écrit : Might be a dumb question but I'm wondering why you're trying with port 38080? Did you change the configuration to use that specific port with a secured instance? Pierre Le mar. 13 août 2019 à 16:00, Nicolas Delsaux

Re: My nifi no more serve admin interface

To go a little further, a test with openssl s_client gives the following nicolas-delsaux@NICOLASDELSAUX C:\Users\nicolas-delsaux $ openssl s_client -host localhost -port 38080 CONNECTED(0164) 416:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl\record

My nifi no more serve admin interface

response containing only the string "�P". In other words, nicolas-delsaux@NICOLASDELSAUX C:\Users\nicolas-delsaux $ curl -v -H "Host: nifi-psh.adeo.com" http://localhost:38080/ --output - *   Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 38080 (#0) >

Re: Continuing my LDAP auth adventures

not lining up with the identities being returned from the LDAP provider. If you entered a full DN, but the LDAP provider returns just the short name, or vice versa, then it doesn't line up. On Fri, Jul 19, 2019 at 9:59 AM Nicolas Delsaux wrote: And indeed, it changed the error nifi-runner_1

Re: Continuing my LDAP auth adventures

(new line etc...) can occasionally cause a problem with the Spring loading. Edward On Fri, Jul 19, 2019 at 10:45 AM Nicolas Delsaux mailto:nicolas.dels...@gmx.fr>> wrote: Is there any way to get a better error ? Le 19/07/2019 à 11:36, Edward Armes a écrit :

Re: Continuing my LDAP auth adventures

. The result is you get this lovely misleading error. The normal reason for the bean not being created I found was because I made a typo in the configuration file(s). Edward On Fri, Jul 19, 2019 at 10:21 AM Nicolas Delsaux mailto:nicolas.dels...@gmx.fr>> wrote: Hi all Now I know how to c

Continuing my LDAP auth adventures

Hi all Now I know how to connect to my LDAP directory, i now have a strange error nifi-runner_1  | org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration':

Re: ldap auth : error code 12 - Unavailable Critical Extension

Extension. Are you sure about the LDAP tree structure you have? is the organization correct 'o=corp.mycompany.com <http://corp.mycompany.com/>'? Thanks, Pierre Le jeu. 18 juil. 2019 à 15:36, Nicolas Delsaux mailto:nicolas.dels...@gmx.fr>> a écrit : Hello, I'm trying to use LDA

ldap auth : error code 12 - Unavailable Critical Extension

Hello, I'm trying to use LDAP authentication and am having a weird exception nifi-runner_1  | 2019-07-18 13:26:03,076 INFO [main] org.eclipse.jetty.server.Server Started @22069ms nifi-runner_1  | 2019-07-18 13:26:03,080 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web

Nifi and SSL offloading

Hi I'm trying to deploy Nifi in Kubernetes with authentcation. In Kubernetes, it is possible (and recommended in my organization) to have SSL managed by cluster at edge route level. Which means request seen by Nifi are http ones. According to nifi documentation, it seems to imply no

Re: Unable to send JSON to BigQuery

9 à 19:51, Denes Arvay a écrit : Yes, and please attach the test cases too. Does this mean that your original issue hasn't been resolved yet by adding the "mode" fields? On Wed, Jul 3, 2019, 19:27 Nicolas Delsaux <mailto:nicolas.dels...@gmx.fr>> wrote: So I have a simp

Re: Unable to send JSON to BigQuery

quot;, "type": "STRING", *"mode": "NULLABLE"* }, ...) Let me know if it solved the issue. If yes, I'll file a Jira ticket to fix it. Best, Denes [1] https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#TableFieldSchema On We

Docker nifi doesn't support OpenID Connect ?

Hi, I've read on Docker hub that nifi docker container doesn't support OpenID Connect. But if I mount the nifi.properties file using a volume, is it possible to have openID Connect working ? or is it replaced by the Docker start.sh script (which invoke secure.sh only for LDAP or two-way SSL) ?

Re: Unable to send JSON to BigQuery

LLABLE"*, "fields": [ { "name": "id", "type": "STRING", *"mode": "NULLABLE"* }, ...) Let me know if it solved the issue. If yes, I'll file a Jira ticket to fix it. Best, Denes [1] https://cloud.google.com/bigquery/docs

Unable to send JSON to BigQuery

 I'm using Apache Nifi 1.9.2 and trying to post JSON content to a BigQuery table. There seems to be something wrong, sicne I get 2019-07-03 08:35:24,964 ERROR [Timer-Driven Process Thread-8] o.a.n.p.gcp.bigquery.PutBigQueryBatch PutBigQueryBatch[id=b2b1c6bf-016b-1000-e8c9-b3f9fb5b417e]