LookupRecord with RestLookupService

2019-12-04 Thread Etienne Jouvin
ody should be retrieve from a record path evaluate on the flowfile body content ? Regards. Etienne Jouvin

Re: LookupRecord with RestLookupService

2019-12-04 Thread Etienne Jouvin
Etienne Jouvin Le mer. 4 déc. 2019 à 20:31, Etienne Jouvin a écrit : > Hello all. > > I am trying a little bit the LookupRecord with RestlookupService. > > When watching the code for RestLookupService, we can see the following in > the lookup function : >

Re: XSL transformation date formating

2019-12-13 Thread Etienne Jouvin
building on the capabilities of the > ICU-J library. The Advanced level also includes APIs which allow additional > languages to be supported. So it is clear, with version HE of Saxon, no way to have localization functions on date in the XSL. Need to check how to do that now in other way

XSL transformation date formating

2019-12-13 Thread Etienne Jouvin
, I just tried to copy the ICU4J library, after download, in the lib folder. But nothing better. I will try to put it in the nar file and see what happens. Does someone have the same requirements ? Regards Etienne Jouvin

Re: Split a flow file to multiples, related by common key and with split count

2019-12-05 Thread Etienne Jouvin
Hello. Why don't you use a JoltTransformation process first to produce multiple element in JSON according value in the array, and duplicate common attributes for all. And then, you do the split. Etienne Le jeu. 5 déc. 2019 à 14:11, James McMahon a écrit : > Daeho and Matt, thank you for all

Re: Split a flow file to multiples, related by common key and with split count

2019-12-05 Thread Etienne Jouvin
Hello. You are right. If it works and you are satisfied, you should keep your solution. By the wya JoltTransformation may be difficult at the very beginning. But it is very powerful and with some pratice, it begins to be easy. For study, you may give it a try. Regards. Etienne Jouvin Le jeu

Re: Split a flow file to multiples, related by common key and with split count

2019-12-05 Thread Etienne Jouvin
that you recommend? > > On Thu, Dec 5, 2019 at 8:57 AM Etienne Jouvin > wrote: > >> Hello. >> >> You are right. If it works and you are satisfied, you should keep your >> solution. >> By the wya JoltTransformation may be difficult at the very beginning

Re: promote NiFi flows from DEV to PROD and controllers

2019-12-06 Thread Etienne Jouvin
Hello. Why don't you use NiFi registry ? Discover this couple if weeks ago, ans ont ils really cool Le ven. 6 déc. 2019 à 19:14, Boris Tyukin a écrit : > Hi, > > We have a single NiFi Registry and DEV/PROD NiFi clusters. When we deploy > changes to PROD NiFi (after initial commit), we often

Re: Merge content Defrag with high activity

2019-10-22 Thread Etienne Jouvin
sions. I also tried to set current task, for the merge to 1 and set a Run Schedule to 0.1 But it was "pretty slow", almost not as fast as I expected. But now, working with a new identifier, and set maximum bean number relly greater than expected possible (6x more) I works as expected. I will surve

Merge content Defrag with high activity

2019-10-22 Thread Etienne Jouvin
0.1 second ? Regards Etienne Jouvin

Merge content Max Bin Age optional ?

2019-10-23 Thread Etienne Jouvin
e bin can be considered as full but it is wrong. First call, with fragement identifier --> FRAG1 Second call with fragement identifier --> FRAG2 But during the second call, the check for first bean will return true. It is not fullEnough, but it is older than the "specify" max age. Do you confirm than ? If yes, this is expected behavior ? Regards. Etienne Jouvin

Jolt specification registry

2019-11-20 Thread Etienne Jouvin
same as schema registry. Regards. Etienne Jouvin

Re: Jolt specification registry

2019-11-20 Thread Etienne Jouvin
that is a great idea, I’d suggest the same thing for protobuf >> specs as well. >> >> Even if the first step is the registry supporting raw bytes access and >> support…. >> >> >> >> >> On November 20, 2019 at 09:28:23, Etienne Jouvin (lapinoujou...@gm

Re: Jolt specification registry

2019-11-20 Thread Etienne Jouvin
gistry controller > service. This is different. > > Thanks > > On Wed, Nov 20, 2019 at 10:26 AM Etienne Jouvin > wrote: > >> Yes it would be a ControllerService as you described. >> >> There is currently three implementation : >>

Re: Jolt specification registry

2019-11-20 Thread Etienne Jouvin
er. It should have no impact on the schema registry controller > service. This is different. > > Thanks > > On Wed, Nov 20, 2019 at 10:26 AM Etienne Jouvin > wrote: > >> Yes it would be a ControllerService as you described. >> >> There is currently three implementati

Re: Question about data provenance menu in processors

2019-12-18 Thread Etienne Jouvin
r > > Am 18.12.2019 um 11:36 schrieb Etienne Jouvin: > > Hello. > > You have this when you right click on the processor and then call the data > provenance. > > Data provenance from the menu is for the instance in global. > > Etienne > > > Le mer. 18 déc.

Re: Question about data provenance menu in processors

2019-12-18 Thread Etienne Jouvin
Hello. You have this when you right click on the processor and then call the data provenance. Data provenance from the menu is for the instance in global. Etienne Le mer. 18 déc. 2019 à 11:21, Dieter Scholz a écrit : > Hello, > > in the past when I opened the data provenance menu of a

Re: Question about data provenance menu in processors

2019-12-18 Thread Etienne Jouvin
lue if any? > > Thanks. > > Dieter > > Am 18.12.2019 um 13:30 schrieb Etienne Jouvin: > > Ok sorry. > > I've never seen that. At least the list is empty sometimes and I have to go > into filters to refresh. > > Did you check search filter after display the data prov

Re: How to preserve the previous flowfile content using InvokeHttp

2020-01-27 Thread Etienne Jouvin
I didn't find how to > share the sessionID after the Notify. > > I just hope that storing the content in the distributed cache is better > than having it as an attribute. > > > El lun., 27 ene. 2020 a las 11:14, Etienne Jouvin (< > lapinoujou...@gmail.com>) escribió: > >

Re: How to preserve the previous flowfile content using InvokeHttp

2020-01-27 Thread Etienne Jouvin
Hello. I manage to do this in two ways : 1. Use Wait / Notify process. Put the original content in the wait, and after your post is finished call the Notify to free the original one. 2. Or use a distribute cache map. Store the original content in the cache, do your stuff and then request the

Re: InvokeHTTP & AttributesToJSON Help

2020-02-01 Thread Etienne Jouvin
Hi. If I well understand, you can configure the InvokeHTTP to always output the response. In this case, the response will be routed to the response relation in any case. Then yo ucan do a RouteOnAttribute to check the response status. And if HTTP code, from attribute, is 500 go to a specific

On ExecuteSQL (1.9.2) failure, infinite loop

2020-02-10 Thread Etienne Jouvin
intermediate processor, LogMessage, but this is the same. Notice that I do not have this in the success relation. Does someone have this also ? Regards Etienne Jouvin

Re: On ExecuteSQL (1.9.2) failure, infinite loop

2020-02-10 Thread Etienne Jouvin
Put 0 seconds to penalized and it goes to the logger processor without waiting. That's fine. I just do not understand weel why there is a penalty or yield failure, but now that I know this, this is ok. Le lun. 10 févr. 2020 à 14:58, Etienne Jouvin a écrit : > Mark, > > Hu

Re: On ExecuteSQL (1.9.2) failure, infinite loop

2020-02-10 Thread Etienne Jouvin
cs/html/user-guide.html#settings-tab > > > On Feb 10, 2020, at 8:47 AM, Etienne Jouvin > wrote: > > > > Hello All. > > > > Here is an extract of my process > > > > > > If the executeSQL failed (invalid SQL for example), the flowfile goes to > the

Re: On ExecuteSQL (1.9.2) failure, infinite loop

2020-02-10 Thread Etienne Jouvin
at case, you don't > want to constantly hit the database and give some time before trying again. > > HTH, > Pierre > > Le lun. 10 févr. 2020 à 06:03, Etienne Jouvin a > écrit : > >> Put 0 seconds to penalized and it goes to the logger processor without >> waiting. &

SSLContextService configuration

2020-01-03 Thread Etienne Jouvin
lem could be the implementation that may produce circular reference. But it is not our fault if user make things stupid. What do you think about that ? Regards Etienne Jouvin

Re: SSLContextService configuration

2020-01-05 Thread Etienne Jouvin
registry-docs/index.html > [3] https://semver.org > [4] https://issues.apache.org/jira/browse/NIFIREG-288 > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jan 3, 2020, at

Apache NiFi 1.9.2 InferAvroSchema on csv file header with :

2020-03-11 Thread Etienne Jouvin
From Data The processor failed because of the name. But if I use a convertRecord with a CSV Reader, that infer schema, and a JSON writer, this is working fine. Not the same algorithm to get infer schema from InferAvroSchema and the reader ? Regards Etienne Jouvin <https://www.avast.com/sig-ema

Re: Apache NiFi 1.9.2 InferAvroSchema on csv file header with :

2020-03-11 Thread Etienne Jouvin
get different results l, while I'm not surprised by this I think > it's worth just flagging up, for future reference. > > Edward > > On Wed, 11 Mar 2020, 09:35 Etienne Jouvin, > wrote: > >> Hello all. >> >> Just in case someone "can test". >>

Re: Adding Nested Properties/JSON

2020-03-31 Thread Etienne Jouvin
Hello. Jolt transformation. Etienne Le mar. 31 mars 2020 à 14:40, Darren Govoni a écrit : > Hi, >I want to use Nifi to design a flow that modifies, updates, etc a > nested JSON document (or that can finally output one at the end). > > For example: > > { >"name":"this and that", >

Re: Adding Nested Properties/JSON

2020-03-31 Thread Etienne Jouvin
t; join them into a single JSON? > > thanks in advance! > Darren > > > -- > *From:* Etienne Jouvin > *Sent:* Tuesday, March 31, 2020 8:52 AM > *To:* users@nifi.apache.org > *Subject:* Re: Adding Nested Properties/JSON > > Hello. > > Jolt transformation

Re: Adding Nested Properties/JSON

2020-03-31 Thread Etienne Jouvin
which adds a record to a sub-field > > { >"name":"this and that", >"field":"value", >"others": [ > {"name":"here and there"} > ] > } > > passes to Processor #3 which also adds a re

Re: POST multipart/form-data with Invokehttp

2020-04-27 Thread Etienne Jouvin
Hello. I did it with a processor ExecuteGroovyScript. The script body is something like : import org.apache.http.entity.mime.MultipartEntityBuilder import org.apache.http.entity.ContentType flowFileList = session.get(100) if(!flowFileList.isEmpty()) { flowFileList.each { flowFile -> def

Re: [Blog] Sending Multipart / Form Data with InvokeHTTP in Apache NiFi 1.12-SNAPSHOT

2020-04-29 Thread Etienne Jouvin
Nice. Will help a lot Le jeu. 30 avr. 2020 à 05:46, Otto Fowler a écrit : > As promised. > > Sending Multipart / Form Data with InvokeHTTP in Apache NiFi 1.12-SNAPSHOT > > > >

Re: Creating parameters from exported template?

2020-05-19 Thread Etienne Jouvin
Hi. In this case why don't you use variables ? Etienne Le mar. 19 mai 2020 à 09:41, James a écrit : > Hi > > For the life of me, I can't find anything related to exporting templates > to new systems/environments along with the parameters. > > I don't need the parameter values necessarily,

Re: Creating parameters from exported template?

2020-05-19 Thread Etienne Jouvin
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables > > "Variables and the nifi.variable.registry.properties property will be > removed in a future release. As a result, it is highly recommended to > switch to Parameters." > > Or am I thinking of the wro

Re: Custom service in NAR generation failure

2020-06-19 Thread Etienne Jouvin
ings. Try posting your > POMs here or as a GitHub gist. > > On Fri, Jun 19, 2020 at 3:36 AM Etienne Jouvin > wrote: > >> Hello all. >> >> Do not know where to post the message, guide me if I should send to >> another mailing list. >> A simple summary

How to upgrade custom nar version

2020-06-19 Thread Etienne Jouvin
, there is like NiFi upgrade, but for now I did not see how to update all process with a new version. Regards Etienne Jouvin

Re: How to upgrade custom nar version

2020-06-19 Thread Etienne Jouvin
Then you go to > each processor from 1.0 and right-click and select "Change Version" and > pick 1.1. > > There is currently no bulk upgrade for processors, but would be a nice > future enhancement. > > > > On Fri, Jun 19, 2020 at 12:46 PM Etienne Jouvin > wrote: &

Re: Custom service in NAR generation failure

2020-06-19 Thread Etienne Jouvin
if this is the correct fix, I will way the pull request review. Etienne Le ven. 19 juin 2020 à 15:19, Bryan Bende a écrit : > If you are not using nifi-nar-bundles as your parent (which is fine), then > you should be explicitly setting versions for nifi-api and > nifi-framework-api. >

Re: Custom service in NAR generation failure

2020-06-26 Thread Etienne Jouvin
t; > We have Maven archetypes for the processor and service bundles. I wonder > if you could compare the resulting projects/poms with yours to see what > seems different? > > > https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions > > > On Fri, Jun 19, 202

Custom service in NAR generation failure

2020-06-19 Thread Etienne Jouvin
Hello all. Do not know where to post the message, guide me if I should send to another mailing list. A simple summary in first step. I created a simple project to build a new service. I extend the nifi-nar-bundles artifact with version 1.11.4. My project version is currently 0.0.1-SNAPSHOT.

Re: Custom service in NAR generation failure

2020-06-19 Thread Etienne Jouvin
r " + > artifact + " when attempting to determine the expected version of NiFi > API", e); > } > } > return null; > } Should not be return dependency.getVersion() ? Because the artifact is the currently parsed artifact and not the depende

Re: Messed with PutS3Object, add signature on top on content

2020-06-26 Thread Etienne Jouvin
Hello all. Got it. It was a matter of signature version. Because I use s3Ninja, I needed to set with V2 : [image: image.png] Le ven. 26 juin 2020 à 15:43, Etienne Jouvin a écrit : > Hello all. > > I am trying to use S3 storage (s3Ninja for "unit test") and I am really

Messed with PutS3Object, add signature on top on content

2020-06-26 Thread Etienne Jouvin
a or input for investigation ? Regards Etienne Jouvin

Need help SSL LDAP Nifi Registry

2020-06-30 Thread Etienne Jouvin
cate error : ERR_BAD_SSL_CLIENT_AUTH_CERT How can I force the authentication to not request a client side certificate ? Thanks for any input. Etienne Jouvin

Re: Need help SSL LDAP Nifi Registry

2020-06-30 Thread Etienne Jouvin
But now, I have NiFi and Registry with secure access (LDAP + SSL) I need to find out how to configure the Registry in NiFi, because for now I did not have to specify login. And even if my first bucket is Public, it is not accessible from NiFi. Le mar. 30 juin 2020 à 11:29, Etienne Jouvin

Re: Need help SSL LDAP Nifi Registry

2020-06-30 Thread Etienne Jouvin
stry.properties»: > > nifi.registry.security.needClientAuth=false > > > > Cheers Josef > > > > > > *From: *Etienne Jouvin > *Reply to: *"users@nifi.apache.org" > *Date: *Tuesday, 30 June 2020 at 10:46 > *To: *"users@nifi.apache.org" > *Subject: *Ne

Re: Need help SSL LDAP Nifi Registry

2020-06-30 Thread Etienne Jouvin
Got it thanks to https://community.cloudera.com/t5/Community-Articles/Setting-Up-a-Secure-NiFi-to-Integrate-with-a-Secure-NiFi/ta-p/247765 Next steps would be to have NiFi and Registry on different hosts and see how connections are made. Le mar. 30 juin 2020 à 11:43, Etienne Jouvin a écrit