Re: Release Dates

Hi, regarding JIRA, there are currently issues with apache servers. This should be fixed very soon. 2016-06-06 9:52 GMT+02:00 ski n : > Today I was checking the release notes at: > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes > > Could the release

Re: Not a Valid JSON content

Hi Anuj, This method supposes your XML input file is not too complex. Have a look on this thread, this should answer your questions: https://mail-archives.apache.org/mod_mbox/nifi-users/201606.mbox/%3CCALo_M1_erYWJkWB=D7kJ17gvpmnyeVGmYiQH9=mqqktb-nh...@mail.gmail.com%3E Hope this helps.

Re: Scheduling using CRON driven on Windows OS

Hi Keith, This is the expected behavior, the first parameter is indeed seconds so that */5 * * * * ? will generate a FF every 5 seconds. In your case, I believe you'd like something like 00 02 10 * * ? Hope this helps. 2016-06-16 19:13 GMT+02:00 Keith Lim : > My

Re: Routing based on conditions in NiFi - ?

Hi Dmitry, I'd suggest to add RouteOnAttribute processor and add a custom property by expected condition : one property with name textxml (for example) and with value the EL you suggested (it supposed that you have the mime type in one of the attribute of your flow file) property with name

HTTP and OAuth

Hi guys, I'm working on a new flow and I'd need to use OAuth for some HTTP requests. Is there something available for this? Or a recommended way to get the job done with existing processors? Thanks! Pierre

Re: HTTP and OAuth

int for further processing. >> >> Andy LoPresto (my go-to guru for all things security, and recently-named >> committer to Apache NiFi) can probably make better recommendations on this >> (sorry in advance if I'm feeding you to the wolves ALP ;) >> >> Regards, &g

Re: Default termination for relationships

Hi, I may be wrong, but I think this is something that has been already discussed for 1.0 branch. However, I had a quick look on JIRA and didn't find anything... Otherwise there is NIFI-1003 to allow a default auto-termination for some

Re: datetime argument to the GetSFTP processor

Hi Ashraf, I believe you are right, RouteOnAttribute is certainly what you should use. Regarding the expression, it sounds good to me. Depending of the filename format characteristics, may be you can directly use ${filename:contains( ${now():toNumber():minus(8640):format('mmdd')} )}

Re: Is it possible to call a HIVE table from a ExecuteScript Processor?

Hi Mike, I never tried but using the JDBC client you should be able to query your Hive table using ExecuteSQL processor. Hope that helps, Pierre 2016-04-26 13:53 GMT+02:00 Mike Harding : > Hi All, > > I have a requirement to access a lookup Hive table to translate a

Re: EvaluateJsonPath processor

Hi, In this case, in your EvaluateJsonPath processor, I would try to change the property "return type" to "json". Hope that helps! Pierre 2016-04-23 23:11 GMT+02:00 Igor Kravzov : > Hi, > > I am using EvaluateJsonPath to extract some properties from tweet JSON >

Re: EvaluateJsonPath processor

atuses_count\":23731,\"created_at\":\"Tue > Jan 05 22:19:11 + > 2016\",\"utc_offset\":null,\"time_zone\":null,\"geo_enabled\":false,\"lang\":\"es\",\"contributors_enabled\":false,\"is_translator\":false,\&quo

Re: problem in windows authentication

Hi Ravisankar, Could you provide full exception stack trace from the log files in NiFi? Thanks, Pierre 2016-05-24 13:05 GMT+02:00 Ravisankar Mani : > Hi All, > > I have trying to move data to sql server using "PutSQL" processor, In this > case i have created a specific DBCP

Re: Exception: .java.sql.SQLException: No suitable driver found for jdbc:sqlserver:

Hi, Where did you place your driver jar? I think the issue is here. You could try to add the driver jar in the /lib directory of NiFi to check if it helps. Pierre 2016-05-24 18:18 GMT+02:00 Kumiko Yada : > Hello, > > > > I’m writing the custom processor to execute the

Re: Rest API invoke

Hi Ravisankar, Depending of the version of OAuth you are looking for, you may be interested by the following article: https://pierrevillard.com/2016/04/12/oauth-1-0a-with-apache-nifi-twitter-api-example/ Pierre 2016-05-02 6:02 GMT+02:00 Ravisankar Mani : > Hi All, > > Is

Re: Using derived attributes in same processor in Apache Nifi

Hi Sarvesh, At the moment, it is not possible to have UpdateAttribute defining new attributes with dependencies between each other. Your best option is, as you said, to use successive processors. - Pierre 2017-01-31 15:53 GMT+01:00 Sarvesh : > I am adding two

Re: Move flow into process group

Hi Banias, By selecting a workflow you want to move, you can click and drag your selection in a process group. You can also (right click on your selection) move your selection to the parent group. This will keep the state unchanged. To make a selection, Shit + click to draw a rectangle over your

Re: How to send back a reply from HandleHTTPRequest

James, If you always want to return 200 as HTTP response whatever the request is, then you could have: HandleHttpRequest directly linked to HandleHttpResponse with property status code to 200, and HandleHttpRequest linked to your flow to perform the expected tasks. Now let's say that you want to

Re: Unable to Start Nifi | Database nay be already in use : “Locked by another process”

Hi Shashi, Any chance you would still have a NiFi instance running? Could you check using something like: ps -aef | grep -i nifi Thanks, Pierre 2017-02-20 7:59 GMT+01:00 Shashi Vishwakarma : > Hi > > I am trying to start nifi but facing h2.jdbc.jdbcSqlException :

Re: ListHDFS ( some suggestions depending on our usage )

Hi Alessio, It sounds like a valid improvement on the existing processor. Please feel free to file a JIRA. Since the directory path is already allowing expression language it wouldn't be a huge change. - Pierre 2017-02-10 9:58 GMT+01:00 Alessio Palma : > Hello

Re: ConvertJSONToSQL in combination with PutCassandraQL

Hi Srinath, Not sure there is an alternate way. However I believe the changes you made to get your use case working are interesting and we should think of a way to get the changes in the processor. Would you mind filing a JIRA? and maybe share your changes through a PR? Thanks 2017-02-16 21:49

Re: Periodic delta pulls from a data source

Hi Selvam, Supposing that your source if a SQL-like source, you should have a look at QueryDatabaseTable [1] processor. It proposes a 'Maximum-value Columns' that gives you the possibility to specify the column containing an ID and or timestamp. The processor will keep track of the maximum value

Re: State management not captured in cluster mode

At the moment the sate management description (if any) of a processor is available by right clicking on it and going into "View state". Pierre 2016-09-28 16:15 GMT+02:00 Bryan Bende : > What I was referring to is in the code of each processor, it is annotated > with something

Re: NiFi operational interface

Hi Roger, I don't know if it answers your needs but you have a search tool bar to find any element on the canvas and quickly access it. In terms of monitoring, you have the "Summary" panel available from the menu. It gives you a way to quickly display information about everything on the canvas.

Re: NiFi operational interface

running, the canvas view is perhaps not the best tool for > someone who needs to monitor and support a production ingestion batch on a > per source basis. > > Regards, > Roger > > > > > > > > Original message > From: Pierre Villard <pi

Re: ExecuteSQL & BigInt fieds

Hi Yari, I think there is a JIRA on this problem at the moment: https://issues.apache.org/jira/browse/NIFI-2811 https://github.com/apache/nifi/pull/1053 If you have the chance you can try the PR and give us your feedback. Pierre 2016-09-27 10:23 GMT+02:00 Yari Marchetti

Re: PutElasticsearchHttp error behaviour

Hi Gaspar, I believe this is because 4294967295 is a too big value that cannot be contained in an integer. I am not a ES expert but you should look at your applicationId type if you can change it. Hope this helps. 2016-11-07 19:09 GMT+01:00 Gaspar, Carson : > It stays in the

Re: Resource directory paths are malformed: docs

Hi Alessio, Could you copy/paste the full stack trace? Thanks! Pierre 2016-11-17 17:03 GMT+01:00 Alessio Palma : > Hello all, > > I rebuild nifi from scratch and after the deploy Jetty won't restartslogs > returns this error: > > > at

Re: Get Ignite Processor - Upstream Connections is invalid

Hi, Bryan is correct, the idea here is to have the Ignite cache key (linked to the content to retrieve from Ignite cache) based on an incoming flow file attribute and that is why the processor requires an upstream connection. I understand it might be confusing regarding the name of the processor

Re: nifi is running out of memory

Quick remark: the fix has also been merged in master and will be in release 1.1.0. Pierre 2016-10-28 15:22 GMT+02:00 Gop Krr : > Thanks Adam. I will try 0.7.1 and update the community on the outcome. If > it works then I can create a patch for 1.x > Thanks > Rai > > On Thu,

Re: DistributedMapCache w/ ListSFTP and FetchSFTP

t; -Nick > > > On Thu, Dec 15, 2016 at 4:21 PM, Pierre Villard < > pierre.villard...@gmail.com> wrote: > >> Hi Nicholas, >> >> You need to configure your ListSFTP processor to only run on the primary >> node (scheduling strategy in processor configuration)

Re: DistributedMapCache w/ ListSFTP and FetchSFTP

Hi Nicholas, You need to configure your ListSFTP processor to only run on the primary node (scheduling strategy in processor configuration), then to send the flow files to a RPG that points to an input port in the cluster itself (so that flow files are distributed over the cluster and do not stay

Re: NiFI server restart blocked

Hi Leo, I'd say that you need to manually update the flow.xml.gz file to fix the XML definition of what is defined on your canvas. Seems like one of the template object is not matching the XSD specs. Hope this helps, Pierre 2017-01-05 12:25 GMT+01:00 Porta Léonard

Re: How to access Controller Service created in UI into Root processors in NiFi-1.1.1?

Hi, I believe you created your controller service by going into "controller settings" in the main menu. This should not be done this way as controller services created here are only to be used by reporting tasks. To create controller services for processors, you need to create it at Process Group

Re: Routing File based on CSV header schema

Hi Austin, I believe the RouteOnContent processor won't try to only match the first line of your flow file but will try to match the whole content. Since your flow file contains multiple lines, the regular expression you are using won't work. I'd do the following: use ExtractText to get the

Re: How to get ftp file according to Current date?

You could use the combination of ListFTP and FetchFTP (this is, most of the time, a better approach), and between the two processors you could do a RouteOnAttribute and only keep the flow files with the filename you are looking for. Pierre. 2017-04-21 13:29 GMT+02:00 prabhu Mahendran

Re: How to get ftp file according to Current date?

FTP file from lakhs. > > Can you guide way to reduce time taken for fetch file? > > On Fri, Apr 21, 2017 at 8:40 PM, Pierre Villard < > pierre.villard...@gmail.com> wrote: > >> You could use the combination of ListFTP and FetchFTP (this is, most of >> the time, a bette

Re: VolatileContentRepository

Hi Margus, I believe that your memory settings are enough. Giving more memory will likely increase duration of garbage collections and won't increase your performances. Others on this mailing list will probably be able to give better recommendations on this one. Also, keep in mind that volatile

Re: VolatileContentRepository

<b.deep.internatio...@gmail.com> : > Pierre, > > Just a clarification on what you said below the potential data loss on > nifi shutdown would only be on provenance information right? > > Or would the data loss affect other repos? > > Thanks > > Brian > > On 10 Aug 2017

Re: MiNiFi on Ubuntu Core

Hi Gabor, I'm not aware of any kind of undergoing effort for that. Maybe Aldrin has a better view on this? Pierre 2017-07-13 17:07 GMT+02:00 Molnár Gábor : > Dear NiFi / MiNiFi Community, > > > > Our team is using NiFi + MiNiFi combination with high satisfaction, and

Re: ListS3 duration

Can you check what's the current state of the processor? (right click / view state) Are you sure there is data to retrieve more recent that what is currently in the processor's state? Pierre 2017-07-20 18:16 GMT+02:00 Laurens Vets : > I'm running 1.3.0 at the moment... I'm

Re: Application is no longer processing files

Hi Noel, Based on the configuration of the processor, NiFi is supposed to delete the original file once retrieved by the processor (keep original file = false). Is the user running NiFi allowed to delete the file? If not, as per the documentation, the files will be ignored. Thanks Pierre

Re: Two pull requests in a weird state

ause we don't want to have to > maintain a separate build of the nifi-mongo bundle. How often are pull > requests evaluated for inclusion into master? > > Thanks, > > Mike > > On Tue, Jun 27, 2017 at 6:49 AM, Pierre Villard < > pierre.villard...@gmail.com> wrote: >

Re: Does PutFile block subsequent files on failure for a flowfile?

Hi Jim, That's correct. When going in the failure relationship, the flow files are penalized with a duration of 30s (by default, this can be changed in the processor configuration) and the processor will try to process the flow files only once this duration expired. Pierre 2017-06-29 14:36

Re: Issues with debugging

Hi, Dummy question but... are you sure the code of the running NiFi is exactly matching the code where you specified breakpoints? If this is not matching, that could be the reason.

Re: Query syntax error in GetMongo processor?

Hi James, The query needs to be a valid JSON. You can have a look into the test class to get some examples: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoTest.java In your case, I

Re: OAuth1.0

Hi Karthik, You might want to have a look here: https://pierrevillard.com/2016/04/12/oauth-1-0a-with-apache-nifi-twitter-api-example/ It's OAuth 1.0A but should give you some ideas. There is no "out-of-the box" processor as far as I know. For OAuth 2.0, few JIRAs are currently in progress:

Re: Cluster is unable to service request to change flow

Mayank, When a node is disconnected, the UI switches to read-only mode. That's why it's saying that action cannot be complete: there is one disconnected node (it does not matter it is the old primary node). If you want to have normal access to the UI, you need to completely remove the node from

Re: Cluster is unable to service request to change flow

ode from the cluster" --> I already > ran stop script on the node. UI shows that node is disconnected. What else > needs to be done to remove node from the cluster? > > > On Mon, Aug 7, 2017 at 1:06 PM, Pierre Villard < > pierre.villard...@gmail.com> wrote: > >&g

Re: VolatileProvenanceRepository problem with 1.2?

Hi Arnaud, I just tried to switch to the volatile provenance repository and didn't get any issue. I think it could be because of a whitespace at the end: *The specified implementation class 'org.apache.nifi.provenance.VolatileProvenanceRepository ' is not known to this nifi.* If that's the

Re: Reliable operation use case

Hi Andy, Regarding bulletins, you can use the new SiteToSiteBulletinReportingTask [1] to achieve what you are expecting (in NiFi 1.2.0). For provenance, you can definitely use the REST API but could also be interested by the SiteToSiteProvenanceReportingTask [2] with the addition of NIFI-3859

Re: Set priority to files based on date time value stored on attribute

Hi Manoj, You may want ot have a look at EnforceOrder processor [1] or simply the prioritizers [2] of the connections (it depends of how your workflow is working). The idea would be to extract the date as an attribute of your flow file, convert into an integer (using expression language) and use

Re: NiFi web interface not starting

Hey, It looks like NiFi is correctly starting: 2017-06-12 18:02:09,570 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: 2017-06-12 18:02:09,570 INFO [main] org.apache.nifi.web.server.JettyServer http://10.248.139.86:9090/nifi

Re: add a property without validator in custom processor

Hi Ben, You can use the following in your property construction: .addValidator(Validator.VALID) Hope this helps. Pierre 2017-06-20 6:35 GMT+02:00 尹文才 : > Hi guys, a very simple question, I'm working on a customized processor and > I'm trying to add a processor property

Re: Checkstyle configuration missing

Hi Mike, The checkstyle rules configuration is included in the root pom. Is that what you are looking for? Pierre 2017-06-26 18:19 GMT+02:00 Mike Thomsen : > The checkstyle configuration seems to be missing from the repo I forked > from github.com/apache/nifi. I ran

Re: some processors runs only once in NiFi

Just a quick remark, the ListHDFS processor won't list files over and over, it'll only list new files since the last execution of the processor. Do you know if new files are generated in the directory your are listing? Screenshots of your configurations would definitely help. 2017-05-24 6:55

Re: Input flowfile for GetHTTP

Hi, As we certainly cannot ask users to look at the code to find out, I think that it's a good idea to add this information into the doc usage of the processor. A generic sentence could be easily added based on the annotation. Would you mind opening a JIRA for this? Thanks! Pierre 2017-05-02

Re: Two pull requests in a weird state

Hi Mike, thanks for the contributions! Don't bother about AppVeyor builds, for now it's broken. We are only looking at Travis builds, and even for this build, we see intermittent failures. Personally I look at the travis details and check that at least one of language-dependent build is OK (and

Re: Does nifi support variable in ExecuteProcess

Hi Hollis, If you go in the processor configuration, for each property you will see a question mark symbol that gives details for each property. In the details it also says if the property supports expression language [1] or not. You can also find this information in the documentation of each

Re: JoltTransformJSON with array of jsons

Hi Chris, Yes it is. For example, if I want to add a field in all JSON of my array - I can do (with default DSL) : { "*": { "mynewfield":"mynewvalue" } } I'm clearly not a Jolt expert, but others can help you if you're trying to do something more complex. Pierre 2017-10-12 11:04

Re: JoltTransformJSON with array of jsons

;> >> Mar.k >> >> On Thu, Oct 12, 2017 at 4:52 PM Chris Herssens <chris.herss...@gmail.com> >> wrote: >> >>> Hello Pierre, >>> >>> I want to flatten the json content. How to I do this ? >>> Now I use the chain DSL with jo

Re: External server other than jetty

Hi Kumar, There is no way to do that as of now. And doing it would require quite a lot of changes in the core code. Thanks! Pierre 2017-09-12 7:32 GMT+02:00 kumar r : > Hi, > > Can i deploy NiFi instance in any other server like Tomcat instead of > embedded jetty. > >

Re: Missing nifi-app.log files

Hi James, Out of curiosity do you have a TailFile processor configured to tail the NiFi log file? Thanks Le 17 août 2017 22:11, "James McMahon" a écrit : Thank you Joe. I agree and will monitor it closely going forward. I suspect there were some external factors at play

Re: NiFi ram usage

As Jeff, I'm a bit surprised by what you are experiencing. I've never changed the default values of 512MB when working with NiFi on my laptop and never hit OOM errors. Are you sure that 1GB is available on the VM before starting NiFi? Pierre 2017-08-31 13:52 GMT+02:00 Mike Thomsen

Re: NPE at ConsumeKafka_0_10

Hi Juan, Could you provide your processor configuration? Thanks! 2017-08-29 14:41 GMT+02:00 Juan Pablo Gardella : > Hi all, > > I was using ConsumerKafka_0_10 processor and yesterday I saw the following > error, the gist >

Re: How to move queue boxes

Hi Eric, You can add a bending point by double clicking on the black line of the relationship (and remove a bending point by double clicking on it). Pierre 2017-10-23 11:38 GMT+02:00 Eric Chaves : > Hi, my flow has a lot processors coming out from a single router processor >

Re: PutHiveStreaming Error : java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwx------

Hi Mohit, As far as I know, the Hive Streaming API is directly writing into HDFS and, based on the stack trace, is trying to write into /tmp/hive. Right now this folder is not writable by your user and it is the reason why it's failing. I would try changing the chmod permissions in HDFS. Pierre

Re: Nifi slowing down the flow in long run

Hi, What implementation are you using for the provenance repository? If you're using a recent version of NiFi (1.2.0+), you can switch to a new implementation which is much more performant and should resolve such issues [1]. The new implementation is

Re: Hanging Processor: ExtractHL7Attributes

Hi, Can you check nifi-app.log file instead of bootstrap? You have a bulletin generated (red icon) and you should have more information in the nifi-app.log file. Thanks! 2017-11-14 14:37 GMT+01:00 Eric Thompson : > I am getting an error of 'Administratively Yielded

Re: Splunk Processor - Re-play

Hi Siva, The processor is storing a "state" in the state management back-end of NiFi (Zookeeper usually). There is no way for you to edit this value. However, some processors expose a property allowing you to manually set the initial value (Example GenerateTableFetch [1]) that the processor will

Re: Metrics on Lineage Duration

Hi Peter, There is not as far as I know. What I usually do is to use ExecuteScript processor to extract the lineage duration of the flow file into an attribute of the flow file where I feel this information is useful for my workflow and I send it to whatever monitoring destination I have. This

Re: NiFi Processor Lifecycle Diagram

Hi, There is actually a JIRA for that [1]. Feel free to add attachments to this JIRA or submit a PR with your work. I'm sure someone will review and get it merged into our documentation! [1] https://issues.apache.org/jira/browse/NIFI-1345 Thanks 2017-11-02 15:51 GMT+01:00 Ryan H

Re: Buttons are greyed out when initial admin account logs in

Hey guys, I'll try to write a new blog with all the new features coming with NiFi 1.4.0. All the new stuff to have LDAP sync is really nice. Pierre 2017-12-03 19:12 GMT+01:00 Kevin Doran : > Hi Mike, > > You also have to enable the LdapUserGroupProvider in authorizes

Re: Nifi connect to AS400 using JT400.jar

Hello, Based on the stack trace, it seems you have a white space at the begining of your driver class name. Can you check? Pierre 2017-12-04 3:54 GMT+01:00 Chee Pei Lim : > Here are the details. > > The jt400.jar is located in Nifi's lib folder. > > Here is the log

Re: Auto Partitioning in PuthiveStreming Processor

Hi Mohit, Can you share a 'describe formatted' of your table in Hive? Thanks! 2017-10-25 9:29 GMT+02:00 : > Hi All, > > > > I am trying to write data to dynamic partitions in Hive using > PutHiveStreaming Processor. > > It fails in case of auto partitioning.

Re: Node won't come back up again

Hi Fabian, What version of NiFi are you running with? I believe you might be facing NIFI-4093 [1] marked as resolved in NiFi 1.4.0. [1] https://issues.apache.org/jira/browse/NIFI-4093 Pierre 2017-10-25 11:21 GMT+02:00 Fabian Werner : > Hi. > > We have a nifi cluster

Re: Auto Partitioning in PuthiveStreming Processor

nd the desc formatted : > > > > > > Regards, > > Mohit > > *From:* Pierre Villard [mailto:pierre.villard...@gmail.com] > *Sent:* 25 October 2017 15:14 > *To:* users@nifi.apache.org > *Subject:* Re: Auto Partitioning in PuthiveStreming Processor > > > >

Re: [Nifi 1.6] PropertiesFileLookupService configuration file reference

Hi, You have to give the path to the properties file (key=value file) that will be used for your lookup service so that when using LookupRecord it will look for the value associated to the given key (with a dynamic property in the processor). Does it make sense? Pierre 2018-04-27 11:25

Re: [Nifi 1.6] PropertiesFileLookupService configuration file reference

esn't sound to be possible right ? > > All the best > > François > > 2018-04-27 11:31 GMT+02:00 Pierre Villard <pierre.villard...@gmail.com>: > >> Hi, >> >> You have to give the path to the properties file (key=value file) that >> will be used for you

Re: Nifi toolkit grape dependecies

Hey Dan, I think it's worth filing a JIRA for it. Pierre 2018-05-22 19:27 GMT+02:00 dan young : > Hello, > > I'm trying to run the Nifi Expression testing groovy tool with 1.6.0 and > am getting the following dep errorNifi 1.4 and 1.5 works fine. I tried > to also

Re: PutDatabaseRecord Error while ingesting to Netezza

Hi, Can we have more details: describe of the Hive table and schema you're using in the processor? Pierre 2018-05-24 8:26 GMT+02:00 Mohit : > Hi all, > > I’m facing the error while writing the data to Netezza using put database > records. I get the following

Re: LookupRecord using Attribute

Hi, Is 'authority' an existing attribute in the incoming flow files or is it only an information available in the content of the flow file ? Pierre 2018-05-25 13:07 GMT+02:00 Thomas P. : > Hello community, > > I am trying to query a MongoDB using LookupRecord

Re: LookupRecord using Attribute

thority, however, will do a lookup on the record and should find the >> value MyAuthority and set a map of { authority => MyAuthority } for the >> lookup service. >> >> On Fri, May 25, 2018 at 8:02 AM Thomas P. <thomaspuschm...@googlemail.com> >> wrote: >

Re: NiFi Performance Analysis Clarification

Hi, What's the version of NiFi you're using? What are the file systems you're using for the repositories? I think that changing the heap won't make any different in this case. I'd keep it to something like 8GB (unless you're doing very specific stuff that are memory consuming) and let the

Re: Hive connection Pool error

Hi, Could you share additional details about the processor/CS configuration as well? Thanks 2018-05-30 7:03 GMT+02:00 Koji Kawamura : > Hello, > > Although I encountered various Kerberos related error, I haven't > encountered that one. > I tried to reproduce the same error by changing Kerberos

Re: Only get file when a set exists.

I'll let Koji give more information about the Wait/Notify, he is clearly the expert here. I'm just jumping in regarding your "and when viewing the queue, the dialog states that the queue is empty.". You're seeing this behavior because, even though the UI shows some flow files in the queue, the

Re: User, Group in LDAP appear to be unknown to PutFile

d the groups to which user nifi has > membership within the ldap. > > On Wed, May 30, 2018 at 4:37 AM, Pierre Villard < > pierre.villard...@gmail.com> wrote: > >> Hi Jim, >> >> LDAP for authentication and authorizations in NiFi has nothing to do with >&

Re: User, Group in LDAP appear to be unknown to PutFile

case I can > indeed resolve at the OS level. That seems like a good thing to me. > > On Wed, May 30, 2018 at 7:53 AM, Pierre Villard < > pierre.villard...@gmail.com> wrote: > >> It depends how your OS is configured, you could leverage tools like SSSD >> to resolve

Re: User, Group in LDAP appear to be unknown to PutFile

ame(owner) within the PutFile is where I believe the > failure is rooted, but I do not understand how that function executes its > lookup. I'm trying to dig deeper into that now. > > On Wed, May 30, 2018 at 8:30 AM, Pierre Villard < > pierre.villard...@gmail.com> wrote: > >&g

Default scheduling

Hi there, Since few versions already, it's now possible to define default values for scheduling of the components using a dedicated annotation. Example @DefaultSchedule(strategy = SchedulingStrategy.TIMER_DRIVEN, period = "1 min") I'm wondering if the community would be OK about setting a

Re: Fetch Contents of HDFS Directory as a Part of a Larger Flow

Hi Shawn, If you know the path of the files to retrieve in HDFS, you could use FetchHDFS processor. If you need to retrieve all the files within the directory created by Hive, I guess you could list the existing files calling the REST API of WebHDFS and then use the FetchHDFS processor. Not sure

Re: FetchSftp with user/password dont move the file

Hi Carlos, Is the user allowed to move the file on the FTP server? Are the permissions on the file correct? Pierre 2017-10-20 12:07 GMT+02:00 Carlos Manuel Fernandes (DSI) < carlos.antonio.fernan...@cgd.pt>: > > > Hi, > > > > I configured a fetchSftp with username/password (no privatekey

Re: NiFi 1.3.0 - java.io.UTFDataFormatException: encoded string too long: 75504 bytes - UpdateAttribute processor

Hi, I think you're hitting the maximum size of the data you can store in a single attribute. How big is the information you want to store? What is your use case? Please note that attributes shouldn't be used to store a lot of content as this information is stored in NiFi's memory. Pierre

Re: PutHiveStreaming + Hive 3.0

Hi Mike, By any chance, could you share the full stack trace from nifi-app.log? Thanks, Pierre 2018-07-30 11:19 GMT+02:00 Michal Tomaszewski : > Hello, > Is PutHiveStreaming processor working with Hive 3.0? > We installed Hortonworks HDP 3.0, compiled newest NiFi 1.8 snapshot and > got errors:

Re: Title: Feature needed - ConvertJSONToCSV processor

>> Thanks in advance >>> >>> On Tue, Aug 7, 2018 at 4:31 PM Mahendra prabhu >>> wrote: >>> >>>> Hi Pierre, >>>> >>>> I need to move nested json data into SQL table after using expression >>>> language. So I t

Re: NiFi processor to execute a Java class

Quick remark: just wondering if (InputStream, OutputStream, Map) would be a better approach to consider flow files attributes + dynamic properties configured at processor level. Pierre 2018-08-08 17:06 GMT+02:00 Matt Burgess : > Vitaly, > > I am indeed back from vacation and I started on the

Re: [EXT] Re: Get all Processors

Hi Karthik, Are you running a cluster or standalone NiFi? What's your default read timeout value in nifi.properties? I believe the default value is a bit low when you start reaching thousands of processors. nifi.cluster.node.read.timeout=5 sec Pierre 2018-08-14 0:12 GMT+02:00 Karthik

Re: Title: Feature needed - ConvertJSONToCSV processor

Hi, I believe it can be closed because it's doable with ConvertRecord processor (which is the recommended approach). Does it answer your requirement ? Thanks, Pierre 2018-08-07 11:43 GMT+02:00 Mahendra prabhu : > Hi Folks, > > Is there is plan to include this feature or dropped - >

Re: CSV Illegal Initial Character

Hi Shawn, Sounds like a legitimate ask, could you file a JIRA for that? Thanks, Pierre Le lun. 20 août 2018 à 16:05, Shawn Weeks a écrit : > That option is ignored if you're deriving the schema from the header. > Since in the CSV standard supports quoting everything then having the > header

Re: GenerateTableFetch -> RPG -> ExecuteSQL fetching duplicate records from Netezza but the count is same.

Hi, What's the configuration of the GTF processor? Is data written to the source table while executing the workflow? How do you check for duplicate rows in Hive? Thanks 2018-07-20 15:12 GMT+02:00 Mohit : > Hi all, > > I am fetching data from Netezza using GenerateTableFetch -> RPG -> >

Re: Reporting connection timeout from InvokeHTTP Processor.

Hi, There are two things here. What is suggested by Ed is in case the server you're requesting is responding with a Timeout. But the processor itself implements a "read timeout" to give up after a given time. In this case, it won't generate a flow file but will just generate a bulletin. I'd

  1   2   3   >