Re: Github write permission

u can make commits with your apache.org if you want to. > > On Wed, Jul 20, 2016 at 11:20 AM, Koji Kawamura <ijokaruma...@gmail.com> > wrote: > >> Hi Aldrin, >> >> Thank you for your prompt reply. >> I understand that I need to commit to the ASF repo. >> &

Github write permission

Hello team, I've just become a NiFi committer from today, got so excited. Thank you for everyone who have been supporting me to made contributions! I was able to setup my email, ssh, and made my first commit to nifi-site project today. However, when I tried to push my commit to the nifi-site on

Re: [ANNOUNCE] New Apache NiFi Committer Koji Kawamura

attyb...@gmail.com> mattyb...@gmail.com>> wrote: >> >> Congratulations and welcome Koji! Your contributions are greatly >> appreciated and I'm looking forward to more in the future! >> >> Regards, >> Matt >> >> On Tue, Jul 19, 2016 at 6:55 PM, Tony

Re: Github write permission

. Koji On Wed, Jul 20, 2016 at 11:56 PM, Aldrin Piri <aldrinp...@gmail.com> wrote: > Hi Koji, > > GitHub is only a mirror of the ASF repos at this time. All commits must be > pushed to the ASF repositories and then are subsequently pushed to the > GitHub repos. > On Wed, Ju

Re: Spring JdbcTemplate, SimpleJdbcCall and DataSource

Hi Toivo, Sorry for the delayed response, I reviewed the PR and posted few comments. Please check the PR. https://github.com/apache/nifi/pull/1417 Thanks, Koji On Fri, Jan 20, 2017 at 4:51 PM, Toivo Adams wrote: > Please, can anyone review PR? >

Re: Apache NIFI Installation on Windows (No Hadoop Installed)

Hi Naresh, When I researched about NiFi authentication mechanism before, Client certificate is the only way for Site-to-Site protocol to authenticate an user who made a Site-to-Site request. You might find this post useful about how NiFi AuthN works.

Re: Discuss: Process Group as a Process / "Single Threaded" Process Group

I searched JIRA for new ControllerService suggestion that works as a MapCacheClient and can be used locally, so I created one, Add LocalMapCacheService https://issues.apache.org/jira/browse/NIFI-3488 On Thu, Feb 16, 2017 at 1:46 PM, Joe Witt wrote: > definitely Koji's route

Re: Discuss: Process Group as a Process / "Single Threaded" Process Group

Hi Peter, Joe Recently, I've worked on NIFI-3452 "Add Wait processor Wait Mode property". Using Wait/Notify combined with Back-pressure FlowFile count 1, it's possible to limit only one FlowFile to be processed in a part of flow. It's already merged into the master. Example is available here:

Re: [VOTE] Release Apache NiFi 1.1.2

+1 (non-binding) Verified signature, checksums Contrib check build with Oracle jdk 1.8.0_111 on macOS Sierra. Tested Site-to-Site connectivity (RAW and HTTP) among following NiFi environments: A. Standalone Non-secured B. Standalone Secured C. Cluster Non-secured D. Cluster Secured On Mon, Feb

Re: [VOTE] Release Apache NiFi 0.7.2

+1 (non-binding) Built from src on OSX Sierra 10.12.3 (16D32), all tests pass Verified sigs and hashes. Ran sample flow. On Mon, Feb 20, 2017 at 5:13 AM, Bryan Rosander wrote: > +1 (non-binding) > > Verified signature, hashes > Contrib-check build against fresh local repo

Re: [ANNOUNCE] New Apache NiFi Committer Jeff Storck

Congratulations Jeff! On Wed, Feb 22, 2017 at 7:52 AM, Andre wrote: > Welcome aboard Jeff! Well deserved > > On Wed, Feb 22, 2017 at 6:41 AM, Aldrin Piri wrote: > >> On behalf of the Apache NiFI PMC, I am very pleased to announce that Jeff >> Storck has

Re: Build failure on Mac OS X with DS_STORE files

nc and they seem to be live. --aldrin On Fri, Feb 10, 2017 at 7:43 PM, Koji Kawamura <ijokaruma...@gmail.com> wrote: > Thanks Otto, I reviewed pr1497 and merged it. > For the getting started page, I assume you were referring this quick start. > https://nifi.apache.org/quickstart.ht

Re: How to get a ProcessorGroup's Version#

Hi Peter, GET /nifi-api/process-groups/{id} will return revision for a Process Group (without /flow/ in the path). # Revision (version & client) is returned curl http://localhost:8080/nifi-api/process-groups/252b08ac-015a-1000-e683-9ed122374494 # Use version to delete the PG curl -i -XDELETE

Re: Build failure on Mac OS X with DS_STORE files

gt; that >> test ), but I’m not sure how you all handle things like this. >> >> I have to ask, does anyone working on NiFi use a mac? The next problems I >> have are not with .DS_Store files or anything ‘mac centric’, they are > with >> Nashorn Java script types in the

Re: [VOTE] Establish Registry, a sub-project of Apache NiFi

+1 (non-binding) On Feb 11, 2017 5:37 AM, "Jennifer Barnabee" wrote: +1 binding Sent from my iPhone > On Feb 10, 2017, at 2:55 PM, Joe Skora wrote: > > +1 binding > > On Fri, Feb 10, 2017 at 2:09 PM, Peter Wicks (pwicks) >

Re: Build failure on Mac OS X with DS_STORE files

nchronized. On Sat, Feb 11, 2017 at 12:17 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > https://github.com/apache/nifi/pull/1497 > > > On February 9, 2017 at 09:19:21, Koji Kawamura (ijokaruma...@gmail.com) > wrote: > > Thanks! Please ping me when the PR is ready. >

Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Registry

Huge +1! I was excited to read the design documents :) Agree with flow versioning at ProcessGroup level. I don't know if this is helpful, but here is an experimental project of mine which tries to achieve the same goal, versioning ProcessGroup.

Re: Build failure on Mac OS X with DS_STORE files

Hi Otto, Thanks for reporting this. I personally haven't encountered this issue, but as described here [1], when I opened the directory that the test uses by Mac Finder application, and changed view as icon and move the icon position, then a .DS_Store file was created. I agree with your

Re: Build failure on Mac OS X with DS_STORE files

Thanks! Please ping me when the PR is ready. On Thu, Feb 9, 2017 at 11:15 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Sure - I see what you mean, that is a much better approach. > I will certainly do that. > > > > On February 9, 2017 at 09:02:05, Koji Kawamura

Re: Custom color for processors.

Hello Alessio, I am not an expert of NiFi UI, but tried to find where to customize to apply a default color. Here is the Javascript function that creates new Processor instance by sending a REST request to NiFi Web API:

Re: [DISCUSS] Idea for visual access control policy application

Hi Andy, I like this idea, too! Enabling security policy for multiple components is definitely useful. While I was thinking about the security zone concept, I was wondering if components those share the same level of security policy stay close in 2 Dimensional position on a flow. Maybe those

Re: nifi 1.0 extending a standard processor

Hello, Have you been able to find a way to extend the standard processors? I was wondering what would be the proper way to do that. So I tested few things: 1. Create new custom nar which depends on nifi-standard-processors First, I tried to add new nar bundle, let's say "nifi-stdpllus-bundle"

Re: [ANNOUNCE] New Apache NiFi PMC Member - Matt Burgess

Congratulations! Matt!! On Sat, Oct 8, 2016, 00:54 Oleg Zhurakousky wrote: > Matthew. . . congrats! > Ohh wait, sorry I am actually referring to you Burgess, not that other one > that is pounding your neighborhood at the moment. > Anyway, stay safe and congrats

Re: Nifi InvokeHttp

Hi Selvam, Did you find a solution for this? Although the inline image couldn't be displayed for me, I assume the problem is how to invoke InvokeHttp using each setSpec.n attribute, and the n can vary. If so, you could design your flow differently, by splitting a XML into multiple flow-files,

Re: [VOTE] Release Apache NiFi 1.0.0 (RC1)

Full clean build w/contrib check good. Ran standalone NiFi and secured standalone. -0.9 (non-binding) Because I found two issues during the test. 1. Confusing log message with tls-toolkit.sh "No hostnames specified" This is a trivial one, and this is NOT the reason why I did negative fraction

Re: Hello, I have a question about nifi and need help

Hello, Sorry to hear that you've encountered the issue. That is a known one, already fixed and will be released in the next version. https://issues.apache.org/jira/browse/NIFI-2824 Regards, Koji On Wed, Nov 9, 2016 at 12:16 PM, 万一 wrote: > I am a developer of nifi. I have a

Re: [VOTE] Release Apache NiFi 0.7.1 (RC1)

+1 (non-binding) - git tag and commit ID verified - Downloaded nifi-0.7.1-source-release.zip and verified hashes - Full build finished without issue - Application runs without issue on Mac with java 1.8.0_111-b14 - Found a trivial typo on release note wiki page, updated the wiki page:

Re: [VOTE] Release Apache NiFi 1.1.0 (RC1)

Tony, Joe, Sorry about the nifi-websocket-bundle Rat check issue, I should have added apache-rat-plugin exclude configuration in its pom.xml like other projects such as nifi-toolkit-tls does. Created a JIRA for that: https://issues.apache.org/jira/browse/NIFI-3103 I'll send a PR immediately. I

Re: [VOTE] Release Apache NiFi 1.1.0 (RC1)

+1 (non-binding) Ran full clean build w/contrib-check Built on OSX Ran a various test on secure/un-secure/cluster/standalone NiFi environments I really like the updated colorful UI icons! Thanks! On Thu, Nov 24, 2016 at 6:04 PM, Andrew Psaltis wrote: > +1

Re: [VOTE] Release Apache NiFi 1.1.0 (RC2)

+1 (non-binding) - Verified checksums - Successfully ran mvn clean install, on OS X and Windows Server 2016, with Oracle JDK 1.8.0_111-b14, mvn 3.3.9 - Run example data flows on OS X and Windows Building and testing on Windows went well without issue. I hope the InvokeScriptedProcessor test

Re: [VOTE] Release Apache NiFi MiNiFI C++ 0.1.0 (RC1)

+1 (non-binding) Build on RaspberryPi 2 without issue, took (30 - 40 minutes) Running sample conf.yml written in README worked without issue However, when I used minifi-toolkit-0.1.0 (included in MiNiFi java, which is under voting, too right now) to convert a NiFi flow template to conf.yml.

Re: [VOTE] Release Apache NiFi 1.1.1 (RC1)

+1 (non-binding) - Built and tested on OSX with contrib-check. - Ran followings with Standalone/Cluster/Secure/Plain NiFi environments - Test Consume/Publish with Kafka 0.10 using demarcator - Create, download, upload and instantiate template - Site-to-Site communication push/pull On Wed,

Re: [VOTE] Release Apache NiFi 1.0.1 (RC1)

+1 (non-binding). - Ran through the release helper - Ran some basic data flows (Java 1.8.0_111, Mac OS X 10.11.6) On Sun, Dec 18, 2016 at 6:28 AM, James Wing wrote: > +1 (non-binding). I ran through the release helper and did some basic > testing of the built binary (Java

Re: [ANNOUNCE] New Apache NiFi Committer Jeremy Dyer

Congratulations Jeremy!! Koji On Dec 20, 2016 9:26 PM, "Rob Moran" wrote: Thank you and congratulations Jeremy! Rob On Tue, Dec 20, 2016 at 7:14 AM, Joe Skora wrote: > Congrats Jeremy, welcome! > > On Tue, Dec 20, 2016 at 8:47 AM, Pierre Villard < >

Re: Help with ExtractText processor and Mime format files

Hi David, If you'd like to extract attributes such as 'To' and 'From', then ExtractEmailHeaders will work for you. ExtractEmailHeaders is added since NiFi 1.0.0. Thanks, Koji On Sun, Jan 15, 2017 at 6:30 AM, DAVID SMITH wrote: > Hi > I have received some mime

Re: Re: we finished a Chinesization & a Japanize version of Nifi 1.1

b.com/apache/nifi/pull/1604 , > the JIRA tickiet url : https://issues.apache.org/jira/browse/NIFI-3603 . > What should I do next ? Any suggestions are welcome . Thanks a lot . > > > Peter Wang > best wishes > ____ > wangj...@weiresearch.com

Re: Re: we finished a Chinesization & a Japanize version of Nifi 1.1

nothing for modified , the branch of NIFI-cn-v1 is the > Chinesization & a Japanize version of Nifi 1.1.0 ,the branch of > NIFI-cn-pro-v1 is our own processor (chineseName,spider,dameng). Any > suggestions are welcome . Thanks a lot . > > > Best regards > Petter Wang > __

Re: [VOTE] Release Apache NiFi nifi-nar-maven-plugin-1.2.0

+1 Release this package as nifi-nar-maven-plugin-1.2.0 Verified checksums and git commit id. Built NiFi using new nar plugin was successful. Created simple NiFi flow, confirmed it works as expected. On Thu, Mar 16, 2017 at 3:31 AM, Scott Aslan wrote: > Built sample NAR,

Re: we finished a Chinesization & a Japanize version of Nifi 1.1

, Mar 10, 2017 at 4:29 PM, Koji Kawamura <ijokaruma...@gmail.com> wrote: > Hi Dustin, > > This is awesome! I am a native Japanese NiFi developer and many people > wish to see local language in the UI. I am going to check the Japanese > NiFi UI out :) > > Thanks, > Koji

Re: we finished a Chinesization & a Japanize version of Nifi 1.1

Hi Dustin, This is awesome! I am a native Japanese NiFi developer and many people wish to see local language in the UI. I am going to check the Japanese NiFi UI out :) Thanks, Koji On Fri, Mar 10, 2017 at 1:26 PM, 王家昉 wrote: > Hi All: > We are glad to announce

Re: Trigger timer-driven processor to run from outside NIFI

Hi Ben, Just an idea, using ListenHTTP or HandleHTTPRequest (or whatever listener type processor you can use) in front of your processor might be helpful. You also need to change your processor to support incoming FlowFile as well if it doesn't currently. This way, the outside application can

Re: Adding properties

Hi, By looking at GetMongo source code and its docs, I believe it doesn't write specified 'Mongo Collection Name' to any attribute of outgoing FlowFiles. So you wouldn't be able to refer it from other processors at downstream of your flow. Please file a JIRA if this is needed.

Re: FlowFile position when transferred to Relationship.SELF

Hi, I think it puts back a FlowFile to its original position but update queued date as implemented here:

Re: Trigger timer-driven processor to run from outside NIFI

e NIFI REST API and it seems that I need to use > Groovy to do it(I don't understand Groovy), is there any Java related > examples which interact with NIFI via REST API? Thanks. > > Regards, > Ben > > 2017-07-14 13:49 GMT+08:00 Koji Kawamura <ijokaruma...@gm

Re: processors ListFile/ListSFTP do not store milliseconds in timestamp

brings back storing file identifiers into state, but only for those having the latest timestamp. Previously, it stores whole identifiers it processed. Could anyone review PRs above? Thanks Koji On Wed, Jun 21, 2017 at 2:01 PM, Koji Kawamura <ijokaruma...@gmail.com> wrote: > Thanks Joe

Re: whats new in nifi 1.2.0 / careful use of marks and being clear about non released features

Hi all, I should have written about the state of the project clearer, and be careful about talking these things. I am sorry that I spread it out without proper naming and disclaimer in the presentation. The presentation has been updated with a disclaimer page and title page now has "NOT RELEASED

Re: get controller service's configuration

Hi Ben, I'm not aware of ways to obtain configurations of a controller from a processor. Those should be encapsulated inside a controller service. If you'd like to create DataSource instance instead of just obtaining a connection, this discussion might be helpful:

Re: How to use ConvertRecord Processor

Hello, If you'd like to load data from CSV file to database table, 1.2.0 also has PutDatabaseRecord processor. Then you can configure PutDatabaseRecord to use CSVReader. Since you have your schema within incoming file as CSV header, no need to setup Schema Registry. However, "random number of

Re: [ANNOUNCE] New Apache NiFi PMC Member - Yolanda Davis

Congratulations Yolanda, well deserved! I have a great respect for your excellent work. On Thu, May 18, 2017 at 11:15 AM, Andy LoPresto wrote: > Congratulations Yolanda. Well deserved and very excited for your continued > excellent contributions to all aspects of the

Re: Which RestAPI can provide the throughput of a nifi cluster?

Hello, Throughput of a NiFi cluster can mean variety of things. Do you find NiFi's "summary" page useful? It can be accessed from the hamburger icon at right top corner. It provides stats for example In/Out size or Read/Write for each Processor, Input/Output port, ProcessGroups ... etc for the

Re: processors ListFile/ListSFTP do not store milliseconds in timestamp

art script for i in {1..10}; do touch ./test_$i; done > > Result: missing 68 and 40 files > > > In all tests listing.timestamp and processed.timestamp still not have > milliseconds > > > > Summary: > 1. Now much better than it was. Thanks Koji for good job! >

Re: Authorization problems of NiFi secured cluster

Hello Takanobu, If the issue doesn't happen with standalone mode, I assume it happens because the security policy does not allow NiFi node to "view the data". When a user sends a request to a node within a cluster, the node proxies the request to other nodes within the same cluster. I'd

Re: Authorization problems of NiFi secured cluster

the Node Identity before stating the cluster at the first > time, it seemed NiFi did not automatically create the policies and I needed > to add the Node Identity to the policy explicitly. > > Thanks again! > Takanobu > > -Original Message- > From: Koji Kawamura [mailt

Re: Authorization problems of NiFi secured cluster

I just created a brand-new secured cluster now. NiFi automatically created a policy "view the data" (and others) with the user defined as "Initial Admin Identity" and "Node Identity" in conf/authorizers.xml. It seems working as expected. Koji On Tue, Jun 27,

Re: [ANNOUNCE] New Apache NiFi PMC Member - Koji Kawamura

On May 26, 2017, at 7:18 PM, Andre <andre-li...@fucs.org> wrote: >> >> >> >> Congratulations Koji!!!! >> >> >> >> Cheers >> >> >> >> >> >> >> >> >> >>> On Sat,

Re: SplitText processor OOM larger input files

Hi Martin, Generally, NiFi processor doesn't load entire content of file and is capable of handling huge files. However, having massive amount of FlowFiles can cause OOM issue as FlowFiles and its Attributes resides on heap. I assume you are using 'Line Split Count' as 1 at SplitText. We

Re: [ANNOUNCE] New Apache NiFi Committer Drew Lim

Congratulations Drew!! On Thu, Jun 1, 2017 at 11:47 AM, Yolanda Davis wrote: > > Congratulations Drew! > > >> On May 31, 2017, at 4:20 PM, Bryan Rosander wrote: >> >> Congrats Drew! >> >> On Wed, May 31, 2017 at 3:51 PM, Andrew Psaltis

Re: Load Balancing in NiFi

Hi Martin, As Kevin mentioned, the recommended way of distribute files depends on your use-case. So, please take this as one possible example to address that challenge. While I was playing with the new RecordReader functionality added since 1.2.0, I came up with an idea to distribute filename

Re: NIFI-3641

Hi Mike, I like the idea of adding gRPC as an option for NiFi to communicate with other NiFi (s2s) or other server endpoint which can talk via gRPC. I had implemented HTTP for s2s before. It was not an easy task (at least for me) to make the new protocol align with existing terminology, behavior

Re: Extract the min and max date of the input file

Hello. Sorry for the late reply. If you are free to use the latest version, Apache NiFi 1.2.0 includes very powerful processors for your use cases: QueryRecords can query record using aggregate functions such as min, max, avg. A NiFi template as an example to calculate min, max and avg from a

Re: processors ListFile/ListSFTP do not store milliseconds in timestamp

Hello Roman, It seems the resolution of last modified timestamp depends on the file system implementation. https://stackoverflow.com/questions/3805201/how-to-get-ubuntu-file-timestamp-in-millisecond I reproduced the same behavior on OS X, which uses HFS that has the same limitation of resolution

Re: [VOTE] Release Apache NiFi 1.3.0

+1 (binding) - ran through release helper - ran clean build, test passed without issue - ran simple flow using standalone/non-secure/secure/clustered NiFi - confirmed few JIRAs data flow those had been addressed within 1.3.0 On Wed, Jun 7, 2017 at 2:15 AM, Pierre Villard

Re: NiFi 1.3.0 Remote ports "disabled" after a while?

Hi Joe, I haven't encountered that issue. Would you tell us more details? So that we can investigate it. 1. Which transport protocol are you using? RAW or HTTP 2. Is this a pull or push transfer? Pull = client NiFi pulls from remote OutputPort, Push = sends to remote InputPort 3. Is compression

Re: processors ListFile/ListSFTP do not store milliseconds in timestamp

be appreciated. Thanks, Koji On Tue, Jun 6, 2017 at 9:18 PM, Koji Kawamura <ijokaruma...@gmail.com> wrote: > Hi Roman, > > I think NIFI-3332 is probably related as I can see timestamps in logs > don't have milliseconds. > > I've been considering how we can support all corner ca

Re: [VOTE] Release Apache NiFi 1.2.0 (RC2)

+1 (non-binding) Full build with contrib check passed on Mac OS X Tested "Provenance Stream Record ReaderWriter" template, and other new features Thanks, Koji On Sun, May 7, 2017 at 7:43 AM, Richard St. John wrote: > +1 non-binding > Built RPM on CentOS 6 > Installed RPM on

Re: [VOTE] Release Apache NiFi MiNiFI C++ 0.2.0 (RC2)

+1 (non-binding) Full build and test finished successfully without any issue on OS X. Here are the things that I look forward in future improvements (didn't check existing JIRAs): -- 1. minifi.sh restart does not working?

Re: Custom processor UI exception

Hello Yuri, >From the stack trace (line numbers) I assume you are using NiFi 1.2.0. If so, I think NullPointer can be thrown if your custom processor returns null from its getRelationships() method. A list of possible relationships from a processor should be returned, if the processor doesn't

Re: Custom processor UI exception

cessor class which I was extending, > and misused init method. > > Thanks again! > > On 5/21/17, 9:38 PM, "Koji Kawamura" <ijokaruma...@gmail.com> wrote: > >>Hello Yuri, >> > >From the stack trace (line numbers) I assume you are using NiFi 1.2.

Re: NiFi 1.2.0 Record processors question

I've updated the JIRA description to cover not only embedded Avro schema but also ones such as derived from CSVReader. https://issues.apache.org/jira/browse/NIFI-3921 Thanks, Koji On Sat, May 20, 2017 at 4:14 AM, Joe Gresock wrote: > Yes, both of your examples help explain

Re: processors ListFile/ListSFTP do not store milliseconds in timestamp

: > Hi there, > > During digging into this issue, I found open issue in jira NIFI-3332 > <https://issues.apache.org/jira/browse/NIFI-3332> . Can it be related to my > situation with missed milliseconds? > > Thanks > Roman > > > Koji Kawamura-2 wrote >> Hello Ro

Re: [VOTE] Release Apache NiFi 0.7.4

+1 (binding) Verified gpg signature. Verified all hashes on source zipfile. Performed mvn clean install -Pcontrib-checks Builds cleanly, all tests pass on Mac OS Sierra (10.12.4) Oracle JDK 1.8.0_121-b13, maven 3.2.5 Ran simple flow and worked as expected. Thanks, Koji On Tue, Jun 6, 2017 at

Re: [VOTE] Release Apache NiFi 1.3.0

Hi Adam, Jeff, StandardFlowSynchronizerSpec relies on System.property so it can pass if other test sets 'nifi.properties.file.path'. I think I found how to fix the test class. I will create a JIRA and send a PR for it. Thanks, Koji On Tue, Jun 6, 2017 at 9:35 AM, Jeff wrote:

Re: [VOTE] Release Apache NiFi 1.3.0

Filed https://issues.apache.org/jira/browse/NIFI-4021 And PullRequest https://github.com/apache/nifi/pull/1892 On Tue, Jun 6, 2017 at 10:54 AM, Koji Kawamura <ijokaruma...@gmail.com> wrote: > Hi Adam, Jeff, > > StandardFlowSynchronizerSpec relies on System.property so it can pass

Re: Authorization problems of NiFi secured cluster

oes not work correctly. > > Thanks again, > Takanobu > > -Original Message- > From: Koji Kawamura [mailto:ijokaruma...@gmail.com] > Sent: Tuesday, June 27, 2017 5:59 PM > To: dev <dev@nifi.apache.org> > Subject: Re: Authorization problems of NiFi secur

Re: [EXT] Re: [DISCUSS} Closing in on a NiFi 1.4.0 release?

Hi Paul, I was able to reproduce the GenerateTableFetch processor issue reported by NIFI-4395. Please go ahead and provide a PR, I can review it. Thanks, Koji On Wed, Sep 20, 2017 at 1:10 PM, Paul Gibeault (pagibeault) wrote: > We have submitted this JIRA ticket: >

Re: Posting Multipart data

Hello Chris, Did you find a solution for this? I looked at the underlying Apache HttpComponents API, and found InputStreamBody class. InputStreamBody doesn't copy whole byte array, but it streams data from specified input stream to the output stream per 4kb. You may find this example useful, the

Re: [EXT] Re: Please refresh my memory on NAR dependencies

Peter, Matt, If the goal is sharing org.apache.nifi.csv.CSVUtils among modules, an alternative approach is moving CSVUtils to nifi-standard-record-util and add ordinary JAR dependency from nifi-poi-processors. How do you think? Thanks, Koji On Mon, Oct 16, 2017 at 12:17 PM, Peter Wicks (pwicks)

Re: Apache NiFI - Atlas Bridge Builders Needed

Hello Lawrance, I've been working on NIFI-3709, adding a Apache NiFi reporting task to report lineage to Apache Atlas. The original PullRequest did not use Provenance events, but I'm now adding it so that it can cover more use-cases. It's still under development and not ready to be reviewed

Re: Ingest data into SQL database using NiFi

Hi Tina, I wonder if the column name is the cause of that issue, because 'date' is a reserved keyword. I wonder whether ConvertJSONToSQL can wrap those columns with square brackets as shown in your example query. If possible, can you try to change the column name to different one such as

Re: Ingest data into SQL database using NiFi

The list of reserved keywords: https://docs.microsoft.com/en-us/sql/t-sql/language-elements/reserved-keywords-transact-sql On Tue, Sep 26, 2017 at 9:31 AM, Koji Kawamura <ijokaruma...@gmail.com> wrote: > Hi Tina, > > I wonder if the column name is the cause of that issue

Re: Ingest data into SQL database using NiFi

Hi Tina, I tested ExecuteSQL -> ConvertAvroToJSON -> ConvertJSONToSQL -> PutSQL flow with my SQL Server. It worked fine, I was able to copy rows from a table to another. One thing to note is that since you're using two different databases, you need to specify 'Catalog Name' and 'Schema Name' at

Re: [VOTE] Release Apache NiFi 1.4.0 (RC2)

+1 (binding) Release this package as nifi-1.4.0 Verified hashes, local build was successful on OS X, confirmed S2S communication with older versions. On Sat, Sep 30, 2017 at 9:27 AM, Andy LoPresto wrote: > +1 (binding) > > Build environment: Mac OS X 10.11.6, Java

Re: Ingest data into SQL database using NiFi

Hi Tina, Glad to hear you were able to get schema. The read size in ExecuteSQL is less because it's serialized with Avro in which data can be written efficiently, and gets bigger after ConvertJSONToSQL because each FlowFile has SQL statement in it. Which version of Apache NiFi are you using? If

Re: Request: Add my account to the JIRA contributors list

Hi Yuri, I've added you to JIRA contributor list, you should be able to assign yourself now. Thanks for your contributions to enhance NiFi UX! Koji On Wed, Oct 11, 2017 at 3:07 AM, Yuri <1969yuri1...@gmail.com> wrote: > Hello, > I'd like to be able to assign JIRA issues to myself. > > My JIRA

Re: [VOTE] Release Apache NiFi MiNiFi 0.3.0 - RC2

Hi Aldrin, I'm verifying the updated RC now. It's working nicely. Just a question before casting my vote. How was the source zip file is created? I am seeing minifi.exe and minifiw.exe are existing in

Re: [VOTE] Release Apache NiFi MiNiFi 0.3.0 - RC3

Thanks Aldrin for updating RC again, and to all devs who contributed to MiNiFi 0.3.0 release! I confirmed: - Hashes are correct - MiNiFi Windows Service works - MiNiFi Toolset works, NiFi template -> MiNiFi config yml - NiFi template -> C2 server -> MiNiFi PullHttpChangeIngestor works nicely +1

Re: NiFi 1.3 - PublishKafka_0_10 - A message in the stream exceeds the maximum allowed message size of 1048576 bytes

Hi Mayank, I've tried to reproduce the issue, but to no avail so far. PublishKafka_0_10 uses the specified Max Request Size as expected and I got the exception if incoming message size exceeds the configured size. And I was able to publish messages whose size is 2.08MB with 10MB Max Request Size.

Re: Run 1 instance of ExecuteStreamCommand constantly

Hi, If the script encounters a while(1) loop when it called from NiFi, then NiFi can not do anything until the loop ends. To achieve what you described (keep using the same instance of a script), I'd recommend to implement an API endpoint in that script, e.g. a simple REST endpoint to receive

Re: Proposal: standard record metadata attributes for data sources

Hi Mike, I agree with the approach that enrich provenance events. In order to do so, we can use several places to embed meta-data: - FlowFile attributes: automatically mapped to a provenance event, but as Andy mentioned, we need to be careful not to put sensitive data. - Transit URI: when I

Re: Put data to Elastic with static settings or index template

Hi Bobby, Elasticsearch creates index if it doesn't exist. I haven't tried it myself yet, but Elasticsearch's Index template might be useful to tweak default settings for indices those are created automatically.

Re: LookupService + flowfile attributes

ibutes as criteria by default. > I'll update the PR accordingly and make the new method default to the > existing one in all of the lookup services that are already there. > > On Sat, Jun 9, 2018 at 8:44 AM Mike Thomsen wrote: > >> https://issues.apache.org/jira/browse/NIFI-5287 &g

How do we want to add DeepLearning capability?

Hi all, A PR is submitted to add to do classification or prediction using a pre-built model with DeepLearning4J library (thanks @mans2singh!). https://github.com/apache/nifi/pull/2686 I found following things can/should be improved so that people can use it easier from a NiFi flow: - To utilize

Re: [VOTE] Release Apache NiFi Registry 0.2.0

+1 (binding) - Run through Release Helper Guide - Tested other database other than H2 - Tested Git persistence provider Few minor feedback: 1. Database user whose password is blank can not be used When I used HSQLDB, the default 'sa' user does not have password. If I configure a blank password

Re: LookupService + flowfile attributes

Thanks Mike for starting the discussion. Yes, I believe that will make LookupService and Schema access strategy much easier, reusable, and useful. What I was imagined is not adding new method signature, but simply copy certain FlowFile attributes into the coordinates map. We can add that at

Re: [EXT] Re: Primary Only Content Migration

There is an existing JIRA submitted by Pierre. I think its goal is the same with what Joe mentioned above. https://issues.apache.org/jira/browse/NIFI-4026 As for hashing and routing data with affinity/correlation, I think 'Consistent Hashing' is the most popular approach to minimize the impact of

Re: Unable to bring up NiFi user interface

Hello, > A black command window screen pops up for a brief second and then closes. Instead of double clicking run-nifi.bat button, you can run the bat file from command prompt. That way, the output of run-nifi.bat will stay in the command prompt and can help debugging what went wrong. 1. Open

Re: Using EL classes with just java Map objects

Hi Mike, In order to evaluate an ExpressionLanguage with Map containing variables, I used Query.prepare, to parse a query String into PreparedQuery. Following code snippet works without issue. Is that something you want to do? final Map map = Collections.singletonMap("name", "John Smith"); final

Re: Restrict WebUI Access based on IP

Hi Ruben, I am not aware of any configuration to do that at NiFi side, I believe NiFi doesn't have that. I usually do access control based on client IP addresses by FireWall. 'iptables' is the standard one for Linux. You can find many examples on the internet to configure iptables. If you are

Re: sFTP Question

Hi Anil, 1. I'd use MonitorActivity, too. Assuming you want to do something when there is no new files listed by ListSFTP at a scheduled time. Then you can add MonitorActivity in between ListSFTP and FetchSFTP. ListSFTP -> MonitorActivity --success--> FetchSFTP

Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.5.0

+1 (binding) - Verified hashes - Build and unit tests succeeded - Run simple flows to send data from MiNiFi CPP to NiFi - on Mac OS 10.13.4 I have a feedback on the release procedure. Apache Release distribution policy has following: "SHOULD NOT supply a MD5 checksum file (because MD5 is too

  1   2   >