How to pass custom attribute to a ListFile and putHDFS

2017-04-25 Thread Vdsa
I have a custom processor which sets a source and destination directories. I need to pass these attributes as a directory to ListFile and PutHDFS. I have added attribute in session like flowFile = session.putAttribute(flowFile, "source dir", source); and I am trying to access it using expressio

Re: Hostname does not match

2017-04-25 Thread Andy LoPresto
Joe, Sorry I missed this before you diagnosed it yourself. A couple of thoughts: * Yes, the SAN is supposed to take priority over the CN for hostname verification. This is specified in RFC 2818 [1] among others. * If the SAN doesn’t match but the CN and hostname do, that error is frustratingly

Re: puts3object super slow

2017-04-25 Thread pradeepbill
thanks Bryan, i took your hint, and looks like the network might be the culprit.I did a test and 95M took almost an hour. -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/puts3object-super-slow-tp15554p15556.html Sent from the Apache NiFi Developer List mai

Re: puts3object super slow

2017-04-25 Thread Bryan Rosander
Hi Pradeep, How does the upload speed compare to uploading without NiFi? Maybe try the enhanced uploader [1] to the same bucket from the same machine to see if the problem is internal to NiFi or maybe due to something else? [1] http://docs.aws.amazon.com/AmazonS3/latest/UG/enhanced-uploader.html

puts3object super slow

2017-04-25 Thread pradeepbill
Hello folks, I have a listenSyslogProcessor -> puts3object processor, and our network bandwidth is 1GBPS, but I see a super slow performance on the s3 writes,our s3 bucket has no policy, here are the stats Total Size written – 17MB Total Time for Queue to empty – 21 mins Min/Max/Mean – 0.0/9.29MB

Re: Making NiFi upgrade smoother

2017-04-25 Thread Adam Lamar
Great idea Yolanda!

Re: Making NiFi upgrade smoother

2017-04-25 Thread Andy LoPresto
Andre, Aldrin and I discussed something similar in the comments of NIFI-2974 last year but didn’t follow up on it. I think you’ve gotten sufficient support here to file a Jira and scope it out. I know as someone who is constantly rebuilding and redeploying the application, I have shortcuts to c

Re: Making NiFi upgrade smoother

2017-04-25 Thread u...@moosheimer.com
👍 Mit freundlichen Grüßen / best regards Kay-Uwe Moosheimer > Am 25.04.2017 um 18:16 schrieb Yolanda Davis : > > Hi Andre, > > Just wanted to add my thoughts on this. One of the things I have been > thinking and working through in creating utilities to automate upgrades > (see https://issues.a

Re: Making NiFi upgrade smoother

2017-04-25 Thread Yolanda Davis
Hi Andre, Realized I responded on the wrong thread... Just wanted to add my thoughts on this. One of the things I have been thinking and working through in creating utilities to automate upgrades (see https://issues.apache.org/jira/browse/NIFI-3663) is creating the ability to migrate to new conf

Re: Making NiFi upgrade smoother

2017-04-25 Thread Yolanda Davis
Hi Andre, Just wanted to add my thoughts on this. One of the things I have been thinking and working through in creating utilities to automate upgrades (see https://issues.apache.org/jira/browse/NIFI-3663) is creating the ability to migrate to new configurations/settings, especially in the instan

Re: Making NiFi upgrade smoother

2017-04-25 Thread u...@moosheimer.com
This is a great idea! Mit freundlichen Grüßen / best regards Kay-Uwe Moosheimer > Am 25.04.2017 um 17:03 schrieb Andre F de Miranda : > > dev, > > What do you think about the idea of allowing an user to configure overrides > for the "master" configuration files in order to smooth upgrades? > >

Re: Making NiFi upgrade smoother

2017-04-25 Thread James Wing
I think this is a great idea. It would also provide a standard integration point for the tls-toolkit command and similar utilities. On Tue, Apr 25, 2017 at 8:03 AM, Andre F de Miranda wrote: > dev, > > What do you think about the idea of allowing an user to configure overrides > for the "maste

Making NiFi upgrade smoother

2017-04-25 Thread Andre F de Miranda
dev, What do you think about the idea of allowing an user to configure overrides for the "master" configuration files in order to smooth upgrades? In my opinion candidates would be_ bootstrap.conf_ and _nifi.properties_ as they contain the links to all other files (logback.xml excluded). Upon

Making NiFi upgrade smoother

2017-04-25 Thread Andre
dev, What do you think about the idea of allowing an user to configure overrides for the "master" configuration files in order to smooth upgrades? In my opinion candidates would be_ bootstrap.conf_ and _nifi.properties_ as they contain the links to all other files (logback.xml excluded). Upon

Re: Closing in on a NiFi 1.2.0 release?

2017-04-25 Thread Russell Bateman
Many thanks to all the industrious folk working on this tool that's become indispensable to so many. The community has my enduring gratitude and admiration! Russ On 04/24/2017 07:16 PM, Joe Witt wrote: Robert, We're about 10 or so JIRAs away at this point and all of them have PRs and progres