Re: GetKafka maximum fetch size

2016-10-13 Thread Jeremy Farbota
max size? > > Thanks in advance. > -- [image: Payoff, Inc.] <http://www.payoff.com/> Jeremy Farbota Software Engineer, Data jfarb...@payoff.com • (217) 898-8110 <(949)+430-0630> I'm a Storyteller. Discover your Financial Personality! <https://www.payoff.com/quiz> [ima

provenance & content repos re infosec

2016-10-20 Thread Jeremy Farbota
igurable server wide? Has there ever been thought to enable encryption at rest of the provenance repo to deal with situations like mine? Thanks in advance. -- [image: Payoff, Inc.] <http://www.payoff.com/> Jeremy Farbota Software Engineer, Data jfarb...@payoff.com • (217) 898-8110 <(949)

Re: provenance & content repos re infosec

2016-10-20 Thread Jeremy Farbota
collaboration on this moving forward. > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Oct 20, 2016, at 2:03 PM, Jeremy Farbota wrote: > > Hello, > > I'm using NiF

Re: How does Nifi ingest large files?

2016-10-27 Thread Jeremy Farbota
liance upon, this information by persons or >> entities other than the intended recipient is prohibited. If you received >> this in error, please contact the sender and delete the material from any >> computer. >> > > -- [image: Payoff, Inc.] <http://www.payof

Re: How does Nifi ingest large files?

2016-10-27 Thread Jeremy Farbota
gt; > On Thu, Oct 27, 2016 at 11:58 AM, Joe Witt wrote: > >> That is correct. >> >> Thanks >> Joe >> >> On Thu, Oct 27, 2016 at 11:55 AM, Jeremy Farbota >> wrote: >> >>> Bryan, >>> >>> If I have the content repo implem

Re: Data SLA

2016-12-06 Thread Jeremy Farbota
> the files not being there in time, and/or the files not following the > correct naming conventions? > > If not, are there any other tools that tie into Nifi to provide help on > the Data SLA? > > Thanks, > Richard > > > -- [image: Payoff, Inc.] <h

Re: Pull big size HTTP responses.

2016-12-12 Thread Jeremy Farbota
e it is processed, or is there a possibility to write this > content to disk while it gets downloaded? > > Thanks, > Pablo. > -- [image: Payoff, Inc.] <http://www.payoff.com/> Jeremy Farbota Software Engineer, Data jfarb...@payoff.com • (217) 898-8110 <(949)+430-0

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Jeremy Farbota
Internet but I don't think we can/should put that > >> unless folks volunteer to have this on the page. So please let us > >> know if you're interested in your company/organizational use of NiFi > >> being mentioned there. > >> > >> Thanks > >> Joe > -- *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110>

Re: Change data capture processor

2017-04-12 Thread Jeremy Farbota
AM Buntu Dev wrote: >> >>> Is the new CaptureChangeMySQL processor available in 1.2 and is there >>> any documentation available for this processor? >>> >>> https://issues.apache.org/jira/browse/NIFI-3413 >>> >>> >>> Thanks! >>> >> > -- *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110>

Re: Change data capture processor

2017-04-12 Thread Jeremy Farbota
ete at this time. However there are some reusable interfaces and > classes that should enable the implementation of other CaptureChange > processors. > > Regards, > Matt > > On Apr 12, 2017, at 1:44 PM, Jeremy Farbota wrote: > > Wow, this is really exciting! > > Co

upgrading cluster from 1.0.0 to 1.3.0

2017-06-15 Thread Jeremy Farbota
e guide someone for clusters on Wiki? We have kafka consumers and other maintenance processes that are running in production so we'd like to make the change without messing with the state of those consumers if possible. Kindly, [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data

Re: upgrading cluster from 1.0.0 to 1.3.0

2017-06-16 Thread Jeremy Farbota
Thanks, after I fired this email I saw that. We'll try a couple different things. I will assume this will not work and will switch the consumers using a route with partition-offsets. If there is any other information, please let me know. [image: Payoff, Inc.] *Jeremy Farbota* Software Eng

Re: issues with Kafka ConsumeKafka_0_10 (1.0.0)

2017-06-23 Thread Jeremy Farbota
I see. That makes some sense. I've been reading about the improvements and it is good to reaffirm the new version will likely help us out a lot. We do not plan on getting this far behind again. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com

Re: issues with Kafka ConsumeKafka_0_10 (1.0.0)

2017-06-23 Thread Jeremy Farbota
off, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110> On Fri, Jun 23, 2017 at 12:46 PM, Joe Witt wrote: > I get it. If there are curveballs with upgrades that you run into and > ideas you have to make it easier please let us know.

Re: Encryption and Hashing in Nifi

2017-06-27 Thread Jeremy Farbota
Andy, Just wanted to note that this would be really useful for us if this was in a processor as well. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110> On Tue, Jun 27, 2017 at 10:29 AM, Andy LoPresto wrote: &g

Re: upgrading cluster from 1.0.0 to 1.3.0

2017-06-29 Thread Jeremy Farbota
this there was a reply with a link to a thread that does not provide any additional info about how to bring the cluster up with the new version. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110> On Thu, Jun 15, 2017

Re: upgrading cluster from 1.0.0 to 1.3.0

2017-06-29 Thread Jeremy Farbota
Andy, Thanks a ton. Huge help. We will report back. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110> On Thu, Jun 29, 2017 at 5:20 PM, Andy LoPresto wrote: > I believe I missed a crucial file — if you are

Re: upgrading cluster from 1.0.0 to 1.3.0

2017-07-06 Thread Jeremy Farbota
g etl. Thanks again! [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110> On Fri, Jun 30, 2017 at 6:04 AM, Matt Gilman wrote: > Thanks for the great summarization Andy. Just wanted to clarify one small > detail. T

FTPS support in FTPTransfer (List/Fetch/Get/PutFTP) in NiFi 1.3

2017-07-17 Thread Jeremy Farbota
.java [3] https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPSClient.html [4] https://commons.apache.org/proper/commons-net/examples/ftp/FTPClientExample.java [5] https://issues.apache.org/jira/browse/NIFI-2278 [image: Payoff, Inc.] *Jeremy Farbota* Software Eng

Re: FTPS support in FTPTransfer (List/Fetch/Get/PutFTP) in NiFi 1.3

2017-07-17 Thread Jeremy Farbota
Andy, Thanks for the response. I will mess with execute script and update. I will consider posting for help in Dev. This does seem like something that'd be useful and fits NiFi's strengths. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com

Re: OAuth1.0

2017-08-09 Thread Jeremy Farbota
Jeremy, This is exciting news! [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com On Wed, Aug 9, 2017 at 3:15 AM, Jeremy Dyer wrote: > Karthik, > > I should have the code finished shortly for the JIRAs that Pierre > referenced. I&#x

Re: HDFS Environments

2017-08-17 Thread Jeremy Farbota
: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110> On Thu, Aug 17, 2017 at 8:48 AM, Steve Champagne wrote: > Hello, > > How would I handle environment separation in HDFS? My initial thought was > to use a direct

accessing environment variables via expression language

2017-08-17 Thread Jeremy Farbota
. The variables come back when we echo them in shell as the nifi run user. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217) 898-8110 <+2178988110>

ListSFTP/FetchSFTP not running after bad password

2017-08-24 Thread Jeremy Farbota
in cluster? I'm hoping to continue using ListSFTP for other things on prod but the processors are not able to be configured or ran at this time on that cluster. When I try an create new processes, they do not respond. Thanks in advance! [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com

Re: ListSFTP/FetchSFTP not running after bad password

2017-08-24 Thread Jeremy Farbota
No problem, I will work on it. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com On Thu, Aug 24, 2017 at 11:38 AM, Joe Witt wrote: > Jeremy > > Can you please file a JIRA to cover this scenario that you're observing. > If yo

NiFi cluster sluggish...fine after VMs rebooted

2017-09-12 Thread Jeremy Farbota
s when there are files heavily queued so we can get alerted when it's happening. Thanks in advance. [1] http://apache-nifi.1125220.n5.nabble.com/Rate-of-the-data-flow-is-exceeding-the-provenance-recording-rate-Slowing-down-flow-to-accomodate-td9213.html [image: Payoff, Inc.] *Jeremy Farb

Re: NiFi cluster sluggish...fine after VMs rebooted

2017-09-12 Thread Jeremy Farbota
x27;ve ran processes before where I've had over a million flow files after a split text and did not have this issue. The dedicated heap is 26g for each of these nodes. It usually is around 3g and it will spike if we get a huge dump from kafka but I rarely see it go over 12g. [image: Payo

Re: ExecuteSQL question: how do I stop long running queries

2017-09-29 Thread Jeremy Farbota
here's a better way. Recently had some PutSQL processor (1.3.0) that hung up after I killed it on the db side in prod. Had to restart the service which was a bit of a paid in prod. Kindly, Jeremy [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com