specific types of data and processing, be sure to post a question back here
> and someone will be able to assist you.
> >
> > Cheers,
> > Kevin
> >
> > [1] https://nifi.apache.org/docs/nifi-docs/components/org.
> apache.nifi/nifi-standard-nar/1.2.0/org.apache.nif
;
> Thanks
>
> On Fri, Jun 2, 2017 at 3:32 PM, Martin Eden <martineden...@gmail.com>
> wrote:
> > Hi everyone,
> >
> > Simple flow in NiFi 1.2.0:
> > ListHDFS -> FetchHDFS -> PutHDFS
> >
> > Just moving files from one HDFS folder to anothe
Hi everyone,
Simple flow in NiFi 1.2.0:
ListHDFS -> FetchHDFS -> PutHDFS
Just moving files from one HDFS folder to another for evaluation purposes,
to see if NiFi can be used for this sort of ETL.
To benchmark I am doing is on a 50 x 1 GB input files dataset.
I am testing out with varying
e writing via PutHDFS.
>
> Andrew
>
> On Thu, Jun 1, 2017, 2:24 AM Martin Eden <martineden...@gmail.com> wrote:
>
> > Hi Joe,
> >
> > Thanks for the explanations. Really useful in understanding how it works.
> > Good to know that in the future t
ther
> use MergeContent to create data bundles of a given size then write
> those to HDFS.
>
> Thanks
> Joe
>
> On Wed, May 31, 2017 at 10:33 AM, Martin Eden <martineden...@gmail.com>
> wrote:
> > Hi Koji,
> >
> > Good to know that it can handle large
plitText splits into each line.
> This way, we can decrease number of FlowFiles at a given time
> requiring less heap.
>
> I hope this helps.
>
> Thanks,
> Koji
>
> On Wed, May 31, 2017 at 6:20 PM, Martin Eden <martineden...@gmail.com>
> wrote:
> > Hi all,
>
Hi all,
I have a vanilla Nifi 1.2.0 node with 1GB of heap.
The flow I am trying to run is:
ListHDFS -> FetchHDFS -> SplitText -> RouteOnContent -> MergeContent ->
PutHDFS
When I give it a 300MB input zip file (2.5GB uncompressed) I am getting
Java OutOfMemoryError as below.
Does NiFi read in
bucket was configured?
> Is it added to one of the project roles, explicitly granted on the bucket,
> or something else?
>
> Thanks,
>
> James
>
> On Fri, May 26, 2017 at 2:04 AM, Martin Eden <martineden...@gmail.com>
> wrote:
>
> > Hi all,
> >
> >
Hi all,
I have a Google Compute instance with a service account configured.
I have installed Nifi 1.2.0 on it.
In the Nifi UI I am describing a flow.
The first processor is a ListGCSBucket processor. It is pointing to a
bucket for which the service account is added as owner and I am not
