Hi Martijn,
I'm not an expert on Jython, but if you already have a python script
using boto3 working fine, then I'd suggest using ExecuteStreamCommand
instead.
For example:
- you can design the python script to print out JSON formatted string
about listed files
- then connect the outputs to
Hello,
Did you try setting 'Max Rows Per Flow File' at ExecuteSQL processor?
If the OOM happened when NiFi writes all results into a single
FlowFile, then the property can help breaking the result set into
several FlowFiles to avoid that.
Thanks,
Koji
On Fri, Sep 21, 2018 at 3:56 PM Dnyaneshwar
I figured it out finally, it was the state management file.
From: Varun Tomar
Reply-To: "users@nifi.apache.org"
Date: Monday, September 24, 2018 at 4:07 PM
To: "users@nifi.apache.org"
Subject: Re: Secure NiFi cluster on kubernetes.
Same issue even with persistent disks I don’t have users.xml
Same issue even with persistent disks I don’t have users.xml any other
suggestion.
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: nifi
spec:
serviceName: nifi
replicas: 3
podManagementPolicy: Parallel
updateStrategy:
type: RollingUpdate
template:
metadata:
You could do it with an external scheduler, but my gut would tell me that there
is probably a way to structure your flow in a way that satisfies your batch
requirement, without having to manage a scheduler.
-Alexandre
-Original Message-
From: Nathan Gough [mailto:thena...@gmail.com]
Typically I would not expect to schedule dataflows in NiFi as it's not the
ideal place for data to stay sitting. For running scheduled batch jobs as you
describe I would expect the data to be constantly flowing to date/time based
directories on HDFS. This allows data to be stored in a place
I will check, thanks. But what about if I would like to run as a cluster? I
cannot follow that approach right?
On Mon, 24 Sep 2018 at 12:59 Mike Thomsen wrote:
> hostname: nifi
>
> Under the nifi declaration
> On Mon, Sep 24, 2018 at 11:07 AM Juan Pablo Gardella <
> gardellajuanpa...@gmail.com>
hostname: nifi
Under the nifi declaration
On Mon, Sep 24, 2018 at 11:07 AM Juan Pablo Gardella <
gardellajuanpa...@gmail.com> wrote:
> How?
>
> On Mon, 24 Sep 2018 at 11:31 David Gallagher
> wrote:
>
>> Hi – not sure if it helps, but you can set a static hostname in your
>> docker-compose.
>>
How?
On Mon, 24 Sep 2018 at 11:31 David Gallagher
wrote:
> Hi – not sure if it helps, but you can set a static hostname in your
> docker-compose.
>
>
>
> Thanks,
>
>
> Dave
>
>
>
> *From:* Juan Pablo Gardella
> *Sent:* Sunday, September 23, 2018 3:43 PM
> *To:* users@nifi.apache.org
>
Hi – not sure if it helps, but you can set a static hostname in your
docker-compose.
Thanks,
Dave
From: Juan Pablo Gardella
Sent: Sunday, September 23, 2018 3:43 PM
To: users@nifi.apache.org
Subject: Nifi with docker and LDAP
Hi all,
I'm using Nifi with docker and it's secure.
I'm facing
Hi,
I don't know what the etiquette on a mailing list is for this, but I'd like to
bump my original question.
Perhaps it's good to add that many of our flows are batch loads and therefore
depend on a schedule to run, once.
Does anyone have experience with remote scheduling in NiFi or do you
11 matches
Mail list logo
