I was able to replicate this on Minio too and have a fix for it. The change is
trivial but it needs more testing.
-joey
> On Aug 17, 2021, at 8:22 AM, Vibhath Ileperuma
> wrote:
>
>
> Hi Joey,
>
> It happens only for 0B files.
> Sorry for the confusion caused.
>
> Vibhath.
Hi Joey,
It happens only for 0B files.
Sorry for the confusion caused.
Vibhath.
:
>>>> Vibhath, there was a change to allow reading by ranges that I’m wondering
>>>> if is causing this.
>>>>
>>>> When you say small, exactly how small are the files?
>>>>
>>>> -joey
>>>>
>>>>>> O
small, exactly how small are the files?
>>>
>>> -joey
>>>
>>> On Aug 16, 2021, at 10:53 PM, Vibhath Ileperuma <
>>> vibhatharunapr...@gmail.com> wrote:
>>>
>>>
>>> Hi All,
>>>
>>> I'm using Nifi 1.14.0 t
hen you say small, exactly how small are the files?
>>
>> -joey
>>
>> On Aug 16, 2021, at 10:53 PM, Vibhath Ileperuma <
>> vibhatharunapr...@gmail.com> wrote:
>>
>>
>> Hi All,
>>
>> I'm using Nifi 1.14.0 to fetch a set of files fro
t; I'm using Nifi 1.14.0 to fetch a set of files from a S3 bucket. The
> 'FetchS3Object' processor throws the following error only for small files.
> I tried to fetch the same set of files using Nifi 1.13.2. There was no
> issue with 1.13.2. Can you please let me know a way to overcome
et of files from a S3 bucket. The
> 'FetchS3Object' processor throws the following error only for small files. I
> tried to fetch the same set of files using Nifi 1.13.2. There was no issue
> with 1.13.2. Can you please let me know a way to overcome this issue.
>
> FetchS3Object[Id:
Hi All,
I'm using Nifi 1.14.0 to fetch a set of files from a S3 bucket. The
'FetchS3Object' processor throws the following error only for small files.
I tried to fetch the same set of files using Nifi 1.13.2. There was no
issue with 1.13.2. Can you please let me know a way to overcome this issue
Using Nifi 1.11.4, the FetchS3Object processor occasionally throws the
following error:
routing to failure: com.amazonaws.SdkClientException: Unable to execute HTTP
request: The target server failed to respond.
The full stack trace looks like:
com.amazonaws.SdkClientException: Unable
you want it.
>
>
>
> Best,
>
> Keren
>
>
>
> *From: *James Wing <jvw...@gmail.com>
> *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Date: *Thursday, October 6, 2016 at 12:43 PM
> *To: *"users@nifi.apache.org" <users@nifi.
Selvam,
Can you describe what is happening when you run FetchS3Object? Does the
processor start? Are flowfiles routed to the failure relationship? Are
there error messages?
To test your configuration, I recommend using ListS3 to get flowfiles with
attributes referencing the S3 objects
in the FetchS3Object component.
See this link for some additional info:
http://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-sharing-logs-assume-role.html
I have communication working manually on the Nifi machines in VPC A when I use
the AWS CLI. The process is as follows:
1. Run sts
On Tue, Jan 12, 2016 at 7:20 PM, Adam Lamar <adamond...@gmail.com> wrote:
> Russell,
>
> Sorry - I meant that I use the FetchS3Object processor, not ListS3.
Ah, that makes more sense, thanks.
> You can
> set the Object Key property with the name of the key you want to down
nyxpoint.com>
> wrote:
> > Ha ha! Well that would do it! :)
> >
>
> I don't know what that would "do" other than confirm that the
> FetchS3Object processor shipped
> with v0.4.1 needs its doc to reflect the fact it's not yet useable
> untl a ListS3*
Russell,
Sorry - I meant that I use the FetchS3Object processor, not ListS3. You
can set the Object Key property with the name of the key you want to
download. This property supports the expression language, so the object
key can be sourced from each flowfile.
I'm unsure if there is a good
2, 2016 at 9:16 PM, Russell Whitaker <
> russell.whita...@gmail.com> wrote:
>
>> On Tue, Jan 12, 2016 at 6:11 PM, Corey Flowers <cflow...@onyxpoint.com>
>> wrote:
>> > Ha ha! Well that would do it! :)
>> >
>>
>> I don't know what that would
I'm running v0.4.1 Nifi, and seeing this (taken from nifi-app.log,
also seeing on mouseover of the "!" icon on the processor on the
canvas):
2016-01-12 17:08:50,357 ERROR [NiFi Web Server-18]
o.a.nifi.groups.StandardProcessGroup Unable to start
FetchS3Object[id=f4253204-a2e2
unning v0.4.1 Nifi, and seeing this (taken from nifi-app.log,
> also seeing on mouseover of the "!" icon on the processor on the
> canvas):
>
> 2016-01-12 17:08:50,357 ERROR [NiFi Web Server-18]
> o.a.nifi.groups.StandardProcessGroup Unable to start
> FetchS3Objec
quot;ListS3Object" processor type which
might hypothetically populate
attributes for FetchS3Object to act upon." I should have made this
doubly explicit that I checked
in the processor creation dialogue.
Also, this:
https://mail-archives.apache.org/mod_mbox/nifi-users/201510.mbo
s processor but I believe it is looking for
>> the S3 list processor to generate the list of objects to fetch. Did
>> you try that yet?
>
> I mentioned this: "There's no "ListS3Object" processor type which
> might hypothetically populate
> attributes for FetchS
On Tue, Jan 12, 2016 at 6:11 PM, Corey Flowers <cflow...@onyxpoint.com> wrote:
> Ha ha! Well that would do it! :)
>
I don't know what that would "do" other than confirm that the
FetchS3Object processor shipped
with v0.4.1 needs its doc to reflect the fact it's not yet
I understand that FetchS3Object processor requires an incoming FlowFile to
trigger it. The problem is that FetchS3Object emits a RECEIVE provenance event
for the existing FlowFile. That event causes following error when I try to open
the lineage chart for a simple flow: GenerateFlowFile
ck.com>> wrote:
I understand that FetchS3Object processor requires an incoming FlowFile to
trigger it. The problem is that FetchS3Object emits a RECEIVE provenance event
for the existing FlowFile. That event causes following error when I try to open
the lineage chart for a simple flow: Gene
Nifi users - FetchS3Object processor do not have option to specify Object file
path, how do you set that ?
Eg : s3:
Thanks,
-Chakri
The information contained in this transmission may contain privileged and
confidential information. It is intended only
Chakri,
The Amazon docs explain here
<http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html> that
S3 doesn't support pathes within a bucket, but you can embed path-like
naming into the "Object Key" parameter to FetchS3Object.
For your example, you would use bucket
Chakri,
The FetchS3Object processor does not act as a source directly and is
instead driven by incoming FlowFiles. The use case was that one could
utilize Expression Language to interact with buckets and paths in a dynamic
fashion. Alternatively, if you are looking to just grab a specific file
Date: Thursday, October 8, 2015 at 11:22 AM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>"
<users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: FetchS3Object processor
Chakri,
The FetchS3Object processor does not act as a source directly and
There is already a ticket (NIFI-840
<https://issues.apache.org/jira/browse/NIFI-840>) in the hopper to create a
ListS3Objects processor that can track bucket contents and trigger
FetchS3Object.
In the meantime, if you want it triggered automatically (as opposed to
generate flow file) y
e: Thursday, October 8, 2015 at 11:39 AM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>"
<users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: FetchS3Object processor
There is already a ticket
(NIFI-840<https://issues.apache.org/jira/browse/NIFI-
29 matches
Mail list logo
