On Tue, Jan 12, 2016 at 6:11 PM, Corey Flowers <[email protected]> wrote: > Ha ha! Well that would do it! :) >
I don't know what that would "do" other than confirm that the FetchS3Object processor shipped with v0.4.1 needs its doc to reflect the fact it's not yet useable untl a ListS3* processor is implemented and included in the narfile for the distribution. Russell > Sent from my iPhone > >> On Jan 12, 2016, at 9:10 PM, Russell Whitaker <[email protected]> >> wrote: >> >>> On Tue, Jan 12, 2016 at 6:02 PM, Corey Flowers <[email protected]> >>> wrote: >>> I haven't worked with this processor but I believe it is looking for >>> the S3 list processor to generate the list of objects to fetch. Did >>> you try that yet? >> >> I mentioned this: "There's no "ListS3Object" processor type which >> might hypothetically populate >> attributes for FetchS3Object to act upon." I should have made this >> doubly explicit that I checked >> in the processor creation dialogue. >> >> Also, this: >> https://mail-archives.apache.org/mod_mbox/nifi-users/201510.mbox/%3cd23c06e8.ca0%[email protected]%3E >> >> "There is already a ticket >> (NIFI-840<https://issues.apache.org/jira/browse/NIFI-840>) >> in the hopper to create a ListS3Objects processor that can track >> bucket contents and trigger >> FetchS3Object." >> >> Oh god, it does appear that v0.4.1 ships with an implemented >> FetchS3Object processor but no >> List processor to feed it: >> >> https://issues.apache.org/jira/browse/NIFI-840 >> >> Status: unresolved >> >> Description: "A processor is needed that can provide an S3 listing to >> use in conjunction with FetchS3Object. This is to provide a similar >> user experience as with the HDFS processors that perform List/Get." >> >> I think this means I'm horked. And the Relationships section of the >> FetchS3Object doc is still wrong. >> >> Russell >> >> >>> Sent from my iPhone >>> >>>> On Jan 12, 2016, at 8:38 PM, Russell Whitaker <[email protected]> >>>> wrote: >>>> >>>> I'm running v0.4.1 Nifi, and seeing this (taken from nifi-app.log, >>>> also seeing on mouseover of the "!" icon on the processor on the >>>> canvas): >>>> >>>> 2016-01-12 17:08:50,357 ERROR [NiFi Web Server-18] >>>> o.a.nifi.groups.StandardProcessGroup Unable to start >>>> FetchS3Object[id=f4253204-a2e2-4ce6-ba09-9415e8024dca] due to {} >>>> java.lang.IllegalStateException: Processor FetchS3Object is not in a >>>> valid state due to ['Upstream Connections' is invalid because >>>> Processor requires an upstream connection but currently has none] >>>> >>>> Per: >>>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.aws.s3.FetchS3Object/index.html, >>>> FetchS3Object "Retrieves the contents of an S3 Object and writes it to >>>> the content of a FlowFile," which would seem to indicate this is an >>>> "edge" processor that doesn't expect a flowfile from an upstream >>>> processor. >>>> >>>> The "Tags" on the doc are: "Amazon, S3, AWS, Get, Fetch" >>>> >>>> The processor configuration settings themselves strongly indicate it >>>> expects to connect to S3 using the supplied >>>> credentials/bucket/objectkey settings, with no upstream processor. >>>> >>>> But I get this error. What am I missing? There's no GetS3Object >>>> anymore; surely this is the edge processor for directly downloading S3 >>>> objects, yes? There's no "ListS3Object" processor type which might >>>> hypothetically populate attributes for FetchS3Object to act upon. >>>> >>>> Also, there are these obviously incorrect copy/paste lines in the >>>> Relationships area of the API doc referenced above: >>>> >>>> "success - FlowFiles are routed to success after being successfully >>>> copied to Amazon S3" >>>> "failure - FlowFiles are routed to failure if unable to be copied to >>>> Amazon S3" >>>> >>>> No, that's obviously lifted from the PutS3Object doc page, where it's >>>> actually correct: >>>> >>>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.aws.s3.PutS3Object/index.html >>>> >>>> Anyone have any insight into this? Thanks in advance. >>>> >>>> Russell >> >> -- >> Russell Whitaker >> http://twitter.com/OrthoNormalRuss >> http://www.linkedin.com/pub/russell-whitaker/0/b86/329 -- Russell Whitaker http://twitter.com/OrthoNormalRuss http://www.linkedin.com/pub/russell-whitaker/0/b86/329
