[jira] [Commented] (NIFI-7579) Create a GetS3Object Processor
[ https://issues.apache.org/jira/browse/NIFI-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148915#comment-17148915 ] ArpStorm1 commented on NIFI-7579: - The problem with the List/Fetch pattern regarding S3 is the need to first list all the objects, and the list operation can be very heavy. S3 is a common standard today of Object storage, and not only Amazon implemented it. Using listS3 processor can create heavy workload on the backend storage, resulting in slow answer which can fail the entire flow process. And sometimes that can be avoided by getting the exact object the user needs. GetS3Object not has to be the solution - maybe implement this logic to the FetchS3Object processor would be enough. > Create a GetS3Object Processor > -- > > Key: NIFI-7579 > URL: https://issues.apache.org/jira/browse/NIFI-7579 > Project: Apache NiFi > Issue Type: New Feature >Reporter: ArpStorm1 >Assignee: YoungGyu Chun >Priority: Major > > Sometimes the client needs to get only specific object or a subset of objects > from its bucket. Now, the only way to do it is using ListS3 Processor and > after that using FetchS3Object processor. Creating a GetS3Object processor > for such cases can be great -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7579) Create a GetS3Object Processor
[ https://issues.apache.org/jira/browse/NIFI-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147862#comment-17147862 ] ArpStorm1 commented on NIFI-7579: - I don't think this behavior should be merged into the FetchS3Object processor. My suggestion is to create a processor that shouldn't need upstream connection for working with S3 > Create a GetS3Object Processor > -- > > Key: NIFI-7579 > URL: https://issues.apache.org/jira/browse/NIFI-7579 > Project: Apache NiFi > Issue Type: New Feature >Reporter: ArpStorm1 >Assignee: YoungGyu Chun >Priority: Major > > Sometimes the client needs to get only specific object or a subset of objects > from its bucket. Now, the only way to do it is using ListS3 Processor and > after that using FetchS3Object processor. Creating a GetS3Object processor > for such cases can be great -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7579) Create a GetS3Object Processor
[ https://issues.apache.org/jira/browse/NIFI-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147223#comment-17147223 ] ArpStorm1 commented on NIFI-7579: - The FetchS3Object depends on upstream connection. You can't start your flow with it. That makes users use processors like ListS3 or something else. But sometimes the user wants to start his workflow with fetching objects > Create a GetS3Object Processor > -- > > Key: NIFI-7579 > URL: https://issues.apache.org/jira/browse/NIFI-7579 > Project: Apache NiFi > Issue Type: New Feature >Reporter: ArpStorm1 >Assignee: YoungGyu Chun >Priority: Major > > Sometimes the client needs to get only specific object or a subset of objects > from its bucket. Now, the only way to do it is using ListS3 Processor and > after that using FetchS3Object processor. Creating a GetS3Object processor > for such cases can be great -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7579) Create a GetS3Object Processor
[ https://issues.apache.org/jira/browse/NIFI-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ArpStorm1 updated NIFI-7579: Description: Sometimes the client needs to get only specific object or a subset of objects from its bucket. Now, the only way to do it is using ListS3 Processor and after that using FetchS3Object processor. Creating a GetS3Object processor for such cases can be great > Create a GetS3Object Processor > -- > > Key: NIFI-7579 > URL: https://issues.apache.org/jira/browse/NIFI-7579 > Project: Apache NiFi > Issue Type: New Feature >Reporter: ArpStorm1 >Priority: Major > > Sometimes the client needs to get only specific object or a subset of objects > from its bucket. Now, the only way to do it is using ListS3 Processor and > after that using FetchS3Object processor. Creating a GetS3Object processor > for such cases can be great -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7579) Create a GetS3Object Processor
[ https://issues.apache.org/jira/browse/NIFI-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ArpStorm1 updated NIFI-7579: Summary: Create a GetS3Object Processor (was: GetS3Object Processor) > Create a GetS3Object Processor > -- > > Key: NIFI-7579 > URL: https://issues.apache.org/jira/browse/NIFI-7579 > Project: Apache NiFi > Issue Type: New Feature >Reporter: ArpStorm1 >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7579) GetS3Object Processor
ArpStorm1 created NIFI-7579: --- Summary: GetS3Object Processor Key: NIFI-7579 URL: https://issues.apache.org/jira/browse/NIFI-7579 Project: Apache NiFi Issue Type: New Feature Reporter: ArpStorm1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
