[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r540875965
##
File path: extensions/aws/processors/DeleteS3Object.cpp
##
@@ -0,0 +1,90 @@
+/**
+ * @file DeleteS3Object.cpp
+ * DeleteS3Object class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "DeleteS3Object.h"
+
+#include
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const core::Property DeleteS3Object::Version(
+ core::PropertyBuilder::createProperty("Version")
+->withDescription("The Version of the Object to delete")
+->supportsExpressionLanguage(true)
+->build());
+
+const core::Relationship DeleteS3Object::Success("success", "FlowFiles are
routed to success relationship");
+const core::Relationship DeleteS3Object::Failure("failure", "FlowFiles are
routed to failure relationship");
+
+void DeleteS3Object::initialize() {
+ // Set the supported properties
+ std::set properties(S3Processor::getSupportedProperties());
Review comment:
Included in
[b065515](https://github.com/apache/nifi-minifi-cpp/pull/931/commits/b06551501eee29d7b8e22133135998079af70647).
The only difference is that I did not introduce a new insertion operation in
the `ConfigurableComponent` but used `updateSupportedProperties` instead.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r540876649
##
File path: extensions/aws/processors/S3Processor.cpp
##
@@ -0,0 +1,263 @@
+/**
+ * @file S3Processor.cpp
+ * Base S3 processor class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "S3Processor.h"
+
+#include
+#include
+#include
+
+#include "S3Wrapper.h"
+#include "AWSCredentialsService.h"
+#include "properties/Properties.h"
+#include "utils/StringUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const std::set S3Processor::REGIONS({region::AF_SOUTH_1,
region::AP_EAST_1, region::AP_NORTHEAST_1,
+ region::AP_NORTHEAST_2, region::AP_NORTHEAST_3, region::AP_SOUTH_1,
region::AP_SOUTHEAST_1, region::AP_SOUTHEAST_2,
+ region::CA_CENTRAL_1, region::CN_NORTH_1, region::CN_NORTHWEST_1,
region::EU_CENTRAL_1, region::EU_NORTH_1,
+ region::EU_SOUTH_1, region::EU_WEST_1, region::EU_WEST_2, region::EU_WEST_3,
region::ME_SOUTH_1, region::SA_EAST_1,
+ region::US_EAST_1, region::US_EAST_2, region::US_GOV_EAST_1,
region::US_GOV_WEST_1, region::US_WEST_1, region::US_WEST_2});
+
+const core::Property S3Processor::ObjectKey(
+ core::PropertyBuilder::createProperty("Object Key")
+->withDescription("The key of the S3 object. If none is given the filename
attribute will be used by default.")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::Bucket(
+ core::PropertyBuilder::createProperty("Bucket")
+->withDescription("The S3 bucket")
+->isRequired(true)
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::AccessKey(
+ core::PropertyBuilder::createProperty("Access Key")
+->withDescription("AWS account access key")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::SecretKey(
+ core::PropertyBuilder::createProperty("Secret Key")
+->withDescription("AWS account secret key")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::CredentialsFile(
+ core::PropertyBuilder::createProperty("Credentials File")
+->withDescription("Path to a file containing AWS access key and secret key
in properties file format. Properties used: accessKey and secretKey")
+->build());
+const core::Property S3Processor::AWSCredentialsProviderService(
+ core::PropertyBuilder::createProperty("AWS Credentials Provider service")
+->withDescription("The name of the AWS Credentials Provider controller
service that is used to obtain AWS credentials.")
+->build());
+const core::Property S3Processor::Region(
+ core::PropertyBuilder::createProperty("Region")
+->isRequired(true)
+->withDefaultValue(region::US_WEST_2)
+->withAllowableValues(S3Processor::REGIONS)
+->withDescription("AWS Region")
+->build());
+const core::Property S3Processor::CommunicationsTimeout(
+ core::PropertyBuilder::createProperty("Communications Timeout")
+->isRequired(true)
+->withDefaultValue("30 sec")
+->withDescription("")
Review comment:
Added description in
[4f3ac8e](https://github.com/apache/nifi-minifi-cpp/pull/931/commits/4f3ac8e4d6f287c7e5e13bad07dc652467e18669)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r540876403
##
File path: extensions/aws/processors/DeleteS3Object.h
##
@@ -0,0 +1,76 @@
+/**
+ * @file DeleteS3Object.h
+ * DeleteS3Object class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+
+#include "S3Processor.h"
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+class DeleteS3Object : public S3Processor {
+ public:
+ static constexpr char const* ProcessorName = "DeleteS3Object";
+
+ // Supported Properties
+ static const core::Property Version;
+
+ // Supported Relationships
+ static const core::Relationship Failure;
+ static const core::Relationship Success;
+
+ explicit DeleteS3Object(std::string name, minifi::utils::Identifier uuid =
minifi::utils::Identifier())
+: S3Processor(std::move(name), uuid,
logging::LoggerFactory::getLogger()) {
+ }
+
+ explicit DeleteS3Object(std::string name, minifi::utils::Identifier uuid,
std::unique_ptr s3_wrapper)
Review comment:
Moved the constructor private in
[b04bddf](https://github.com/apache/nifi-minifi-cpp/pull/931/commits/b04bddf30e848db09f1aa5e8a606a53a34dd3d81)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r540875965
##
File path: extensions/aws/processors/DeleteS3Object.cpp
##
@@ -0,0 +1,90 @@
+/**
+ * @file DeleteS3Object.cpp
+ * DeleteS3Object class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "DeleteS3Object.h"
+
+#include
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const core::Property DeleteS3Object::Version(
+ core::PropertyBuilder::createProperty("Version")
+->withDescription("The Version of the Object to delete")
+->supportsExpressionLanguage(true)
+->build());
+
+const core::Relationship DeleteS3Object::Success("success", "FlowFiles are
routed to success relationship");
+const core::Relationship DeleteS3Object::Failure("failure", "FlowFiles are
routed to failure relationship");
+
+void DeleteS3Object::initialize() {
+ // Set the supported properties
+ std::set properties(S3Processor::getSupportedProperties());
Review comment:
Included in
[1a59b59](https://github.com/apache/nifi-minifi-cpp/pull/931/commits/1a59b59b1141edaa82d45bae920f732c385e78fd).
The only difference is that I did not introduce a new insertion operation in
the `ConfigurableComponent` but used `updateSupportedProperties` instead.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r540875020
##
File path: extensions/aws/processors/DeleteS3Object.cpp
##
@@ -0,0 +1,90 @@
+/**
+ * @file DeleteS3Object.cpp
+ * DeleteS3Object class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "DeleteS3Object.h"
+
+#include
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const core::Property DeleteS3Object::Version(
+ core::PropertyBuilder::createProperty("Version")
+->withDescription("The Version of the Object to delete")
+->supportsExpressionLanguage(true)
+->build());
+
+const core::Relationship DeleteS3Object::Success("success", "FlowFiles are
routed to success relationship");
+const core::Relationship DeleteS3Object::Failure("failure", "FlowFiles are
routed to failure relationship");
+
+void DeleteS3Object::initialize() {
+ // Set the supported properties
+ std::set properties(S3Processor::getSupportedProperties());
+ properties.insert(Version);
+ setSupportedProperties(properties);
+ // Set the supported relationships
+ setSupportedRelationships({Failure, Success});
+}
+
+bool DeleteS3Object::getExpressionLanguageSupportedProperties(
+const std::shared_ptr ,
+const std::shared_ptr _file) {
+ if (!S3Processor::getExpressionLanguageSupportedProperties(context,
flow_file)) {
+return false;
+ }
+
+ context->getProperty(Version, version_, flow_file);
+ logger_->log_debug("DeleteS3Object: Version [%s]", version_);
+ return true;
+}
+
+void DeleteS3Object::onTrigger(const std::shared_ptr
, const std::shared_ptr ) {
+ logger_->log_debug("DeleteS3Object onTrigger");
+ std::shared_ptr flow_file = session->get();
+ if (!flow_file) {
+context->yield();
+return;
+ }
+
+ if (!getExpressionLanguageSupportedProperties(context, flow_file)) {
+context->yield();
+return;
Review comment:
Fixed in
[af2501b](https://github.com/apache/nifi-minifi-cpp/pull/931/commits/af2501ba0dcd880a7e78ae4ac21d56badca7dd18)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r540874499
##
File path: extensions/aws/processors/S3Processor.cpp
##
@@ -0,0 +1,263 @@
+/**
+ * @file S3Processor.cpp
+ * Base S3 processor class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "S3Processor.h"
+
+#include
+#include
+#include
+
+#include "S3Wrapper.h"
+#include "AWSCredentialsService.h"
+#include "properties/Properties.h"
+#include "utils/StringUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const std::set S3Processor::REGIONS({region::AF_SOUTH_1,
region::AP_EAST_1, region::AP_NORTHEAST_1,
+ region::AP_NORTHEAST_2, region::AP_NORTHEAST_3, region::AP_SOUTH_1,
region::AP_SOUTHEAST_1, region::AP_SOUTHEAST_2,
+ region::CA_CENTRAL_1, region::CN_NORTH_1, region::CN_NORTHWEST_1,
region::EU_CENTRAL_1, region::EU_NORTH_1,
+ region::EU_SOUTH_1, region::EU_WEST_1, region::EU_WEST_2, region::EU_WEST_3,
region::ME_SOUTH_1, region::SA_EAST_1,
+ region::US_EAST_1, region::US_EAST_2, region::US_GOV_EAST_1,
region::US_GOV_WEST_1, region::US_WEST_1, region::US_WEST_2});
+
+const core::Property S3Processor::ObjectKey(
+ core::PropertyBuilder::createProperty("Object Key")
+->withDescription("The key of the S3 object. If none is given the filename
attribute will be used by default.")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::Bucket(
+ core::PropertyBuilder::createProperty("Bucket")
+->withDescription("The S3 bucket")
+->isRequired(true)
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::AccessKey(
+ core::PropertyBuilder::createProperty("Access Key")
+->withDescription("AWS account access key")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::SecretKey(
+ core::PropertyBuilder::createProperty("Secret Key")
+->withDescription("AWS account secret key")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::CredentialsFile(
+ core::PropertyBuilder::createProperty("Credentials File")
+->withDescription("Path to a file containing AWS access key and secret key
in properties file format. Properties used: accessKey and secretKey")
+->build());
+const core::Property S3Processor::AWSCredentialsProviderService(
+ core::PropertyBuilder::createProperty("AWS Credentials Provider service")
+->withDescription("The name of the AWS Credentials Provider controller
service that is used to obtain AWS credentials.")
+->build());
+const core::Property S3Processor::Region(
+ core::PropertyBuilder::createProperty("Region")
+->isRequired(true)
+->withDefaultValue(region::US_WEST_2)
+->withAllowableValues(S3Processor::REGIONS)
+->withDescription("AWS Region")
+->build());
+const core::Property S3Processor::CommunicationsTimeout(
+ core::PropertyBuilder::createProperty("Communications Timeout")
+->isRequired(true)
+->withDefaultValue("30 sec")
+->withDescription("")
+->build());
+const core::Property S3Processor::EndpointOverrideURL(
+ core::PropertyBuilder::createProperty("Endpoint Override URL")
+->withDescription("Endpoint URL to use instead of the AWS default
including scheme, host, "
+ "port, and path. The AWS libraries select an endpoint
URL based on the AWS "
+ "region, but this property overrides the selected
endpoint URL, allowing use "
+ "with other S3-compatible endpoints.")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::ProxyHost(
+ core::PropertyBuilder::createProperty("Proxy Host")
+->withDescription("Proxy host name or IP")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property S3Processor::ProxyPort(
+ core::PropertyBuilder::createProperty("Proxy Port")
+->withDescription("The port number of the proxy host")
+->supportsExpressionLanguage(true)
+->build());
+const core::Property
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r540772490
##
File path: extensions/aws/processors/DeleteS3Object.cpp
##
@@ -0,0 +1,90 @@
+/**
+ * @file DeleteS3Object.cpp
+ * DeleteS3Object class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "DeleteS3Object.h"
+
+#include
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const core::Property DeleteS3Object::Version(
+ core::PropertyBuilder::createProperty("Version")
+->withDescription("The Version of the Object to delete")
+->supportsExpressionLanguage(true)
+->build());
+
+const core::Relationship DeleteS3Object::Success("success", "FlowFiles are
routed to success relationship");
+const core::Relationship DeleteS3Object::Failure("failure", "FlowFiles are
routed to failure relationship");
+
+void DeleteS3Object::initialize() {
+ // Set the supported properties
+ std::set properties(S3Processor::getSupportedProperties());
+ properties.insert(Version);
+ setSupportedProperties(properties);
+ // Set the supported relationships
+ setSupportedRelationships({Failure, Success});
+}
+
+bool DeleteS3Object::getExpressionLanguageSupportedProperties(
+const std::shared_ptr ,
+const std::shared_ptr _file) {
+ if (!S3Processor::getExpressionLanguageSupportedProperties(context,
flow_file)) {
+return false;
+ }
+
+ context->getProperty(Version, version_, flow_file);
+ logger_->log_debug("DeleteS3Object: Version [%s]", version_);
+ return true;
+}
+
+void DeleteS3Object::onTrigger(const std::shared_ptr
, const std::shared_ptr ) {
+ logger_->log_debug("DeleteS3Object onTrigger");
+ std::shared_ptr flow_file = session->get();
+ if (!flow_file) {
+context->yield();
+return;
+ }
+
+ if (!getExpressionLanguageSupportedProperties(context, flow_file)) {
+context->yield();
+return;
Review comment:
I think you are right, as if this fails we cannot continue with this
flow file as one of the required properties is missing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r539132879
##
File path: extensions/aws/processors/DeleteS3Object.h
##
@@ -0,0 +1,76 @@
+/**
+ * @file DeleteS3Object.h
+ * DeleteS3Object class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+
+#include "S3Processor.h"
+#include "utils/GeneralUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+class DeleteS3Object : public S3Processor {
+ public:
+ static constexpr char const* ProcessorName = "DeleteS3Object";
+
+ // Supported Properties
+ static const core::Property Version;
+
+ // Supported Relationships
+ static const core::Relationship Failure;
+ static const core::Relationship Success;
+
+ explicit DeleteS3Object(std::string name, minifi::utils::Identifier uuid =
minifi::utils::Identifier())
+: S3Processor(std::move(name), uuid,
logging::LoggerFactory::getLogger()) {
+ }
+
+ explicit DeleteS3Object(std::string name, minifi::utils::Identifier uuid,
std::unique_ptr s3_wrapper)
Review comment:
Yes, this constructor is used in the tests to pass the mocked version of
the S3 wrapper.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931: URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r539132249 ## File path: PROCESSORS.md ## @@ -208,6 +209,39 @@ In the list below, the names of required properties appear in bold. Any other pr |success|FlowFiles that are sent successfully to the destination are transferred to this relationship| +## DeleteS3Object + +### Description + +Deletes FlowFiles on an Amazon S3 Bucket. If attempting to delete a file that does not exist, FlowFile is routed to success. Review comment: We started discussing this with @arpadboda and @adebreceni in a comment above, where @adebreceni noted that it can be logical if we interpret it the following way: "if you define "success" as the post-condition "the object is not in the S3 bucket", it makes sense to transfer to "success"". So if the goal is to have the object non-existent in that bucket we succeed by doing nothing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r521488405
##
File path: extensions/aws/processors/DeleteS3Object.cpp
##
@@ -0,0 +1,92 @@
+/**
+ * @file DeleteS3Object.cpp
+ * DeleteS3Object class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "DeleteS3Object.h"
+
+#include
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const core::Property DeleteS3Object::Version(
+ core::PropertyBuilder::createProperty("Version")
+->withDescription("The Version of the Object to delete")
+->supportsExpressionLanguage(true)
+->build());
+
+const core::Relationship DeleteS3Object::Success("success", "FlowFiles are
routed to success relationship");
+const core::Relationship DeleteS3Object::Failure("failure", "FlowFiles are
routed to failure relationship");
+
+void DeleteS3Object::initialize() {
+ // Set the supported properties
+ std::set properties(S3Processor::getSupportedProperties());
+ properties.insert(Version);
+ setSupportedProperties(properties);
+ // Set the supported relationships
+ std::set relationships;
+ relationships.insert(Failure);
+ relationships.insert(Success);
+ setSupportedRelationships(relationships);
Review comment:
Done in
[a2ba9ac7](https://github.com/apache/nifi-minifi-cpp/pull/931/commits/a2ba9ac73f38d032a678d546ed343afd63092150)
for PutS3Object as well
##
File path: extensions/aws/processors/DeleteS3Object.cpp
##
@@ -0,0 +1,92 @@
+/**
+ * @file DeleteS3Object.cpp
+ * DeleteS3Object class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "DeleteS3Object.h"
+
+#include
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace aws {
+namespace processors {
+
+const core::Property DeleteS3Object::Version(
+ core::PropertyBuilder::createProperty("Version")
+->withDescription("The Version of the Object to delete")
+->supportsExpressionLanguage(true)
+->build());
+
+const core::Relationship DeleteS3Object::Success("success", "FlowFiles are
routed to success relationship");
+const core::Relationship DeleteS3Object::Failure("failure", "FlowFiles are
routed to failure relationship");
+
+void DeleteS3Object::initialize() {
+ // Set the supported properties
+ std::set properties(S3Processor::getSupportedProperties());
+ properties.insert(Version);
+ setSupportedProperties(properties);
+ // Set the supported relationships
+ std::set relationships;
+ relationships.insert(Failure);
+ relationships.insert(Success);
+ setSupportedRelationships(relationships);
+}
+
+bool DeleteS3Object::getExpressionLanguageSupportedProperties(
+const std::shared_ptr ,
+const std::shared_ptr _file) {
+ if (!S3Processor::getExpressionLanguageSupportedProperties(context,
flow_file)) {
+return false;
+ }
+
+ context->getProperty(Version, version_, flow_file);
+ logger_->log_debug("DeleteS3Object: Version [%s]", version_);
+ return true;
+}
+
+void DeleteS3Object::onTrigger(const std::shared_ptr
, const std::shared_ptr ) {
+ logger_->log_debug("DeleteS3Object onTrigger");
+ std::shared_ptr flow_file = session->get();
+ if (!flow_file) {
+return;
Review comment:
Done in
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #931: MINIFICPP-1390 Create DeleteS3Object processor
lordgamez commented on a change in pull request #931:
URL: https://github.com/apache/nifi-minifi-cpp/pull/931#discussion_r521469706
##
File path: extensions/aws/s3/S3Wrapper.cpp
##
@@ -38,8 +38,24 @@ minifi::utils::optional
S3Wrapper::sendPutObjec
logger_->log_info("Added S3 object %s to bucket %s", request.GetKey(),
request.GetBucket());
return outcome.GetResultWithOwnership();
} else {
- logger_->log_error("PutS3Object failed with the following: '%s'",
outcome.GetError().GetMessage());
- return minifi::utils::nullopt;
+logger_->log_error("PutS3Object failed with the following: '%s'",
outcome.GetError().GetMessage());
+return minifi::utils::nullopt;
+ }
+}
+
+bool S3Wrapper::sendDeleteObjectRequest(const
Aws::S3::Model::DeleteObjectRequest& request) {
+ Aws::S3::S3Client s3_client(credentials_, client_config_);
+ Aws::S3::Model::DeleteObjectOutcome outcome =
s3_client.DeleteObject(request);
+
+ if (outcome.IsSuccess()) {
+logger_->log_info("Deleted S3 object %s from bucket %s", request.GetKey(),
request.GetBucket());
+return true;
+ } else if (outcome.GetError().GetErrorType() ==
Aws::S3::S3Errors::NO_SUCH_KEY) {
+logger_->log_info("S3 object %s was not found in bucket %s",
request.GetKey(), request.GetBucket());
Review comment:
In NiFi the processor's
[description](https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.5.0/org.apache.nifi.processors.aws.s3.DeleteS3Object/index.html)
says "If attempting to delete a file that does not exist, FlowFile is routed
to success". It's a bit strange to me as well, but I wanted to be consistent
with the Nifi implementation. We could change it in our case if you prefer.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
