[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi-cpp/pull/156


---

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r148075278
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,218 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive (touch, remove, copy, move).", "");
+core::Property ManipulateArchive::Target("Target", "An existing entry 
within the archive to perform the operation on.", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (touch, move or copy) which result in new entries.", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property.", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property.", "");
+core::Relationship ManipulateArchive::Success("success", "FlowFiles will 
be transferred to the success relationship if the operation succeeds.");
+core::Relationship ManipulateArchive::Failure("failure", "FlowFiles will 
be transferred to the failure relationship if the operation fails.");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+relationships.insert(Failure);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onSchedule(core::ProcessContext *context, 
core::ProcessSessionFactory *sessionFactory) {
+context->getProperty(Operation.getName(), operation_);
+std::transform(operation_.begin(), operation_.end(), 
operation_.begin(), ::tolower);
+
+bool op_create = operation_ == OPERATION_COPY ||
+ operation_ == OPERATION_MOVE ||
+ operation_ == OPERATION_TOUCH;
+
+// Operation must be one of copy, move, touch or remove
+assert(op_create || (operation_ == OPERATION_REMOVE));
--- End diff --

The usage of assert here is intriguing. Do you not want to avoid running 
this processor and log an error by throwing an exception if NDEBUG is defined? 


---

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r148075762
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,218 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive (touch, remove, copy, move).", "");
+core::Property ManipulateArchive::Target("Target", "An existing entry 
within the archive to perform the operation on.", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (touch, move or copy) which result in new entries.", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property.", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property.", "");
+core::Relationship ManipulateArchive::Success("success", "FlowFiles will 
be transferred to the success relationship if the operation succeeds.");
+core::Relationship ManipulateArchive::Failure("failure", "FlowFiles will 
be transferred to the failure relationship if the operation fails.");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+relationships.insert(Failure);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onSchedule(core::ProcessContext *context, 
core::ProcessSessionFactory *sessionFactory) {
+context->getProperty(Operation.getName(), operation_);
+std::transform(operation_.begin(), operation_.end(), 
operation_.begin(), ::tolower);
+
+bool op_create = operation_ == OPERATION_COPY ||
+ operation_ == OPERATION_MOVE ||
+ operation_ == OPERATION_TOUCH;
+
+// Operation must be one of copy, move, touch or remove
+assert(op_create || (operation_ == OPERATION_REMOVE));
+
+context->getProperty(Target.getName(), targetEntry_);
+context->getProperty(Destination.getName(), destination_);
+
+// All operations which create new entries require a set destination
+assert(op_create != destination_.empty());
+
+// The only operation that doesn't require an existing target is touch
+assert((operation_ == OPERATION_TOUCH) == targetEntry_.empty());
+
+context->getProperty(Before.getName(), before_);
+context->getProperty(After.getName(), after_);
+
+// Users may specify one or none of before or after, but never both.
+assert(!(before_.size() && after_.size()));
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr 

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[calebj]

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r148094619
  
--- Diff: CMakeLists.txt ---
@@ -111,38 +111,38 @@ add_subdirectory(libminifi)
 #function(createExtension extensionCondition extensionGuard extensionName 
description dirName)
 
 ## Add http-curl extensions
-createExtension(DISABLE_CURL 
-   HTTP-CURL 
-   "HTTP CURL" 
-   "This enables RESTProtocol, InvokeHTTP, and the 
HTTPClient for Site to Site" 
-   "extensions/http-curl"
-   "${TEST_DIR}/curl-tests")
+createExtension(DISABLE_CURL
+HTTP-CURL
--- End diff --

I don't follow. What do you need me to change?


---

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r148073780
  
--- Diff: CMakeLists.txt ---
@@ -111,38 +111,38 @@ add_subdirectory(libminifi)
 #function(createExtension extensionCondition extensionGuard extensionName 
description dirName)
 
 ## Add http-curl extensions
-createExtension(DISABLE_CURL 
-   HTTP-CURL 
-   "HTTP CURL" 
-   "This enables RESTProtocol, InvokeHTTP, and the 
HTTPClient for Site to Site" 
-   "extensions/http-curl"
-   "${TEST_DIR}/curl-tests")
+createExtension(DISABLE_CURL
+HTTP-CURL
--- End diff --

we need to add a skip to this so this doesn't happen when the linter runs. 
thanks


---

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[calebj]

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147534705
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the 
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property", "");
+core::Relationship ManipulateArchive::Success("success", "success 
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, );
+session->read(flowFile, );
+
+logger_->log_info("ManipulateArchive performing operation %s on %s", 
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it != 
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for 
removal", targetEntry.c_str());
+std::remove((*it).tmpFileName.c_str());
--- End diff --

I'm leaning towards yes. There's no surefire way to determine the 
uncompressed size of input files.


---

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[calebj]

Github user calebj commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147534442
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the 
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property", "");
+core::Relationship ManipulateArchive::Success("success", "success 
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, );
+session->read(flowFile, );
+
+logger_->log_info("ManipulateArchive performing operation %s on %s", 
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it != 
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for 
removal", targetEntry.c_str());
+std::remove((*it).tmpFileName.c_str());
+it = archiveMetadata.entryMetadata.erase(it);
+break;
+} else {
+it++;
+}
+}
+}
+
+// Perform operation: COPY
+if (operation 

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147533673
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the 
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property", "");
+core::Relationship ManipulateArchive::Success("success", "success 
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, );
+session->read(flowFile, );
+
+logger_->log_info("ManipulateArchive performing operation %s on %s", 
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it != 
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for 
removal", targetEntry.c_str());
+std::remove((*it).tmpFileName.c_str());
+it = archiveMetadata.entryMetadata.erase(it);
+break;
+} else {
+it++;
+}
+}
+}
+
+// Perform operation: COPY
+if 

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147532903
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the 
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property", "");
+core::Relationship ManipulateArchive::Success("success", "success 
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
--- End diff --

Can these be gathered once, in onSchedule, and validated there?


---

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147533391
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the 
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property", "");
+core::Relationship ManipulateArchive::Success("success", "success 
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, );
+session->read(flowFile, );
+
+logger_->log_info("ManipulateArchive performing operation %s on %s", 
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it != 
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for 
removal", targetEntry.c_str());
+std::remove((*it).tmpFileName.c_str());
--- End diff --

In the event that volatile repositories are used in environments without 
persistent storage should we disable this suite of extensions? As I understand 
it, these operations can be performed in memory, right @minifirocks?

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147532806
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the 
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property", "");
+core::Relationship ManipulateArchive::Success("success", "success 
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, );
+session->read(flowFile, );
+
+logger_->log_info("ManipulateArchive performing operation %s on %s", 
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it != 
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for 
removal", targetEntry.c_str());
+std::remove((*it).tmpFileName.c_str());
+it = archiveMetadata.entryMetadata.erase(it);
+break;
+} else {
+it++;
+}
+}
+}
+
+// Perform operation: COPY
+if 

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[phrocker]

Github user phrocker commented on a diff in the pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147533090
  
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to 
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the 
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination 
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which 
result in new entries, places the new entry before the entry specified by this 
property", "");
+core::Property ManipulateArchive::After("After", "For operations which 
result in new entries, places the new entry after the entry specified by this 
property", "");
+core::Relationship ManipulateArchive::Success("success", "success 
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY =   "copy";
+char const* ManipulateArchive::OPERATION_MOVE =   "move";
+char const* ManipulateArchive::OPERATION_TOUCH =  "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context, 
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, );
+session->read(flowFile, );
+
+logger_->log_info("ManipulateArchive performing operation %s on %s", 
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it != 
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for 
removal", targetEntry.c_str());
+std::remove((*it).tmpFileName.c_str());
+it = archiveMetadata.entryMetadata.erase(it);
+break;
+} else {
+it++;
+}
+}
+}
+
+// Perform operation: COPY
+if 

[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...

[calebj]

GitHub user calebj opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/156

MINIFICPP-268 Implement ManipulateArchive processor

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [x] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NiFiLocal/nifi-minifi-cpp ManipulateArchive

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/156.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #156






---