[GitHub] nifi-minifi-cpp pull request #44: MINIFI-189: Check log level before buffer ...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/44 MINIFI-189: Check log level before buffer operations Checking the log level before attempting buffer operations has the impact of avoiding memory and functional calls that are unnecessary due to the configured level excluding this log message Trivial change that checks the log level before logging. I noticed this while creating a test processor that logged a lot. That logging created a noticeable impact, hence I'm submitting a PR for this low hanging fruit. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-189 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/44.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #44 commit da8ee8ecb067e227fff177b36622f0fe079b1b98 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-01T11:50:09Z MINIFI-189: Check log level before buffer operations Checking the log level before attempting buffer operations has the impact of avoiding memory and functional calls that are unnecessary due to the configured level excluding this log message --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #37: MINIFI-171 Dynamic Properties support for ...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/37#discussion_r98902210
--- Diff: libminifi/src/Processor.cpp ---
@@ -206,6 +207,29 @@ bool Processor::getProperty(std::string name,
std::string )
}
}
+bool Processor::getDynamicProperty(std::string name, std::string )
+{
+ if (isRunning())
+ // Because set property only allowed in non running state, we
need to obtain lock avoid rack condition
+ _mtx.lock();
--- End diff --
Hi, any particular reason you did not use a lock guard ?
http://en.cppreference.com/w/cpp/thread/lock_guard
It's scoped on the stack pop so you don't need to unlock.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #38: MINIFI-175 Create official Apache NiFi MiNiFi C++...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/38 Going through some PRs to learn the lay of the land. I can't approve but I really like this one. It excites me +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #37: MINIFI-171 Dynamic Properties support for ...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/37#discussion_r98902765
--- Diff: libminifi/src/Processor.cpp ---
@@ -226,6 +250,39 @@ bool Processor::setProperty(std::string name,
std::string value)
}
}
+bool Processor::setDynamicProperty(std::string name, std::string value)
+{
+ std::lock_guard lock(_mtx);
+ std::map<std::string, Property>::iterator it =
_dynamicProperties.find(name);
--- End diff --
Just a general comment about iterators. We could make this simpler with
C++11 type deduction:
auto it = _dynamicProperties.find(name).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #37: MINIFI-171 Dynamic Properties support for ...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/37#discussion_r98902595
--- Diff: libminifi/src/Processor.cpp ---
@@ -206,6 +207,29 @@ bool Processor::getProperty(std::string name,
std::string )
}
}
+bool Processor::getDynamicProperty(std::string name, std::string )
+{
+ if (isRunning())
+ // Because set property only allowed in non running state, we
need to obtain lock avoid rack condition
+ _mtx.lock();
+
+ std::map<std::string, Property>::iterator it =
_dynamicProperties.find(name);
+ if (it != _dynamicProperties.end())
+ {
+ Property item = it->second;
+ value = item.getValue();
+ if (!isRunning())
+ _mtx.unlock();
+ return true;
+ }
+ else
+ {
+ if (!isRunning())
--- End diff --
I'm new to this, so forgive the question, but why only unlock when it is
not running if the above conditional works only when it is running? This would
imply the mutex is locked when you are running. I likely missed something,
though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99357485 --- Diff: cmake/FindOpenSSL.cmake --- @@ -0,0 +1,28 @@ +# OPENSSL_ROOT_DIR - Set this variable to the root installation of OpenSSL +# +# Read-Only variables: +# OPENSSL_FOUND - system has the OpenSSL library +# OPENSSL_INCLUDE_DIR - the OpenSSL include directory +# OPENSSL_LIBRARIES - The libraries needed to use OpenSSL +# OPENSSL_VERSION - This is set to $major.$minor.$revision$path (eg. 0.9.8s) + +find_path(OPENSSL_INCLUDE_DIR +NAMES openssl/ssl.h --- End diff -- Did you consider BOTAN? That's a purely C++/Crypto library ( one of many ). Any particular reason we are introducing a C library when C++ libraries exist? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99357721
--- Diff: libminifi/src/FlowController.cpp ---
@@ -75,22 +78,123 @@ FlowController::FlowController(std::string name)
}
}
-path = realpath(adjustedFilename.c_str(), full_path);
+ path = realpath(adjustedFilename.c_str(), full_path);
if (!path)
{
_logger->log_error("Could not locate path from provided
configuration file name (%s). Exiting.", full_path);
exit(1);
}
-std::string pathString(path);
-_configurationFileName = pathString;
-_logger->log_info("FlowController NiFi Configuration file %s",
pathString.c_str());
+ std::string pathString(path);
--- End diff --
Would it make sense to break this out into a separate class to reduce
coupling with the security library within the FlowController?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99357571 --- Diff: libminifi/include/FlowController.h --- @@ -90,6 +93,25 @@ class FlowController } return _flowController; } + //! passphase for the private file callback --- End diff -- Could we extract SSL Functionality into separate classes so do we don't couple the controller with security? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #45: MINIFI-187: Remove XML References and deps...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/45 MINIFI-187: Remove XML References and deps. Remove XML references and dependencies. This includes changing the reload functionality to use YAML instead of XML. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-187 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/45.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #45 commit affe73089009316ff229e0eb2cfeec89ae928cb7 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-03T00:41:38Z MINIFI-187: Remove XML References and deps. Remove XML references and dependencies. This includes changing the reload functionality to use YAML instead of XML. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99655455 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- Thanks! I'll fix that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99686039 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- I see this same typo in other files. I'll make that change in either this PR or the next. Apparently I propagated the typo. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99691327 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- I updated the commit to include a blurb about it so that it was squashed into a single commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99691182 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- Oh I already force pushed the change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/46 MINIFI-188: Incorporate CATCH and example tests This PR introduces catch and a few example tests. Additional tests will follow, but they currently cause several segfaults without a large number of corresponding changes in the code. Therefore to minimize the bootstrapping I will include the cmake changes in addition to the dep. to run type make; make test You can also run ./tests in the build directory for more information CATCH is Boost license, let me know if this is a problem. Thanks! You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-188-A Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/46.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #46 commit b5a0cec0e681a0f2ed13331a036cbfd105103b1f Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-03T14:43:25Z MINIFI-188: Incorporate CATCH and example tests This PR introduces catch and a few example tests. Additional tests will follow, but they currently cause several segfaults without a large number of corresponding changes in the code. Therefore to minimize the bootstrapping I will include the cmake changes in addition to the dep. to run type make; make test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #54: MINIFI-206: Implement Extensible Comm Mechanism
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/54 @apiri I tested it locally. I would like to add tests for it in unit tests ala the java client. That may require a little bit more work though to stand up a server, but that will be a subsequent PR @jdye64 I will make this change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #54: MINIFI-206: Implement Extensible Comm Mechanism
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/54 @apiri FYI the test protocol I used was to test s2s, bidirectionally, in secure and non-secure mode. I tested in linux and under mac. It inspired me to try and build an integration test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #57: MINIFI-83: Allow appenders to be specified...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/57 MINIFI-83: Allow appenders to be specified and auto configured. This PR will modify logging so that we can have configurable appenders and have them be modified via custom configuration for that appender. Eventually these can be modified real time. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-83 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/57.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #57 commit e1b05ccc68ca4084b4ac861bd5de0eb21e5d44e5 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-24T15:06:19Z MINIFI-83: Allow appenders to be specified and auto configured. This PR will modify logging so that we can have configurable appenders and have them be modified via custom configuration for that appender. Eventually these can be modified real time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #54: MINIFI-206: Implement Extensible Comm Mech...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/54#discussion_r102715874
--- Diff: libminifi/CMakeLists.txt ---
@@ -36,18 +36,34 @@ ENDIF(POLICY CMP0048)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
+include(CheckCXXCompilerFlag)
+CHECK_CXX_COMPILER_FLAG("-std=c++11" COMPILER_SUPPORTS_CXX11)
+CHECK_CXX_COMPILER_FLAG("-std=c++0x" COMPILER_SUPPORTS_CXX0X)
+if(COMPILER_SUPPORTS_CXX11)
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -g")
+elseif(COMPILER_SUPPORTS_CXX0X)
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x")
+else()
+ message(STATUS "The compiler ${CMAKE_CXX_COMPILER} has no C++11 support.
Please use a different C++ compiler.")
+endif()
+
include_directories(../include)
include_directories(../thirdparty/yaml-cpp-yaml-cpp-0.5.3/include)
include_directories(include)
-file(GLOB SOURCES "src/*.cpp")
+file(GLOB_RECURSE SOURCES "src/*.cpp")
--- End diff --
I generally don't like to see this. Since we only have two sub directories,
we can call them out.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/52#discussion_r102284495
--- Diff: libminifi/src/gps/GetGPS.cpp ---
@@ -0,0 +1,158 @@
+/**
+ * @file GetGPS.cpp
+ * GetGPS class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "TimeUtil.h"
+#include "GetGPS.h"
+#include "ProcessContext.h"
+#include "ProcessSession.h"
+
+#include
+
+using namespace std;
+
+const std::string GetGPS::ProcessorName("GetGPS");
+Relationship GetGPS::Success("success", "All files are routed to success");
+Property GetGPS::GPSDHost("GPSD Host", "The host running the GPSD daemon",
"localhost");
+Property GetGPS::GPSDPort("GPSD Port", "The GPSD daemon port", "2947");
+Property GetGPS::GPSDWaitTime("GPSD Wait Time", "Timeout value for waiting
for data from the GPSD instance", "5000");
+void GetGPS::initialize()
+{
+ //! Set the supported properties
+ std::set properties;
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(Success);
+ setSupportedRelationships(relationships);
+}
+
+void GetGPS::onTrigger(ProcessContext *context, ProcessSession *session)
+{
+ try
+ {
+
+ std::string value;
+
+ _logger->log_debug("onTrigger GetGPS");
+ if (context->getProperty(GPSDHost.getName(), value))
+ {
+ _gpsdHost = value;
+ }
+
+ if (context->getProperty(GPSDPort.getName(), value))
+ {
+ _gpsdPort = value;
+ }
+
+ if (context->getProperty(GPSDWaitTime.getName(), value))
+ {
+ Property::StringToInt(value, _gpsdWaitTime);
+ }
+
+ gpsmm gps_rec(_gpsdHost.c_str(), _gpsdPort.c_str());
+
+ if (gps_rec.stream(WATCH_ENABLE|WATCH_JSON) == NULL) {
+ _logger->log_error("No GPSD running.");
+ return;
+ }
+
+ for (;;) {
--- End diff --
Could you wait on a condition in case this processor is forced shutdown.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #52: Support for GPSD integration
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/52#discussion_r102284269
--- Diff: libminifi/include/GetGPS.h ---
@@ -0,0 +1,73 @@
+/**
+ * @file GetGPS.h
+ * GetGPS class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __GET_GPS_H__
+#define __GET_GPS_H__
+
+#include "FlowFileRecord.h"
+#include "Processor.h"
+#include "ProcessSession.h"
+
+//! GetGPS Class
+class GetGPS : public Processor
+{
+public:
+ //! Constructor
+ /*!
+* Create a new processor
+*/
+ GetGPS(std::string name, uuid_t uuid = NULL)
+ : Processor(name, uuid)
+ {
+ _logger = Logger::getLogger();
+ _gpsdHost = "localhost";
+ _gpsdPort = "2947";
+ _gpsdWaitTime = 5000;
+ }
+ //! Destructor
+ virtual ~GetGPS()
+ {
+ }
+ //! Processor Name
+ static const std::string ProcessorName;
+ //! Supported Properties
+ static Property GPSDHost;
+ static Property GPSDPort;
+ static Property GPSDWaitTime;
+
+ //! Supported Relationships
+ static Relationship Success;
+
+public:
+ //! OnTrigger method, implemented by NiFi GetGPS
+ virtual void onTrigger(ProcessContext *context, ProcessSession
*session);
+ //! Initialize, over write by NiFi GetGPS
+ virtual void initialize(void);
+
+protected:
+
+private:
+ //! Logger
+ Logger *_logger;
--- End diff --
logger_ is the google cpp style guide approach. Same with all of these.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request #72: MINIFI-216 - Allowing override of nifi.propert...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/72#discussion_r102290405 --- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java --- @@ -704,7 +691,7 @@ protected static void addTextElement(final Element element, final String name, f } public static final String PROPERTIES_FILE_APACHE_2_0_LICENSE = -"# Licensed to the Apache Software Foundation (ASF) under one or more\n" + +" Licensed to the Apache Software Foundation (ASF) under one or more\n" + --- End diff -- Why did you remove the comment markdown (octothorp -- #)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi pull request #72: MINIFI-216 - Allowing override of nifi.propert...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/72#discussion_r102289560 --- Diff: minifi-bootstrap/src/test/resources/MINIFI-216/config.yml --- @@ -0,0 +1,106 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more --- End diff -- +1 I think we should be able to share these test resources amongst cpp and java and have the same results or at least used in similar tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #54: MINIFI-206: Implement Extensible Comm Mech...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/54 MINIFI-206: Implement Extensible Comm Mechanism Create initial communication mechanism that's initially based on BSD sockets. MINIFI-201: Change how hostname is resolved and ensure that it is not null MINIFI-199: Utilize zlib's crc32 implementation, which is more portable and faster. we can explore alternatives when that time arrives. MINIFI-200: Put host "get" into initialization. Any change wouldn't be picked up by a running process most likely. MINIFI-222: Upgrade spdlog You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-206 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/54.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #54 commit 9bbe352caa9ac319786de42d129cf6ea31e603a8 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-14T13:38:05Z MINIFI-206: Implement Extensible Comm Mechanism Create initial communication mechanism that's initially based on BSD sockets. MINIFI-201: Change how hostname is resolved and ensure that it is not null MINIFI-199: Utilize zlib's crc32 implementation, which is more portable and faster. we can explore alternatives when that time arrives. MINIFI-200: Put host "get" into initialization. Any change wouldn't be picked up by a running process most likely. MINIFI-222: Upgrade spdlog --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #55: MINIFI-222: Upgrade spdlog to latest relea...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/55 MINIFI-222: Upgrade spdlog to latest release. Modified Logger so that it removed the previously required flush argument. Copyright headers changed but copyright license is the same. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-222 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/55.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #55 commit 6cd9eba6b747207e58b5a13eb275f0690b071fcb Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-22T15:06:20Z MINIFI-222: Upgrade spdlog to latest release. Modified Logger so that it removed the previously required flush argument --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #55: MINIFI-222: Upgrade spdlog to latest release.
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/55 I put the authors with the corresponding license multiple times. I feel like it would be better to include the Copyrights and the license once, but I've seen it done like this in other apache projects. advise if the alternative is preferred. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/43 @achristianson I thought I created a ticket to break these out...or maybe it was an E-mail, but I think that's something we could discuss. When I sent a message to @apiri earlier I said that the ExternalProject comment was mostly a desire and that this PR followed precedent and as such it's not a deal breaker, it's just a direction I would like to see things go. With that said, the operative word is suggestion and we can thus have a ticket to discuss this further. I'll create that ticket and we can discuss it there. TL;DR on my thoughts: having people build offline isn't a big deal because that likely wouldn't happen very frequently. In cases where it is anticipated, you would likely need to move code to said device, and in this case we could codify the download so that it's easy to stage. In my opinion, worrying about those types of systems also brings the worry about those which don't have OpenSSL or boost readily available. Since we don't package those and those certainly aren't standard fare on many lightweight distros, are we really breaking the mold to have ExternalProjects? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/43 Created MINIFI-213 to have the CMake 3rd party discussion. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #47: MINIFI-190: Creating initial commit with addition...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/47 @apiri Yes. I just noticed a typo, but it should be ready. It'll probably have some adjustments along the review pipeline, but I don't plan anything else in this PR unless there are comments or things such as typos. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #47: MINIFI-190: Creating initial commit with addition...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/47 @apiri Okay, cool. I think reviewing these changes then allow the subsequent changes to be in a separate PR ( it's coming ) is probably better since those tests encompass other parts of the code too. Plus getting this through sooner with any changes as a result of comments is probably more advantageous. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #42: MINIFI-186 Updating Copyright statements to refle...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/42 +1 totally a fan of 2017 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #50: MINIFI-195: Convert uuid_unparse usage to ...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/50 MINIFI-195: Convert uuid_unparse usage to uuid_unparse_lower This will ensure that we use lower case UUIDs. Undefining the macro won't work and we likely don't want to change the default behavior. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-195 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/50.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #50 commit 0f0c6c561c0d07d8cdd791584a8456ec317a41f9 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-13T15:49:45Z MINIFI-195: Convert uuid_unparse usage to uuid_unparse_lower This will ensure that we use lower case UUIDs. Undefining the macro won't work and we likely don't want to change the default behavior. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100846349 --- Diff: libminifi/src/ThreadedSchedulingAgent.cpp --- @@ -64,11 +68,18 @@ void ThreadedSchedulingAgent::schedule(Processor *processor) return; } + auto processContext = std::make_shared(processor); --- End diff -- Why are you getting the raw (stored -- not managed ) pointer if you're making a shared ptr? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100844559
--- Diff: libminifi/include/ProcessSessionFactory.h ---
@@ -0,0 +1,52 @@
+/**
+ * @file ProcessSessionFactory.h
+ * ProcessSessionFactory class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __PROCESS_SESSION_FACTORY_H__
+#define __PROCESS_SESSION_FACTORY_H__
+
+#include
+
+#include "ProcessContext.h"
+#include "ProcessSession.h"
+
+//! ProcessSessionFactory Class
+class ProcessSessionFactory
+{
+public:
+ //! Constructor
+ /*!
+* Create a new process session factory
+*/
+ ProcessSessionFactory(ProcessContext *processContext) :
_processContext(processContext) {}
+
+ //! Create the session
+ std::unique_ptr createSession();
--- End diff --
It is my goal to make pointer usage clear. Thanks for making this
exceptionally clear with unique_ptr here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100847768
--- Diff: libminifi/src/ListenHTTP.cpp ---
@@ -0,0 +1,395 @@
+/**
+ * @file ListenHTTP.cpp
+ * ListenHTTP class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include "ListenHTTP.h"
+
+#include "TimeUtil.h"
+#include "ProcessContext.h"
+#include "ProcessSession.h"
+#include "ProcessSessionFactory.h"
+
+const std::string ListenHTTP::ProcessorName("ListenHTTP");
+
+Property ListenHTTP::BasePath("Base Path", "Base path for incoming
connections", "contentListener");
+Property ListenHTTP::Port("Listening Port", "The Port to listen on for
incoming connections", "");
+Property ListenHTTP::AuthorizedDNPattern("Authorized DN Pattern", "A
Regular Expression to apply against the Distinguished Name of incoming
connections. If the Pattern does not match the DN, the connection will be
refused.", ".*");
+Property ListenHTTP::SSLCertificate("SSL Certificate", "File containing
PEM-formatted file including TLS/SSL certificate and key", "");
+Property ListenHTTP::SSLCertificateAuthority("SSL Certificate Authority",
"File containing trusted PEM-formatted certificates", "");
+Property ListenHTTP::SSLVerifyPeer("SSL Verify Peer", "Whether or not to
verify the client's certificate (yes/no)", "no");
+Property ListenHTTP::SSLMinimumVersion("SSL Minimum Version", "Minimum
TLS/SSL version allowed (SSL2, SSL3, TLS1.0, TLS1.1, TLS1.2)", "SSL2");
+Property ListenHTTP::HeadersAsAttributesRegex("HTTP Headers to receive as
Attributes (Regex)", "Specifies the Regular Expression that determines the
names of HTTP Headers that should be passed along as FlowFile attributes", "");
+
+Relationship ListenHTTP::Success("success", "All files are routed to
success");
+
+void ListenHTTP::initialize()
+{
+ _logger->log_info("Initializing ListenHTTP");
+
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(BasePath);
+ properties.insert(Port);
+ properties.insert(AuthorizedDNPattern);
+ properties.insert(SSLCertificate);
+ properties.insert(SSLCertificateAuthority);
+ properties.insert(SSLVerifyPeer);
+ properties.insert(SSLMinimumVersion);
+ properties.insert(HeadersAsAttributesRegex);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(Success);
+ setSupportedRelationships(relationships);
+}
+
+void ListenHTTP::onSchedule(ProcessContext *context, ProcessSessionFactory
*sessionFactory)
+{
+
+ std::string basePath;
+
+ if (!context->getProperty(BasePath.getName(), basePath))
+ {
+ _logger->log_info("%s attribute is missing, so default value of
%s will be used",
+ BasePath.getName().c_str(),
+ BasePath.getValue().c_str());
+ basePath = BasePath.getValue();
+ }
+
+ basePath.insert(0, "/");
+
+ std::string listeningPort;
+
+ if (!context->getProperty(Port.getName(), listeningPort))
+ {
+ _logger->log_error("%s attribute is missing or invalid",
+ Port.getName().c_str());
+ return;
+ }
+
+ std::string authDNPattern;
+
+ if (context->getProperty(AuthorizedDNPattern.getName(), authDNPattern)
&& !a
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100844761
--- Diff: libminifi/include/ProcessSessionFactory.h ---
@@ -0,0 +1,52 @@
+/**
+ * @file ProcessSessionFactory.h
+ * ProcessSessionFactory class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __PROCESS_SESSION_FACTORY_H__
+#define __PROCESS_SESSION_FACTORY_H__
+
+#include
+
+#include "ProcessContext.h"
+#include "ProcessSession.h"
+
+//! ProcessSessionFactory Class
+class ProcessSessionFactory
+{
+public:
+ //! Constructor
+ /*!
+* Create a new process session factory
+*/
+ ProcessSessionFactory(ProcessContext *processContext) :
_processContext(processContext) {}
--- End diff --
Do you want to mark this explicit?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100845905
--- Diff: libminifi/CMakeLists.txt ---
@@ -48,6 +49,19 @@ file(GLOB SPD_SOURCES "../include/spdlog/*")
add_library(spdlog STATIC ${SPD_SOURCES})
add_library(minifi STATIC ${SOURCES})
+# Include Boost System
+find_package(Boost COMPONENTS system REQUIRED)
--- End diff --
There seemed to be a little hesitation about using boost. My guess is that
-Os added to flags won't actually cause the expected increase; however, it does
potentially cause some bloat.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100846654 --- Diff: CMakeLists.txt --- @@ -63,6 +66,8 @@ find_package(UUID REQUIRED) file(GLOB SPD_SOURCES "include/spdlog/*") add_subdirectory(thirdparty/yaml-cpp-yaml-cpp-0.5.3) +set(CIVETWEB_ENABLE_CXX ON CACHE BOOL "Enable civet C++ library") --- End diff -- Could we download and then build civet using ExternalProject instead of bundling it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100846792
--- Diff: libminifi/CMakeLists.txt ---
@@ -48,6 +49,19 @@ file(GLOB SPD_SOURCES "../include/spdlog/*")
add_library(spdlog STATIC ${SPD_SOURCES})
add_library(minifi STATIC ${SOURCES})
+# Include Boost System
+find_package(Boost COMPONENTS system REQUIRED)
--- End diff --
I assume this was included for civet? all uses of pointer management in
libminifi appeared to be std::
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100845047
--- Diff: libminifi/src/Processor.cpp ---
@@ -439,31 +447,25 @@ bool Processor::flowFilesOutGoingFull()
return false;
}
-void Processor::onTrigger()
+void Processor::onTrigger(ProcessContext *context, ProcessSessionFactory
*sessionFactory)
{
- ProcessContext *context = new ProcessContext(this);
- ProcessSession *session = new ProcessSession(context);
+ auto session = sessionFactory->createSession();
--- End diff --
Thanks. I can remove this off of my my todo. fantastic!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/43 If you happen to update try -Os ( don't feel like you need to in this PR ) https://issues.apache.org/jira/browse/MINIFI-197 is a corresponding ticket for it. I was going to do this and prune out warnings that weren't important. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #43: MINIFI-183 Implemented ListenHTTP
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/43#discussion_r100844292
--- Diff: libminifi/include/ListenHTTP.h ---
@@ -0,0 +1,116 @@
+/**
+ * @file ListenHTTP.h
+ * ListenHTTP class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __LISTEN_HTTP_H__
+#define __LISTEN_HTTP_H__
+
+#include
+#include
+
+#include
+
+#include "FlowFileRecord.h"
+#include "Processor.h"
+#include "ProcessSession.h"
+
+
+//! ListenHTTP Class
+class ListenHTTP : public Processor
+{
+public:
+
+ //! Constructor
+ /*!
+* Create a new processor
+*/
+ ListenHTTP(std::string name, uuid_t uuid = NULL)
+ : Processor(name, uuid)
+ {
+ _logger = Logger::getLogger();
+ }
+ //! Destructor
+ ~ListenHTTP()
+ {
+ }
+ //! Processor Name
+ static const std::string ProcessorName;
+ //! Supported Properties
+ static Property BasePath;
+ static Property Port;
+ static Property AuthorizedDNPattern;
+ static Property SSLCertificate;
+ static Property SSLCertificateAuthority;
+ static Property SSLVerifyPeer;
+ static Property SSLMinimumVersion;
+ static Property HeadersAsAttributesRegex;
+ //! Supported Relationships
+ static Relationship Success;
+
+ void onTrigger(ProcessContext *context, ProcessSession *session);
+ void initialize();
+ void onSchedule(ProcessContext *context, ProcessSessionFactory
*sessionFactory);
+
+ //! HTTP request handler
+ class Handler : public CivetHandler
+ {
+ public:
+ Handler(ProcessContext *context,
+ ProcessSessionFactory *sessionFactory,
+ std::string &,
+ std::string &);
+ bool handlePost(CivetServer *server, struct mg_connection
*conn);
+
+ private:
+ //! Send HTTP 500 error response to client
+ void sendErrorResponse(struct mg_connection *conn);
+ //! Logger
+ Logger *_logger;
+
+ std::regex _authDNRegex;
+ std::regex _headersAsAttributesRegex;
+ ProcessContext *_processContext;
+ ProcessSessionFactory *_processSessionFactory;
+ };
+
+ //! Write callback for transferring data from HTTP request to content
repo
+ class WriteCallback : public OutputStreamCallback
+ {
+ public:
+ WriteCallback(struct mg_connection *conn, const struct
mg_request_info *reqInfo);
+ void process(std::ofstream *stream);
+
+ private:
+ //! Logger
+ Logger *_logger;
+
+ struct mg_connection *_conn;
+ const struct mg_request_info *_reqInfo;
+ };
+
+protected:
+
+private:
+ //! Logger
+ Logger *_logger;
+
+ std::unique_ptr _server;
--- End diff --
You have no idea how happy I am to see you use unique_ptr :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #47: MINIFI-190: Creating initial commit with addition...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/47 I have found that the flow repository isn't cleaning up after itself. Will resolve that first. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #47: MINIFI-190: Creating initial commit with a...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/47 MINIFI-190: Creating initial commit with additional tests Additional tests will be updated in separate tickets as needed. Resolves MINIFI-194 by using a semaphore in place of the FlowController instance. Stop is performed outside of the signal handler to avoid synchronicity issues. Resolves MINIFI-192 by using lock_guard based on a conditional Resolves issues found with MINIFI-190 regarding GetFile. Added pragma definitions for GCC < 4.9 You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-190 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/47.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #47 commit e3ed8da1739288774a018c8b2422073802362014 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-03T14:23:42Z MINIFI-190: Creating initial commit with changes found Resolves MINIFI-194 by using a semaphore in place of the FlowController instance. Stop is performed outside of the signal handler to avoid synchronicity issues. Resolves MINIFI-192 by using lock_guard based on a conditional Resolves issues found with MINIFI-190 regarding GetFile. Added pragma definitions for GCC < 4.9 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #47: MINIFI-190: Creating initial commit with addition...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/47 I'll be happy to break this apart --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #48: MINIFI-195: Update root CMake so testing c...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/48 MINIFI-195: Update root CMake so testing can occur Update root CMake to bring OpenSSL location to top level. Keeping find_package(...) in libminifi CMakeLists.txt to avoid issues building only libminifi. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-195 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/48.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #48 commit 5565f8f3295883de10614baec1cab8f36b726442 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-13T13:40:10Z MINIFI-195: Update root CMAKE so testing can occur --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #48: MINIFI-195: Update root CMake so testing c...
Github user phrocker closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/48 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #49: MINIFI-198: Update root CMAKE so testing c...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/49 MINIFI-198: Update root CMAKE so testing can occur Update root CMake to bring OpenSSL location to top level. Keeping find_package(...) in libminifi CMakeLists.txt to avoid issues building only libminifi. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-198 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/49.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #49 commit 3d40dcf0f66b7ab107f42221fac1db65f2bd7379 Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-13T13:40:10Z MINIFI-198: Update root CMAKE so testing can occur Update root CMake to bring OpenSSL location to top level. Keeping find_package(...) in libminifi CMakeLists.txt to avoid issues building only libminifi. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #60: MINIFI-225: Apply first round of CPP linter and c...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/60 I'm going to do this one file at a time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #60: MINIFI-225: Apply first round of CPP linte...
Github user phrocker closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/60 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #60: MINIFI-225: Apply first round of CPP linte...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/60 MINIFI-225: Apply first round of CPP linter and code style fixes Included and applied Google code style CPP formatter for eclipse. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-225 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/60.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #60 commit 506818c4b80a533b55613cc624490cd27abc073f Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-27T19:53:41Z MINIFI-225: Apply first round of CPP linter and code style fixes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #60: MINIFI-225: Apply first round of CPP linter and c...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/60 Added xml formatter in thirdparty/codestyle dir. Usually I would add this to /contrib. Please advise. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #57: MINIFI-83: Allow appenders to be specified...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/57#discussion_r103284177
--- Diff: libminifi/include/LogAppenders.h ---
@@ -0,0 +1,292 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_LOGAPPENDERS_H_
+#define LIBMINIFI_INCLUDE_LOGAPPENDERS_H_
+
+#include "BaseLogger.h"
+#include "spdlog/sinks/null_sink.h"
+#include "spdlog/sinks/ostream_sink.h"
+#include
+#include "Configure.h"
+
+template
+static std::string getUniqueName() {
+ std::string name = LOG_NAME;
+ name += " -- ";
+ name += abi::__cxa_demangle(typeid(T).name(), 0, 0, 0);
+ spdlog::drop(name);
+ return name;
+}
+
+/**
+ * Null appender sets a null sink, thereby performing no logging.
+ */
+class NullAppender: public BaseLogger {
+public:
+ /**
+* Base constructor that creates the null sink.
+*/
+ explicit NullAppender() :
+ ::BaseLogger("off") {
+ auto null_sink =
std::make_shared();
+ std::string unique_name = getUniqueName();
+ logger_ = std::make_shared(unique_name,
null_sink);
+ configured_level_ = off;
+ setLogLevel();
+ }
+
+ /**
+* Move constructor for the null appender.
+*/
+ explicit NullAppender(const NullAppender &) :
+ ::BaseLogger(std::move(other)) {
+
+ }
+
+};
+
+/**
+ * Basic output stream configuration that uses a supplied ostream
+ *
+ * Design : extends LoggerConfiguration using the logger and log level
+ * encapsulated within the base configuration class.
+ */
+class OutputStreamAppender: public BaseLogger {
+
+public:
+
+ static const char *nifi_log_output_stream_error_stderr;
+
+ /**
+* Output stream move constructor.
+*/
+ explicit OutputStreamAppender(const OutputStreamAppender &) :
+ ::BaseLogger(std::move(other)) {
+
+ }
+
+ /**
+* Base constructor. Creates a ostream sink.
+* @param stream incoming stream reference.
+* @param config configuration.
+*/
+ explicit OutputStreamAppender(Configure *config) :
+ ::BaseLogger("info") {
+ auto ostream_sink =
std::make_shared(
+ std::cout);
+
+ std::string unique_name = getUniqueName();
+ logger_ = std::make_shared(unique_name,
ostream_sink);
+
+ std::string use_std_err;
+
+ if (NULL != config
+ &&
config->get(nifi_log_output_stream_error_stderr,
+ use_std_err)) {
+
+ std::transform(use_std_err.begin(), use_std_err.end(),
+ use_std_err.begin(), ::tolower);
+
+ if (use_std_err == "true") {
+ std::string err_unique_name =
+
getUniqueName();
+ auto error_ostream_sink = std::make_shared<
+
spdlog::sinks::ostream_sink_mt>(std::cerr);
+ stderr_ =
std::make_shared(err_unique_name,
+ error_ostream_sink);
+ }
+ } else
--- End diff --
Good catch, thanks. I'll update this when you're finished commenting so I
don't make your life harder.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #57: MINIFI-83: Allow appenders to be specified...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/57#discussion_r103284475
--- Diff: libminifi/include/LogAppenders.h ---
@@ -0,0 +1,292 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_LOGAPPENDERS_H_
+#define LIBMINIFI_INCLUDE_LOGAPPENDERS_H_
+
+#include "BaseLogger.h"
+#include "spdlog/sinks/null_sink.h"
+#include "spdlog/sinks/ostream_sink.h"
+#include
+#include "Configure.h"
+
+template
+static std::string getUniqueName() {
+ std::string name = LOG_NAME;
+ name += " -- ";
+ name += abi::__cxa_demangle(typeid(T).name(), 0, 0, 0);
+ spdlog::drop(name);
+ return name;
+}
+
+/**
+ * Null appender sets a null sink, thereby performing no logging.
+ */
+class NullAppender: public BaseLogger {
+public:
+ /**
+* Base constructor that creates the null sink.
+*/
+ explicit NullAppender() :
+ ::BaseLogger("off") {
+ auto null_sink =
std::make_shared();
+ std::string unique_name = getUniqueName();
+ logger_ = std::make_shared(unique_name,
null_sink);
+ configured_level_ = off;
+ setLogLevel();
+ }
+
+ /**
+* Move constructor for the null appender.
+*/
+ explicit NullAppender(const NullAppender &) :
+ ::BaseLogger(std::move(other)) {
+
+ }
+
+};
+
+/**
+ * Basic output stream configuration that uses a supplied ostream
+ *
+ * Design : extends LoggerConfiguration using the logger and log level
+ * encapsulated within the base configuration class.
+ */
+class OutputStreamAppender: public BaseLogger {
+
+public:
+
+ static const char *nifi_log_output_stream_error_stderr;
+
+ /**
+* Output stream move constructor.
+*/
+ explicit OutputStreamAppender(const OutputStreamAppender &) :
+ ::BaseLogger(std::move(other)) {
+
+ }
+
+ /**
+* Base constructor. Creates a ostream sink.
+* @param stream incoming stream reference.
+* @param config configuration.
+*/
+ explicit OutputStreamAppender(Configure *config) :
+ ::BaseLogger("info") {
+ auto ostream_sink =
std::make_shared(
+ std::cout);
+
+ std::string unique_name = getUniqueName();
+ logger_ = std::make_shared(unique_name,
ostream_sink);
+
+ std::string use_std_err;
+
+ if (NULL != config
+ &&
config->get(nifi_log_output_stream_error_stderr,
+ use_std_err)) {
+
+ std::transform(use_std_err.begin(), use_std_err.end(),
+ use_std_err.begin(), ::tolower);
+
+ if (use_std_err == "true") {
+ std::string err_unique_name =
+
getUniqueName();
+ auto error_ostream_sink = std::make_shared<
+
spdlog::sinks::ostream_sink_mt>(std::cerr);
+ stderr_ =
std::make_shared(err_unique_name,
+ error_ostream_sink);
+ }
+ } else
--- End diff --
I did that in a few places apparently...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #70: MINIFI-226
Github user phrocker closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/70 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1618#discussion_r107912643
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/impl/mysql/event/io/AbstractBinlogTableEventWriter.java
---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db.impl.mysql.event.io;
+
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processors.standard.GetChangeDataCaptureMySQL;
+import
org.apache.nifi.processors.standard.db.impl.mysql.event.BinlogTableEventInfo;
+
+import java.io.IOException;
+
+/**
+ * An abstract base class for writing MYSQL binlog events into flow
file(s), e.g.
+ */
+public abstract class AbstractBinlogTableEventWriter extends AbstractBinlogEventWriter {
+
+protected void writeJson(T event) throws IOException {
+super.writeJson(event);
+if (event.getDatabaseName() != null) {
+jsonGenerator.writeStringField("database",
event.getDatabaseName());
+} else {
+jsonGenerator.writeNullField("database");
+}
+if (event.getTableName() != null) {
+jsonGenerator.writeStringField("table_name",
event.getTableName());
+} else {
+jsonGenerator.writeNullField("table_name");
+}
+if (event.getTableId() != null) {
+jsonGenerator.writeNumberField("table_id", event.getTableId());
+} else {
+jsonGenerator.writeNullField("table_id");
+}
+}
+
+// Default implementation for binlog events
+@Override
+public long writeEvent(ProcessSession session, T eventInfo, long
currentSequenceId) {
+FlowFile flowFile = session.create();
+flowFile = session.write(flowFile, (outputStream) -> {
+super.startJson(outputStream, eventInfo);
+super.writeJson(eventInfo);
--- End diff --
Isn't the super.writeJson(event) encapsulaed by
AbstractBinlogEventWriter.writeEvent(...) when it calls writeJson(T)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1618#discussion_r107910775
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/event/TableInfoCacheKey.java
---
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db.event;
+
+import
org.apache.nifi.distributed.cache.client.exception.SerializationException;
+
+import java.io.IOException;
+import java.io.OutputStream;
+
+import static
org.apache.nifi.processors.standard.db.event.TableInfo.DB_TABLE_NAME_DELIMITER;
+
+/**
+ * This class represents a key in a cache that contains information
(column definitions, e.g.) for a database table
+ */
+public class TableInfoCacheKey {
+
+private final String databaseName;
+private final String tableName;
+private final long tableId;
+private final String uuidPrefix;
+
+public TableInfoCacheKey(String uuidPrefix, String databaseName,
String tableName, long tableId) {
+this.uuidPrefix = uuidPrefix;
+this.databaseName = databaseName;
+this.tableName = tableName;
+this.tableId = tableId;
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) return true;
+if (o == null || getClass() != o.getClass()) return false;
+
+TableInfoCacheKey that = (TableInfoCacheKey) o;
+
+if (tableId != that.tableId) return false;
+if (databaseName != null ? !databaseName.equals(that.databaseName)
: that.databaseName != null) return false;
--- End diff --
what if that.databaseName is null? I don't believe java.lang.String checks
null. Could use EqualsBuilder from commons-lang, which does the null check
since commons-lang3 is in the pom.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi/pull/1618#discussion_r107920928 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetChangeDataCaptureMySQL.java --- @@ -0,0 +1,879 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.standard; + +import com.github.shyiko.mysql.binlog.BinaryLogClient; +import com.github.shyiko.mysql.binlog.event.Event; +import com.github.shyiko.mysql.binlog.event.EventHeaderV4; +import com.github.shyiko.mysql.binlog.event.EventType; +import com.github.shyiko.mysql.binlog.event.QueryEventData; +import com.github.shyiko.mysql.binlog.event.RotateEventData; +import com.github.shyiko.mysql.binlog.event.TableMapEventData; +import org.apache.nifi.annotation.behavior.DynamicProperties; +import org.apache.nifi.annotation.behavior.DynamicProperty; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.distributed.cache.client.Deserializer; +import org.apache.nifi.distributed.cache.client.DistributedMapCacheClient; +import org.apache.nifi.distributed.cache.client.Serializer; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractSessionFactoryProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.ProcessSessionFactory; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.standard.db.CDCException; +import org.apache.nifi.processors.standard.db.event.ColumnDefinition; +import org.apache.nifi.processors.standard.db.event.RowEventException; +import org.apache.nifi.processors.standard.db.event.TableInfo; +import org.apache.nifi.processors.standard.db.event.TableInfoCacheKey; +import org.apache.nifi.processors.standard.db.event.io.EventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BeginTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.RawBinlogEvent; +import org.apache.nifi.processors.standard.db.impl.mysql.BinlogEventListener; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BinlogEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.CommitTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.DeleteRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.SchemaChangeEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.UpdateRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.InsertRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.BeginTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.CommitTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.DeleteRowsWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.InsertRow
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1618#discussion_r107917776
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/impl/mysql/event/io/InsertRowsWriter.java
---
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db.impl.mysql.event.io;
+
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processors.standard.GetChangeDataCaptureMySQL;
+import org.apache.nifi.processors.standard.db.event.ColumnDefinition;
+import
org.apache.nifi.processors.standard.db.impl.mysql.event.InsertRowsEventInfo;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.BitSet;
+import java.util.concurrent.atomic.AtomicLong;
+
+import static
org.apache.nifi.processors.standard.db.impl.mysql.MySQLCDCUtils.getWritableObject;
+
+/**
+ * A writer class to output MySQL binlog "write rows" (aka INSERT) events
to flow file(s).
+ */
+public class InsertRowsWriter extends
AbstractBinlogTableEventWriter {
+
+/**
+ * Creates and transfers a new flow file whose contents are the
JSON-serialized value of the specified event, and the sequence ID attribute set
+ *
+ * @param session A reference to a ProcessSession from which the
flow file(s) will be created and transferred
+ * @param eventInfo An event whose value will become the contents of
the flow file
+ * @return The next available CDC sequence ID for use by the CDC
processor
+ */
+public long writeEvent(final ProcessSession session, final
InsertRowsEventInfo eventInfo, final long currentSequenceId) {
+final AtomicLong seqId = new AtomicLong(currentSequenceId);
+for (Serializable[] row : eventInfo.getRows()) {
+
+FlowFile flowFile = session.create();
+flowFile = session.write(flowFile, outputStream -> {
+
+super.startJson(outputStream, eventInfo);
+super.writeJson(eventInfo);
+
+final BitSet bitSet = eventInfo.getIncludedColumns();
+writeRow(eventInfo, row, bitSet);
+
+super.endJson();
+});
+
+flowFile = session.putAllAttributes(flowFile,
getCommonAttributes(seqId.get(), eventInfo));
+session.transfer(flowFile,
GetChangeDataCaptureMySQL.REL_SUCCESS);
+seqId.getAndIncrement();
+}
+return seqId.get();
+}
+
+protected void writeRow(InsertRowsEventInfo event, Serializable[] row,
BitSet includedColumns) throws IOException {
+jsonGenerator.writeArrayFieldStart("columns");
+int i = includedColumns.nextSetBit(0);
+while (i != -1) {
+jsonGenerator.writeStartObject();
+jsonGenerator.writeNumberField("id", i + 1);
+ColumnDefinition columnDefinition = event.getColumnByIndex(i);
+Integer columnType = null;
+if (columnDefinition != null) {
+jsonGenerator.writeStringField("name",
columnDefinition.getName());
+columnType = (int) columnDefinition.getType();
--- End diff --
I left a comment earlier about this. Could we avoid the autoboxing and make
make type a Long instead of an enumeration?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi/pull/1618#discussion_r107921330 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetChangeDataCaptureMySQL.java --- @@ -0,0 +1,879 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.standard; + +import com.github.shyiko.mysql.binlog.BinaryLogClient; +import com.github.shyiko.mysql.binlog.event.Event; +import com.github.shyiko.mysql.binlog.event.EventHeaderV4; +import com.github.shyiko.mysql.binlog.event.EventType; +import com.github.shyiko.mysql.binlog.event.QueryEventData; +import com.github.shyiko.mysql.binlog.event.RotateEventData; +import com.github.shyiko.mysql.binlog.event.TableMapEventData; +import org.apache.nifi.annotation.behavior.DynamicProperties; +import org.apache.nifi.annotation.behavior.DynamicProperty; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.distributed.cache.client.Deserializer; +import org.apache.nifi.distributed.cache.client.DistributedMapCacheClient; +import org.apache.nifi.distributed.cache.client.Serializer; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractSessionFactoryProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.ProcessSessionFactory; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.standard.db.CDCException; +import org.apache.nifi.processors.standard.db.event.ColumnDefinition; +import org.apache.nifi.processors.standard.db.event.RowEventException; +import org.apache.nifi.processors.standard.db.event.TableInfo; +import org.apache.nifi.processors.standard.db.event.TableInfoCacheKey; +import org.apache.nifi.processors.standard.db.event.io.EventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BeginTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.RawBinlogEvent; +import org.apache.nifi.processors.standard.db.impl.mysql.BinlogEventListener; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BinlogEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.CommitTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.DeleteRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.SchemaChangeEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.UpdateRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.InsertRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.BeginTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.CommitTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.DeleteRowsWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.InsertRow
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1618#discussion_r107909853
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/event/TableInfo.java
---
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db.event;
+
+import
org.apache.nifi.distributed.cache.client.exception.DeserializationException;
+import
org.apache.nifi.distributed.cache.client.exception.SerializationException;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * A POJO for holding table information related to update events.
+ */
+public class TableInfo {
+
+final static String DB_TABLE_NAME_DELIMITER = "@!@";
+
+private String databaseName;
+private String tableName;
+private Long tableId;
+private List columns;
+
+public TableInfo(String databaseName, String tableName, Long tableId,
List columns) {
+this.databaseName = databaseName;
+this.tableName = tableName;
+this.tableId = tableId;
+this.columns = columns;
+}
+
+public String getDatabaseName() {
+return databaseName;
+}
+
+public String getTableName() {
+return tableName;
+}
+
+public void setTableName(String tableName) {
+this.tableName = tableName;
+}
+
+public Long getTableId() {
+return tableId;
+}
+
+public List getColumns() {
+return columns;
+}
+
+public void setColumns(List columns) {
+this.columns = columns;
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) return true;
+if (o == null || getClass() != o.getClass()) return false;
+
+TableInfo tableInfo = (TableInfo) o;
+
+if (!databaseName.equals(tableInfo.databaseName)) return false;
+if (!tableName.equals(tableInfo.tableName)) return false;
+if (!tableId.equals(tableInfo.tableId)) return false;
+return columns != null ? columns.equals(tableInfo.columns) :
tableInfo.columns == null;
+}
+
+@Override
+public int hashCode() {
+int result = databaseName.hashCode();
+result = 31 * result + tableName.hashCode();
+result = 31 * result + tableId.hashCode();
+result = 31 * result + (columns != null ? columns.hashCode() : 0);
+return result;
+}
+
+public static class Serializer implements
org.apache.nifi.distributed.cache.client.Serializer {
+
+@Override
+public void serialize(TableInfo value, OutputStream output) throws
SerializationException, IOException {
+StringBuilder sb = new StringBuilder(value.getDatabaseName());
+sb.append(DB_TABLE_NAME_DELIMITER);
+sb.append(value.getTableName());
+sb.append(DB_TABLE_NAME_DELIMITER);
+sb.append(value.getTableId());
+List columnDefinitions = value.getColumns();
+if (columnDefinitions != null && !columnDefinitions.isEmpty())
{
+sb.append(DB_TABLE_NAME_DELIMITER);
+sb.append(columnDefinitions.stream().map((col) ->
col.getName() + DB_TABLE_NAME_DELIMITER +
col.getType()).collect(Collectors.joining(DB_TABLE_NAME_DELIMITER)));
+}
+output.write(sb.toString().getBytes());
+}
+}
+
+public static class Deserializer implements
org.apache.nifi.distributed.cache.client.Deserializer {
+
+@Override
+public TableInfo deserialize(byte[] input) thro
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi/pull/1618#discussion_r107922204 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetChangeDataCaptureMySQL.java --- @@ -0,0 +1,879 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.standard; + +import com.github.shyiko.mysql.binlog.BinaryLogClient; +import com.github.shyiko.mysql.binlog.event.Event; +import com.github.shyiko.mysql.binlog.event.EventHeaderV4; +import com.github.shyiko.mysql.binlog.event.EventType; +import com.github.shyiko.mysql.binlog.event.QueryEventData; +import com.github.shyiko.mysql.binlog.event.RotateEventData; +import com.github.shyiko.mysql.binlog.event.TableMapEventData; +import org.apache.nifi.annotation.behavior.DynamicProperties; +import org.apache.nifi.annotation.behavior.DynamicProperty; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.distributed.cache.client.Deserializer; +import org.apache.nifi.distributed.cache.client.DistributedMapCacheClient; +import org.apache.nifi.distributed.cache.client.Serializer; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractSessionFactoryProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.ProcessSessionFactory; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.standard.db.CDCException; +import org.apache.nifi.processors.standard.db.event.ColumnDefinition; +import org.apache.nifi.processors.standard.db.event.RowEventException; +import org.apache.nifi.processors.standard.db.event.TableInfo; +import org.apache.nifi.processors.standard.db.event.TableInfoCacheKey; +import org.apache.nifi.processors.standard.db.event.io.EventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BeginTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.RawBinlogEvent; +import org.apache.nifi.processors.standard.db.impl.mysql.BinlogEventListener; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BinlogEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.CommitTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.DeleteRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.SchemaChangeEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.UpdateRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.InsertRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.BeginTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.CommitTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.DeleteRowsWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.InsertRow
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi/pull/1618#discussion_r107921926 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetChangeDataCaptureMySQL.java --- @@ -0,0 +1,879 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.standard; + +import com.github.shyiko.mysql.binlog.BinaryLogClient; +import com.github.shyiko.mysql.binlog.event.Event; +import com.github.shyiko.mysql.binlog.event.EventHeaderV4; +import com.github.shyiko.mysql.binlog.event.EventType; +import com.github.shyiko.mysql.binlog.event.QueryEventData; +import com.github.shyiko.mysql.binlog.event.RotateEventData; +import com.github.shyiko.mysql.binlog.event.TableMapEventData; +import org.apache.nifi.annotation.behavior.DynamicProperties; +import org.apache.nifi.annotation.behavior.DynamicProperty; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.Stateful; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyValue; +import org.apache.nifi.components.state.Scope; +import org.apache.nifi.components.state.StateManager; +import org.apache.nifi.components.state.StateMap; +import org.apache.nifi.distributed.cache.client.Deserializer; +import org.apache.nifi.distributed.cache.client.DistributedMapCacheClient; +import org.apache.nifi.distributed.cache.client.Serializer; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.AbstractSessionFactoryProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.ProcessSessionFactory; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.standard.db.CDCException; +import org.apache.nifi.processors.standard.db.event.ColumnDefinition; +import org.apache.nifi.processors.standard.db.event.RowEventException; +import org.apache.nifi.processors.standard.db.event.TableInfo; +import org.apache.nifi.processors.standard.db.event.TableInfoCacheKey; +import org.apache.nifi.processors.standard.db.event.io.EventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BeginTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.RawBinlogEvent; +import org.apache.nifi.processors.standard.db.impl.mysql.BinlogEventListener; +import org.apache.nifi.processors.standard.db.impl.mysql.event.BinlogEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.CommitTransactionEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.DeleteRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.SchemaChangeEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.UpdateRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.InsertRowsEventInfo; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.BeginTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.CommitTransactionEventWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.DeleteRowsWriter; +import org.apache.nifi.processors.standard.db.impl.mysql.event.io.InsertRow
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1618#discussion_r107914632
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/impl/mysql/event/io/DeleteRowsWriter.java
---
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db.impl.mysql.event.io;
+
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processors.standard.GetChangeDataCaptureMySQL;
+import org.apache.nifi.processors.standard.db.event.ColumnDefinition;
+import
org.apache.nifi.processors.standard.db.impl.mysql.event.DeleteRowsEventInfo;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.BitSet;
+import java.util.concurrent.atomic.AtomicLong;
+
+import static
org.apache.nifi.processors.standard.db.impl.mysql.MySQLCDCUtils.getWritableObject;
+
+/**
+ * A writer class to output MySQL binlog "delete rows" events to flow
file(s).
+ */
+public class DeleteRowsWriter extends
AbstractBinlogTableEventWriter {
+
+/**
+ * Creates and transfers a new flow file whose contents are the
JSON-serialized value of the specified event, and the sequence ID attribute set
+ *
+ * @param session A reference to a ProcessSession from which the
flow file(s) will be created and transferred
+ * @param eventInfo An event whose value will become the contents of
the flow file
+ * @return The next available CDC sequence ID for use by the CDC
processor
+ */
+public long writeEvent(final ProcessSession session, final
DeleteRowsEventInfo eventInfo, final long currentSequenceId) {
+final AtomicLong seqId = new AtomicLong(currentSequenceId);
--- End diff --
The correct answer is that I'm usually missing something...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1618#discussion_r107908219
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/event/ColumnDefinition.java
---
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db.event;
+
+/**
+ * A class that specifies a definition for a relational table column,
including type, name, etc.
+ */
+public class ColumnDefinition {
+
+private byte type;
+private String name = "";
+
+public ColumnDefinition(byte type) {
--- End diff --
Is there a way to avoid a byte and use an enumeration? I may answer this
myself as I continue the review...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on the issue: https://github.com/apache/nifi/pull/1618 @mattyb149 I began with an empty password for my user and thus set an empty string. I saw an NPE in the logs, which did go away once I set a password for my user and subsequently set it in processor config. 2017-03-24 18:19:53,150 ERROR [blc-localhost:3306] org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[blc-localhost:3306,5,main]: java.lang.NullPointerException 2017-03-24 18:19:53,155 ERROR [blc-localhost:3306] org.apache.nifi.NiFi java.lang.NullPointerException: null   at com.github.shyiko.mysql.binlog.network.protocol.command.AuthenticateCommand.passwordCompatibleWithMySQL411(AuthenticateCommand.java:89) ~[mysql-binlog-connector-java-0.8.1.jar:na]   at com.github.shyiko.mysql.binlog.network.protocol.command.AuthenticateCommand.toByteArray(AuthenticateCommand.java:70) ~[mysql-binlog-connector-java-0.8.1.jar:na]   at com.github.shyiko.mysql.binlog.network.protocol.PacketChannel.write(PacketChannel.java:65) ~[mysql-binlog-connector-java-0.8.1.jar:na]   at com.github.shyiko.mysql.binlog.BinaryLogClient.authenticate(BinaryLogClient.java:546) ~[mysql-binlog-connector-java-0.8.1.jar:na]   at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:407) ~[mysql-binlog-connector-java-0.8.1.jar:na]   at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:635) ~[mysql-binlog-connector-java-0.8.1.jar:na]   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121] --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on the issue: https://github.com/apache/nifi/pull/1618 @mattyb149 Cool stuff. I used your template because it was an easier place to start. Seems pretty good. I'll continue to test at my leisure but I'm happy with it presently. I tried to break it by stopping mysql mid cycle and BinaryLogClient handled it gracefully and all was well. Besides the NPE above I'm happy thus far. I'll continue to play with it for my own edification. Let me know if you wish me to test the NPE again or not. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108924803
--- Diff: libminifi/include/processors/GetFile.h ---
@@ -79,16 +80,27 @@ class GetFile : public core::Processor {
virtual void onTrigger(
core::ProcessContext *context,
core::ProcessSession *session);
+ /**
+ * Function that's executed when the processor is scheduled.
+ * @param context process context.
+ * @param sessionFactory process session factory that is used when
creating
+ * ProcessSession objects.
+ */
+ void onSchedule(
+core::ProcessContext *context,
+core::ProcessSessionFactory *sessionFactory);
// Initialize, over write by NiFi GetFile
virtual void initialize(void);
// perform directory listing
- void performListing(std::string dir);
+ void performListing(std::string dir,const GetFileRequest );
+
--- End diff --
comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108926011
--- Diff: libminifi/src/processors/TailFile.cpp ---
@@ -145,10 +145,10 @@ static bool
sortTailMatchedFileItem(TailMatchedFileItem i,
TailMatchedFileItem j) {
return (i.modifiedTime < j.modifiedTime);
}
-void TailFile::checkRollOver() {
+void TailFile::checkRollOver(std::string fileLocation, std::string
fileName) {
--- End diff --
const arguments?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108924490
--- Diff: CMakeLists.txt ---
@@ -136,6 +136,21 @@ enable_testing(test)
target_include_directories(tests PRIVATE BEFORE
"libminifi/include/provenance")
target_link_libraries(tests ${CMAKE_THREAD_LIBS_INIT}
${UUID_LIBRARIES} ${LEVELDB_LIBRARIES} ${OPENSSL_LIBRARIES} minifi yaml-cpp
c-library civetweb-cpp)
add_test(NAME LibMinifiTests COMMAND tests)
+
--- End diff --
Can we try and make this less duplicative?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108925441
--- Diff: libminifi/src/core/ProcessSession.cpp ---
@@ -530,6 +530,91 @@ void
ProcessSession::read(std::shared_ptr &,
}
}
+void ProcessSession::importFrom(io::DataStream ,
+std::shared_ptr &) {
+ std::shared_ptr claim = std::make_shared();
+
+ int max_read = getpagesize();
--- End diff --
is this portable?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108924995
--- Diff: libminifi/include/utils/ThreadPool.h ---
@@ -0,0 +1,287 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_THREAD_POOL_H
+#define LIBMINIFI_INCLUDE_THREAD_POOL_H
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+/**
+ * Worker task
+ * purpose: Provides a wrapper for the functor
+ * and returns a future based on the template argument.
+ */
+template< typename T>
+class Worker{
+public:
+ explicit Worker(std::function<T()> ) : task(task)
+ {
+promise = std::make_shared<std::promise>();
+ }
+
+ /**
+ * Move constructor for worker tasks
+ */
+ Worker(Worker &) : task (std::move(other.task)),
+ promise(other.promise)
+ {
+ }
+
+
+ /**
+ * Runs the task and takes the output from the funtor
+ * setting the result into the promise
+ */
+ void run()
+ {
+T result = task();
+promise->set_value(result);
+ }
+
+ Worker(const Worker&) = delete;
+Worker& operator = (const Worker&) = delete;
+
+ Worker& operator = (Worker&&) ;
+
+ std::shared_ptr<std::promise> getPromise();
+
+private:
+ std::function<T()> task;
+ std::shared_ptr<std::promise> promise;
+};
+
+template< typename T>
--- End diff --
comment to remind people that && will take ownership of Worker
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108924646
--- Diff: libminifi/include/core/ProcessSession.h ---
@@ -122,6 +128,7 @@ class ProcessSession {
void penalize(std::shared_ptr );
void penalize(std::shared_ptr &);
// Import the existed file into the flow
+ void importFrom(io::DataStream , std::shared_ptr
&);
--- End diff --
better comment?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108819408 --- Diff: libminifi/src/core/ProcessSession.cpp --- @@ -530,6 +530,91 @@ void ProcessSession::read(std::shared_ptr &, } } +void ProcessSession::importFrom(io::DataStream , --- End diff -- Add comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108926131
--- Diff: libminifi/test/TestExecuteProcess.cpp ---
@@ -0,0 +1,136 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "FlowController.h"
+#include "unit/ProvenanceTestHelper.h"
+#include "core/logging/LogAppenders.h"
+#include "core/logging/BaseLogger.h"
+#include "processors/GetFile.h"
+#include "core/core.h"
+#include "core/FlowFile.h"
+#include "core/Processor.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/ProcessorNode.h"
+
+int main(int argc, char **argv)
+{
+
+ std::ostringstream oss;
+ std::unique_ptr outputLogger = std::unique_ptr<
+ logging::BaseLogger>(
+ new
org::apache::nifi::minifi::core::logging::OutputStreamAppender(oss,
+
0));
+ std::shared_ptr logger = logging::Logger::getLogger();
+ logger->updateLogger(std::move(outputLogger));
+
+
+ outputLogger = std::unique_ptr(
+new org::apache::nifi::minifi::core::logging::NullAppender());
+logger->updateLogger(std::move(outputLogger));
+
+ std::shared_ptr processor = std::make_shared<
+
org::apache::nifi::minifi::processors::ExecuteProcess>("executeProcess");
+ processor->setMaxConcurrentTasks(1);
+
+ std::shared_ptr test_repo =
+ std::make_shared();
+
+ std::shared_ptr repo =
+ std::static_pointer_cast(test_repo);
+ std::shared_ptr controller = std::make_shared<
+ TestFlowController>(test_repo, test_repo);
+
+ uuid_t processoruuid;
+ assert(true == processor->getUUID(processoruuid));
+
+ std::shared_ptr connection = std::make_shared<
+ minifi::Connection>(test_repo, "executeProcessConnection");
+ connection->setRelationship(core::Relationship("success",
"description"));
+
+ // link the connections so that we can test results at the end for this
+ connection->setSource(processor);
+ connection->setDestination(processor);
+
+ connection->setSourceUUID(processoruuid);
+ connection->setDestinationUUID(processoruuid);
+
+ processor->addConnection(connection);
+ assert(processor->getName() == "executeProcess");
+
+ std::shared_ptr record;
+ processor->setScheduledState(core::ScheduledState::RUNNING);
+
+ processor->initialize();
+
+ std::atomic is_ready(false);
+
+ std::vector processor_workers;
+
+ core::ProcessorNode node2(processor);
+ std::shared_ptr contextset = std::make_shared<
+ core::ProcessContext>(node2, test_repo);
+ core::ProcessSessionFactory factory(contextset.get());
+ // processor->onSchedule(contextset.get(), );
--- End diff --
uncomment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/71#discussion_r108926160
--- Diff: libminifi/test/TestExecuteProcess.cpp ---
@@ -0,0 +1,136 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include "FlowController.h"
+#include "unit/ProvenanceTestHelper.h"
+#include "core/logging/LogAppenders.h"
+#include "core/logging/BaseLogger.h"
+#include "processors/GetFile.h"
+#include "core/core.h"
+#include "core/FlowFile.h"
+#include "core/Processor.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/ProcessorNode.h"
+
+int main(int argc, char **argv)
+{
+
+ std::ostringstream oss;
+ std::unique_ptr outputLogger = std::unique_ptr<
+ logging::BaseLogger>(
+ new
org::apache::nifi::minifi::core::logging::OutputStreamAppender(oss,
+
0));
+ std::shared_ptr logger = logging::Logger::getLogger();
+ logger->updateLogger(std::move(outputLogger));
+
+
+ outputLogger = std::unique_ptr(
+new org::apache::nifi::minifi::core::logging::NullAppender());
+logger->updateLogger(std::move(outputLogger));
+
+ std::shared_ptr processor = std::make_shared<
+
org::apache::nifi::minifi::processors::ExecuteProcess>("executeProcess");
+ processor->setMaxConcurrentTasks(1);
+
+ std::shared_ptr test_repo =
+ std::make_shared();
+
+ std::shared_ptr repo =
+ std::static_pointer_cast(test_repo);
+ std::shared_ptr controller = std::make_shared<
+ TestFlowController>(test_repo, test_repo);
+
+ uuid_t processoruuid;
+ assert(true == processor->getUUID(processoruuid));
+
+ std::shared_ptr connection = std::make_shared<
+ minifi::Connection>(test_repo, "executeProcessConnection");
+ connection->setRelationship(core::Relationship("success",
"description"));
+
+ // link the connections so that we can test results at the end for this
+ connection->setSource(processor);
+ connection->setDestination(processor);
+
+ connection->setSourceUUID(processoruuid);
+ connection->setDestinationUUID(processoruuid);
+
+ processor->addConnection(connection);
+ assert(processor->getName() == "executeProcess");
+
+ std::shared_ptr record;
+ processor->setScheduledState(core::ScheduledState::RUNNING);
+
+ processor->initialize();
+
+ std::atomic is_ready(false);
+
+ std::vector processor_workers;
+
+ core::ProcessorNode node2(processor);
+ std::shared_ptr contextset = std::make_shared<
+ core::ProcessContext>(node2, test_repo);
+ core::ProcessSessionFactory factory(contextset.get());
+ // processor->onSchedule(contextset.get(), );
--- End diff --
unless this should be removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #72: MINIFI-246: Adding tests that were used to...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/72 MINIFI-246: Adding tests that were used to find bug. Solved by MINIFI… …-193 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-246 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/72.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #72 commit 1a915801ef075e6431a2068e5eaa739ed698aa58 Author: Marc Parisi <phroc...@apache.org> Date: 2017-03-30T16:16:03Z MINIFI-246: Adding tests that were used to find bug. Solved by MINIFI-193 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #63: MINIFI-217: Implement namespaces.
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/63 Updated to facilitate changes for #67 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #71: MINIFI-236: Make GetFile, PutFile, TailFil...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/71 MINIFI-236: Make GetFile, PutFile, TailFile, and ExecuteProcess ThreadSafe Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-236-b Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/71.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #71 commit 9d0bcd08906bacc2fdcc39b5be25346f60bd08cf Author: Marc Parisi <phroc...@apache.org> Date: 2017-03-28T19:50:48Z MINIFI-236: Make GetFile, PutFile, TailFile, and ExecuteProcess thread safe --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #69: MINIFI-225: Add Linter for Google style gu...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/69#discussion_r107930194 --- Diff: thirdparty/google-styleguide/cppguide.html --- @@ -0,0 +1,6170 @@ + --- End diff -- Agreed. Will change this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1618: NIFI-3413: Add GetChangeDataCaptureMySQL processor
Github user phrocker commented on the issue: https://github.com/apache/nifi/pull/1618 @mattyb149 I've left some minor comments. I'm going to pull this down and test this. I'm super excited. Looks cool. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #68: MINIFI-193: Ensure safe UTF encoding
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/68 MINIFI-193: Ensure safe UTF encoding Since the C++ library is agnostic of UTF-8 we can safely write the bytes. Since we won't be interpreting the UTF-8 code in the core library we do not need any additional dependencies. Nor do we need to worry about encoding beyond proper serialization and deserialization of the byte array. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-193 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/68.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #68 commit be5064c82d8c9c98cc06abf7ec5c9470e4dca105 Author: Marc Parisi <phroc...@apache.org> Date: 2017-03-17T17:11:21Z MINIFI-193: Ensure safe UTF encoding Since the C++ library is agnostic of UTF-8 we can safely write the bytes. Since we won't be interpreting the UTF-8 code in the core library we do not need any additional dependencies. Nor do we need to worry about encoding beyond proper serialization and deserialization of the byte array. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #70: MINIFI-226
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/70 MINIFI-226 DO NOT MERGE: This is an early review of changes for MINIFI-226. First step will be to separate Configuring components from the FlowController. Next Controller services will be configurable and returned via the FlowConfiguration Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-226 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/70.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #70 commit 0db1116d5544409413f7302a645aa2475c9871dd Author: Marc Parisi <phroc...@apache.org> Date: 2017-02-27T19:02:58Z MINIFI-217: First commit. Updates namespaces and removes use of raw pointers for user facing API. commit edfea1ac2ac2ad82c185a32384406023accf57f3 Author: Marc Parisi <phroc...@apache.org> Date: 2017-03-13T14:47:05Z MINIFI-217: Update based on PR and rebasing against master commit d6774b32b40e36afbea80dd09495cceaa5db5233 Author: Marc Parisi <phroc...@apache.org> Date: 2017-03-20T19:12:51Z MINIFI-226: First Step is to Separate YamlConfiguration from FlowController. Next step will be to Create Controller services, which will be easier with the seperated FlowConfiguration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #69: MINIFI-225: Add Linter for Google style guide
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/69 Please provide feedback for this approach. I will update the readme to make it a bit clear if more is needed. I kept it intentionally short until we can make this a required process. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #69: MINIFI-225: Add Linter for Google style gu...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/69 MINIFI-225: Add Linter for Google style guide Please provide feedback for this approach. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-225 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/69.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #69 commit f480adb21ca993637636bb3717513c9cc7a85b22 Author: Marc Parisi <phroc...@apache.org> Date: 2017-03-20T14:25:37Z MINIFI-225: Add Linter for Google style guide --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #73: MINIFI-254: Incremental update for linter ...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/73 MINIFI-254: Incremental update for linter changes This particular commit reduces the number < 200. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-254 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/73.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #73 commit 2717886b316af480912e4a50192899e570c28356 Author: Marc Parisi <phroc...@apache.org> Date: 2017-04-03T20:27:24Z MINIFI-254: Incremental update for linter changes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1586: NIFI-3586: Fix for retrieving ProcessID for NiFi un...
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1586#discussion_r109555028
--- Diff:
nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/util/OSUtil.java ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.bootstrap.util;
+
+import java.lang.reflect.Field;
+
+import org.slf4j.Logger;
+import com.sun.jna.Pointer;
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.WinNT;
+
+/**
+ * OS specific utilities with generic method interfaces
+ */
+public final class OSUtil {
--- End diff --
@trkurc From what I can tell you are correct. A quick dive into
java-native-access shows an MIT style license. Top level POM includes the
native module, which runs an ant build ( maven-antrun-plugin in the target
compile-native) which builds the subtree, libff -- where @trkurc saw the MIT
license. Ant will build the native libraries through the make files, but that
still appears bundled.
To further verify this you can look at the jar from Maven central, which
shows the packaged shared objects.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109791553 --- Diff: thirdparty/jsoncpp/devtools/batchbuild.py --- @@ -0,0 +1,278 @@ +from __future__ import print_function --- End diff -- Remove the code if it is unnecessary. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109792191
--- Diff: libminifi/src/provenance/ProvenanceTaskReport.cpp ---
@@ -0,0 +1,221 @@
+/**
+ * @file ProvenanceTaskReport.cpp
+ * ProvenanceTaskReport class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "provenance/ProvenanceTaskReport.h"
+#include "../include/io/StreamFactory.h"
+#include "io/ClientSocket.h"
+#include "utils/TimeUtil.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "provenance/Provenance.h"
+#include "FlowController.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace provenance{
+
+const std::string
ProvenanceTaskReport::ProcessorName("ProvenanceTaskReport");
+core::Property ProvenanceTaskReport::hostName("Host Name", "Remote Host
Name.", "localhost");
+core::Property ProvenanceTaskReport::port("Port", "Remote Port", "");
+core::Property ProvenanceTaskReport::batchSize("Batch Size", "Specifies
how many records to send in a single batch, at most.", "100");
+core::Relationship ProvenanceTaskReport::relation;
+
+void ProvenanceTaskReport::initialize()
+{
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(hostName);
+ properties.insert(port);
+ properties.insert(batchSize);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(relation);
+ setSupportedRelationships(relationships);
+}
+
+std::unique_ptr
ProvenanceTaskReport::getNextProtocol()
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ if (available_protocols_.empty())
+ return nullptr;
+ std::unique_ptr return_pointer =
std::move(available_protocols_.top());
+ available_protocols_.pop();
+ return std::move(return_pointer);
+}
+
+void ProvenanceTaskReport::returnProtocol(
+ std::unique_ptr return_protocol)
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ available_protocols_.push(std::move(return_protocol));
+}
+
+void ProvenanceTaskReport::onTrigger(core::ProcessContext *context,
core::ProcessSession *session)
+{
+ std::string value;
+ int64_t lvalue;
+
+ std::unique_ptr protocol_ = getNextProtocol();
+
+ if (protocol_ == nullptr)
+ {
+ protocol_ = std::unique_ptr(
+ new Site2SiteClientProtocol(0));
+ protocol_->setPortId(protocol_uuid_);
+
+ std::string host = "";
+ uint16_t sport = 0;
+
+ if (context->getProperty(hostName.getName(), value)) {
+ host = value;
+ }
+ if (context->getProperty(port.getName(), value)
+ && core::Property::StringToInt(value, lvalue)) {
+ sport = (uint16_t) lvalue;
+ }
+ std::unique_ptr str =
+ std::unique_ptr(
+ org::apache::nifi::minifi::io::StreamFactory::getInstance()
+ ->createSocket(host, sport));
+
+ std::unique_ptr peer_ =
std::unique_ptr(
+ new Site2SitePeer(std::move(str), host, sport));
+
+ protocol_->setPeer(std::move(peer_));
+ }
+
+ if (!protocol_->bootstrap())
+ {
+ // bootstrap the client protocol if needeed
+ context->yield();
+ std::shared_ptr processo
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109792168 --- Diff: libminifi/src/Site2SiteClientProtocol.cpp --- @@ -1240,6 +1257,88 @@ void Site2SiteClientProtocol::transferFlowFiles( return; } +void Site2SiteClientProtocol::transferBytes(core::ProcessContext *context, core::ProcessSession *session, uint8_t *payload, int length, --- End diff -- No @benqiu2016 that is why std::move(...) exists. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109792275 --- Diff: libminifi/src/Site2SiteClientProtocol.cpp --- @@ -682,6 +682,7 @@ bool Site2SiteClientProtocol::receive(std::string transactionID, bool Site2SiteClientProtocol::send( std::string transactionID, DataPacket *packet, std::shared_ptr flowFile, + uint8_t *payload, int length, --- End diff -- As someone reading this that does not make sense. Why not make DataPacket hold the "packet"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109791492
--- Diff: libminifi/src/provenance/ProvenanceTaskReport.cpp ---
@@ -0,0 +1,221 @@
+/**
+ * @file ProvenanceTaskReport.cpp
+ * ProvenanceTaskReport class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "provenance/ProvenanceTaskReport.h"
+#include "../include/io/StreamFactory.h"
+#include "io/ClientSocket.h"
+#include "utils/TimeUtil.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "provenance/Provenance.h"
+#include "FlowController.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace provenance{
+
+const std::string
ProvenanceTaskReport::ProcessorName("ProvenanceTaskReport");
+core::Property ProvenanceTaskReport::hostName("Host Name", "Remote Host
Name.", "localhost");
+core::Property ProvenanceTaskReport::port("Port", "Remote Port", "");
+core::Property ProvenanceTaskReport::batchSize("Batch Size", "Specifies
how many records to send in a single batch, at most.", "100");
+core::Relationship ProvenanceTaskReport::relation;
+
+void ProvenanceTaskReport::initialize()
+{
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(hostName);
+ properties.insert(port);
+ properties.insert(batchSize);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(relation);
+ setSupportedRelationships(relationships);
+}
+
--- End diff --
I don't follow. How does duplicating code decouple and add flexibility?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109792063
--- Diff: libminifi/src/provenance/ProvenanceRepository.cpp ---
@@ -36,6 +36,7 @@ void ProvenanceRepository::run() {
uint64_t curTime = getTimeMillis();
uint64_t size = repoSize();
if (size >= purgeThreshold) {
+ std::lock_guard lock(mutex_);
--- End diff --
If LevelDB is thread safe and is being picked up that means at that moment
in the thread which picks it it is not eligible to be purged and is eligible to
be sent. That means you can atomically remove said item and thread A will never
get it to send OR lock all access. If you need locking here above LevelDB we
should change the access pattern.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109742314
--- Diff: libminifi/src/provenance/ProvenanceTaskReport.cpp ---
@@ -0,0 +1,221 @@
+/**
+ * @file ProvenanceTaskReport.cpp
+ * ProvenanceTaskReport class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "provenance/ProvenanceTaskReport.h"
+#include "../include/io/StreamFactory.h"
+#include "io/ClientSocket.h"
+#include "utils/TimeUtil.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "provenance/Provenance.h"
+#include "FlowController.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace provenance{
+
+const std::string
ProvenanceTaskReport::ProcessorName("ProvenanceTaskReport");
+core::Property ProvenanceTaskReport::hostName("Host Name", "Remote Host
Name.", "localhost");
+core::Property ProvenanceTaskReport::port("Port", "Remote Port", "");
+core::Property ProvenanceTaskReport::batchSize("Batch Size", "Specifies
how many records to send in a single batch, at most.", "100");
+core::Relationship ProvenanceTaskReport::relation;
+
+void ProvenanceTaskReport::initialize()
+{
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(hostName);
+ properties.insert(port);
+ properties.insert(batchSize);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(relation);
+ setSupportedRelationships(relationships);
+}
+
+std::unique_ptr
ProvenanceTaskReport::getNextProtocol()
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ if (available_protocols_.empty())
+ return nullptr;
+ std::unique_ptr return_pointer =
std::move(available_protocols_.top());
+ available_protocols_.pop();
+ return std::move(return_pointer);
+}
+
+void ProvenanceTaskReport::returnProtocol(
+ std::unique_ptr return_protocol)
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ available_protocols_.push(std::move(return_protocol));
+}
+
+void ProvenanceTaskReport::onTrigger(core::ProcessContext *context,
core::ProcessSession *session)
+{
+ std::string value;
+ int64_t lvalue;
+
+ std::unique_ptr protocol_ = getNextProtocol();
+
+ if (protocol_ == nullptr)
+ {
+ protocol_ = std::unique_ptr(
+ new Site2SiteClientProtocol(0));
+ protocol_->setPortId(protocol_uuid_);
+
+ std::string host = "";
+ uint16_t sport = 0;
+
+ if (context->getProperty(hostName.getName(), value)) {
+ host = value;
+ }
+ if (context->getProperty(port.getName(), value)
+ && core::Property::StringToInt(value, lvalue)) {
+ sport = (uint16_t) lvalue;
+ }
+ std::unique_ptr str =
+ std::unique_ptr(
+ org::apache::nifi::minifi::io::StreamFactory::getInstance()
+ ->createSocket(host, sport));
+
+ std::unique_ptr peer_ =
std::unique_ptr(
+ new Site2SitePeer(std::move(str), host, sport));
+
+ protocol_->setPeer(std::move(peer_));
+ }
+
+ if (!protocol_->bootstrap())
+ {
+ // bootstrap the client protocol if needeed
+ context->yield();
+ std::shared_ptr processo
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109745089
--- Diff: libminifi/src/provenance/ProvenanceTaskReport.cpp ---
@@ -0,0 +1,221 @@
+/**
+ * @file ProvenanceTaskReport.cpp
+ * ProvenanceTaskReport class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "provenance/ProvenanceTaskReport.h"
+#include "../include/io/StreamFactory.h"
+#include "io/ClientSocket.h"
+#include "utils/TimeUtil.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "provenance/Provenance.h"
+#include "FlowController.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace provenance{
+
+const std::string
ProvenanceTaskReport::ProcessorName("ProvenanceTaskReport");
+core::Property ProvenanceTaskReport::hostName("Host Name", "Remote Host
Name.", "localhost");
+core::Property ProvenanceTaskReport::port("Port", "Remote Port", "");
+core::Property ProvenanceTaskReport::batchSize("Batch Size", "Specifies
how many records to send in a single batch, at most.", "100");
+core::Relationship ProvenanceTaskReport::relation;
+
+void ProvenanceTaskReport::initialize()
+{
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(hostName);
+ properties.insert(port);
+ properties.insert(batchSize);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(relation);
+ setSupportedRelationships(relationships);
+}
+
+std::unique_ptr
ProvenanceTaskReport::getNextProtocol()
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ if (available_protocols_.empty())
+ return nullptr;
+ std::unique_ptr return_pointer =
std::move(available_protocols_.top());
+ available_protocols_.pop();
+ return std::move(return_pointer);
+}
+
+void ProvenanceTaskReport::returnProtocol(
+ std::unique_ptr return_protocol)
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ available_protocols_.push(std::move(return_protocol));
+}
+
+void ProvenanceTaskReport::onTrigger(core::ProcessContext *context,
core::ProcessSession *session)
+{
+ std::string value;
+ int64_t lvalue;
+
+ std::unique_ptr protocol_ = getNextProtocol();
+
+ if (protocol_ == nullptr)
+ {
+ protocol_ = std::unique_ptr(
+ new Site2SiteClientProtocol(0));
+ protocol_->setPortId(protocol_uuid_);
+
+ std::string host = "";
+ uint16_t sport = 0;
+
+ if (context->getProperty(hostName.getName(), value)) {
+ host = value;
+ }
+ if (context->getProperty(port.getName(), value)
+ && core::Property::StringToInt(value, lvalue)) {
+ sport = (uint16_t) lvalue;
+ }
+ std::unique_ptr str =
+ std::unique_ptr(
+ org::apache::nifi::minifi::io::StreamFactory::getInstance()
+ ->createSocket(host, sport));
+
+ std::unique_ptr peer_ =
std::unique_ptr(
+ new Site2SitePeer(std::move(str), host, sport));
+
+ protocol_->setPeer(std::move(peer_));
+ }
+
+ if (!protocol_->bootstrap())
+ {
+ // bootstrap the client protocol if needeed
+ context->yield();
+ std::shared_ptr processo
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109741638
--- Diff: libminifi/src/provenance/ProvenanceRepository.cpp ---
@@ -36,6 +36,7 @@ void ProvenanceRepository::run() {
uint64_t curTime = getTimeMillis();
uint64_t size = repoSize();
if (size >= purgeThreshold) {
+ std::lock_guard lock(mutex_);
--- End diff --
What makes this necessary? is there a way to avoid locking? LevelDB is
thread safe due to process level locking.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109742049
--- Diff: libminifi/src/provenance/ProvenanceTaskReport.cpp ---
@@ -0,0 +1,221 @@
+/**
+ * @file ProvenanceTaskReport.cpp
+ * ProvenanceTaskReport class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "provenance/ProvenanceTaskReport.h"
+#include "../include/io/StreamFactory.h"
+#include "io/ClientSocket.h"
+#include "utils/TimeUtil.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "provenance/Provenance.h"
+#include "FlowController.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace provenance{
+
+const std::string
ProvenanceTaskReport::ProcessorName("ProvenanceTaskReport");
+core::Property ProvenanceTaskReport::hostName("Host Name", "Remote Host
Name.", "localhost");
+core::Property ProvenanceTaskReport::port("Port", "Remote Port", "");
+core::Property ProvenanceTaskReport::batchSize("Batch Size", "Specifies
how many records to send in a single batch, at most.", "100");
+core::Relationship ProvenanceTaskReport::relation;
+
+void ProvenanceTaskReport::initialize()
+{
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(hostName);
+ properties.insert(port);
+ properties.insert(batchSize);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(relation);
+ setSupportedRelationships(relationships);
+}
+
+std::unique_ptr
ProvenanceTaskReport::getNextProtocol()
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ if (available_protocols_.empty())
+ return nullptr;
+ std::unique_ptr return_pointer =
std::move(available_protocols_.top());
+ available_protocols_.pop();
+ return std::move(return_pointer);
+}
+
+void ProvenanceTaskReport::returnProtocol(
+ std::unique_ptr return_protocol)
+{
+ std::lock_guard protocol_lock_(protocol_mutex_);
+ available_protocols_.push(std::move(return_protocol));
+}
+
+void ProvenanceTaskReport::onTrigger(core::ProcessContext *context,
core::ProcessSession *session)
+{
+ std::string value;
+ int64_t lvalue;
+
+ std::unique_ptr protocol_ = getNextProtocol();
+
+ if (protocol_ == nullptr)
+ {
+ protocol_ = std::unique_ptr(
+ new Site2SiteClientProtocol(0));
+ protocol_->setPortId(protocol_uuid_);
+
+ std::string host = "";
+ uint16_t sport = 0;
+
+ if (context->getProperty(hostName.getName(), value)) {
+ host = value;
+ }
+ if (context->getProperty(port.getName(), value)
+ && core::Property::StringToInt(value, lvalue)) {
+ sport = (uint16_t) lvalue;
+ }
+ std::unique_ptr str =
+ std::unique_ptr(
+ org::apache::nifi::minifi::io::StreamFactory::getInstance()
+ ->createSocket(host, sport));
+
+ std::unique_ptr peer_ =
std::unique_ptr(
+ new Site2SitePeer(std::move(str), host, sport));
+
+ protocol_->setPeer(std::move(peer_));
+ }
+
+ if (!protocol_->bootstrap())
+ {
+ // bootstrap the client protocol if needeed
+ context->yield();
+ std::shared_ptr processo
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109740706 --- Diff: libminifi/src/Site2SiteClientProtocol.cpp --- @@ -682,6 +682,7 @@ bool Site2SiteClientProtocol::receive(std::string transactionID, bool Site2SiteClientProtocol::send( std::string transactionID, DataPacket *packet, std::shared_ptr flowFile, + uint8_t *payload, int length, --- End diff -- Can payload be embedded into DataPacket, thus simplifying this function? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109742909
--- Diff: libminifi/include/provenance/ProvenanceTaskReport.h ---
@@ -0,0 +1,88 @@
+/**
+ * @file ProvenanceTaskReport.h
+ * ProvenanceTaskReport class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __PROVENANCE_TASK_REPORT_H__
+#define __PROVENANCE_TASK_REPORT_H__
+
+#include
+#include
+#include
+#include "FlowFileRecord.h"
+#include "core/Processor.h"
+#include "core/ProcessSession.h"
+#include "Site2SiteClientProtocol.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace provenance {
+
+//! ProvenanceTaskReport Class
+class ProvenanceTaskReport: public core::Processor {
--- End diff --
Would a more applicable name be ProvenanceTaskReporter?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109725610
--- Diff: libminifi/src/provenance/ProvenanceTaskReport.cpp ---
@@ -0,0 +1,221 @@
+/**
+ * @file ProvenanceTaskReport.cpp
+ * ProvenanceTaskReport class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "provenance/ProvenanceTaskReport.h"
+#include "../include/io/StreamFactory.h"
+#include "io/ClientSocket.h"
+#include "utils/TimeUtil.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "provenance/Provenance.h"
+#include "FlowController.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace provenance{
+
+const std::string
ProvenanceTaskReport::ProcessorName("ProvenanceTaskReport");
+core::Property ProvenanceTaskReport::hostName("Host Name", "Remote Host
Name.", "localhost");
+core::Property ProvenanceTaskReport::port("Port", "Remote Port", "");
+core::Property ProvenanceTaskReport::batchSize("Batch Size", "Specifies
how many records to send in a single batch, at most.", "100");
+core::Relationship ProvenanceTaskReport::relation;
+
+void ProvenanceTaskReport::initialize()
+{
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(hostName);
+ properties.insert(port);
+ properties.insert(batchSize);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(relation);
+ setSupportedRelationships(relationships);
+}
+
--- End diff --
This is identical to code in RemoteProcessGroup, so this should likely be
abstracted elsewhere.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #74: Minifi 227
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/74#discussion_r109726099 --- Diff: thirdparty/jsoncpp/devtools/batchbuild.py --- @@ -0,0 +1,278 @@ +from __future__ import print_function --- End diff -- Are the tests or devtools or tests needed for jsoncpp? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
