lordgamez commented on code in PR #1712:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1712#discussion_r1500708936
##########
extensions/python/ExecutePythonProcessor.cpp:
##########
@@ -146,11 +151,38 @@ std::unique_ptr<PythonScriptEngine>
ExecutePythonProcessor::createScriptEngine()
auto engine = std::make_unique<PythonScriptEngine>();
python_logger_ =
core::logging::LoggerFactory<ExecutePythonProcessor>::getAliasedLogger(getName());
- engine->initialize(Success, Failure, python_logger_);
+ engine->initialize(Success, Failure, Original, python_logger_);
return engine;
}
+core::Property* ExecutePythonProcessor::findProperty(const std::string& name)
const {
+ if (auto prop_ptr = core::ConfigurableComponent::findProperty(name)) {
+ return prop_ptr;
+ }
+
+ auto it = ranges::find_if(python_properties_, [&name](const auto& item){
+ return item.getName() == name;
+ });
+ if (it != python_properties_.end()) {
+ return const_cast<core::Property*>(&*it);
+ }
+
+ return nullptr;
+}
+
+std::map<std::string, core::Property> ExecutePythonProcessor::getProperties()
const {
+ auto result = ConfigurableComponent::getProperties();
+
+ std::lock_guard<std::mutex> lock(configuration_mutex_);
Review Comment:
You are right, it should be a separate mutex for the `python_properties_`,
it didn't really make sense this way. Updated in
6593d9b63514dd2a3b2060b747680f37691abecd
##########
extensions/python/PYTHON.md:
##########
@@ -106,20 +127,39 @@ class VaderSentiment(object):
To enable python Processor capabilities, the following options need to be
provided in minifi.properties. The directory specified
can contain processors. Note that the processor name will be the reference in
your flow. Directories are treated like package names.
Therefore if the nifi.python.processor.dir is /tmp/ and you have a
subdirectory named packagedir with the file name file.py, it will
-produce a processor with the name
org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory
will append a package
-to the reference class name.
+produce a processor with the name
org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory
will append a package
+to the reference class name.
in minifi.properties
#directory where processors exist
nifi.python.processor.dir=XXXX
-
-
+
+
## Processors
The python directory (extensions/pythonprocessors) contains implementations
that will be available for flows if the required dependencies
exist.
-
-## Sentiment Analysis
+
+### Sentiment Analysis
The SentimentAnalysis processor will perform a Vader Sentiment Analysis. This
requires that you install nltk and VaderSentiment
pip install nltk
pip install VaderSentiment
+
+## Using NiFi Python Processors
+
+MiNiFi C++ supports the use of NiFi Python processors, that are inherited from
the FlowFileTransform base class. To use these processors, you must copy the
Python processor module under the nifi_python_processors directory located
under the python directory (by default it can be found at
${minifi_root}/minifi-python/nifi_python_processors). To see how to write NiFi
Python processors, please refer to the Python Developer Guide under the [Apache
NiFi documentation](https://nifi.apache.org/documentation/v2/).
+
+In the flow configuration these Python processors can be referenced by their
fully qualified class name, which looks like this:
org.apache.nifi.minifi.processors.nifi_python_processors.<package_name>.<processor_name>.
For example, the fully qualified class name of the PromptChatGPT processor
implemented in the file nifi_python_processors/PromptChatGPT.py is
org.apache.nifi.minifi.processors.nifi_python_processors.PromptChatGPT. If a
processor is copied under a subdirectory, because it is part of a python
submodule, the submodule name will be appended to the fully qualified class
name. For example, if the QueryPinecone processor is implemented in the
QueryPinecone.py file that is copied to
nifi_python_processors/vectorstores/QueryPinecone.py, the fully qualified class
name will be
org.apache.nifi.minifi.processors.nifi_python_processors.vectorstores.QueryPinecone
in the configuration file.
+
+**NOTE:** The name of the NiFi Python processor file should match the class
name in the file, otherwise the processor will not be found.
+
+Due to some differences between the NiFi and MiNiFi C++ processors and
implementation, there are some limitations using the NiFi Python processors:
+- Record based processors are not yet supported in MiNiFi C++, so the NiFi
Python processors inherited from RecordTransform are not supported.
+- Virtualenv support is not yet available in MiNiFi C++, so all required
packaged must be installed on the system.
Review Comment:
Updated in 6593d9b63514dd2a3b2060b747680f37691abecd
##########
extensions/python/PYTHON.md:
##########
@@ -106,20 +127,39 @@ class VaderSentiment(object):
To enable python Processor capabilities, the following options need to be
provided in minifi.properties. The directory specified
can contain processors. Note that the processor name will be the reference in
your flow. Directories are treated like package names.
Therefore if the nifi.python.processor.dir is /tmp/ and you have a
subdirectory named packagedir with the file name file.py, it will
-produce a processor with the name
org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory
will append a package
-to the reference class name.
+produce a processor with the name
org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory
will append a package
+to the reference class name.
in minifi.properties
#directory where processors exist
nifi.python.processor.dir=XXXX
-
-
+
+
## Processors
The python directory (extensions/pythonprocessors) contains implementations
that will be available for flows if the required dependencies
exist.
-
-## Sentiment Analysis
+
+### Sentiment Analysis
The SentimentAnalysis processor will perform a Vader Sentiment Analysis. This
requires that you install nltk and VaderSentiment
pip install nltk
pip install VaderSentiment
+
+## Using NiFi Python Processors
+
+MiNiFi C++ supports the use of NiFi Python processors, that are inherited from
the FlowFileTransform base class. To use these processors, you must copy the
Python processor module under the nifi_python_processors directory located
under the python directory (by default it can be found at
${minifi_root}/minifi-python/nifi_python_processors). To see how to write NiFi
Python processors, please refer to the Python Developer Guide under the [Apache
NiFi documentation](https://nifi.apache.org/documentation/v2/).
Review Comment:
Updated in 6593d9b63514dd2a3b2060b747680f37691abecd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
