[
https://issues.apache.org/jira/browse/MINIFICPP-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Hunyadi updated MINIFICPP-1355:
------------------------------------
Description:
*Acceptance criteria:*
GIVEN a flow set up in EFM illustrated below
WHEN the flow is with a python script set to add a new attribute to a flow file
THEN no error is produced and the newly added attribute is logged in
LogAttribute
Example script:
{code:c++|title=Trace of where the property is overridden}
def describe(processor):
processor.setDescription("Adds an attribute to your flow files")
def onInitialize(processor):
processor.setSupportsDynamicProperties()
def onTrigger(context, session):
flow_file = session.get()
if flow_file is not None:
flow_file.addAttribute("Python attribute","attributevalue")
session.transfer(flow_file, REL_SUCCESS)
{code}
*Background:*
Currently, even though the tests for ExecutePythonProcessor are passing, if I
were to try and load up a configuration that contains an
ExecutePythonProcessor, it fails due to trying to load an incorrect script file.
Sample flow:
!Screenshot 2020-09-04 at 16.02.41.png|width=467,height=100!
When trying to check in debugger, it seems like the processors script file is
always replaced with an incorrect one, and the processor fails to start.
!https://files.slack.com/files-pri/T024BEHTP-F01942KD4BV/screenshot_2020-08-19_at_13.08.46.png|width=1427,height=288!
This is how it is set:
{code:c++|title=Trace of where the property is overridden}
ConfigurableComponent::setProperty()
std::shared_ptr<core::CoreComponent> create()
ClassLoader::instantiate()
PythonCreator::configure() <- here the first element of classpaths_ is read to
overwrite the config
FlowController::initializeExternalComponents()
{code}
When trying to perform the same thing on the 0.7.0 release version, the startup
already shows some kind of errors, although they seem different:
{code:python|title=Error log}
[2020-09-04 15:49:53.424]
[org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] [error]
Caught Exception Mod uleNotFoundError: No module named 'google'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
<module>
[2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator]
[warning] Cannot load SentimentAnalyzer because of ModuleNotFoundError: No
module named 'google'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
<module>
[2020-09-04 15:49:53.424]
[org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] [error]
Caught Exception ModuleNotFoundError: No module named 'vaderSentiment'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
<module>
[2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator]
[warning] Cannot load SentimentAnalysis because of ModuleNotFoundError: No
module named 'vaderSentiment'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
<module>
{code}
*Proposal:*
One should investigate and fix the error.
was:
*Background:*
Currently, even though the tests for ExecutePythonProcessor are passing, if I
were to try and load up a configuration that contains an
ExecutePythonProcessor, it fails due to trying to load an incorrect script file.
Sample flow:
!Screenshot 2020-09-04 at 16.02.41.png|width=467,height=100!
When trying to check in debugger, it seems like the processors script file is
always replaced with an incorrect one, and the processor fails to start.
!https://files.slack.com/files-pri/T024BEHTP-F01942KD4BV/screenshot_2020-08-19_at_13.08.46.png|width=1427,height=288!
This is how it is set:
{code:c++|title=Trace of where the property is overridden}
ConfigurableComponent::setProperty()
std::shared_ptr<core::CoreComponent> create()
ClassLoader::instantiate()
PythonCreator::configure() <- here the first element of classpaths_ is read to
overwrite the config
FlowController::initializeExternalComponents()
{code}
When trying to perform the same thing on the 0.7.0 release version, the startup
already shows some kind of errors, although they seem different:
{code:python|title=Error log}
[2020-09-04 15:49:53.424]
[org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] [error]
Caught Exception Mod uleNotFoundError: No module named 'google'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
<module>
[2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator]
[warning] Cannot load SentimentAnalyzer because of ModuleNotFoundError: No
module named 'google'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
<module>
[2020-09-04 15:49:53.424]
[org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] [error]
Caught Exception ModuleNotFoundError: No module named 'vaderSentiment'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
<module>
[2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator]
[warning] Cannot load SentimentAnalysis because of ModuleNotFoundError: No
module named 'vaderSentiment'
At:
/Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
<module>
{code}
> Investigate and fix the initialization of ExecutePythonProcessor
> ----------------------------------------------------------------
>
> Key: MINIFICPP-1355
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1355
> Project: Apache NiFi MiNiFi C++
> Issue Type: Task
> Affects Versions: 0.7.0
> Reporter: Adam Hunyadi
> Priority: Minor
> Labels: MiNiFi-CPP-Hygiene
> Fix For: 1.0.0
>
> Attachments: Screenshot 2020-09-04 at 16.02.41.png
>
>
> *Acceptance criteria:*
> GIVEN a flow set up in EFM illustrated below
> WHEN the flow is with a python script set to add a new attribute to a flow
> file
> THEN no error is produced and the newly added attribute is logged in
> LogAttribute
> Example script:
> {code:c++|title=Trace of where the property is overridden}
> def describe(processor):
> processor.setDescription("Adds an attribute to your flow files")
> def onInitialize(processor):
> processor.setSupportsDynamicProperties()
> def onTrigger(context, session):
> flow_file = session.get()
> if flow_file is not None:
> flow_file.addAttribute("Python attribute","attributevalue")
> session.transfer(flow_file, REL_SUCCESS)
> {code}
> *Background:*
> Currently, even though the tests for ExecutePythonProcessor are passing, if I
> were to try and load up a configuration that contains an
> ExecutePythonProcessor, it fails due to trying to load an incorrect script
> file.
> Sample flow:
> !Screenshot 2020-09-04 at 16.02.41.png|width=467,height=100!
> When trying to check in debugger, it seems like the processors script file is
> always replaced with an incorrect one, and the processor fails to start.
> !https://files.slack.com/files-pri/T024BEHTP-F01942KD4BV/screenshot_2020-08-19_at_13.08.46.png|width=1427,height=288!
> This is how it is set:
> {code:c++|title=Trace of where the property is overridden}
> ConfigurableComponent::setProperty()
> std::shared_ptr<core::CoreComponent> create()
> ClassLoader::instantiate()
> PythonCreator::configure() <- here the first element of classpaths_ is read
> to overwrite the config
> FlowController::initializeExternalComponents()
> {code}
> When trying to perform the same thing on the 0.7.0 release version, the
> startup already shows some kind of errors, although they seem different:
> {code:python|title=Error log}
> [2020-09-04 15:49:53.424]
> [org::apache::nifi::minifi::python::processors::ExecutePythonProcessor]
> [error] Caught Exception Mod uleNotFoundError: No module named 'google'
> At:
>
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
> <module>
> [2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator]
> [warning] Cannot load SentimentAnalyzer because of ModuleNotFoundError: No
> module named 'google'
> At:
>
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
> <module>
> [2020-09-04 15:49:53.424]
> [org::apache::nifi::minifi::python::processors::ExecutePythonProcessor]
> [error] Caught Exception ModuleNotFoundError: No module named 'vaderSentiment'
> At:
>
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
> <module>
> [2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator]
> [warning] Cannot load SentimentAnalysis because of ModuleNotFoundError: No
> module named 'vaderSentiment'
> At:
>
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
> <module>
> {code}
> *Proposal:*
> One should investigate and fix the error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
