[jira] [Resolved] (MINIFICPP-994) Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it should be part of MiNiFi build.

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-994.
-
Fix Version/s: 0.11.0
   Resolution: Fixed

> Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it 
> should be part of MiNiFi build.
> 
>
> Key: MINIFICPP-994
> URL: https://issues.apache.org/jira/browse/MINIFICPP-994
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.6.0
>Reporter: Arpad Boda
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Opencv is built when executing a cmake command (for eg. using bootstrap), 
> which seems to be wrong as:
>  * This build is single-threaded, takes a lot of time
>  * Linker warnings are generated when building MiNiFI:
> {code}
> ld: warning: direct access in function '___cxx_global_var_init.35' from file 
> '../thirdparty/opencv/opencv-external/src/opencv-external-build/lib/libopencv_flann.a(miniflann.cpp.o)'
>  to global weak symbol 'guard variable for 
> cvflann::anyimpl::SinglePolicy std::__1::char_traits, std::__1::allocator > >::policy' from file 
> 'CMakeFiles/CaptureRTSPFrameTest.dir/CaptureRTSPFrameTest.cpp.o' means the 
> weak symbol cannot be overridden at runtime. This was likely caused by 
> different translation units being compiled with different visibility settings.
> {code}
> Static linking should be kept when integrating opencv build to the build 
> process of MiNiFi.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1367) Add cmake flag for disabling building nanofi

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1367.
--
Fix Version/s: 0.11.0
   Resolution: Fixed

> Add cmake flag for disabling building nanofi
> 
>
> Key: MINIFICPP-1367
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1367
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 1.0.0, 0.11.0
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> *Background:*
> Currently nanofi automatically builds alongside minifi contributing to its 
> build time.
> *Proposal:*
> We should define a cmake flag (and add an option to toggle it to 
> bootstrap.sh), so that we can skip building it when we do not need it. We can 
> have the option default to building nanofi or disable it but make make sure 
> that at least one CI jobs performs a nanofi build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1601) Add flowfile examples to repository

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1601.
--
Fix Version/s: 0.11.0
   Resolution: Fixed

> Add flowfile examples to repository
> ---
>
> Key: MINIFICPP-1601
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1601
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Task
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Add some config yaml files to the repository as examples for common use cases.
> The examples should include processors like: TailFile, GetFile, PutFile, 
> LogAttribute, 
> PerformanceDataMonitor, ConsumeWindowsEventLog



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1355) Investigate and fix the initialization of ExecutePythonProcessor

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1355:
-
Status: Patch Available  (was: In Progress)

> Investigate and fix the initialization of ExecutePythonProcessor
> 
>
> Key: MINIFICPP-1355
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1355
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 1.0.0
>
> Attachments: Screenshot 2020-09-04 at 16.02.41.png
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> *Acceptance criteria:*
>  - GIVEN a flow set up as highlighted in blue below
>  - WHEN the flow is with a python script set to add a new attribute to a flow 
> file
>  - THEN no error is produced and the newly added attribute is logged in 
> LogAttribute
> {code:c++|title=Example script}
> def describe(processor):
> processor.setDescription("Adds an attribute to your flow files")
> def onInitialize(processor):
>   processor.setSupportsDynamicProperties()
> def onTrigger(context, session):
>   flow_file = session.get()
>   if flow_file is not None:
> flow_file.addAttribute("Python attribute","attributevalue")
> session.transfer(flow_file, REL_SUCCESS)
> {code}
> *Background:*
> Currently, even though the tests for ExecutePythonProcessor are passing, if I 
> were to try and load up a configuration that contains an 
> ExecutePythonProcessor, it fails due to trying to load an incorrect script 
> file.
> Sample flow:
>  {color:#0747a6}GenerateFlowFile -(success)-> ExecutePythonProcessor 
> -(success,failure)-> LogAttribute{color}
> When trying to check in debugger, it seems like the processors script file is 
> always replaced with an incorrect one, and the processor fails to start.
> This is how it is set:
> {code:c++|title=Trace of where the property is overridden}
> ConfigurableComponent::setProperty()
> std::shared_ptr create()
> ClassLoader::instantiate()
> PythonCreator::configure() <- here the first element of classpaths_ is read 
> to overwrite the config
> FlowController::initializeExternalComponents()
> {code}
> When trying to perform the same thing on the 0.7.0 release version, the 
> startup already shows some kind of errors, although they seem different:
> {code:python|title=Error log}
> [2020-09-04 15:49:53.424] 
> [org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] 
> [error] Caught Exception ModuleNotFoundError: No module named 'google'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
>  
> [2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator] 
> [warning] Cannot load SentimentAnalyzer because of ModuleNotFoundError: No 
> module named 'google'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
>  
> [2020-09-04 15:49:53.424] 
> [org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] 
> [error] Caught Exception ModuleNotFoundError: No module named 'vaderSentiment'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
>  
> [2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator] 
> [warning] Cannot load SentimentAnalysis because of ModuleNotFoundError: No 
> module named 'vaderSentiment'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
>  
> {code}
> *Proposal:*
> One should investigate and fix the error.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1596) Conceptual problems in docker integration tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1596:
-
Status: Patch Available  (was: In Progress)

> Conceptual problems in docker integration tests
> ---
>
> Key: MINIFICPP-1596
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1596
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Test
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The concept "cluster" is misused in docker integration tests. There should be 
> a single cluster containing all the containers on a single network. As there 
> are multiple clusters now with the same network the cleanup process is 
> convoluted and flaky.
> The "cluster" and the "container" concepts are also mixed up in places, so 
> there should be an explicit Container concept in the code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1587) The windows-event-log extension does not compile if there is a space in the path

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1587.
--
Resolution: Fixed

> The windows-event-log extension does not compile if there is a space in the 
> path
> 
>
> Key: MINIFICPP-1587
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1587
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> If you check out the MiNiFi code into a directory which contains a space 
> (e.g. {{C:\Users\Jane Doe\src\minifi}}), then the {{windows-event-log}} 
> extension fails to compile with an error at 
> {{CWELCustomProviderTests.cpp:37}}, with the message "error C1083: Cannot 
> open include file: 'custom-provider/unit-test-provider.h': No such file or 
> directory".
> This is because the {{generate-and-register.bat}} script in 
> {{extensions/windows-event-log/tests/CMakeLists.txt}} fails, with an error 
> similar to
> {noformat}
> Doe\src\minifi\extensions\windows-event-log\tests\custom-provider"/unit-test-provider.man"
>  was unexpected at this time.
> {noformat}
> I suspect more quoting or escaping is needed in {{generate-and-register.bat}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1516) Fix transient issues related to docker container cleanup

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1516:
-
Status: Patch Available  (was: In Progress)

> Fix transient issues related to docker container cleanup
> 
>
> Key: MINIFICPP-1516
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1516
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.9.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 1.0.0
>
>
> *Background:*
> There are still issues due to the python docker API rarely not properly 
> cleaning up containers.
> *Proposal:*
> We should add retry attempts when starting docker containers and potentially 
> try and improve the process of cleaning.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1374) Implement security protocol support for ConsumeKafka

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1374:
-
Status: Patch Available  (was: In Progress)

> Implement security protocol support for ConsumeKafka
> 
>
> Key: MINIFICPP-1374
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1374
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Sub-task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 1.0.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> *Acceptance Criteria:*
> SSL:
> {code:python|title=Example feature definition}
> Scenario Outline: ConsumeKafka receives data via SSL
>   Given a ConsumeKafka processor set up in a "kafka-consumer-flow" flow
>   And ssl certificates are placed in "/tmp/resources" with cert name 
> "cert.pem" and key name "key.pem"
>   And these processor properties are set:
> | processor name | property name | property value |
> | ConsumeKafka | Kafka Brokers | kafka-broker:9093 |
> | ConsumeKafka | Security Protocol | ssl |
> | ConsumeKafka | Security CA | ... |
> | ConsumeKafka | Security Cert | ... |
>   And a PutFile processor with the "Directory" property set to "/tmp/output" 
> in the "kafka-consumer-flow" flow
>   And the "success" relationship of the ConsumeKafka processor is connected 
> to the PutFile
>   And a kafka broker "broker" set up to communicate via SSL is set up in 
> correspondence with the third-party kafka publisher
>   When all instances start up
>   And a message with content "Alice's Adventures in Wonderland" is published 
> to the "ConsumeKafkaTest" topic using an ssl connection
>   And a message with content "Lewis Carroll" is published to the 
> "ConsumeKafkaTest" topic using an ssl connection
>   Then two flowfiles with the contents "Alice's Adventures in Wonderland" and 
> "Lewis Carroll" are placed in the monitored directory in less than 45 seconds
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1541) Move ODBC dependent SQL testing to the docker integration tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1541:
-
Status: Patch Available  (was: In Progress)

> Move ODBC dependent SQL testing to the docker integration tests
> ---
>
> Key: MINIFICPP-1541
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1541
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Adam Debreceni
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> SQL tests currently use platform-dependent ODBC drivers which are possibly 
> missing on the user's machine, this results in test failure. We should mock 
> the ODBCService on Linux and macOS, and use the actual implementation in 
> docker-based integration tests. On Windows we should use the actual 
> implementation but only enable it in the CI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1373) Implement and test a simplified ConsumeKafka processor without security protocols

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1373.
--
Resolution: Fixed

> Implement and test a simplified ConsumeKafka processor without security 
> protocols
> -
>
> Key: MINIFICPP-1373
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1373
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Sub-task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 1.0.0, 0.11.0
>
> Attachments: ConsumeKafka_test_matrix.numbers, 
> ConsumeKafka_test_matrix.pdf
>
>  Time Spent: 36h 50m
>  Remaining Estimate: 0h
>
> *Acceptance Criteria:*
> *{color:#de350b}See attached test matrix plan.{color}*
> Additional test (that require multiple Kafka consumers):
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}different group ids{color} subscribed to the same topic
>  {color:#505f79}*WHEN*{color} a message is published to the topic
>  {color:#505f79}*THEN*{color} both of the ConsumeKafka processors should 
> produce identical flowfiles
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic
>  {color:#505f79}*WHEN*{color} a message is published to the topic
>  {color:#505f79}*THEN*{color} only one of the ConsumeKafka processors should 
> produce a flowfile
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}earliest{color}.
>  {color:#505f79}*WHEN*{color} a messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should process duplicates of the 
> messages that originally came to the second (at_least_once delivery)
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}latest{color}.
>  {color:#505f79}*WHEN a*{color} messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should {color:#0747a6}not{color} 
> process duplicates of the messages that originally came to the second 
> (at_most_once delivery)
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}none{color}.
>  {color:#505f79}*WHEN*{color} a messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should throw an exception
> {quote}
> *Background:*
> See parent task.
> *Proposal:*
> This should be the first part of the implementation, the second being adding 
> and testing multiple security protocols.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1607) AttributesToJson processor

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1607:
-
Status: Patch Available  (was: In Progress)

> AttributesToJson processor
> --
>
> Key: MINIFICPP-1607
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1607
> Project: Apache NiFi MiNiFi C++
>  Issue Type: New Feature
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Create an AttributesToJson processor similar to the Java version.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1607) AttributesToJson processor

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1607:


 Summary: AttributesToJson processor
 Key: MINIFICPP-1607
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1607
 Project: Apache NiFi MiNiFi C++
  Issue Type: New Feature
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


Create an AttributesToJson processor similar to the Java version.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1605) Renewing AWS credentials not working when using instance profiles

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1605.
--
Resolution: Fixed

> Renewing AWS credentials not working when using instance profiles
> -
>
> Key: MINIFICPP-1605
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1605
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> When using AWS instance profile for AWS processors through AWS Credentials 
> Service the credentials can expire after a few days. According to the AWS 
> developer support (https://github.com/aws/aws-sdk-cpp/issues/1684) when using 
> instance profiles we should not cache the credentials as due to a time skew 
> problem the credentials may expire on AWS side even though it can be 
> non-expired when checked on client side.
> When using default credentials chain we should let the AWS SDK handle the 
> caching and renewing the credentials and not cache manually on client side.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1605) Renewing AWS credentials not working when using instance profiles

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1605:


 Summary: Renewing AWS credentials not working when using instance 
profiles
 Key: MINIFICPP-1605
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1605
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


When using AWS instance profile for AWS processors through AWS Credentials 
Service the credentials can expire after a few days. According to the AWS 
developer support (https://github.com/aws/aws-sdk-cpp/issues/1684) when using 
instance profiles we should not cache the credentials as due to a time skew 
problem the credentials may expire on AWS side even though it can be 
non-expired when checked on client side.

When using default credentials chain we should let the AWS SDK handle the 
caching and renewing the credentials and not cache manually on client side.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1603) Support Funnel element in flow definition

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1603:


 Summary: Support Funnel element in flow definition
 Key: MINIFICPP-1603
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1603
 Project: Apache NiFi MiNiFi C++
  Issue Type: New Feature
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


In NiFi we have the possibility to add a funnel element to the flow definition 
to merge multiple incoming connections. MiNiFi agent currently does not support 
this which needs to be added. The handling of a new "Funnels" element in the 
flow configuration yaml file should be added:

 
{code:java}
Funnels:
- id: 88d4eb63-73a9-4e24-93e0-b0af3ef4e010
{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1599) Patch prompt can halt Azure build

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1599.
--
Fix Version/s: 0.11.0
   Resolution: Fixed

> Patch prompt can halt Azure build
> -
>
> Key: MINIFICPP-1599
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1599
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Checking patch application in Azure build may show a prompt that can halt the 
> build process.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1598) Kafka Offset Reset test scenario transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1598.
--
Fix Version/s: 0.11.0
   Resolution: Fixed

> Kafka Offset Reset test scenario transiently fails
> --
>
> Key: MINIFICPP-1598
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1598
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
> Attachments: consumekafka_failure.log
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> "Scenario: ConsumeKafka consumes only messages after MiNiFi startup when 
> "Offset Reset" property is set to latest" transiently fails due to a timing 
> issue in synchronization between the kafka broker and the consumer.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1492) The bundled PCAP++ is not compatible with linux kernel 5.5(or higher)

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1492:
-
Fix Version/s: 0.11.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

> The bundled PCAP++ is not compatible with linux kernel 5.5(or higher)
> -
>
> Key: MINIFICPP-1492
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1492
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Martin Zink
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The bundled pcap++ is not compatible with linux kernel 5.5 (or higher), due 
> to the removal of sys/sysctl.h 
> [https://man7.org/linux/man-pages/man2/sysctl.2.html]
> Causing a build error:
> {code:java}
> src/PcapLiveDevice.cpp:25:10: fatal error: sys/sysctl.h: No such file or 
> directory 25 | #include 
> {code}
> This is fixed in newer PCAP++ versions, it also pretty easy to backport to 
> our version, 
> [https://github.com/seladb/PcapPlusPlus/commit/f474316b2fcdcd3b2d6ab6143ef4c3b2b5ec9ea8]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1601) Add flowfile examples to repository

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1601:


 Summary: Add flowfile examples to repository
 Key: MINIFICPP-1601
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1601
 Project: Apache NiFi MiNiFi C++
  Issue Type: Task
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


Add some config yaml files to the repository as examples for common use cases.

The examples should include processors like: TailFile, GetFile, PutFile, 
LogAttribute, 
PerformanceDataMonitor, ConsumeWindowsEventLog



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1599) Patch prompt can halt Azure build

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1599:


 Summary: Patch prompt can halt Azure build
 Key: MINIFICPP-1599
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1599
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


Checking patch application in Azure build may show a prompt that can halt the 
build process.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1367) Add cmake flag for disabling building nanofi

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1367:


Assignee: Gabor Gyimesi

> Add cmake flag for disabling building nanofi
> 
>
> Key: MINIFICPP-1367
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1367
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 1.0.0
>
>
> *Background:*
> Currently nanofi automatically builds alongside minifi contributing to its 
> build time.
> *Proposal:*
> We should define a cmake flag (and add an option to toggle it to 
> bootstrap.sh), so that we can skip building it when we do not need it. We can 
> have the option default to building nanofi or disable it but make make sure 
> that at least one CI jobs performs a nanofi build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1597) Release binaries for CentOS 7

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1597.
--
Resolution: Fixed

> Release binaries for CentOS 7
> -
>
> Key: MINIFICPP-1597
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1597
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> CentOS base image for building CentOS binaries was upgraded to version 8, but 
> we still want to support CentOS 7, this should be reverted.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1597) Release binaries for CentOS 7

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1597:
-
Fix Version/s: 0.11.0

> Release binaries for CentOS 7
> -
>
> Key: MINIFICPP-1597
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1597
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 0.11.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> CentOS base image for building CentOS binaries was upgraded to version 8, but 
> we still want to support CentOS 7, this should be reverted.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1516) Fix transient issues related to docker container cleanup

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1516:


Assignee: Gabor Gyimesi  (was: Adam Hunyadi)

> Fix transient issues related to docker container cleanup
> 
>
> Key: MINIFICPP-1516
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1516
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.9.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 1.0.0
>
>
> *Background:*
> There are still issues due to the python docker API rarely not properly 
> cleaning up containers.
> *Proposal:*
> We should add retry attempts when starting docker containers and potentially 
> try and improve the process of cleaning.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1598) Kafka Offset Reset test scenario transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1598:
-
Attachment: consumekafka_failure.log

> Kafka Offset Reset test scenario transiently fails
> --
>
> Key: MINIFICPP-1598
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1598
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Attachments: consumekafka_failure.log
>
>
> "Scenario: ConsumeKafka consumes only messages after MiNiFi startup when 
> "Offset Reset" property is set to latest" transiently fails due to a timing 
> issue in synchronization between the kafka broker and the consumer.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1598) Kafka Offset Reset test scenario transiently fails

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1598:


 Summary: Kafka Offset Reset test scenario transiently fails
 Key: MINIFICPP-1598
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1598
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


"Scenario: ConsumeKafka consumes only messages after MiNiFi startup when 
"Offset Reset" property is set to latest" transiently fails due to a timing 
issue in synchronization between the kafka broker and the consumer.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1355) Investigate and fix the initialization of ExecutePythonProcessor

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1355:


Assignee: Gabor Gyimesi

> Investigate and fix the initialization of ExecutePythonProcessor
> 
>
> Key: MINIFICPP-1355
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1355
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 1.0.0
>
> Attachments: Screenshot 2020-09-04 at 16.02.41.png
>
>
> *Acceptance criteria:*
>  - GIVEN a flow set up as highlighted in blue below
>  - WHEN the flow is with a python script set to add a new attribute to a flow 
> file
>  - THEN no error is produced and the newly added attribute is logged in 
> LogAttribute
> {code:c++|title=Example script}
> def describe(processor):
> processor.setDescription("Adds an attribute to your flow files")
> def onInitialize(processor):
>   processor.setSupportsDynamicProperties()
> def onTrigger(context, session):
>   flow_file = session.get()
>   if flow_file is not None:
> flow_file.addAttribute("Python attribute","attributevalue")
> session.transfer(flow_file, REL_SUCCESS)
> {code}
> *Background:*
> Currently, even though the tests for ExecutePythonProcessor are passing, if I 
> were to try and load up a configuration that contains an 
> ExecutePythonProcessor, it fails due to trying to load an incorrect script 
> file.
> Sample flow:
>  {color:#0747a6}GenerateFlowFile -(success)-> ExecutePythonProcessor 
> -(success,failure)-> LogAttribute{color}
> When trying to check in debugger, it seems like the processors script file is 
> always replaced with an incorrect one, and the processor fails to start.
> This is how it is set:
> {code:c++|title=Trace of where the property is overridden}
> ConfigurableComponent::setProperty()
> std::shared_ptr create()
> ClassLoader::instantiate()
> PythonCreator::configure() <- here the first element of classpaths_ is read 
> to overwrite the config
> FlowController::initializeExternalComponents()
> {code}
> When trying to perform the same thing on the 0.7.0 release version, the 
> startup already shows some kind of errors, although they seem different:
> {code:python|title=Error log}
> [2020-09-04 15:49:53.424] 
> [org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] 
> [error] Caught Exception ModuleNotFoundError: No module named 'google'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
>  
> [2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator] 
> [warning] Cannot load SentimentAnalyzer because of ModuleNotFoundError: No 
> module named 'google'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//google/SentimentAnalyzer.py(28):
>  
> [2020-09-04 15:49:53.424] 
> [org::apache::nifi::minifi::python::processors::ExecutePythonProcessor] 
> [error] Caught Exception ModuleNotFoundError: No module named 'vaderSentiment'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
>  
> [2020-09-04 15:49:53.424] [org::apache::nifi::minifi::python::PythonCreator] 
> [warning] Cannot load SentimentAnalysis because of ModuleNotFoundError: No 
> module named 'vaderSentiment'
> At:
>  
> /Users/adamhunyadi/Documents/Projects/integration_tests/minifi_agent_02/build/nifi-minifi-cpp-0.7.0/minifi-python//examples/SentimentAnalysis.py(17):
>  
> {code}
> *Proposal:*
> One should investigate and fix the error.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1587) The windows-event-log extension does not compile if there is a space in the path

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1587:


Assignee: Gabor Gyimesi

> The windows-event-log extension does not compile if there is a space in the 
> path
> 
>
> Key: MINIFICPP-1587
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1587
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>
> If you check out the MiNiFi code into a directory which contains a space 
> (e.g. {{C:\Users\Jane Doe\src\minifi}}), then the {{windows-event-log}} 
> extension fails to compile with an error at 
> {{CWELCustomProviderTests.cpp:37}}, with the message "error C1083: Cannot 
> open include file: 'custom-provider/unit-test-provider.h': No such file or 
> directory".
> This is because the {{generate-and-register.bat}} script in 
> {{extensions/windows-event-log/tests/CMakeLists.txt}} fails, with an error 
> similar to
> {noformat}
> Doe\src\minifi\extensions\windows-event-log\tests\custom-provider"/unit-test-provider.man"
>  was unexpected at this time.
> {noformat}
> I suspect more quoting or escaping is needed in {{generate-and-register.bat}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1492) The bundled PCAP++ is not compatible with linux kernel 5.5(or higher)

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1492:
-
Status: Patch Available  (was: In Progress)

> The bundled PCAP++ is not compatible with linux kernel 5.5(or higher)
> -
>
> Key: MINIFICPP-1492
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1492
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Martin Zink
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The bundled pcap++ is not compatible with linux kernel 5.5 (or higher), due 
> to the removal of sys/sysctl.h 
> [https://man7.org/linux/man-pages/man2/sysctl.2.html]
> Causing a build error:
> {code:java}
> src/PcapLiveDevice.cpp:25:10: fatal error: sys/sysctl.h: No such file or 
> directory 25 | #include 
> {code}
> This is fixed in newer PCAP++ versions, it also pretty easy to backport to 
> our version, 
> [https://github.com/seladb/PcapPlusPlus/commit/f474316b2fcdcd3b2d6ab6143ef4c3b2b5ec9ea8]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1492) The bundled PCAP++ is not compatible with linux kernel 5.5(or higher)

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1492:


Assignee: Gabor Gyimesi

> The bundled PCAP++ is not compatible with linux kernel 5.5(or higher)
> -
>
> Key: MINIFICPP-1492
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1492
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Martin Zink
>Assignee: Gabor Gyimesi
>Priority: Minor
>
> The bundled pcap++ is not compatible with linux kernel 5.5 (or higher), due 
> to the removal of sys/sysctl.h 
> [https://man7.org/linux/man-pages/man2/sysctl.2.html]
> Causing a build error:
> {code:java}
> src/PcapLiveDevice.cpp:25:10: fatal error: sys/sysctl.h: No such file or 
> directory 25 | #include 
> {code}
> This is fixed in newer PCAP++ versions, it also pretty easy to backport to 
> our version, 
> [https://github.com/seladb/PcapPlusPlus/commit/f474316b2fcdcd3b2d6ab6143ef4c3b2b5ec9ea8]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1597) Release binaries for CentOS 7

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1597:


 Summary: Release binaries for CentOS 7
 Key: MINIFICPP-1597
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1597
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


CentOS base image for building CentOS binaries was upgraded to version 8, but 
we still want to support CentOS 7, this should be reverted.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1580) Fix Tensorflow extension build

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1580:
-
Fix Version/s: 0.11.0

> Fix Tensorflow extension build
> --
>
> Key: MINIFICPP-1580
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1580
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1576) Allow build of all extensions in docker images

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1576.
--
Resolution: Fixed

> Allow build of all extensions in docker images
> --
>
> Key: MINIFICPP-1576
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1576
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Build and runtime dependencies of the minifi docker image should be corrected 
> in order to be able to build a docker image with any supported extension.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1580) Fix Tensorflow extension build

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1580.
--
Resolution: Fixed

> Fix Tensorflow extension build
> --
>
> Key: MINIFICPP-1580
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1580
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1576) Allow build of all extensions in docker images

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1576:
-
Fix Version/s: 0.11.0

> Allow build of all extensions in docker images
> --
>
> Key: MINIFICPP-1576
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1576
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Build and runtime dependencies of the minifi docker image should be corrected 
> in order to be able to build a docker image with any supported extension.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1595) Behave tests crash with newer M2Crypto version

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1595:
-
Fix Version/s: 0.11.0

> Behave tests crash with newer M2Crypto version
> --
>
> Key: MINIFICPP-1595
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1595
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 0.11.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Integration tests use the latest version of all pip packages. The latest 
> version of M2Crypto package is not compatible with the latest Behave package 
> so the tests crash at start. Versions should be pinned for every pip package 
> to avoid similar issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1595) Behave tests crash with newer M2Crypto version

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1595.
--
Resolution: Fixed

> Behave tests crash with newer M2Crypto version
> --
>
> Key: MINIFICPP-1595
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1595
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 0.11.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Integration tests use the latest version of all pip packages. The latest 
> version of M2Crypto package is not compatible with the latest Behave package 
> so the tests crash at start. Versions should be pinned for every pip package 
> to avoid similar issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1592:
-
Fix Version/s: 0.11.0

> TailFileTests transiently fails
> ---
>
> Key: MINIFICPP-1592
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1592
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 0.11.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Test case "TailFile reads from a single file when Initial Start Position is 
> set to Current Time with rollover" of TailFileTests suite transiently fails. 
> Failure found in ubuntu16.04-gcc4.8 build but can be reproduced on 
> ubuntu20.04 environment as well, but only in Release build.
> {code:java}
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::TailFile to trace
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::LogAttribute to debug
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Found no stored state
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 
> not found (this is OK)
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Logged 0 flow 
> files
> [2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found 
> Logged 0 flow files in log output.
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
> minifi-tmpfile.txt, 0, 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] State of the TailFile processor tailfileProc:
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
> position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Searching for files rolled over after last read time: 1623743337002
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Performing file listing against /tmp/gt.HkGp2w
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
> filename pattern minifi-tmpfile\..*
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Looking for delimiter 0xA
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Read 17 bytes of input
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] TailFile minifi-tmpfile.txt for 17 bytes
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] 1 flowfiles were received from TailFile input
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [info] Logging for flow 
> file 
> --
> Standard FlowFile Attributes
> UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
> EntryDate:2021-06-15 07:48:57.002
> lineageStartDate:2021-06-15 07:48:57.002
> Size:17 Offset:0
> FlowFile Attributes Map Content
> key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
> key:filename value:minifi-tmpfile.0-16.txt
> key:flow.id value:test
> key:path value:/tmp/gt.HkGp2w
> FlowFile Resource Claim Content
> Content Claim:.//1623743337000-0
> --
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] 

[jira] [Updated] (MINIFICPP-1590) Extract logs of exited containers

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1590:
-
Fix Version/s: 0.11.0

> Extract logs of exited containers
> -
>
> Key: MINIFICPP-1590
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1590
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> If a dockerized integration test fails we only get the logs of the running 
> containers. If the MiNiFi process stops or crashes we don't have any means of 
> seeing the logs of the container. In this case we need to extract and log the 
> application log entries before cleaning up the container so we can debug the 
> underlying issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1557) Transient parse failure in HTTPHandler in C2 tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1557:
-
Fix Version/s: 0.11.0

> Transient parse failure in HTTPHandler in C2 tests
> --
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 0.11.0
>
> Attachments: C2DescribeCoreComponentStateTest_falure_ubuntu20.04.log, 
> C2LogHeartbeatTest_failiure_ubuntu20.04.log, 
> C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1589) Azure docker tests fail due to new restrictions

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1589:
-
Fix Version/s: 0.11.0

> Azure docker tests fail due to new restrictions
> ---
>
> Key: MINIFICPP-1589
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1589
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 0.11.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Azure docker tests fail with the following message: "400 The specifed 
> resource name contains invalid characters."
> This is due to new Azure naming conventions that do not meet our test blob 
> and container names:
>  * Container name must be *_lowercase_*.
>  * Container names must be > 3 character and < 63 characters.
>  * Container names must only contain letters, numbers, and if needed (-) 
> character is allowed.
> The test data should be fixed, and also any non-pinned docker image version 
> should be pinned to a specific version to avoid similar issues later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1494) InvokeHTTP should behave in the same way as NiFi

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1494:
-
Fix Version/s: 0.11.0

> InvokeHTTP should behave in the same way as NiFi
> 
>
> Key: MINIFICPP-1494
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1494
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 0.11.0
>
> Attachments: flow which reproduces the issue.png
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> There are some differences between the NiFi and MiNiFi implementations of 
> InvokeHTTP.  I suspect these are not intentional, and we should change MiNiFi 
> to behave in the same way as NiFi.
>  #  When InvokeHTTP is timer-driven and the Method is GET, then NiFi does not 
> require an incoming flow file, but MiNiFi does.
>  # The NiFi version of InvokeHTTP has an output relationship called Original. 
>  This is called Success in MiNiFi; it should probably be renamed to Original.
> TODO: any others?
> *These changes could break some existing flows; discuss whether this is 
> acceptable.*
> EDIT: in (1), the problem is not really with InvokeHTTP but with 
> SchedulingAgent::hasWorkToDo():
> {noformat}
> bool SchedulingAgent::hasWorkToDo(const std::shared_ptr& 
> processor) {
>   if (processor->getTriggerWhenEmpty() || 
> !processor->hasIncomingConnections() || processor->isWorkAvailable())
> return true;
>   else
> return false;
> }{noformat}
> InvokeHTTP does not set _triggerWhenEmpty, so {{getTriggerWhenEmpty()}} is 
> false; *if it has no real incoming connection but there is a Retry 
> self-loop*, then {{hasIncomingConnections()}} is true, and 
> {{isWorkAvailable()}} is false unless there are flow files waiting to be 
> retried.  As a result, {{InvokeHTTP::onTrigger()}} will never get called.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1494) InvokeHTTP should behave in the same way as NiFi

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1494.
--
Resolution: Fixed

> InvokeHTTP should behave in the same way as NiFi
> 
>
> Key: MINIFICPP-1494
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1494
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Assignee: Gabor Gyimesi
>Priority: Minor
> Attachments: flow which reproduces the issue.png
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> There are some differences between the NiFi and MiNiFi implementations of 
> InvokeHTTP.  I suspect these are not intentional, and we should change MiNiFi 
> to behave in the same way as NiFi.
>  #  When InvokeHTTP is timer-driven and the Method is GET, then NiFi does not 
> require an incoming flow file, but MiNiFi does.
>  # The NiFi version of InvokeHTTP has an output relationship called Original. 
>  This is called Success in MiNiFi; it should probably be renamed to Original.
> TODO: any others?
> *These changes could break some existing flows; discuss whether this is 
> acceptable.*
> EDIT: in (1), the problem is not really with InvokeHTTP but with 
> SchedulingAgent::hasWorkToDo():
> {noformat}
> bool SchedulingAgent::hasWorkToDo(const std::shared_ptr& 
> processor) {
>   if (processor->getTriggerWhenEmpty() || 
> !processor->hasIncomingConnections() || processor->isWorkAvailable())
> return true;
>   else
> return false;
> }{noformat}
> InvokeHTTP does not set _triggerWhenEmpty, so {{getTriggerWhenEmpty()}} is 
> false; *if it has no real incoming connection but there is a Retry 
> self-loop*, then {{hasIncomingConnections()}} is true, and 
> {{isWorkAvailable()}} is false unless there are flow files waiting to be 
> retried.  As a result, {{InvokeHTTP::onTrigger()}} will never get called.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1596) Conceptual problems in docker integration tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1596:


Assignee: Gabor Gyimesi

> Conceptual problems in docker integration tests
> ---
>
> Key: MINIFICPP-1596
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1596
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Test
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>
> The concept "cluster" is misused in docker integration tests. There should be 
> a single cluster containing all the containers on a single network. As there 
> are multiple clusters now with the same network the cleanup process is 
> convoluted and flaky.
> The "cluster" and the "container" concepts are also mixed up in places, so 
> there should be an explicit Container concept in the code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1596) Conceptual problems in docker integration tests

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1596:


 Summary: Conceptual problems in docker integration tests
 Key: MINIFICPP-1596
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1596
 Project: Apache NiFi MiNiFi C++
  Issue Type: Test
Reporter: Gabor Gyimesi


The concept "cluster" is misused in docker integration tests. There should be a 
single cluster containing all the containers on a single network. As there are 
multiple clusters now with the same network the cleanup process is convoluted 
and flaky.

The "cluster" and the "container" concepts are also mixed up in places, so 
there should be an explicit Container concept in the code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1592.
--
Resolution: Fixed

> TailFileTests transiently fails
> ---
>
> Key: MINIFICPP-1592
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1592
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Test case "TailFile reads from a single file when Initial Start Position is 
> set to Current Time with rollover" of TailFileTests suite transiently fails. 
> Failure found in ubuntu16.04-gcc4.8 build but can be reproduced on 
> ubuntu20.04 environment as well, but only in Release build.
> {code:java}
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::TailFile to trace
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::LogAttribute to debug
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Found no stored state
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 
> not found (this is OK)
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Logged 0 flow 
> files
> [2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found 
> Logged 0 flow files in log output.
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
> minifi-tmpfile.txt, 0, 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] State of the TailFile processor tailfileProc:
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
> position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Searching for files rolled over after last read time: 1623743337002
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Performing file listing against /tmp/gt.HkGp2w
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
> filename pattern minifi-tmpfile\..*
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Looking for delimiter 0xA
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Read 17 bytes of input
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] TailFile minifi-tmpfile.txt for 17 bytes
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] 1 flowfiles were received from TailFile input
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [info] Logging for flow 
> file 
> --
> Standard FlowFile Attributes
> UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
> EntryDate:2021-06-15 07:48:57.002
> lineageStartDate:2021-06-15 07:48:57.002
> Size:17 Offset:0
> FlowFile Attributes Map Content
> key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
> key:filename value:minifi-tmpfile.0-16.txt
> key:flow.id value:test
> key:path value:/tmp/gt.HkGp2w
> FlowFile Resource Claim Content
> Content Claim:.//1623743337000-0
> --
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Logged 1 flow 
> files
> 

[jira] [Resolved] (MINIFICPP-1590) Extract logs of exited containers

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1590.
--
Resolution: Fixed

> Extract logs of exited containers
> -
>
> Key: MINIFICPP-1590
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1590
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> If a dockerized integration test fails we only get the logs of the running 
> containers. If the MiNiFi process stops or crashes we don't have any means of 
> seeing the logs of the container. In this case we need to extract and log the 
> application log entries before cleaning up the container so we can debug the 
> underlying issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1595) Behave tests crash with newer M2Crypto version

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1595:


 Summary: Behave tests crash with newer M2Crypto version
 Key: MINIFICPP-1595
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1595
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi


Integration tests use the latest version of all pip packages. The latest 
version of M2Crypto package is not compatible with the latest Behave package so 
the tests crash at start. Versions should be pinned for every pip package to 
avoid similar issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1595) Behave tests crash with newer M2Crypto version

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1595:


Assignee: Gabor Gyimesi

> Behave tests crash with newer M2Crypto version
> --
>
> Key: MINIFICPP-1595
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1595
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>
> Integration tests use the latest version of all pip packages. The latest 
> version of M2Crypto package is not compatible with the latest Behave package 
> so the tests crash at start. Versions should be pinned for every pip package 
> to avoid similar issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1592:
-
Issue Type: Bug  (was: Improvement)

> TailFileTests transiently fails
> ---
>
> Key: MINIFICPP-1592
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1592
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Test case "TailFile reads from a single file when Initial Start Position is 
> set to Current Time with rollover" of TailFileTests suite transiently fails. 
> Failure found in ubuntu16.04-gcc4.8 build but can be reproduced on 
> ubuntu20.04 environment as well, but only in Release build.
> {code:java}
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::TailFile to trace
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::LogAttribute to debug
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Found no stored state
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 
> not found (this is OK)
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Logged 0 flow 
> files
> [2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found 
> Logged 0 flow files in log output.
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
> minifi-tmpfile.txt, 0, 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] State of the TailFile processor tailfileProc:
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
> position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Searching for files rolled over after last read time: 1623743337002
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Performing file listing against /tmp/gt.HkGp2w
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
> filename pattern minifi-tmpfile\..*
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Looking for delimiter 0xA
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Read 17 bytes of input
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] TailFile minifi-tmpfile.txt for 17 bytes
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] 1 flowfiles were received from TailFile input
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [info] Logging for flow 
> file 
> --
> Standard FlowFile Attributes
> UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
> EntryDate:2021-06-15 07:48:57.002
> lineageStartDate:2021-06-15 07:48:57.002
> Size:17 Offset:0
> FlowFile Attributes Map Content
> key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
> key:filename value:minifi-tmpfile.0-16.txt
> key:flow.id value:test
> key:path value:/tmp/gt.HkGp2w
> FlowFile Resource Claim Content
> Content Claim:.//1623743337000-0
> --
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Logged 1 

[jira] [Updated] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1592:
-
Description: 
Test case "TailFile reads from a single file when Initial Start Position is set 
to Current Time with rollover" of TailFileTests suite transiently fails. 
Failure found in ubuntu16.04-gcc4.8 build but can be reproduced on ubuntu20.04 
environment as well, but only in Release build.
{code:java}
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::TailFile to trace
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::LogAttribute to debug
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Found no stored state
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 not 
found (this is OK)
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 0 flow files
[2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found Logged 
0 flow files in log output.
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
minifi-tmpfile.txt, 0, 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] State of the TailFile processor tailfileProc:
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Searching for files rolled over after last read time: 1623743337002
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Performing file listing against /tmp/gt.HkGp2w
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
filename pattern minifi-tmpfile\..*
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Looking for delimiter 0xA
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Read 17 bytes of input
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] TailFile minifi-tmpfile.txt for 17 bytes
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] 1 flowfiles were received from TailFile input
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[info] Logging for flow file 
--
Standard FlowFile Attributes
UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
EntryDate:2021-06-15 07:48:57.002
lineageStartDate:2021-06-15 07:48:57.002
Size:17 Offset:0
FlowFile Attributes Map Content
key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
key:filename value:minifi-tmpfile.0-16.txt
key:flow.id value:test
key:path value:/tmp/gt.HkGp2w
FlowFile Resource Claim Content
Content Claim:.//1623743337000-0
--
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 1 flow files
[2021-06-15 07:49:00.005] [LogTestController] [info] Failed to find Logged 2 
flow files in log 
output.~~~
TailFileTests is a Catch v1.6.1 host application.
Run with -? for 
options---
TailFile reads from a single file when Initial Start Position is set to Current
Time with rollover
---
/nifi-minifi-cpp/extensions/standard-processors/tests/unit/TailFileTests.cpp:1596

[jira] [Updated] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1592:
-
Description: 
Test case "TailFile reads from a single file when Initial Start Position is set 
to Current Time with rollover" of TailFileTests suite transiently fails. 
Failure found in 16.04-gcc4.8 build.
{code:java}
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::TailFile to trace
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::LogAttribute to debug
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Found no stored state
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 not 
found (this is OK)
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 0 flow files
[2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found Logged 
0 flow files in log output.
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
minifi-tmpfile.txt, 0, 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] State of the TailFile processor tailfileProc:
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Searching for files rolled over after last read time: 1623743337002
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Performing file listing against /tmp/gt.HkGp2w
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
filename pattern minifi-tmpfile\..*
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Looking for delimiter 0xA
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Read 17 bytes of input
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] TailFile minifi-tmpfile.txt for 17 bytes
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] 1 flowfiles were received from TailFile input
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[info] Logging for flow file 
--
Standard FlowFile Attributes
UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
EntryDate:2021-06-15 07:48:57.002
lineageStartDate:2021-06-15 07:48:57.002
Size:17 Offset:0
FlowFile Attributes Map Content
key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
key:filename value:minifi-tmpfile.0-16.txt
key:flow.id value:test
key:path value:/tmp/gt.HkGp2w
FlowFile Resource Claim Content
Content Claim:.//1623743337000-0
--
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 1 flow files
[2021-06-15 07:49:00.005] [LogTestController] [info] Failed to find Logged 2 
flow files in log 
output.~~~
TailFileTests is a Catch v1.6.1 host application.
Run with -? for 
options---
TailFile reads from a single file when Initial Start Position is set to Current
Time with rollover
---
/nifi-minifi-cpp/extensions/standard-processors/tests/unit/TailFileTests.cpp:1596

[jira] [Updated] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1592:
-
Description: 
Test case "TailFile reads from a single file when Initial Start Position is set 
to Current Time with rollover" of TailFileTests suite transiently fails. 
Failure found in 16.04-gcc4.8 build.

 
{code:java}
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::TailFile to trace
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::LogAttribute to debug
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Found no stored state
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 not 
found (this is OK)
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 0 flow files
[2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found Logged 
0 flow files in log output.
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
minifi-tmpfile.txt, 0, 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] State of the TailFile processor tailfileProc:
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Searching for files rolled over after last read time: 1623743337002
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Performing file listing against /tmp/gt.HkGp2w
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
filename pattern minifi-tmpfile\..*
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Looking for delimiter 0xA
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Read 17 bytes of input
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] TailFile minifi-tmpfile.txt for 17 bytes
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] 1 flowfiles were received from TailFile input
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[info] Logging for flow file 
--
Standard FlowFile Attributes
UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
EntryDate:2021-06-15 07:48:57.002
lineageStartDate:2021-06-15 07:48:57.002
Size:17 Offset:0
FlowFile Attributes Map Content
key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
key:filename value:minifi-tmpfile.0-16.txt
key:flow.id value:test
key:path value:/tmp/gt.HkGp2w
FlowFile Resource Claim Content
Content Claim:.//1623743337000-0
--
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 1 flow files
[2021-06-15 07:49:00.005] [LogTestController] [info] Failed to find Logged 2 
flow files in log 
output.~~~
TailFileTests is a Catch v1.6.1 host application.
Run with -? for 
options---
TailFile reads from a single file when Initial Start Position is set to Current
Time with rollover
---
/nifi-minifi-cpp/extensions/standard-processors/tests/unit/TailFileTests.cpp:1596

[jira] [Updated] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1592:
-
Labels: MiNiFi-CPP-Hygiene  (was: )

> TailFileTests transiently fails
> ---
>
> Key: MINIFICPP-1592
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1592
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
>
> Test case "TailFile reads from a single file when Initial Start Position is 
> set to Current Time with rollover" of TailFileTests suite transiently fails:
>  
> {code:java}
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::TailFile to trace
> [2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
> org::apache::nifi::minifi::processors::LogAttribute to debug
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Found no stored state
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 
> not found (this is OK)
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.002] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Logged 0 flow 
> files
> [2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found 
> Logged 0 flow files in log output.
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
> minifi-tmpfile.txt, 0, 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] State of the TailFile processor tailfileProc:
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
> position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Searching for files rolled over after last read time: 1623743337002
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Performing file listing against /tmp/gt.HkGp2w
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
> filename pattern minifi-tmpfile\..*
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Looking for delimiter 0xA
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [trace] Read 17 bytes of input
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] TailFile minifi-tmpfile.txt for 17 bytes
> [2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
> [info] 1 flowfiles were received from TailFile input
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] FlowFiles To 
> Log: 0
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Maximum Payload 
> Line Length: 80
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [info] Logging for flow 
> file 
> --
> Standard FlowFile Attributes
> UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
> EntryDate:2021-06-15 07:48:57.002
> lineageStartDate:2021-06-15 07:48:57.002
> Size:17 Offset:0
> FlowFile Attributes Map Content
> key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
> key:filename value:minifi-tmpfile.0-16.txt
> key:flow.id value:test
> key:path value:/tmp/gt.HkGp2w
> FlowFile Resource Claim Content
> Content Claim:.//1623743337000-0
> --
> [2021-06-15 07:48:57.003] 
> [org::apache::nifi::minifi::processors::LogAttribute] [debug] Logged 1 flow 
> files
> [2021-06-15 07:49:00.005] [LogTestController] [info] Failed to find Logged 2 
> flow files in log 
> 

[jira] [Created] (MINIFICPP-1592) TailFileTests transiently fails

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1592:


 Summary: TailFileTests transiently fails
 Key: MINIFICPP-1592
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1592
 Project: Apache NiFi MiNiFi C++
  Issue Type: Improvement
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


Test case "TailFile reads from a single file when Initial Start Position is set 
to Current Time with rollover" of TailFileTests suite transiently fails:

 
{code:java}
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::TailFile to trace
[2021-06-15 07:48:57.001] [LogTestController] [info] Setting log level for 
org::apache::nifi::minifi::processors::LogAttribute to debug
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Found no stored state
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] Legacy state file TailFileState.2396d624-cdae-11eb-9b23-0242ac110002 not 
found (this is OK)
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 0 flow files
[2021-06-15 07:48:57.002] [LogTestController] [info] Successfully found Logged 
0 flow files in log output.
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Received path /tmp/gt.HkGp2w, file minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] TailState /tmp/gt.HkGp2w/minifi-tmpfile.txt: /tmp/gt.HkGp2w, 
minifi-tmpfile.txt, 0, 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] State of the TailFile processor tailfileProc:
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] /tmp/gt.HkGp2w/minifi-tmpfile.txt => { name: minifi-tmpfile.txt, 
position: 34, checksum: 1535622921, last_read_time: 1623743337002 }
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Searching for files rolled over after last read time: 1623743337002
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Performing file listing against /tmp/gt.HkGp2w
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] File minifi-tmpfile.txt.old.1 with mtime 1623743336 matches rolling 
filename pattern minifi-tmpfile\..*
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Tailing file /tmp/gt.HkGp2w/minifi-tmpfile.txt from 0
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Looking for delimiter 0xA
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[debug] Opening /tmp/gt.HkGp2w/minifi-tmpfile.txt
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[trace] Read 17 bytes of input
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] TailFile minifi-tmpfile.txt for 17 bytes
[2021-06-15 07:48:57.002] [org::apache::nifi::minifi::processors::TailFile] 
[info] 1 flowfiles were received from TailFile input
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] FlowFiles To Log: 0
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Maximum Payload Line Length: 80
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[info] Logging for flow file 
--
Standard FlowFile Attributes
UUID:2396ed26-cdae-11eb-9b2d-0242ac110002
EntryDate:2021-06-15 07:48:57.002
lineageStartDate:2021-06-15 07:48:57.002
Size:17 Offset:0
FlowFile Attributes Map Content
key:absolute.path value:/tmp/gt.HkGp2w/minifi-tmpfile.txt
key:filename value:minifi-tmpfile.0-16.txt
key:flow.id value:test
key:path value:/tmp/gt.HkGp2w
FlowFile Resource Claim Content
Content Claim:.//1623743337000-0
--
[2021-06-15 07:48:57.003] [org::apache::nifi::minifi::processors::LogAttribute] 
[debug] Logged 1 flow files
[2021-06-15 07:49:00.005] [LogTestController] [info] Failed to find Logged 2 
flow files in log 
output.~~~
TailFileTests is a Catch v1.6.1 host application.
Run with -? for 
options---
TailFile reads from a single file when Initial Start Position is set to Current
Time with rollover
---

[jira] [Resolved] (MINIFICPP-1557) Transient parse failure in HTTPHandler in C2 tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1557.
--
Resolution: Fixed

> Transient parse failure in HTTPHandler in C2 tests
> --
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: C2DescribeCoreComponentStateTest_falure_ubuntu20.04.log, 
> C2LogHeartbeatTest_failiure_ubuntu20.04.log, 
> C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1589) Azure docker tests fail due to new restrictions

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1589.
--
Resolution: Fixed

> Azure docker tests fail due to new restrictions
> ---
>
> Key: MINIFICPP-1589
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1589
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Azure docker tests fail with the following message: "400 The specifed 
> resource name contains invalid characters."
> This is due to new Azure naming conventions that do not meet our test blob 
> and container names:
>  * Container name must be *_lowercase_*.
>  * Container names must be > 3 character and < 63 characters.
>  * Container names must only contain letters, numbers, and if needed (-) 
> character is allowed.
> The test data should be fixed, and also any non-pinned docker image version 
> should be pinned to a specific version to avoid similar issues later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1568) PerformanceDataMonitorTests transiently fail

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1568.
--
Resolution: Fixed

> PerformanceDataMonitorTests transiently fail
> 
>
> Key: MINIFICPP-1568
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1568
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Fix For: 0.10.0
>
> Attachments: PerformanceDataMonitorTests_failure_win_vs2017.log
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-994) Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it should be part of MiNiFi build.

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-994:
---

Assignee: Gabor Gyimesi  (was: Nghia Le)

> Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it 
> should be part of MiNiFi build.
> 
>
> Key: MINIFICPP-994
> URL: https://issues.apache.org/jira/browse/MINIFICPP-994
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Affects Versions: 0.6.0
>Reporter: Arpad Boda
>Assignee: Gabor Gyimesi
>Priority: Minor
>
> Opencv is built when executing a cmake command (for eg. using bootstrap), 
> which seems to be wrong as:
>  * This build is single-threaded, takes a lot of time
>  * Linker warnings are generated when building MiNiFI:
> {code}
> ld: warning: direct access in function '___cxx_global_var_init.35' from file 
> '../thirdparty/opencv/opencv-external/src/opencv-external-build/lib/libopencv_flann.a(miniflann.cpp.o)'
>  to global weak symbol 'guard variable for 
> cvflann::anyimpl::SinglePolicy std::__1::char_traits, std::__1::allocator > >::policy' from file 
> 'CMakeFiles/CaptureRTSPFrameTest.dir/CaptureRTSPFrameTest.cpp.o' means the 
> weak symbol cannot be overridden at runtime. This was likely caused by 
> different translation units being compiled with different visibility settings.
> {code}
> Static linking should be kept when integrating opencv build to the build 
> process of MiNiFi.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1590) Extract logs of exited containers

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1590:


 Summary: Extract logs of exited containers
 Key: MINIFICPP-1590
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1590
 Project: Apache NiFi MiNiFi C++
  Issue Type: Improvement
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


If a dockerized integration test fails we only get the logs of the running 
containers. If the MiNiFi process stops or crashes we don't have any means of 
seeing the logs of the container. In this case we need to extract and log the 
application log entries before cleaning up the container so we can debug the 
underlying issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1588) Test multiple files transferred via HTTP

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1588:
-
Description: 
Previously an HTTP test scenario was proposed to be tested, but was removed 
because of it's failure. The scenario should be tested and fixed if needed:
{code:java}
  # Failing scenario, needs to be fixed
  @skip
  Scenario: Multiple files transferred via HTTP are received and transferred 
only once
Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input"
And the "Keep Source File" of the GetFile processor is set to "false"
And two files with content "first message" and "second message" are placed 
in "/tmp/input"
And a InvokeHTTP processor with the "Remote URL" property set to 
"http://secondary:8080/contentListener;
And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
And the "success" relationship of the GetFile processor is connected to the 
InvokeHTTPAnd a ListenHTTP processor with the "Listening Port" property set 
to "8080" in a "secondary" flow
And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "secondary" flow
And the "success" relationship of the ListenHTTP processor is connected to 
the PutFileWhen both instances start up
Then two flowfiles with the contents "first message" and "second message" 
are placed in the monitored directory in less than 120 seconds
{code}
where a possible implementation of the prerequisites would be:
{code:python}
@given("two files with content \"{content_1}\" and \"{content_2}\" are placed 
in \"{path}\"")
def step_impl(context, content_1, content_2, path):
context.execute_steps("""
given a file with the content \"{content_1}\" is present in \"{path}\"
and a file with the content \"{content_2}\" is present in 
\"{path}\.format(content_1=content_1, content_2=content_2, path=path))
{code}

  was:
Previously an HTTP test scenario was proposed to be tested, but was removed 
because of it's failure. The scenario should be tested and fixed if needed:
{code:java}
  # Failing scenario, needs to be fixed
  @skip
  Scenario: Multiple files transferred via HTTP are received and transferred 
only once
Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input"
And the "Keep Source File" of the GetFile processor is set to "false"
And two files with content "first message" and "second message" are placed 
in "/tmp/input"
And a InvokeHTTP processor with the "Remote URL" property set to 
"http://secondary:8080/contentListener;
And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
And the "success" relationship of the GetFile processor is connected to the 
InvokeHTTPAnd a ListenHTTP processor with the "Listening Port" property set 
to "8080" in a "secondary" flow
And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "secondary" flow
And the "success" relationship of the ListenHTTP processor is connected to 
the PutFileWhen both instances start up
Then two flowfiles with the contents "first message" and "second message" 
are placed in the monitored directory in less than 120 seconds
{code}

where

{code:python}
@given("two files with content \"{content_1}\" and \"{content_2}\" are placed 
in \"{path}\"")
def step_impl(context, content_1, content_2, path):
context.execute_steps("""
given a file with the content \"{content_1}\" is present in \"{path}\"
and a file with the content \"{content_2}\" is present in 
\"{path}\.format(content_1=content_1, content_2=content_2, path=path))
{code}


> Test multiple files transferred via HTTP
> 
>
> Key: MINIFICPP-1588
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1588
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Priority: Minor
>
> Previously an HTTP test scenario was proposed to be tested, but was removed 
> because of it's failure. The scenario should be tested and fixed if needed:
> {code:java}
>   # Failing scenario, needs to be fixed
>   @skip
>   Scenario: Multiple files transferred via HTTP are received and transferred 
> only once
> Given a GetFile processor with the "Input Directory" property set to 
> "/tmp/input"
> And the "Keep Source File" of the GetFile processor is set to "false"
> And two files with content "first message" and "second message" are 
> placed in "/tmp/input"
> And a InvokeHTTP processor with the "Remote URL" property set to 
> "http://secondary:8080/contentListener;
> And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
> And the "success" relationship of the GetFile processor is connected to 
> the InvokeHTTPAnd a ListenHTTP 

[jira] [Updated] (MINIFICPP-1588) Test multiple files transferred via HTTP

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1588:
-
Description: 
Previously an HTTP test scenario was proposed to be tested, but was removed 
because of it's failure. The scenario should be tested and fixed if needed:
{code:java}
  # Failing scenario, needs to be fixed
  @skip
  Scenario: Multiple files transferred via HTTP are received and transferred 
only once
Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input"
And the "Keep Source File" of the GetFile processor is set to "false"
And two files with content "first message" and "second message" are placed 
in "/tmp/input"
And a InvokeHTTP processor with the "Remote URL" property set to 
"http://secondary:8080/contentListener;
And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
And the "success" relationship of the GetFile processor is connected to the 
InvokeHTTPAnd a ListenHTTP processor with the "Listening Port" property set 
to "8080" in a "secondary" flow
And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "secondary" flow
And the "success" relationship of the ListenHTTP processor is connected to 
the PutFileWhen both instances start up
Then two flowfiles with the contents "first message" and "second message" 
are placed in the monitored directory in less than 120 seconds
{code}

where

{code:python}
@given("two files with content \"{content_1}\" and \"{content_2}\" are placed 
in \"{path}\"")
def step_impl(context, content_1, content_2, path):
context.execute_steps("""
given a file with the content \"{content_1}\" is present in \"{path}\"
and a file with the content \"{content_2}\" is present in 
\"{path}\.format(content_1=content_1, content_2=content_2, path=path))
{code}

  was:
Previously an HTTP test scenario was proposed to be tested, but was removed 
because of it's failure. The scenario should be tested and fixed if needed:
{code:java}
  # Failing scenario, needs to be fixed
  @skip
  Scenario: Multiple files transferred via HTTP are received and transferred 
only once
Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input"
And the "Keep Source File" of the GetFile processor is set to "false"
And two files with content "first message" and "second message" are placed 
in "/tmp/input"
And a InvokeHTTP processor with the "Remote URL" property set to 
"http://secondary:8080/contentListener;
And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
And the "success" relationship of the GetFile processor is connected to the 
InvokeHTTPAnd a ListenHTTP processor with the "Listening Port" property set 
to "8080" in a "secondary" flow
And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "secondary" flow
And the "success" relationship of the ListenHTTP processor is connected to 
the PutFileWhen both instances start up
Then two flowfiles with the contents "first message" and "second message" 
are placed in the monitored directory in less than 120 seconds
{code}


> Test multiple files transferred via HTTP
> 
>
> Key: MINIFICPP-1588
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1588
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Priority: Minor
>
> Previously an HTTP test scenario was proposed to be tested, but was removed 
> because of it's failure. The scenario should be tested and fixed if needed:
> {code:java}
>   # Failing scenario, needs to be fixed
>   @skip
>   Scenario: Multiple files transferred via HTTP are received and transferred 
> only once
> Given a GetFile processor with the "Input Directory" property set to 
> "/tmp/input"
> And the "Keep Source File" of the GetFile processor is set to "false"
> And two files with content "first message" and "second message" are 
> placed in "/tmp/input"
> And a InvokeHTTP processor with the "Remote URL" property set to 
> "http://secondary:8080/contentListener;
> And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
> And the "success" relationship of the GetFile processor is connected to 
> the InvokeHTTPAnd a ListenHTTP processor with the "Listening Port" 
> property set to "8080" in a "secondary" flow
> And a PutFile processor with the "Directory" property set to 
> "/tmp/output" in the "secondary" flow
> And the "success" relationship of the ListenHTTP processor is connected 
> to the PutFileWhen both instances start up
> Then two flowfiles with the contents "first message" and "second message" 
> are placed in the monitored directory in less than 120 seconds
> {code}
> where
> 

[jira] [Assigned] (MINIFICPP-1589) Azure docker tests fail due to new restrictions

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1589:


Assignee: Gabor Gyimesi

> Azure docker tests fail due to new restrictions
> ---
>
> Key: MINIFICPP-1589
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1589
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>
> Azure docker tests fail with the following message: "400 The specifed 
> resource name contains invalid characters."
> This is due to new Azure naming conventions that do not meet our test blob 
> and container names:
>  * Container name must be *_lowercase_*.
>  * Container names must be > 3 character and < 63 characters.
>  * Container names must only contain letters, numbers, and if needed (-) 
> character is allowed.
> The test data should be fixed, and also any non-pinned docker image version 
> should be pinned to a specific version to avoid similar issues later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1589) Azure docker tests fail due to new restrictions

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1589:


 Summary: Azure docker tests fail due to new restrictions
 Key: MINIFICPP-1589
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1589
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi


Azure docker tests fail with the following message: "400 The specifed resource 
name contains invalid characters."

This is due to new Azure naming conventions that do not meet our test blob and 
container names:
 * Container name must be *_lowercase_*.
 * Container names must be > 3 character and < 63 characters.
 * Container names must only contain letters, numbers, and if needed (-) 
character is allowed.

The test data should be fixed, and also any non-pinned docker image version 
should be pinned to a specific version to avoid similar issues later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1588) Test multiple files transferred via HTTP

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1588:


 Summary: Test multiple files transferred via HTTP
 Key: MINIFICPP-1588
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1588
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi


Previously an HTTP test scenario was proposed to be tested, but was removed 
because of it's failure. The scenario should be tested and fixed if needed:
{code:java}
  # Failing scenario, needs to be fixed
  @skip
  Scenario: Multiple files transferred via HTTP are received and transferred 
only once
Given a GetFile processor with the "Input Directory" property set to 
"/tmp/input"
And the "Keep Source File" of the GetFile processor is set to "false"
And two files with content "first message" and "second message" are placed 
in "/tmp/input"
And a InvokeHTTP processor with the "Remote URL" property set to 
"http://secondary:8080/contentListener;
And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
And the "success" relationship of the GetFile processor is connected to the 
InvokeHTTPAnd a ListenHTTP processor with the "Listening Port" property set 
to "8080" in a "secondary" flow
And a PutFile processor with the "Directory" property set to "/tmp/output" 
in the "secondary" flow
And the "success" relationship of the ListenHTTP processor is connected to 
the PutFileWhen both instances start up
Then two flowfiles with the contents "first message" and "second message" 
are placed in the monitored directory in less than 120 seconds
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1580) Fix Tensorflow extension build

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1580:


 Summary: Fix Tensorflow extension build
 Key: MINIFICPP-1580
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1580
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1557) Transient parse failure in HTTPHandler in C2 tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1557:
-
Attachment: C2DescribeCoreComponentStateTest_falure_ubuntu20.04.log

> Transient parse failure in HTTPHandler in C2 tests
> --
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: C2DescribeCoreComponentStateTest_falure_ubuntu20.04.log, 
> C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1557) Transient parse failure in HTTPHandler in C2 tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1557:
-
Summary: Transient parse failure in HTTPHandler in C2 tests  (was: 
C2VerifyHeartbeatAndStop and C2VerifyResourceConsumptionInHeartbeat transient 
failure)

> Transient parse failure in HTTPHandler in C2 tests
> --
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: C2DescribeCoreComponentStateTest_falure_ubuntu20.04.log, 
> C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1557) Transient parse failure in HTTPHandler in C2 tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1557:


Assignee: Gabor Gyimesi

> Transient parse failure in HTTPHandler in C2 tests
> --
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: C2DescribeCoreComponentStateTest_falure_ubuntu20.04.log, 
> C2LogHeartbeatTest_failiure_ubuntu20.04.log, 
> C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1557) Transient parse failure in HTTPHandler in C2 tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1557:
-
Attachment: C2LogHeartbeatTest_failiure_ubuntu20.04.log

> Transient parse failure in HTTPHandler in C2 tests
> --
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: C2DescribeCoreComponentStateTest_falure_ubuntu20.04.log, 
> C2LogHeartbeatTest_failiure_ubuntu20.04.log, 
> C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1562) Failing CI due to corrupted cache on Windows-2019

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1562.
--
Resolution: Fixed

> Failing CI due to corrupted cache on Windows-2019
> -
>
> Key: MINIFICPP-1562
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1562
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Martin Zink
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The windows-vs2019 CI is failing due to corrupted clcache.
> Unfortunetly github doesnt have clean way to clear the cache.
> {code:java}
> GitHub will remove any cache entries that have not been accessed in over 7 
> days. There is no limit on the number of caches you can store, but the total 
> size of all caches in a repository is limited to 5 GB. If you exceed this 
> limit, GitHub will save your cache but will begin evicting caches until the 
> total size is less than 5 GB.
> {code}
> But since the cache is used I dont think github will clear it.
> A quick and dirty fix is: if we were to change the key in the .github/ci.yml 
> file, it would fix the issue.
> [https://github.community/t/how-to-clear-cache-in-github-actions/129038/3]
> If this issue comes up again maybe we could implement a workflow that clears 
> the caches, with 
> [https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1572) Fix -Wmaybe-uninitialized warnings

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1572.
--
Resolution: Fixed

> Fix -Wmaybe-uninitialized warnings
> --
>
> Key: MINIFICPP-1572
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1572
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Trivial
>  Labels: MiNiFi-CPP-Hygiene
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1569) Windows jobs do not run tests in CI

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1569.
--
Resolution: Fixed

> Windows jobs do not run tests in CI
> ---
>
> Key: MINIFICPP-1569
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1569
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Since MINIFICPP-1084 was merged to main tests are skipped in the Windows 
> builds. The original intent was to not run them as part of the build phase 
> only later in the test running phase, but unfortunately the build was also 
> removed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1576) Allow build of all extensions in docker images

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1576:


 Summary: Allow build of all extensions in docker images
 Key: MINIFICPP-1576
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1576
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi


Build and runtime dependencies of the minifi docker image should be corrected 
in order to be able to build a docker image with any supported extension.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1572) Fix -Wmaybe-uninitialized warnings

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1572:


 Summary: Fix -Wmaybe-uninitialized warnings
 Key: MINIFICPP-1572
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1572
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1557) C2VerifyHeartbeatAndStop transient failure

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1557:
-
Attachment: C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt

> C2VerifyHeartbeatAndStop transient failure
> --
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1557) C2VerifyHeartbeatAndStop and C2VerifyResourceConsumptionInHeartbeat transient failure

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1557:
-
Summary: C2VerifyHeartbeatAndStop and 
C2VerifyResourceConsumptionInHeartbeat transient failure  (was: 
C2VerifyHeartbeatAndStop transient failure)

> C2VerifyHeartbeatAndStop and C2VerifyResourceConsumptionInHeartbeat transient 
> failure
> -
>
> Key: MINIFICPP-1557
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt, 
> C2VerifyResourceConsumptionInHeartbeat_failure_ubuntu20.04.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1569) Windows jobs do not run tests in CI

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1569:
-
Description: Since MINIFICPP-1084 was merged to main tests are skipped in 
the Windows builds. The original intent was to not run them as part of the 
build phase only later in the test running phase, but unfortunately the build 
was also removed.

> Windows jobs do not run tests in CI
> ---
>
> Key: MINIFICPP-1569
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1569
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>
> Since MINIFICPP-1084 was merged to main tests are skipped in the Windows 
> builds. The original intent was to not run them as part of the build phase 
> only later in the test running phase, but unfortunately the build was also 
> removed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1569) Windows jobs do not run tests in CI

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1569:


 Summary: Windows jobs do not run tests in CI
 Key: MINIFICPP-1569
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1569
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
Assignee: Gabor Gyimesi






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1568) PerformanceDataMonitorTests transiently fail

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1568:


Assignee: Gabor Gyimesi

> PerformanceDataMonitorTests transiently fail
> 
>
> Key: MINIFICPP-1568
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1568
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
> Attachments: PerformanceDataMonitorTests_failure_win_vs2017.log
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1403) Create TagS3Object processor

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1403:
-
Description: 
Create new processor to tag S3 objects of a bucket with similar functionality 
defined in Nifi:

[https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.13.2/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.9.0/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html]

  was:
Create new processor to tag S3 objects of a bucket with similar functionality 
defined in Nifi:

[https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.9.0/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html]


> Create TagS3Object processor
> 
>
> Key: MINIFICPP-1403
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1403
> Project: Apache NiFi MiNiFi C++
>  Issue Type: New Feature
>Reporter: Gabor Gyimesi
>Priority: Minor
>
> Create new processor to tag S3 objects of a bucket with similar functionality 
> defined in Nifi:
> [https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.13.2/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.9.0/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1454) Reduce duplication of CMake parameters in docker arguments

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1454.
--
Resolution: Fixed

> Reduce duplication of CMake parameters in docker arguments
> --
>
> Key: MINIFICPP-1454
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1454
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: MiNiFi-CPP-Hygiene
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> MINIFICPP-1452 introduced the ability for Docker builds to consume the 
> configured CMake options. This way CMake options set regarding extension 
> availability are passed to Docker builds when the docker target is built. 
> Unfortunately these options need to be defined and passed through 
> DockerConfig.cmake, DockerBuild.sh and finally the Dockerfile, which causes 
> duplication of these arguments. We need to find a way to reduced this 
> duplication.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1360) Make scheduling strategy configurable in integration tests

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1360.
--
  Assignee: Gabor Gyimesi
Resolution: Fixed

With the new design all processors can get a 
schedule=\{'scheduling strategy': 'EVENT_DRIVEN'} parameter to change the 
default scheduling strategy if needed.

> Make scheduling strategy configurable in integration tests
> --
>
> Key: MINIFICPP-1360
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1360
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In docker/test/integration/minifi/__init__.py the scheduling strategy is hard 
> coded to be TIMER_DRIVEN for all processors. This should be configurable and 
> updated for all processors used by integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1568) PerformanceDataMonitorTests transiently fail

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1568:


 Summary: PerformanceDataMonitorTests transiently fail
 Key: MINIFICPP-1568
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1568
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
 Attachments: PerformanceDataMonitorTests_failure_win_vs2017.log





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (MINIFICPP-1562) Failing CI due to corrupted cache on Windows-2019

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reopened MINIFICPP-1562:
--
  Assignee: Gabor Gyimesi  (was: Martin Zink)

> Failing CI due to corrupted cache on Windows-2019
> -
>
> Key: MINIFICPP-1562
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1562
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Martin Zink
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> The windows-vs2019 CI is failing due to corrupted clcache.
> Unfortunetly github doesnt have clean way to clear the cache.
> {code:java}
> GitHub will remove any cache entries that have not been accessed in over 7 
> days. There is no limit on the number of caches you can store, but the total 
> size of all caches in a repository is limited to 5 GB. If you exceed this 
> limit, GitHub will save your cache but will begin evicting caches until the 
> total size is less than 5 GB.
> {code}
> But since the cache is used I dont think github will clear it.
> A quick and dirty fix is: if we were to change the key in the .github/ci.yml 
> file, it would fix the issue.
> [https://github.community/t/how-to-clear-cache-in-github-actions/129038/3]
> If this issue comes up again maybe we could implement a workflow that clears 
> the caches, with 
> [https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1518) Migrate integration tests (without secutity protocol settings) for ConsumeKafka to behave

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1518:


Assignee: Gabor Gyimesi  (was: Adam Hunyadi)

> Migrate integration tests (without secutity protocol settings) for 
> ConsumeKafka to behave
> -
>
> Key: MINIFICPP-1518
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1518
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Test
>Affects Versions: 0.9.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Fix For: 1.0.0
>
>
> *Background:*
> Currently ConsumeKafka tests are implemented as tests to be run manually that 
> assume the presence of a running broker on the network.
> *Proposal:*
> We should move these tests to the integration test framework.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (MINIFICPP-1369) Implement ConsumeKafka processor

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi updated MINIFICPP-1369:
-
Description: 
Implement ConsumeKafka processor to support consuming Kafka topic(s).

Hint: 
[https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-kafka-2-0-nar/1.9.0/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/index.html]

  was:
Implenent ConsumeKafka processor to support consuming Kafka topic(s).

Hint: 
https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-kafka-2-0-nar/1.9.0/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/index.html


> Implement ConsumeKafka processor
> 
>
> Key: MINIFICPP-1369
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1369
> Project: Apache NiFi MiNiFi C++
>  Issue Type: New Feature
>Affects Versions: 0.9.0
>Reporter: Arpad Boda
>Assignee: Adam Hunyadi
>Priority: Major
>
> Implement ConsumeKafka processor to support consuming Kafka topic(s).
> Hint: 
> [https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-kafka-2-0-nar/1.9.0/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/index.html]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1369) Implement ConsumeKafka processor

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1369:


Assignee: Gabor Gyimesi  (was: Adam Hunyadi)

> Implement ConsumeKafka processor
> 
>
> Key: MINIFICPP-1369
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1369
> Project: Apache NiFi MiNiFi C++
>  Issue Type: New Feature
>Affects Versions: 0.9.0
>Reporter: Arpad Boda
>Assignee: Gabor Gyimesi
>Priority: Major
>
> Implement ConsumeKafka processor to support consuming Kafka topic(s).
> Hint: 
> [https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-kafka-2-0-nar/1.9.0/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/index.html]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1373) Implement and test a simplified ConsumeKafka processor without security protocols

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1373:


Assignee: Gabor Gyimesi  (was: Adam Hunyadi)

> Implement and test a simplified ConsumeKafka processor without security 
> protocols
> -
>
> Key: MINIFICPP-1373
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1373
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Sub-task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ConsumeKafka_test_matrix.numbers, 
> ConsumeKafka_test_matrix.pdf
>
>  Time Spent: 33h 10m
>  Remaining Estimate: 0h
>
> *Acceptance Criteria:*
> *{color:#de350b}See attached test matrix plan.{color}*
> Additional test (that require multiple Kafka consumers):
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}different group ids{color} subscribed to the same topic
>  {color:#505f79}*WHEN*{color} a message is published to the topic
>  {color:#505f79}*THEN*{color} both of the ConsumeKafka processors should 
> produce identical flowfiles
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic
>  {color:#505f79}*WHEN*{color} a message is published to the topic
>  {color:#505f79}*THEN*{color} only one of the ConsumeKafka processors should 
> produce a flowfile
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}earliest{color}.
>  {color:#505f79}*WHEN*{color} a messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should process duplicates of the 
> messages that originally came to the second (at_least_once delivery)
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}latest{color}.
>  {color:#505f79}*WHEN a*{color} messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should {color:#0747a6}not{color} 
> process duplicates of the messages that originally came to the second 
> (at_most_once delivery)
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}none{color}.
>  {color:#505f79}*WHEN*{color} a messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should throw an exception
> {quote}
> *Background:*
> See parent task.
> *Proposal:*
> This should be the first part of the implementation, the second being adding 
> and testing multiple security protocols.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (MINIFICPP-1373) Implement and test a simplified ConsumeKafka processor without security protocols

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reopened MINIFICPP-1373:
--

> Implement and test a simplified ConsumeKafka processor without security 
> protocols
> -
>
> Key: MINIFICPP-1373
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1373
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Sub-task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ConsumeKafka_test_matrix.numbers, 
> ConsumeKafka_test_matrix.pdf
>
>  Time Spent: 33h 10m
>  Remaining Estimate: 0h
>
> *Acceptance Criteria:*
> *{color:#de350b}See attached test matrix plan.{color}*
> Additional test (that require multiple Kafka consumers):
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}different group ids{color} subscribed to the same topic
>  {color:#505f79}*WHEN*{color} a message is published to the topic
>  {color:#505f79}*THEN*{color} both of the ConsumeKafka processors should 
> produce identical flowfiles
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic
>  {color:#505f79}*WHEN*{color} a message is published to the topic
>  {color:#505f79}*THEN*{color} only one of the ConsumeKafka processors should 
> produce a flowfile
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}earliest{color}.
>  {color:#505f79}*WHEN*{color} a messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should process duplicates of the 
> messages that originally came to the second (at_least_once delivery)
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}latest{color}.
>  {color:#505f79}*WHEN a*{color} messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should {color:#0747a6}not{color} 
> process duplicates of the messages that originally came to the second 
> (at_most_once delivery)
> {quote}
> {quote}{color:#505f79}*GIVEN*{color} two ConsumeKafkas with 
> {color:#0747a6}the same group id{color} subscribed to the same topic with 
> exactly two partitions with {color:#0747a6}Offset Reset{color} set to 
> {color:#0747a6}none{color}.
>  {color:#505f79}*WHEN*{color} a messages were already present on both 
> partitions and the second one crashes
>  {color:#505f79}*THEN*{color} the first one should throw an exception
> {quote}
> *Background:*
> See parent task.
> *Proposal:*
> This should be the first part of the implementation, the second being adding 
> and testing multiple security protocols.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1374) Implement security protocol support for ConsumeKafka

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1374:


Assignee: Gabor Gyimesi  (was: Adam Hunyadi)

> Implement security protocol support for ConsumeKafka
> 
>
> Key: MINIFICPP-1374
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1374
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Sub-task
>Affects Versions: 0.7.0
>Reporter: Adam Hunyadi
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 1.0.0
>
>
> *Acceptance Criteria:*
> SSL:
> {code:python|title=Example feature definition}
> Scenario Outline: ConsumeKafka receives data via SSL
>   Given a ConsumeKafka processor set up in a "kafka-consumer-flow" flow
>   And ssl certificates are placed in "/tmp/resources" with cert name 
> "cert.pem" and key name "key.pem"
>   And these processor properties are set:
> | processor name | property name | property value |
> | ConsumeKafka | Kafka Brokers | kafka-broker:9093 |
> | ConsumeKafka | Security Protocol | ssl |
> | ConsumeKafka | Security CA | ... |
> | ConsumeKafka | Security Cert | ... |
>   And a PutFile processor with the "Directory" property set to "/tmp/output" 
> in the "kafka-consumer-flow" flow
>   And the "success" relationship of the ConsumeKafka processor is connected 
> to the PutFile
>   And a kafka broker "broker" set up to communicate via SSL is set up in 
> correspondence with the third-party kafka publisher
>   When all instances start up
>   And a message with content "Alice's Adventures in Wonderland" is published 
> to the "ConsumeKafkaTest" topic using an ssl connection
>   And a message with content "Lewis Carroll" is published to the 
> "ConsumeKafkaTest" topic using an ssl connection
>   Then two flowfiles with the contents "Alice's Adventures in Wonderland" and 
> "Lewis Carroll" are placed in the monitored directory in less than 45 seconds
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1558) Remove s3 bucket objects test transiently fails

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1558:


 Summary: Remove s3 bucket objects test transiently fails
 Key: MINIFICPP-1558
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1558
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
 Attachments: remove_object_from_s3_bucket_failure.log





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1244) Support the Initial Start Position property in TailFile

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1244.
--
Resolution: Fixed

> Support the Initial Start Position property in TailFile
> ---
>
> Key: MINIFICPP-1244
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1244
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Labels: selected-for-development
> Fix For: 0.10.0
>
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> The TailFile processor in NiFi Java has a property called *Initial Start 
> Position*, with allowed values:
>  * *Beginning of Time*: Start with the oldest data that matches the Rolling 
> Filename Pattern and then begin reading from the File to Tail
>  * *Beginning of File*: Start with the beginning of the File to Tail. Do not 
> ingest any data that has already been rolled over
>  * *Current Time*: Start with the data at the end of the File to Tail. Do not 
> ingest any data that has already been rolled over or any data in the File to 
> Tail that has already been written.
> MiNiFi C++ does not have this property, and always behaves as if Beginning of 
> File was selected. Add the *Initial Start Position* property to MiNiFi C++, 
> with the three allowed values and behaviors as above.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (MINIFICPP-1556) Setting partial credentials in AWS credentials service may set other properties

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi reassigned MINIFICPP-1556:


Assignee: Gabor Gyimesi

> Setting partial credentials in AWS credentials service may set other 
> properties
> ---
>
> Key: MINIFICPP-1556
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1556
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Setting only one of authentication key or secret key of AWS credentials can 
> lead to setting AWS credential file property to the previous auth key/secret 
> key value if the credentials file property is empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1556) Setting partial credentials in AWS credentials service may set other properties

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1556.
--
Resolution: Fixed

> Setting partial credentials in AWS credentials service may set other 
> properties
> ---
>
> Key: MINIFICPP-1556
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1556
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Setting only one of authentication key or secret key of AWS credentials can 
> lead to setting AWS credential file property to the previous auth key/secret 
> key value if the credentials file property is empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1553) AWSCredentialsService should be able to refresh its credentials after expiry

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1553.
--
Resolution: Fixed

> AWSCredentialsService should be able to refresh its credentials  after expiry
> -
>
> Key: MINIFICPP-1553
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1553
> Project: Apache NiFi MiNiFi C++
>  Issue Type: New Feature
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The AWSCredentialService in MiNiFi only invokes the AWS credential libraries 
> within its onEnabled method and caches the credentials for later use. It 
> doesn't have a way to refresh the credential after it's been persisted in 
> memory, this leads to the credentials to expire after a while.  Optimally 
> we'd want to be able to ensure the credential is refreshed (which can be 
> something that's property driven).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1557) C2VerifyHeartbeatAndStop transient failure

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1557:


 Summary: C2VerifyHeartbeatAndStop transient failure
 Key: MINIFICPP-1557
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1557
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi
 Attachments: C2VerifyHeartbeatAndStop_failure_ubuntu20.04.txt





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MINIFICPP-1556) Setting partial credentials in AWS credentials service may set other properties

[Gabor Gyimesi (Jira)]

Gabor Gyimesi created MINIFICPP-1556:


 Summary: Setting partial credentials in AWS credentials service 
may set other properties
 Key: MINIFICPP-1556
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1556
 Project: Apache NiFi MiNiFi C++
  Issue Type: Bug
Reporter: Gabor Gyimesi


Setting only one of authentication key or secret key of AWS credentials can 
lead to setting AWS credential file property to the previous auth key/secret 
key value if the credentials file property is empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (MINIFICPP-1529) Improve penalisation in S3 processors

[Gabor Gyimesi (Jira)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338944#comment-17338944
 ] 

Gabor Gyimesi commented on MINIFICPP-1529:
--

Retry is done automatically by AWS SDK

> Improve penalisation in S3 processors
> -
>
> Key: MINIFICPP-1529
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1529
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Gabor Gyimesi
>Priority: Major
>
> Fetch, Delete and Put should penalise flowfiles sent to failure. 
> This way we can introduce retry loops without busy wait. 
> Also worth checking if there are scenarios with non-recoverable errors. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (MINIFICPP-1529) Improve penalisation in S3 processors

[Gabor Gyimesi (Jira)]

 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Gyimesi resolved MINIFICPP-1529.
--
Resolution: Won't Fix

> Improve penalisation in S3 processors
> -
>
> Key: MINIFICPP-1529
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1529
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Arpad Boda
>Assignee: Gabor Gyimesi
>Priority: Major
>
> Fetch, Delete and Put should penalise flowfiles sent to failure. 
> This way we can introduce retry loops without busy wait. 
> Also worth checking if there are scenarios with non-recoverable errors. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)