[jira] [Updated] (MINIFICPP-1041) Re-order connection queues in RESTFuil c2 response
[ https://issues.apache.org/jira/browse/MINIFICPP-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1041: -- Description: Currently the JSON response has the queue name as the top level node ( in jSON this is the object key ). We don't want to change the AST and impact upstream changes. Therefore we will simply change the JSON response in line. > Re-order connection queues in RESTFuil c2 response > --- > > Key: MINIFICPP-1041 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1041 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > > Currently the JSON response has the queue name as the top level node ( in > jSON this is the object key ). We don't want to change the AST and impact > upstream changes. Therefore we will simply change the JSON response in line. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1041) Re-order connection queues in RESTFuil c2 response
Mr TheSegfault created MINIFICPP-1041: - Summary: Re-order connection queues in RESTFuil c2 response Key: MINIFICPP-1041 URL: https://issues.apache.org/jira/browse/MINIFICPP-1041 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault Assignee: Mr TheSegfault Fix For: 0.7.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-651) Lazily start scheduling agent threads
[ https://issues.apache.org/jira/browse/MINIFICPP-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-651: - Fix Version/s: (was: 0.7.0) 0.8.0 > Lazily start scheduling agent threads > - > > Key: MINIFICPP-651 > URL: https://issues.apache.org/jira/browse/MINIFICPP-651 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.8.0 > > > Right now we start these threads up. We could allow functionality that > doesn't start a thread until they are needed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-964) Exception thrown from onTrigger seems to make the C2 reload fail
[ https://issues.apache.org/jira/browse/MINIFICPP-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-964: - Priority: Major (was: Blocker) > Exception thrown from onTrigger seems to make the C2 reload fail > > > Key: MINIFICPP-964 > URL: https://issues.apache.org/jira/browse/MINIFICPP-964 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Nghia Le >Priority: Major > Fix For: 0.7.0 > > > I tried to publish new config for a class which contains CaptureRTSPFrame > processor. In the onTrigger function of that processor, there are 2 lines: > video_capture_.open(rtsp_url_); > video_backend_driver_ = video_capture_.getBackendName(); > > Which can cause an exception. > It leads to a block in the middle of reloading config process. Thus, the > config.yml will not be updated. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-1017) Improve logging of docker tests
[ https://issues.apache.org/jira/browse/MINIFICPP-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-1017. --- Resolution: Fixed PR Merged > Improve logging of docker tests > --- > > Key: MINIFICPP-1017 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1017 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 0.7.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Docker tests are not logging MiNiFi and NiFi app logs and stdout properly > (bytestreams are printed). > Logs should be printed in a readable way. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-1028) http url parsing without port fails
[ https://issues.apache.org/jira/browse/MINIFICPP-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-1028. --- Resolution: Fixed > http url parsing without port fails > --- > > Key: MINIFICPP-1028 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1028 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > Time Spent: 40m > Remaining Estimate: 0h > > In case URL is provided without port, like "http://nifi.io/nifi;, port is > chosen to be a random port instead of 80, host is not parsed properly > (cutting "/nifi" fails) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-14) Templates should be removed or pruned when logging class names
[ https://issues.apache.org/jira/browse/MINIFICPP-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-14: Resolution: Fixed Status: Resolved (was: Patch Available) > Templates should be removed or pruned when logging class names > -- > > Key: MINIFICPP-14 > URL: https://issues.apache.org/jira/browse/MINIFICPP-14 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Labels: newbie > Fix For: 0.7.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > The logger doesn't prune or de-munge the template names from the log > statements. This makes reading the log difficult. Additionally, we may want > to drop org::apache::nifi since the logged class names will likely include > this prefix. > This should only be done if this doesn't negatively impact performance. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (MINIFICPP-1027) Move getclassname to ClassUtils
Mr TheSegfault created MINIFICPP-1027: - Summary: Move getclassname to ClassUtils Key: MINIFICPP-1027 URL: https://issues.apache.org/jira/browse/MINIFICPP-1027 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-14) Templates should be removed or pruned when logging class names
[ https://issues.apache.org/jira/browse/MINIFICPP-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-14: Status: Patch Available (was: Open) > Templates should be removed or pruned when logging class names > -- > > Key: MINIFICPP-14 > URL: https://issues.apache.org/jira/browse/MINIFICPP-14 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Labels: newbie > Fix For: 0.7.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The logger doesn't prune or de-munge the template names from the log > statements. This makes reading the log difficult. Additionally, we may want > to drop org::apache::nifi since the logged class names will likely include > this prefix. > This should only be done if this doesn't negatively impact performance. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (MINIFICPP-14) Templates should be removed or pruned when logging class names
[ https://issues.apache.org/jira/browse/MINIFICPP-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-14: --- Assignee: Mr TheSegfault > Templates should be removed or pruned when logging class names > -- > > Key: MINIFICPP-14 > URL: https://issues.apache.org/jira/browse/MINIFICPP-14 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Labels: newbie > Fix For: 0.7.0 > > > The logger doesn't prune or de-munge the template names from the log > statements. This makes reading the log difficult. Additionally, we may want > to drop org::apache::nifi since the logged class names will likely include > this prefix. > This should only be done if this doesn't negatively impact performance. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1026) Invalid license on base64, need to move to third party lib
[ https://issues.apache.org/jira/browse/MINIFICPP-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1026: -- Priority: Blocker (was: Major) > Invalid license on base64, need to move to third party lib > -- > > Key: MINIFICPP-1026 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1026 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Daniel Bakai >Priority: Blocker > Fix For: 0.7.0 > > > This license, while valid, should be moved to our third party directory and > referenced from there. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1026) Invalid license on base64, need to move to third party lib
[ https://issues.apache.org/jira/browse/MINIFICPP-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1026: -- Issue Type: Bug (was: New Feature) > Invalid license on base64, need to move to third party lib > -- > > Key: MINIFICPP-1026 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1026 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Daniel Bakai >Priority: Major > Fix For: 0.7.0 > > > This license, while valid, should be moved to our third party directory and > referenced from there. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (MINIFICPP-1026) Invalid license on base64, need to move to third party lib
Mr TheSegfault created MINIFICPP-1026: - Summary: Invalid license on base64, need to move to third party lib Key: MINIFICPP-1026 URL: https://issues.apache.org/jira/browse/MINIFICPP-1026 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Mr TheSegfault Assignee: Daniel Bakai Fix For: 0.7.0 This license, while valid, should be moved to our third party directory and referenced from there. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-619) Improve C2 interaction with C API
[ https://issues.apache.org/jira/browse/MINIFICPP-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-619. -- Resolution: Duplicate MINIFICPP-1007 was created later, so setting the fix version of that ticket and simply closing this one. > Improve C2 interaction with C API > - > > Key: MINIFICPP-619 > URL: https://issues.apache.org/jira/browse/MINIFICPP-619 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.7.0 > > > Currently implemented as callbacks, we should be more discrete about our > interaction with the C2 API when writing C code. Make the interactions > distinct and ensure that we have full control over how we interact with a C2 > server implementation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-1007) CoAP/C2 API integration for nanofi tailfile ecu
[ https://issues.apache.org/jira/browse/MINIFICPP-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924172#comment-16924172 ] Mr TheSegfault commented on MINIFICPP-1007: --- Closing MINIFICPP-619 in lieu of this. > CoAP/C2 API integration for nanofi tailfile ecu > --- > > Key: MINIFICPP-1007 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1007 > Project: Apache NiFi MiNiFi C++ > Issue Type: Story >Reporter: Murtuza Shareef >Assignee: Murtuza Shareef >Priority: Major > Labels: CAPI, ECU, nanofi > Fix For: 0.7.0 > > > The goal of this story is to implement a C2 protocol C API over CoAP > protocol. At the end of this story we should be able to demonstrate the > following: > # Configure tailfile ecu with input and output parameters. > # Start and Stop the tailfile ecu via C2 protocol. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1007) CoAP/C2 API integration for nanofi tailfile ecu
[ https://issues.apache.org/jira/browse/MINIFICPP-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1007: -- Fix Version/s: 0.7.0 > CoAP/C2 API integration for nanofi tailfile ecu > --- > > Key: MINIFICPP-1007 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1007 > Project: Apache NiFi MiNiFi C++ > Issue Type: Story >Reporter: Murtuza Shareef >Assignee: Murtuza Shareef >Priority: Major > Labels: CAPI, ECU, nanofi > Fix For: 0.7.0 > > > The goal of this story is to implement a C2 protocol C API over CoAP > protocol. At the end of this story we should be able to demonstrate the > following: > # Configure tailfile ecu with input and output parameters. > # Start and Stop the tailfile ecu via C2 protocol. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-921) Incorrect assignment of flags
[ https://issues.apache.org/jira/browse/MINIFICPP-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-921. -- Resolution: Fixed Resolved through other efforts. I think [~bakaid] may have solved this. either way it appears to be fixed. > Incorrect assignment of flags > - > > Key: MINIFICPP-921 > URL: https://issues.apache.org/jira/browse/MINIFICPP-921 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Mr TheSegfault >Priority: Blocker > Fix For: 0.7.0 > > > Evidently I made a mistake when putting the cxx and c flags into cmake. > [https://github.com/apache/nifi-minifi-cpp/blob/master/CMakeLists.txt#L302] > is the culprit. If this runs on a system whose user has spaces ( I have one > for testing purposes ) – it will cause a failure while building curl. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-815) Add FileUtils Tests
[ https://issues.apache.org/jira/browse/MINIFICPP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924166#comment-16924166 ] Mr TheSegfault commented on MINIFICPP-815: -- Going through tickets. I would agree this is not resolved. > Add FileUtils Tests > --- > > Key: MINIFICPP-815 > URL: https://issues.apache.org/jira/browse/MINIFICPP-815 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > > Formerly titled: > [Dog fooded test should use a separate test function to validate the lamdba > function|https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > [ > https://github.com/apache/nifi-minifi-cpp/pull/531/files#diff-fda52d649a6d955419ada205e8dfeea7R376] > is a dog fooded test. > > We should probably make a FileUtils suite of tests as a focal point than > specifically addressing the above class's tests. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-821) Add Metrics to all processors
[ https://issues.apache.org/jira/browse/MINIFICPP-821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-821: - Fix Version/s: (was: 0.7.0) 0.8.0 > Add Metrics to all processors > - > > Key: MINIFICPP-821 > URL: https://issues.apache.org/jira/browse/MINIFICPP-821 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Priority: Major > Fix For: 0.8.0 > > > GetFile and GetTCP Metrics are examples where we can gather processor time > metrics to inject into command and control responses. We should add these to > processors where it makes sense. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (MINIFICPP-844) We should drastically reduce the number of create_ff functions in NanoFi
[ https://issues.apache.org/jira/browse/MINIFICPP-844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-844: Assignee: Murtuza Shareef (was: Daniel Bakai) > We should drastically reduce the number of create_ff functions in NanoFi > - > > Key: MINIFICPP-844 > URL: https://issues.apache.org/jira/browse/MINIFICPP-844 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Murtuza Shareef >Priority: Major > Labels: nanofi > Fix For: 0.7.0 > > > The usage is a bit confusing, and the variants are unnecessary. We should > deprecate those which don't have a clear and explicit purpose for use cases. > Can break some of these out into an extended API if desired. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-854) Capture RTSP Frame
[ https://issues.apache.org/jira/browse/MINIFICPP-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924163#comment-16924163 ] Mr TheSegfault commented on MINIFICPP-854: -- [~le.nghia] Is this ticket resolved? If not I've changed the fix version so that we can push any additional PRs to 0.8.0 or later so that we can focus on other items within the backlog of 0.7.0 tickets. Thanks > Capture RTSP Frame > -- > > Key: MINIFICPP-854 > URL: https://issues.apache.org/jira/browse/MINIFICPP-854 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Nghia Le >Priority: Major > Fix For: 0.8.0 > > Time Spent: 7.5h > Remaining Estimate: 0h > > Need the ability to capture individual frames from a RTSP stream on an IP > camera. This is not full video recording but rather just have the processor > onTrigger to grab the frame from the RTSP enabled camera. The individual > frame is encoding as an image and placed in the flowfile content where > further analysis can be performed on the image. There are lots of other > things I want to add here but want to handle it in bit sized chunks. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-854) Capture RTSP Frame
[ https://issues.apache.org/jira/browse/MINIFICPP-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-854: - Fix Version/s: (was: 0.7.0) 0.8.0 > Capture RTSP Frame > -- > > Key: MINIFICPP-854 > URL: https://issues.apache.org/jira/browse/MINIFICPP-854 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Nghia Le >Priority: Major > Fix For: 0.8.0 > > Time Spent: 7.5h > Remaining Estimate: 0h > > Need the ability to capture individual frames from a RTSP stream on an IP > camera. This is not full video recording but rather just have the processor > onTrigger to grab the frame from the RTSP enabled camera. The individual > frame is encoding as an image and placed in the flowfile content where > further analysis can be performed on the image. There are lots of other > things I want to add here but want to handle it in bit sized chunks. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-916) OpenCV builds java, this isn't necessary
[ https://issues.apache.org/jira/browse/MINIFICPP-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924161#comment-16924161 ] Mr TheSegfault commented on MINIFICPP-916: -- [~le.nghia] Is this still an issue? > OpenCV builds java, this isn't necessary > > > Key: MINIFICPP-916 > URL: https://issues.apache.org/jira/browse/MINIFICPP-916 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Nghia Le >Priority: Major > Fix For: 0.7.0 > > > Remove java builds on OpenCV. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-962) C2 tests should use random port
[ https://issues.apache.org/jira/browse/MINIFICPP-962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-962. -- Resolution: Fixed > C2 tests should use random port > --- > > Key: MINIFICPP-962 > URL: https://issues.apache.org/jira/browse/MINIFICPP-962 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Some tests still fail (most probably due to port collision) randomly on > Travis: > The following tests FAILED: > [0;31m 59 - C2VerifyHeartbeatAndStop (SEGFAULT)[0;0m > Source: > [https://api.travis-ci.org/v3/job/556744709/log.txt] > > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1005) Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections
[ https://issues.apache.org/jira/browse/MINIFICPP-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1005: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections > --- > > Key: MINIFICPP-1005 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1005 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Daniel Bakai >Assignee: Daniel Bakai >Priority: Blocker > Fix For: 0.7.0 > > > This affects multiple processors, like ListenHTTP and InvokeHTTP, and C2. All > instances should be considered. > I am creating this as a separate blocker ticket and not including it in > MINIFICPP-814 to make that eaiser to backport to 0.6.0. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-963) Zero byte (no content) flow files cannot be received via HTTP S2S
[
https://issues.apache.org/jira/browse/MINIFICPP-963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mr TheSegfault resolved MINIFICPP-963.
--
Resolution: Fixed
> Zero byte (no content) flow files cannot be received via HTTP S2S
> -
>
> Key: MINIFICPP-963
> URL: https://issues.apache.org/jira/browse/MINIFICPP-963
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: 0.6.0
>Reporter: Arpad Boda
>Assignee: Arpad Boda
>Priority: Critical
> Fix For: 0.7.0
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> MiNiFi fails to receive zero byte flow files:
>
> {code}
> [2019-07-10 22:34:41.336]
> [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [debug] Site2Site
> transaction 82c938d9-0524-46e1-80a5-4c9d6073a9e6 receives attribute key 3
> [2019-07-10 22:34:41.336]
> [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [debug] Site2Site
> transaction 82c938d9-0524-46e1-80a5-4c9d6073a9e6 receives attribute key path
> value .
> [2019-07-10 22:34:41.336]
> [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [debug] Site2Site
> transaction 82c938d9-0524-46e1-80a5-4c9d6073a9e6 receives attribute key
> filename value 704.txt
> [2019-07-10 22:34:41.336]
> [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [debug] Site2Site
> transaction 82c938d9-0524-46e1-80a5-4c9d6073a9e6 receives attribute key uuid
> value e2d63ab8-838c-4ede-8fbd-d53fc053aec3
> [2019-07-10 22:34:41.336]
> [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [debug] Site2Site
> transaction 82c938d9-0524-46e1-80a5-4c9d6073a9e6 receives attribute ?
> [2019-07-10 22:34:41.336]
> [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site
> transaction 82c938d9-0524-46e1-80a5-4c9d6073a9e6 received flow record 0, with
> content size 0 bytes
> [2019-07-10 22:34:41.336]
> [org::apache::nifi::minifi::sitetosite::HttpSiteToSiteClient] [info] Site to
> Site closed transaction 82c938d9-0524-46e1-80a5-4c9d6073a9e6
> [2019-07-10 22:34:41.339] [org::apache::nifi::minifi::utils::HTTPClient]
> [error] curl_easy_perform() failed Failed writing received data to
> disk/application on
> [http://ec2-18-144-59-54.us-west-1.compute.amazonaws.com:8080/nifi-api/data-transfer/output-ports/dd25aa81-016b-1000-fbdb-6f2293428f21/transactions/82c938d9-0524-46e1-80a5-4c9d6073a9e6/flow-files]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (MINIFICPP-1025) CWEL should output event messages
Mr TheSegfault created MINIFICPP-1025: - Summary: CWEL should output event messages Key: MINIFICPP-1025 URL: https://issues.apache.org/jira/browse/MINIFICPP-1025 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Mr TheSegfault Assignee: Mr TheSegfault CWEL should output event messages -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (MINIFICPP-796) Enable Tests on Windows builds
[ https://issues.apache.org/jira/browse/MINIFICPP-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-796: Assignee: Mr TheSegfault (was: Alex Marmer) > Enable Tests on Windows builds > -- > > Key: MINIFICPP-796 > URL: https://issues.apache.org/jira/browse/MINIFICPP-796 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-796) Enable Tests on Windows builds
[ https://issues.apache.org/jira/browse/MINIFICPP-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-796. -- Resolution: Fixed > Enable Tests on Windows builds > -- > > Key: MINIFICPP-796 > URL: https://issues.apache.org/jira/browse/MINIFICPP-796 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-795) Add docker builds and improve flow of agent build
[ https://issues.apache.org/jira/browse/MINIFICPP-795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-795. -- Resolution: Fixed > Add docker builds and improve flow of agent build > - > > Key: MINIFICPP-795 > URL: https://issues.apache.org/jira/browse/MINIFICPP-795 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Priority: Minor > Fix For: 0.7.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Add docker builds and improve flow of agent build -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-721) Add time in queue/repo
[ https://issues.apache.org/jira/browse/MINIFICPP-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-721: - Fix Version/s: (was: 0.7.0) 0.8.0 > Add time in queue/repo > -- > > Key: MINIFICPP-721 > URL: https://issues.apache.org/jira/browse/MINIFICPP-721 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Labels: newbie > Fix For: 0.8.0 > > > Add metrics for time in queue to better debug various parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-932) Bootsrap incorrectly enforces NSS even when we are statically linking libressl
[ https://issues.apache.org/jira/browse/MINIFICPP-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-932. -- Resolution: Duplicate Closed in lieu of the work in https://github.com/apache/nifi-minifi-cpp/pull/610 > Bootsrap incorrectly enforces NSS even when we are statically linking libressl > -- > > Key: MINIFICPP-932 > URL: https://issues.apache.org/jira/browse/MINIFICPP-932 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Blocker > Fix For: 0.7.0 > > > This causes us to connect through libressl as we should. We likely should not > rely on the curl command to make this determination when we are statically > building libressl -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1017) Improve logging of docker tests
[ https://issues.apache.org/jira/browse/MINIFICPP-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1017: -- Fix Version/s: 0.7.0 > Improve logging of docker tests > --- > > Key: MINIFICPP-1017 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1017 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 0.7.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Docker tests are not logging MiNiFi and NiFi app logs and stdout properly > (bytestreams are printed). > Logs should be printed in a readable way. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-892) OPC UA Server Node Event
[ https://issues.apache.org/jira/browse/MINIFICPP-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916672#comment-16916672 ] Mr TheSegfault commented on MINIFICPP-892: -- [~aboda] Any update on these OPC tickets? Would love to try something out. > OPC UA Server Node Event > > > Key: MINIFICPP-892 > URL: https://issues.apache.org/jira/browse/MINIFICPP-892 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > > An OPC UA server can holds values from 1-247 unique Nodes. It is often > beneficial to allow for one of those unique nodes to trigger an event. > Meaning the node might generate a data point above a configured threshold. In > this case the ua server should provide a mechanism for capturing those node > events and forwarding them on to the upstream clients. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-53) Get/Put GPIO support
[ https://issues.apache.org/jira/browse/MINIFICPP-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-53: Fix Version/s: (was: 0.7.0) 0.8.0 > Get/Put GPIO support > > > Key: MINIFICPP-53 > URL: https://issues.apache.org/jira/browse/MINIFICPP-53 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Jeremy Dyer >Priority: Major > Labels: newbie > Fix For: 0.8.0 > > > Provide processors for getting the current value of a GPIO pin or setting the > value of a GPIO pin. These functionality should be optional and will not be > built by default. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-622) Transmit binary information for CAPI implementations in heartbeat
[ https://issues.apache.org/jira/browse/MINIFICPP-622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916670#comment-16916670 ] Mr TheSegfault commented on MINIFICPP-622: -- [~mshareef]I wanted to make you aware of this ticket. Given the work in nanofi [~bakaid] do you have any objections to assigning this to [~mshareef] and merging this with those efforts? We may be able to close this if the work will be done in the other tickets. > Transmit binary information for CAPI implementations in heartbeat > - > > Key: MINIFICPP-622 > URL: https://issues.apache.org/jira/browse/MINIFICPP-622 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Daniel Bakai >Priority: Major > Labels: CAPI, nanofi > Fix For: 0.7.0 > > > Heartbeats transmit a great deal of information as per our defined spec: > [https://cwiki.apache.org/confluence/display/MINIFI/C2+Design+Proposal] > This ticket will be responsible for figuring out what can and cannot be sent > via these binaries created using the CAPI > > For CAPI : probably need the AgentBuild information, but also potentially > information from a consumer's program – is there a way to capture information > about THEIR binary information > > May include information like python version, GCC version, etc that's NOT > about our library ( and not captured in AgentBuild) – but rather about > theirs. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-520) Create .deb as part of build process
[ https://issues.apache.org/jira/browse/MINIFICPP-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-520: - Fix Version/s: (was: 0.7.0) 0.8.0 > Create .deb as part of build process > > > Key: MINIFICPP-520 > URL: https://issues.apache.org/jira/browse/MINIFICPP-520 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > Fix For: 0.8.0 > > > We already offer a debian based binary build. Adding the .deb packaging step > would significantly improve the user installation experience by allowing > users on debian based systems to quickly install on their devices. This would > also be helpful to have for quickly bootstrapping devices like Raspberry PIs > without having to physically build on them which often takes a good bit of > time. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-780) Change GenerateFlowFile to allow 0b content FlowFIles
[ https://issues.apache.org/jira/browse/MINIFICPP-780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916668#comment-16916668 ] Mr TheSegfault commented on MINIFICPP-780: -- [~aboda] was this done on our side? I assigned from me to you because I thought you might know. feel free to unassign. thanks! > Change GenerateFlowFile to allow 0b content FlowFIles > - > > Key: MINIFICPP-780 > URL: https://issues.apache.org/jira/browse/MINIFICPP-780 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > > Would also behoove us to fill some gaps with this processor since it's used > so frequently -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-855) Provide an implementation for all S3 aspects of the AWS C++ SDK
[ https://issues.apache.org/jira/browse/MINIFICPP-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-855: - Fix Version/s: (was: 0.7.0) > Provide an implementation for all S3 aspects of the AWS C++ SDK > --- > > Key: MINIFICPP-855 > URL: https://issues.apache.org/jira/browse/MINIFICPP-855 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Provide an implementation for fully utilizing all aspects of the AWS C++ SDK. > This includes downloading and building the SDK locally as part of the build > and then implementing processors for the major S3 operations such as > GetObject, ListObject, FetchObject, PutObject, DeleteObject, and CopyObject. > As part of this epic the work should include a CredentialsController service > that allows for centralized configuration of the AWS credentials so they need > not be provided directly in the processors themselves. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-862) Create CopyS3Object Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-862: - Fix Version/s: (was: 0.7.0) > Create CopyS3Object Processor > - > > Key: MINIFICPP-862 > URL: https://issues.apache.org/jira/browse/MINIFICPP-862 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Create a processor that allows for an incoming flowfile to trigger the "copy" > of an object in S3. When the incoming flowfile enters this processor it > should contain the information in its attributes or content that inform the > processor of the src, destination, tags, and ACLs for the object that should > be copied in S3 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (MINIFICPP-780) Change GenerateFlowFile to allow 0b content FlowFIles
[ https://issues.apache.org/jira/browse/MINIFICPP-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-780: Assignee: Arpad Boda (was: Mr TheSegfault) > Change GenerateFlowFile to allow 0b content FlowFIles > - > > Key: MINIFICPP-780 > URL: https://issues.apache.org/jira/browse/MINIFICPP-780 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > > Would also behoove us to fill some gaps with this processor since it's used > so frequently -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-863) Create DeleteS3Object Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-863: - Fix Version/s: (was: 0.7.0) > Create DeleteS3Object Processor > --- > > Key: MINIFICPP-863 > URL: https://issues.apache.org/jira/browse/MINIFICPP-863 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Create a processor that will delete the specified object from S3. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-883) Add windows release script
[ https://issues.apache.org/jira/browse/MINIFICPP-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-883. -- Resolution: Fixed > Add windows release script > -- > > Key: MINIFICPP-883 > URL: https://issues.apache.org/jira/browse/MINIFICPP-883 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-816) remove debug target fix dll inclusion
[ https://issues.apache.org/jira/browse/MINIFICPP-816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-816. -- Resolution: Fixed > remove debug target fix dll inclusion > - > > Key: MINIFICPP-816 > URL: https://issues.apache.org/jira/browse/MINIFICPP-816 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test > Environment: Windows >Reporter: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-848) Break out Expression Language into true extension
[ https://issues.apache.org/jira/browse/MINIFICPP-848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-848. -- Resolution: Fixed > Break out Expression Language into true extension > - > > Key: MINIFICPP-848 > URL: https://issues.apache.org/jira/browse/MINIFICPP-848 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Priority: Major > Fix For: 0.7.0 > > > Needs to be done in preparation for allowing tests to build on windows. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-964) Exception thrown from onTrigger seems to make the C2 reload fail
[ https://issues.apache.org/jira/browse/MINIFICPP-964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916662#comment-16916662 ] Mr TheSegfault commented on MINIFICPP-964: -- [~nghiaxlee] Can you provide the full log? We get very aggressive on reload and will/should roll back on any failures.; however, as I see that code appears to log an exception. Is this with a change? Are you suggesting that the process blocks entirely or that the config will not be updated and thusly we'll report a failure through command and control? > Exception thrown from onTrigger seems to make the C2 reload fail > > > Key: MINIFICPP-964 > URL: https://issues.apache.org/jira/browse/MINIFICPP-964 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Nghia Le >Priority: Blocker > Fix For: 0.7.0 > > > I tried to publish new config for a class which contains CaptureRTSPFrame > processor. In the onTrigger function of that processor, there are 2 lines: > video_capture_.open(rtsp_url_); > video_backend_driver_ = video_capture_.getBackendName(); > > Which can cause an exception. > It leads to a block in the middle of reloading config process. Thus, the > config.yml will not be updated. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-964) Exception thrown from onTrigger seems to make the C2 reload fail
[ https://issues.apache.org/jira/browse/MINIFICPP-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-964: - Description: I tried to publish new config for a class which contains CaptureRTSPFrame processor. In the onTrigger function of that processor, there are 2 lines: video_capture_.open(rtsp_url_); video_backend_driver_ = video_capture_.getBackendName(); Which can cause an exception. It leads to a block in the middle of reloading config process. Thus, the config.yml will not be updated. was: I tried to publish new config from EFM for a class which contains CaptureRTSPFrame processor. In the onTrigger function of that processor, there are 2 lines: video_capture_.open(rtsp_url_); video_backend_driver_ = video_capture_.getBackendName(); Which can cause an exception. It leads to a block in the middle of reloading config process. Thus, the config.yml will not be updated. > Exception thrown from onTrigger seems to make the C2 reload fail > > > Key: MINIFICPP-964 > URL: https://issues.apache.org/jira/browse/MINIFICPP-964 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Nghia Le >Priority: Blocker > Fix For: 0.7.0 > > > I tried to publish new config for a class which contains CaptureRTSPFrame > processor. In the onTrigger function of that processor, there are 2 lines: > video_capture_.open(rtsp_url_); > video_backend_driver_ = video_capture_.getBackendName(); > > Which can cause an exception. > It leads to a block in the middle of reloading config process. Thus, the > config.yml will not be updated. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (MINIFICPP-1006) Wix toolchain should separate x64 and x86.
[ https://issues.apache.org/jira/browse/MINIFICPP-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-1006. --- Resolution: Fixed > Wix toolchain should separate x64 and x86. > --- > > Key: MINIFICPP-1006 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1006 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Critical > Fix For: 0.7.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > As part of the work to get the tests going we should add an x64 wix template -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-858) Create FetchS3Object Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-858: - Fix Version/s: (was: 0.7.0) > Create FetchS3Object Processor > -- > > Key: MINIFICPP-858 > URL: https://issues.apache.org/jira/browse/MINIFICPP-858 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Create a processor that can read the contents of an incoming flowfile to > determine the object that should be fetched from S3. This processor should > require input as it would be expected that ListS3 would be ran before this > processor to gather the list of objects in a bucket that should be retrieved. > If a single particular object is wished to be retrieved than GetS3Object > would be the correct processor. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-870) DeleteAzureBlob Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-870: - Fix Version/s: (was: 0.7.0) > DeleteAzureBlob Processor > - > > Key: MINIFICPP-870 > URL: https://issues.apache.org/jira/browse/MINIFICPP-870 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Create a processor for deleting an Azure blob object using the python SDK > method > [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#delete-blob-container-name--blob-name--snapshot-none--lease-id-none--delete-snapshots-none--if-modified-since-none--if-unmodified-since-none--if-match-none--if-none-match-none--timeout-none-] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-860) Create GetS3Object Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-860: - Fix Version/s: (was: 0.7.0) > Create GetS3Object Processor > > > Key: MINIFICPP-860 > URL: https://issues.apache.org/jira/browse/MINIFICPP-860 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Create a processor that Gets a predefined and specific object from an S3 > bucket in AWS. This processor would not accept input and would rather need to > use expression language or a hardcoded object key for the object that should > be retrieved. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-871) GetAzureBlobToStream Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-871: - Fix Version/s: (was: 0.7.0) > GetAzureBlobToStream Processor > -- > > Key: MINIFICPP-871 > URL: https://issues.apache.org/jira/browse/MINIFICPP-871 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Get an Azure blob object and stream the contents of that object into a new > flowfile using the Azure Python SDK method > [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#get-blob-to-stream-container-name--blob-name--stream--snapshot-none--start-range-none--end-range-none--validate-content-false--progress-callback-none--max-connections-2--lease-id-none--if-modified-since-none--if-unmodified-since-none--if-match-none--if-none-match-none--timeout-none-] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-872) CopyAzureBlob Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-872: - Fix Version/s: (was: 0.7.0) > CopyAzureBlob Processor > --- > > Key: MINIFICPP-872 > URL: https://issues.apache.org/jira/browse/MINIFICPP-872 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Processor for copying an Azure blob object asynchronously using the Azure > Python SDK > [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#copy-blob-container-name--blob-name--copy-source--metadata-none--source-if-modified-since-none--source-if-unmodified-since-none--source-if-match-none--source-if-none-match-none--destination-if-modified-since-none--destination-if-unmodified-since-none--destination-if-match-none--destination-if-none-match-none--destination-lease-id-none--source-lease-id-none--timeout-none-] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-861) Create PutS3Object
[ https://issues.apache.org/jira/browse/MINIFICPP-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-861: - Fix Version/s: (was: 0.7.0) > Create PutS3Object > -- > > Key: MINIFICPP-861 > URL: https://issues.apache.org/jira/browse/MINIFICPP-861 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Create a processor that allows for uploading a local flowfile to S3 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-869) ListAzureBlobs Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-869: - Fix Version/s: (was: 0.7.0) > ListAzureBlobs Processor > > > Key: MINIFICPP-869 > URL: https://issues.apache.org/jira/browse/MINIFICPP-869 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Create a new processor to list the blobs that are available in Azure blob > storage using the python SDK method. > [https://docs.microsoft.com/en-au/python/api/azure-storage-blob/azure.storage.blob.baseblobservice.baseblobservice?view=azure-python#list-blobs-container-name--prefix-none--num-results-none--include-none--delimiter-none--marker-none--timeout-none-] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1016) Resolve CWEL fields in XML
[ https://issues.apache.org/jira/browse/MINIFICPP-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1016: -- Description: Fields should be resolved to match what windows produces for the detailed view. We can handle this in line with the XML > Resolve CWEL fields in XML > -- > > Key: MINIFICPP-1016 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1016 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > Fields should be resolved to match what windows produces for the detailed > view. We can handle this in line with the XML -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (MINIFICPP-1016) Resolve CWEL fields in XML
Mr TheSegfault created MINIFICPP-1016: - Summary: Resolve CWEL fields in XML Key: MINIFICPP-1016 URL: https://issues.apache.org/jira/browse/MINIFICPP-1016 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Mr TheSegfault Assignee: Mr TheSegfault -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1013) Add ExecuteSQL capability for MSSQL
[ https://issues.apache.org/jira/browse/MINIFICPP-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1013: -- Labels: Windows (was: ) > Add ExecuteSQL capability for MSSQL > --- > > Key: MINIFICPP-1013 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1013 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Alex Marmer >Priority: Major > Labels: Windows > > Model this after ExecuteSQL that exists now, extend by using controller > service to integrate into the type of backing and use a library to query > MSSQL or potentially any other variant. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (MINIFICPP-1013) Add ExecuteSQL capability for MSSQL
[ https://issues.apache.org/jira/browse/MINIFICPP-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-1013: -- Description: Model this after ExecuteSQL that exists now, extend by using controller service to integrate into the type of backing and use a library to query MSSQL or potentially any other variant. > Add ExecuteSQL capability for MSSQL > --- > > Key: MINIFICPP-1013 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1013 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Assignee: Alex Marmer >Priority: Major > > Model this after ExecuteSQL that exists now, extend by using controller > service to integrate into the type of backing and use a library to query > MSSQL or potentially any other variant. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (MINIFICPP-1013) Add ExecuteSQL capability for MSSQL
Mr TheSegfault created MINIFICPP-1013: - Summary: Add ExecuteSQL capability for MSSQL Key: MINIFICPP-1013 URL: https://issues.apache.org/jira/browse/MINIFICPP-1013 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Mr TheSegfault Assignee: Alex Marmer -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (MINIFICPP-1005) Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections
[ https://issues.apache.org/jira/browse/MINIFICPP-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909104#comment-16909104 ] Mr TheSegfault commented on MINIFICPP-1005: --- Good call. I don't know if backporting is necessary. We can make it very clear in our documentation that this is the case for 0.7.0 moving forward. That also sets a precedent that we may need to go back further; however, having a separate ticket, but we typically squash when merging – we'll have to make sure not to do that. > Only allow >=TLSv1.2 incoming and >=TLSv1.0 outgoing secure connections > --- > > Key: MINIFICPP-1005 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1005 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Daniel Bakai >Assignee: Daniel Bakai >Priority: Blocker > Fix For: 0.7.0 > > > This affects multiple processors, like ListenHTTP and InvokeHTTP, and C2. All > instances should be considered. > I am creating this as a separate blocker ticket and not including it in > MINIFICPP-814 to make that eaiser to backport to 0.6.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (MINIFICPP-1001) Provide better logging when a stoppage delay occurs
Mr TheSegfault created MINIFICPP-1001: - Summary: Provide better logging when a stoppage delay occurs Key: MINIFICPP-1001 URL: https://issues.apache.org/jira/browse/MINIFICPP-1001 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault I experienced a delay because part of the RPG code was waiting to complete. We can probably make this a bit more clear to the user in the logs that we're waiting on this "processor" to stop. [2019-08-13 08:37:55.453] [org::apache::nifi::minifi::FlowController] [info] Started Flow Controller [2019-08-13 08:37:55.453] [main] [info] MiNiFi started [2019-08-13 08:37:55.453] [org::apache::nifi::minifi::FlowController] [info] Stop Flow Controller [2019-08-13 08:37:55.848] [org::apache::nifi::minifi::processors::TailFile] [error] store state file failed [2019-08-13 08:37:57.865] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Timeout was reached on http://192.168.1.172:10080/efm/api/c2-protocol/heartbeat [2019-08-13 08:38:00.274] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Timeout was reached on http://192.168.1.172:10080/efm/api/c2-protocol/heartbeat [2019-08-13 08:38:02.683] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Timeout was reached on http://192.168.1.172:10080/efm/api/c2-protocol/heartbeat [2019-08-13 08:38:05.091] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Timeout was reached on http://192.168.1.172:10080/efm/api/c2-protocol/heartbeat [2019-08-13 08:38:05.447] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Timeout was reached on http://192.168.1.172:8080/nifi-api/site-to-site [2019-08-13 08:38:05.447] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error] ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed , response code 0 [2019-08-13 08:38:05.447] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [info] no protocol, yielding [2019-08-13 08:38:05.447] [org::apache::nifi::minifi::FlowController] [info] Unload Flow Controller [2019-08-13 08:38:05.448] [main] [info] MiNiFi exit -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (MINIFICPP-997) findProcessor should use UUID in addition to name
Mr TheSegfault created MINIFICPP-997: Summary: findProcessor should use UUID in addition to name Key: MINIFICPP-997 URL: https://issues.apache.org/jira/browse/MINIFICPP-997 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Mr TheSegfault Assignee: Mr TheSegfault *findProcessor in PG should search by UUID primarily, then name.* -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-994) Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it should be part of MiNiFi build.
[
https://issues.apache.org/jira/browse/MINIFICPP-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899174#comment-16899174
]
Mr TheSegfault commented on MINIFICPP-994:
--
[~aboda] I just happened to notice an affects version of 0.6.0. This extension
didn't exist in that release.
> Opencv shouldn't be build by bootstrap (cmake), just cloned. Building it
> should be part of MiNiFi build.
>
>
> Key: MINIFICPP-994
> URL: https://issues.apache.org/jira/browse/MINIFICPP-994
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: 0.6.0
>Reporter: Arpad Boda
>Assignee: Nghia Le
>Priority: Minor
>
> Opencv is built when executing a cmake command (for eg. using bootstrap),
> which seems to be wrong as:
> * This build is single-threaded, takes a lot of time
> * Linker warnings are generated when building MiNiFI:
> {code}
> ld: warning: direct access in function '___cxx_global_var_init.35' from file
> '../thirdparty/opencv/opencv-external/src/opencv-external-build/lib/libopencv_flann.a(miniflann.cpp.o)'
> to global weak symbol 'guard variable for
> cvflann::anyimpl::SinglePolicy std::__1::char_traits, std::__1::allocator > >::policy' from file
> 'CMakeFiles/CaptureRTSPFrameTest.dir/CaptureRTSPFrameTest.cpp.o' means the
> weak symbol cannot be overridden at runtime. This was likely caused by
> different translation units being compiled with different visibility settings.
> {code}
> Static linking should be kept when integrating opencv build to the build
> process of MiNiFi.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (MINIFICPP-996) Output port keeps printing info message implying transmission
Mr TheSegfault created MINIFICPP-996: Summary: Output port keeps printing info message implying transmission Key: MINIFICPP-996 URL: https://issues.apache.org/jira/browse/MINIFICPP-996 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Mr TheSegfault Here is an example of the issue. It seems that even though no data is being transmitted we are going through a code path. [~aboda] did you see this? 2019-08-02 15:51:43.164] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site transaction f3b2dad4-b55e-11e9-9df5-74e5f944bfe7 received flow record 0, with content size 0 bytes [2019-08-02 15:51:45.178] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site transaction f4e6260e-b55e-11e9-9df5-74e5f944bfe7 received flow record 0, with content size 0 bytes [2019-08-02 15:51:47.189] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site transaction f618f57e-b55e-11e9-9df5-74e5f944bfe7 received flow record 0, with content size 0 bytes [2019-08-02 15:51:49.211] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site transaction f74d8c52-b55e-11e9-9df5-74e5f944bfe7 received flow record 0, with content size 0 bytes [2019-08-02 15:51:51.227] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site transaction f8811d64-b55e-11e9-9df5-74e5f944bfe7 received flow record 0, with content size 0 bytes [2019-08-02 15:51:53.245] [org::apache::nifi::minifi::sitetosite::SiteToSiteClient] [info] Site to Site transaction f9b50b96-b55e-11e9-9df5-74e5f944bfe7 received flow record 0, with content size 0 bytes -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (MINIFICPP-993) Use JNI to leverage NiFi tests for V on MiNiFi components
Mr TheSegfault created MINIFICPP-993: Summary: Use JNI to leverage NiFi tests for V on MiNiFi components Key: MINIFICPP-993 URL: https://issues.apache.org/jira/browse/MINIFICPP-993 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault We should be able to use the JNI extension to test MiNiFi against NiFi tests that already exist and are trusted -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-990) Segfault on SIGINT after shutdown initiated
[ https://issues.apache.org/jira/browse/MINIFICPP-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897484#comment-16897484 ] Mr TheSegfault edited comment on MINIFICPP-990 at 7/31/19 8:08 PM: --- Thanks for debugging. I always assumed the issue is that we wait on the sem, close it then unlink it before unloading flow controller, meaning when we sem_post, the semaphore won't exist. We should likely move those sem close and sem unlink after "MiNiFi exit" – but I haven't actually debugged it myself. I always made a mental note to "look into it" but never did. I see it a lot when I'm waiting for the repos to clear out and get impatient ( hitting ctrl+c multiple times)so this could potentially cause data loss. was (Author: phrocker): Thanks for debugging. I always assumed the issue is that we wait on the sem, close it then unlink it before unloading flow controller, meaning when we sem_post, the semaphore won't exist. We should likely move those sem close and sem unlink after "MiNiFi exit" – but I haven't actually debugged it myself. I always made a mental note to "look into it" but never did. > Segfault on SIGINT after shutdown initiated > > > Key: MINIFICPP-990 > URL: https://issues.apache.org/jira/browse/MINIFICPP-990 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Priority: Major > Fix For: 0.7.0 > > > If I send a SIGINT (ctrl-c) to MiNiFI, MiNiFi begins shutdown. If I send > another SIGINT while that shutdown is in progress, MiNiFi crashes with a > segfault. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (MINIFICPP-990) Segfault on SIGINT after shutdown initiated
[ https://issues.apache.org/jira/browse/MINIFICPP-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-990: - Fix Version/s: 0.7.0 > Segfault on SIGINT after shutdown initiated > > > Key: MINIFICPP-990 > URL: https://issues.apache.org/jira/browse/MINIFICPP-990 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Priority: Major > Fix For: 0.7.0 > > > If I send a SIGINT (ctrl-c) to MiNiFI, MiNiFi begins shutdown. If I send > another SIGINT while that shutdown is in progress, MiNiFi crashes with a > segfault. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-990) Segfault on SIGINT after shutdown initiated
[ https://issues.apache.org/jira/browse/MINIFICPP-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897484#comment-16897484 ] Mr TheSegfault commented on MINIFICPP-990: -- Thanks for debugging. I always assumed the issue is that we wait on the sem, close it then unlink it before unloading flow controller, meaning when we sem_post, the semaphore won't exist. We should likely move those sem close and sem unlink after "MiNiFi exit" – but I haven't actually debugged it myself. I always made a mental note to "look into it" but never did. > Segfault on SIGINT after shutdown initiated > > > Key: MINIFICPP-990 > URL: https://issues.apache.org/jira/browse/MINIFICPP-990 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Priority: Major > > If I send a SIGINT (ctrl-c) to MiNiFI, MiNiFi begins shutdown. If I send > another SIGINT while that shutdown is in progress, MiNiFi crashes with a > segfault. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (MINIFICPP-992) Move name of connection queue inside object
Mr TheSegfault created MINIFICPP-992: Summary: Move name of connection queue inside object Key: MINIFICPP-992 URL: https://issues.apache.org/jira/browse/MINIFICPP-992 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault Assignee: Mr TheSegfault Use this as an opportunity to clean up FlowInformation.h too -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-976) Add Kafka broker to docker verify
[ https://issues.apache.org/jira/browse/MINIFICPP-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896164#comment-16896164 ] Mr TheSegfault commented on MINIFICPP-976: -- [~aboda] I took this one out of the pile as it may prove useful for another ticket. > Add Kafka broker to docker verify > -- > > Key: MINIFICPP-976 > URL: https://issues.apache.org/jira/browse/MINIFICPP-976 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Nghia Le >Priority: Major > > Add docker kafka tests -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-959) Review librdkafka thread safety
[ https://issues.apache.org/jira/browse/MINIFICPP-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896152#comment-16896152 ] Mr TheSegfault commented on MINIFICPP-959: -- finally [~le.nghia] this would be a good opportunity to break out some of the classes into their own source files so that they can be more verified that tests exist through an automated framework. > Review librdkafka thread safety > --- > > Key: MINIFICPP-959 > URL: https://issues.apache.org/jira/browse/MINIFICPP-959 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Daniel Bakai >Assignee: Nghia Le >Priority: Major > > I don't think librdkafka is doing what it is supposed to around the > KafkaLease and the atomic spinlock combined with mutexes. > It is thread safe in a way that resources are only accessed by one thread at > a time, but the behaviour otherwise is I think not what was intended > (multiple threads can initialize a KafkaConnection after each other, for > example). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-959) Review librdkafka thread safety
[ https://issues.apache.org/jira/browse/MINIFICPP-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896148#comment-16896148 ] Mr TheSegfault edited comment on MINIFICPP-959 at 7/30/19 1:50 PM: --- It should also be noted that if I have my way: https://issues.apache.org/jira/browse/MINIFICPP-974 we would force PublishKafka ( and many others ) to be removed from the release list without sufficient tesets. was (Author: phrocker): It should also be noted that if I have my way: https://issues.apache.org/jira/browse/MINIFICPP-974 we would force PublishKafka ( and many others ) to be removed from the release list. > Review librdkafka thread safety > --- > > Key: MINIFICPP-959 > URL: https://issues.apache.org/jira/browse/MINIFICPP-959 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Daniel Bakai >Assignee: Nghia Le >Priority: Major > > I don't think librdkafka is doing what it is supposed to around the > KafkaLease and the atomic spinlock combined with mutexes. > It is thread safe in a way that resources are only accessed by one thread at > a time, but the behaviour otherwise is I think not what was intended > (multiple threads can initialize a KafkaConnection after each other, for > example). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-959) Review librdkafka thread safety
[ https://issues.apache.org/jira/browse/MINIFICPP-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896148#comment-16896148 ] Mr TheSegfault commented on MINIFICPP-959: -- It should also be noted that if I have my way: https://issues.apache.org/jira/browse/MINIFICPP-974 we would force PublishKafka ( and many others ) to be removed from the release list. > Review librdkafka thread safety > --- > > Key: MINIFICPP-959 > URL: https://issues.apache.org/jira/browse/MINIFICPP-959 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Daniel Bakai >Assignee: Nghia Le >Priority: Major > > I don't think librdkafka is doing what it is supposed to around the > KafkaLease and the atomic spinlock combined with mutexes. > It is thread safe in a way that resources are only accessed by one thread at > a time, but the behaviour otherwise is I think not what was intended > (multiple threads can initialize a KafkaConnection after each other, for > example). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-974) Ensure that non-docker tested processors are not included into the make packaged docker builds
[ https://issues.apache.org/jira/browse/MINIFICPP-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896146#comment-16896146 ] Mr TheSegfault commented on MINIFICPP-974: -- [~aboda] and [~aldrin] what do you think about this? I think we should draw a line in the sand that if something is contributed without sufficient tests in either unit and/or docker we don't release it with our Apache process. Obviously we can't pull the wool out from the processors and features we have, so this would have to be staged, but I'd like to set the stage for controlling what we release. > Ensure that non-docker tested processors are not included into the make > packaged docker builds > -- > > Key: MINIFICPP-974 > URL: https://issues.apache.org/jira/browse/MINIFICPP-974 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Mr TheSegfault >Priority: Major > > This ensures that we don't trust and deploy artifacts that don't have some > type of regression test framework as it relates to our SITs. There may be > exceptions, so we can feel this desire out. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-959) Review librdkafka thread safety
[ https://issues.apache.org/jira/browse/MINIFICPP-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896144#comment-16896144 ] Mr TheSegfault commented on MINIFICPP-959: -- [~le.nghia] This I think would be easier to verify if we actually had tests on kafka's usage. PublishKafka was a quick implementation and we really need to beef tests under it. I think we need to link this to MINIFICPP-976, because docker will be a very easy environment to pull down a docker image with a kafka broker. I don't think we should create integration tests in our normal test suite > Review librdkafka thread safety > --- > > Key: MINIFICPP-959 > URL: https://issues.apache.org/jira/browse/MINIFICPP-959 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Daniel Bakai >Assignee: Nghia Le >Priority: Major > > I don't think librdkafka is doing what it is supposed to around the > KafkaLease and the atomic spinlock combined with mutexes. > It is thread safe in a way that resources are only accessed by one thread at > a time, but the behaviour otherwise is I think not what was intended > (multiple threads can initialize a KafkaConnection after each other, for > example). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (MINIFICPP-959) Review librdkafka thread safety
[ https://issues.apache.org/jira/browse/MINIFICPP-959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-959: Assignee: Nghia Le > Review librdkafka thread safety > --- > > Key: MINIFICPP-959 > URL: https://issues.apache.org/jira/browse/MINIFICPP-959 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Daniel Bakai >Assignee: Nghia Le >Priority: Major > > I don't think librdkafka is doing what it is supposed to around the > KafkaLease and the atomic spinlock combined with mutexes. > It is thread safe in a way that resources are only accessed by one thread at > a time, but the behaviour otherwise is I think not what was intended > (multiple threads can initialize a KafkaConnection after each other, for > example). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (MINIFICPP-976) Add Kafka broker to docker verify
[ https://issues.apache.org/jira/browse/MINIFICPP-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-976: Assignee: Nghia Le > Add Kafka broker to docker verify > -- > > Key: MINIFICPP-976 > URL: https://issues.apache.org/jira/browse/MINIFICPP-976 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Nghia Le >Priority: Major > > Add docker kafka tests -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (MINIFICPP-678) Improve const correctness of code base.
[ https://issues.apache.org/jira/browse/MINIFICPP-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-678: - Fix Version/s: 0.8.0 0.9.0 > Improve const correctness of code base. > > > Key: MINIFICPP-678 > URL: https://issues.apache.org/jira/browse/MINIFICPP-678 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Minor > Fix For: 0.9.0, 0.8.0, 0.7.0 > > > As per MINIFICPP-676 we should improve our long term const correctness. I > agree that this is a good idea ( thanks to [~aboda] for calling this out in > his PR), especially one that I haven't put a lot of emphasis on changing in > code that has been around a while. > For non stream or memory transfer objects, we want to maintain const > correctness. In some cases it makes sense, especially for equality > comparisons and getters to make changes. In some places it may require some > evaluation and tested. Since we are an established open source projects, > might need to safeguard certain changes and do full integration testing when > we can. > > In all cases, we should test all extensions before any merge. We can split > these efforts up as we see fit, but all tickets are relatively minor unless > there is an established bug resulting from that behavior. I'd like to start a > release of 0.6.0 soon so we should attempt to minimize overall impact to this > release and take great care in what we change. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (MINIFICPP-867) Provide an implementation for all Blob storage aspects of the Azure
[ https://issues.apache.org/jira/browse/MINIFICPP-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-867: - Fix Version/s: (was: 0.7.0) > Provide an implementation for all Blob storage aspects of the Azure > --- > > Key: MINIFICPP-867 > URL: https://issues.apache.org/jira/browse/MINIFICPP-867 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Provide an implementation for utilizing Azure Blob storage. Azure currently > does not have a C++ SDK. However they do have a good python library. Since > MiNifi C++ now supports native python processors it seems logical that we > offer out of the box processors and controller services so that we can Put, > Get, List, Fetch, Copy, and Delete objects in Azure blob storage. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (MINIFICPP-868) Create AzureCredentialsService
[ https://issues.apache.org/jira/browse/MINIFICPP-868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-868: - Fix Version/s: (was: 0.7.0) > Create AzureCredentialsService > -- > > Key: MINIFICPP-868 > URL: https://issues.apache.org/jira/browse/MINIFICPP-868 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > > Controller service for managing the Azure credentials. This service will > house the credentials in a secure manner and proxy the credentials to the > configured processor when requested. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-947) Fix all -Wall warnings, gradually fix or disable warnings from -Weverything
[ https://issues.apache.org/jira/browse/MINIFICPP-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894006#comment-16894006 ] Mr TheSegfault edited comment on MINIFICPP-947 at 7/26/19 5:41 PM: --- Definitely another long lived task I think. Though this one has the potential to have a stopping point with the arrival at warning free base – so hopeful we can do this over time. Extensions may need to have a pass on this since we will have many owners. We can draw a line in the sand, but like NiFi there may be an extension that for whatever reason abides by different principles. I don't want to preclude that necessarily but open the discussion on that. In my opinion we should branch out and have a discussion on what it means to provide an extension. Without extension registry they exist in our code base. But some extensions we don't own/know/update so they are not truly "ours" We own libminifi and nanofi. There are some extensions that we update and support, so perhaps we can define a way that registers extensions as apache released – similar to maven profiles. Can make a decree that those extensions must be warning free to be released but won't fail compilation otherwise. was (Author: phrocker): Definitely another long lived task I think. Though this one has the potential to have a stopping point with the arrival at warning free base – so hopeful we can do this over time. Extensions may need to have a pass on this since we will have many owners. We can draw a line in the sand, but like NiFi there may be an extension that for whatever reason abides by different principles. I don't want to preclude that necessarily but open the discussion on that. In my opinion we should branch out and have a discussion on what it means to provide an extension. Without extension registry they exist in our code base. But some extensions we don't own/know/update so they are not truly "ours" We own libminifi and nanofi. There are some extensions that we update and support, so perhaps we can define a way that registers extensions as apache released – similar to maven profiles. those extensions must be warning free to be released but won't fail compilation otherwise. > Fix all -Wall warnings, gradually fix or disable warnings from -Weverything > --- > > Key: MINIFICPP-947 > URL: https://issues.apache.org/jira/browse/MINIFICPP-947 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > * Create macros for locally enabling and disabling warnings (pragma warning > pop, push, etc.) > * Compile third-parties with different warning settings than our code > * Make serious warnings errors > * Once we are warning-free, only merge code that does not introduce new > warnings -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-947) Fix all -Wall warnings, gradually fix or disable warnings from -Weverything
[ https://issues.apache.org/jira/browse/MINIFICPP-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894006#comment-16894006 ] Mr TheSegfault commented on MINIFICPP-947: -- Definitely another long lived task I think. Though this one has the potential to have a stopping point with the arrival at warning free base – so hopeful we can do this over time. Extensions may need to have a pass on this since we will have many owners. We can draw a line in the sand, but like NiFi there may be an extension that for whatever reason abides by different principles. I don't want to preclude that necessarily but open the discussion on that. In my opinion we should branch out and have a discussion on what it means to provide an extension. Without extension registry they exist in our code base. But some extensions we don't own/know/update so they are not truly "ours" We own libminifi and nanofi. There are some extensions that we update and support, so perhaps we can define a way that registers extensions as apache released – similar to maven profiles. those extensions must be warning free to be released but won't fail compilation otherwise. > Fix all -Wall warnings, gradually fix or disable warnings from -Weverything > --- > > Key: MINIFICPP-947 > URL: https://issues.apache.org/jira/browse/MINIFICPP-947 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > * Create macros for locally enabling and disabling warnings (pragma warning > pop, push, etc.) > * Compile third-parties with different warning settings than our code > * Make serious warnings errors > * Once we are warning-free, only merge code that does not introduce new > warnings -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-953) Review the code for proper integral type usage
[ https://issues.apache.org/jira/browse/MINIFICPP-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894000#comment-16894000 ] Mr TheSegfault edited comment on MINIFICPP-953 at 7/26/19 5:18 PM: --- This particular ticket seems like one which will outlive the rest. I don't think we should go scouring code looking for this but rather keep this open and use it to track the efforts as they arise. I have seen tickets like this from myself and [~aboda] , so I hope this doesn't get lost in time and is used to address these concerns over time – particularly to help with tracking was (Author: phrocker): This particular ticket seems like one which will outlive the rest. I don't think we should go scouring code looking for this but rather keep this open and use it to track the efforts as they arise. I have seen tickets like this from myself and [~aboda] , so I hope this doesn't get lost in time and is used to address these concerns over time. > Review the code for proper integral type usage > -- > > Key: MINIFICPP-953 > URL: https://issues.apache.org/jira/browse/MINIFICPP-953 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > * There are many cases of improper narrowings, badly chosen integer types, > unnoticed implicit casts and potential overflows > * These can cause bugs and are a huge potential security issue -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-953) Review the code for proper integral type usage
[ https://issues.apache.org/jira/browse/MINIFICPP-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894000#comment-16894000 ] Mr TheSegfault commented on MINIFICPP-953: -- This particular ticket seems like one which will outlive the rest. I don't think we should go scouring code looking for this but rather keep this open and use it to track the efforts as they arise. I have seen tickets like this from myself and [~aboda] , so I hope this doesn't get lost in time and is used to address these concerns over time. > Review the code for proper integral type usage > -- > > Key: MINIFICPP-953 > URL: https://issues.apache.org/jira/browse/MINIFICPP-953 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > * There are many cases of improper narrowings, badly chosen integer types, > unnoticed implicit casts and potential overflows > * These can cause bugs and are a huge potential security issue -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-941) Ideas for code quality, architectural or process improvements
[ https://issues.apache.org/jira/browse/MINIFICPP-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893992#comment-16893992 ] Mr TheSegfault edited comment on MINIFICPP-941 at 7/26/19 5:15 PM: --- [~bakaid] Thanks for making the effort to compile this list. I love the initiative and I look forward to planning/implementing some of these. Thanks! Lot of great ideas! was (Author: phrocker): [~bakaid] Thanks for making the effort to compile this list. I love the initiative and I look forward to planning/implementing these. Thanks! Lot of great ideas! > Ideas for code quality, architectural or process improvements > - > > Key: MINIFICPP-941 > URL: https://issues.apache.org/jira/browse/MINIFICPP-941 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Daniel Bakai >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-944) Run with Valgrind
[ https://issues.apache.org/jira/browse/MINIFICPP-944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893998#comment-16893998 ] Mr TheSegfault commented on MINIFICPP-944: -- yes yes yes! > Run with Valgrind > - > > Key: MINIFICPP-944 > URL: https://issues.apache.org/jira/browse/MINIFICPP-944 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > Preferably in CI as well. > We also have to fix long double-using ExpressionLanguage tests which > currently fail under Valgrind -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-956) Support compiling and linking our own libc++ and depend on libc only as a dynamic dependency.
[ https://issues.apache.org/jira/browse/MINIFICPP-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893996#comment-16893996 ] Mr TheSegfault commented on MINIFICPP-956: -- We can and should support doing this. I have a branch based on a NiFi registry PR ( for bootstrapping agents ) that does just this ( i have my toolchain compiled b/c of cross compilation needs ). It works quite well, but required a few changes – wasn't too bad – however, it's predicated upon selecting an option. As a result we will always have Apache releases that support this and other ways of building. This will work to increase the number of travis targets and increase the complexity. How do you think we can balance those complexities? Do you think that's something we'll have to figure out over time or do you have a plan to address those various deployments? I agree this is a lovely goal that I hope we implement sooner rather than later but am concerned with the potential implications or complexities of supporting this – not in a way that we shouldn't do this but rather can we mitigate those concerns up front? > Support compiling and linking our own libc++ and depend on libc only as a > dynamic dependency. > - > > Key: MINIFICPP-956 > URL: https://issues.apache.org/jira/browse/MINIFICPP-956 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > * It would enable us to use C++14/C++17 on all platforms without losing > compatibility with older Linux systems -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-941) Ideas for code quality, architectural or process improvements
[ https://issues.apache.org/jira/browse/MINIFICPP-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893992#comment-16893992 ] Mr TheSegfault edited comment on MINIFICPP-941 at 7/26/19 5:15 PM: --- [~bakaid] Thanks for making the effort to compile this list. I love the initiative and I look forward to planning/implementing these. Thanks! Lot of great ideas! was (Author: phrocker): [~bakaid] Thanks for making the effort to compile this list. I love the initiative and I look forward to planning these. Thanks! > Ideas for code quality, architectural or process improvements > - > > Key: MINIFICPP-941 > URL: https://issues.apache.org/jira/browse/MINIFICPP-941 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Daniel Bakai >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-950) Create a strongly-typed file system util
[ https://issues.apache.org/jira/browse/MINIFICPP-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893991#comment-16893991 ] Mr TheSegfault commented on MINIFICPP-950: -- Forgive me but if we moved to statically linking our own glibc, could we not use std::filesystem instead of doing this? > Create a strongly-typed file system util > > > Key: MINIFICPP-950 > URL: https://issues.apache.org/jira/browse/MINIFICPP-950 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > * Move all filesystem operations here and make everything use it > * Local and remote paths should have different types > * Handle absolute and relative paths differently > * Add path concatenation, root search, etc. > * Use W APIs with UNC paths on Windows -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-954) Consider moving the C2 line protocol to Google Protobuf
[ https://issues.apache.org/jira/browse/MINIFICPP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893988#comment-16893988 ] Mr TheSegfault edited comment on MINIFICPP-954 at 7/26/19 5:04 PM: --- [~bakaid] I like some of the concepts you present but prior CVEs were why we moved away from protobuf initially in community discussions. I think offloading the testing to protobuf maintainers is a reason to have Protobuf but I think both will need to exist. We also considered flatbuffers and capn proto – would love to see that added to this discussion. I think this ultimately would be either adding protobuf/flatbuffers/capn as a selectable line format. We have a very simple line format that isn't going to change ( this is because our line formats are designed to use the RESTFul API for the complicated stuff ). The line format is simply for sending a very tiny subset of information. We also want to use the same code for nanofi, where I think our consumers are more likely to appreciate not including protobuf. As a result we can add this as an extension ( in fact, I love the idea ) – especially since we can mitigate concerns of others by having it be configurable in the agent ( and compile time for nanofi ECUs ). If we went down the route of cap'n proto we can rely on the embedded RPC and use that as a separate C2 protocol. I have used Cap'n Proto, for example, and we could easily define those RPC calls to lessen the load, but I would likely change this ticket to exploring alternative protocols/wire formats and then making a decision as an addition to what we have now as opposed to a replacement. We will be forced to maintain it but we have no choice either as we have users who specifically don't want protobuf and we have released it so we are forced to maintain it and improve its testing. Keep in mind that whomever tackles this will need to balance maintaining what we already have ( not replacing ) so we then increase what we must test. We can rely on capn proto or protobuf's tests, but then we still need to perform our test in the venn diagram of testing. Would that effort be better spent improving tests for our work since we already need to do that and must maintain it? I'll leave that question up to the implementor – but it's certainly not an affront against doing this work, more a word of caution. I'm glad we decided against protobuf at the time, though – would have made that decision again. It was a calculated risk with many thoughts in mind – but would love to see something else like protobuf/flatbuffers/capn . Aside from this comment I would love the community to weigh in and make a choice. I've made my points known and have no intention of implementing this myself due to other technical debt I feel is more important so thoughts on what to do are up to the implementation seeker. was (Author: phrocker): [~bakaid] I like some of the concepts you present but prior CVEs were why we moved away from protobuf initially in community discussions. I think offloading the testing to protobuf maintainers is a reason to have Protobuf but I think both will need to exist. We also considered flatbuffers and capn proto. I think this ultimately would be either adding protobuf/flatbuffers/capn as a selectable line format. We have a very simple line format that isn't going to change ( this is because our line formats are designed to use the RESTFul API for the complicated stuff ). The line format is simply for sending a very tiny subset of information. We also want to use the same code for nanofi, where I think our consumers are more likely to appreciate not including protobuf. As a result we can add this as an extension ( in fact, I love the idea ) – especially since we can mitigate concerns of others by having it be configurable in the agent ( and compile time for nanofi ECUs ). If we went down the route of cap'n proto we can rely on the embedded RPC and use that as a separate C2 protocol. I have used Cap'n Proto, for example, and we could easily define those RPC calls to lessen the load, but I would likely change this ticket to exploring alternative protocols/wire formats and then making a decision as an addition to what we have now as opposed to a replacement. We will be forced to maintain it but we have no choice either as we have users who specifically don't want protobuf and we have released it so we are forced to maintain it and improve its testing. Keep in mind that whomever tackles this will need to balance maintaining what we already have ( not replacing ) so we then increase what we must test. We can rely on capn proto or protobuf's tests, but then we still need to perform our test in the venn diagram of testing. Would that effort be better spent improving tests for our work since we already need to do that and must maintain it? I'll
[jira] [Commented] (MINIFICPP-954) Consider moving the C2 line protocol to Google Protobuf
[ https://issues.apache.org/jira/browse/MINIFICPP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893988#comment-16893988 ] Mr TheSegfault commented on MINIFICPP-954: -- [~bakaid] I like some of the concepts you present but prior CVEs were why we moved away from protobuf initially in community discussions. I think offloading the testing to protobuf maintainers is a reason to have Protobuf but I think both will need to exist. We also considered flatbuffers and capn proto. I think this ultimately would be either adding protobuf/flatbuffers/capn as a selectable line format. We have a very simple line format that isn't going to change ( this is because our line formats are designed to use the RESTFul API for the complicated stuff ). The line format is simply for sending a very tiny subset of information. We also want to use the same code for nanofi, where I think our consumers are more likely to appreciate not including protobuf. As a result we can add this as an extension ( in fact, I love the idea ) – especially since we can mitigate concerns of others by having it be configurable in the agent ( and compile time for nanofi ECUs ). If we went down the route of cap'n proto we can rely on the embedded RPC and use that as a separate C2 protocol. I have used Cap'n Proto, for example, and we could easily define those RPC calls to lessen the load, but I would likely change this ticket to exploring alternative protocols/wire formats and then making a decision as an addition to what we have now as opposed to a replacement. We will be forced to maintain it but we have no choice either as we have users who specifically don't want protobuf and we have released it so we are forced to maintain it and improve its testing. Keep in mind that whomever tackles this will need to balance maintaining what we already have ( not replacing ) so we then increase what we must test. We can rely on capn proto or protobuf's tests, but then we still need to perform our test in the venn diagram of testing. Would that effort be better spent improving tests for our work since we already need to do that and must maintain it? I'll leave that question up to the implementor – but it's certainly not an affront against doing this work, more a word of caution. I'm glad we decided against protobuf at the time, though – would have made that decision again. It was a calculated risk with many thoughts in mind – but would love to see something else like protobuf/flatbuffers/capn . > Consider moving the C2 line protocol to Google Protobuf > --- > > Key: MINIFICPP-954 > URL: https://issues.apache.org/jira/browse/MINIFICPP-954 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Daniel Bakai >Priority: Major > > * Very compact serialized format > * Easy protocol description > * Support for generating parsers for many languages from the proto file > * Has good versioning support > * Thoroughly tested, fuzzed > * Less chance of introducing security vulnerabilities with hand-written > parsers -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-954) Consider moving the C2 line protocol to Google Protobuf
[ https://issues.apache.org/jira/browse/MINIFICPP-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893988#comment-16893988 ] Mr TheSegfault edited comment on MINIFICPP-954 at 7/26/19 5:04 PM: --- [~bakaid] I like some of the concepts you present but prior CVEs were why we moved away from protobuf initially in community discussions. I think offloading the testing to protobuf maintainers is a reason to have Protobuf but I think both will need to exist. We also considered flatbuffers and capn proto – would love to see that added to this discussion. I think this ultimately would be either adding protobuf/flatbuffers/capn as a selectable line format. We have a very simple line format that isn't going to change ( this is because our line formats are designed to use the RESTFul API for the complicated stuff ). The line format is simply for sending a very tiny subset of information. We also want to use the same code for nanofi, where I think our consumers are more likely to appreciate not including protobuf. As a result we can add this as an extension ( in fact, I love the idea ) – especially since we can mitigate concerns of others by having it be configurable in the agent ( and compile time for nanofi ECUs ). If we went down the route of cap'n proto we can rely on the embedded RPC and use that as a separate C2 protocol. I have used Cap'n Proto, for example, and we could easily define those RPC calls to lessen the load, but I would likely change this ticket to exploring alternative protocols/wire formats and then making a decision as an addition to what we have now as opposed to a replacement. We will be forced to maintain it but we have no choice either as we have users who specifically don't want protobuf and we have released it so we are forced to maintain it and improve its testing. Keep in mind that whomever tackles this will need to balance maintaining what we already have ( not replacing ) so we then increase what we must test. We can rely on capn proto or protobuf's tests, but then we still need to perform our test in the venn diagram of testing. Would that effort be better spent improving tests for our work since we already need to do that and must maintain it? I'll leave that question up to the implementor – but it's certainly not an affront against doing this work, more a word of caution. I'm glad we decided against protobuf at the time, though – would have made that decision again. It was a calculated risk with many thoughts in mind – but would love to see something else like protobuf/flatbuffers/capn . Aside from this comment I would love the community to weigh in and make a choice. I've made my points known and have no intention of implementing this myself due to other technical debt I feel is more important so thoughts on what to do are up to the implementation seeker. was (Author: phrocker): [~bakaid] I like some of the concepts you present but prior CVEs were why we moved away from protobuf initially in community discussions. I think offloading the testing to protobuf maintainers is a reason to have Protobuf but I think both will need to exist. We also considered flatbuffers and capn proto – would love to see that added to this discussion. I think this ultimately would be either adding protobuf/flatbuffers/capn as a selectable line format. We have a very simple line format that isn't going to change ( this is because our line formats are designed to use the RESTFul API for the complicated stuff ). The line format is simply for sending a very tiny subset of information. We also want to use the same code for nanofi, where I think our consumers are more likely to appreciate not including protobuf. As a result we can add this as an extension ( in fact, I love the idea ) – especially since we can mitigate concerns of others by having it be configurable in the agent ( and compile time for nanofi ECUs ). If we went down the route of cap'n proto we can rely on the embedded RPC and use that as a separate C2 protocol. I have used Cap'n Proto, for example, and we could easily define those RPC calls to lessen the load, but I would likely change this ticket to exploring alternative protocols/wire formats and then making a decision as an addition to what we have now as opposed to a replacement. We will be forced to maintain it but we have no choice either as we have users who specifically don't want protobuf and we have released it so we are forced to maintain it and improve its testing. Keep in mind that whomever tackles this will need to balance maintaining what we already have ( not replacing ) so we then increase what we must test. We can rely on capn proto or protobuf's tests, but then we still need to perform our test in the venn diagram of testing. Would that effort be better spent improving tests for our work since we
[jira] [Commented] (MINIFICPP-941) Ideas for code quality, architectural or process improvements
[ https://issues.apache.org/jira/browse/MINIFICPP-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893992#comment-16893992 ] Mr TheSegfault commented on MINIFICPP-941: -- [~bakaid] Thanks for making the effort to compile this list. I love the initiative and I look forward to planning these. Thanks! > Ideas for code quality, architectural or process improvements > - > > Key: MINIFICPP-941 > URL: https://issues.apache.org/jira/browse/MINIFICPP-941 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Daniel Bakai >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-876) Changes to Socket code may have broken Raw Site to Site
[ https://issues.apache.org/jira/browse/MINIFICPP-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893839#comment-16893839 ] Mr TheSegfault edited comment on MINIFICPP-876 at 7/26/19 2:39 PM: --- [~aboda] I'm closing this as a duplicate since I think this is a duplicateof MINIFICPP-919 . Can re-open later if need be. was (Author: phrocker): [~aboda] I'm closing this as a duplicate since I think this is a duplicate issue. > Changes to Socket code may have broken Raw Site to Site > > > Key: MINIFICPP-876 > URL: https://issues.apache.org/jira/browse/MINIFICPP-876 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Priority: Blocker > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (MINIFICPP-876) Changes to Socket code may have broken Raw Site to Site
[ https://issues.apache.org/jira/browse/MINIFICPP-876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault resolved MINIFICPP-876. -- Resolution: Duplicate [~aboda] I'm closing this as a duplicate since I think this is a duplicate issue. > Changes to Socket code may have broken Raw Site to Site > > > Key: MINIFICPP-876 > URL: https://issues.apache.org/jira/browse/MINIFICPP-876 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Mr TheSegfault >Priority: Blocker > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (MINIFICPP-988) Evaluate Allowable values for Kafka options
Mr TheSegfault created MINIFICPP-988: Summary: Evaluate Allowable values for Kafka options Key: MINIFICPP-988 URL: https://issues.apache.org/jira/browse/MINIFICPP-988 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault Assignee: Nghia Le -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (MINIFICPP-985) Implement listvalidators
[ https://issues.apache.org/jira/browse/MINIFICPP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893799#comment-16893799 ] Mr TheSegfault edited comment on MINIFICPP-985 at 7/26/19 12:52 PM: Maybe this will help me resolve my confusion. It sounds like allowable values cannot be used in cases where we have a defined set of choices? Why is that? Also this seems like a good thing to discuss for 0.8.0 or 0.9.0 as we seem to have a small disagreement about risk ( mine is semantic risk ) – so I removed the version modifier. I saw the POC pr, appreciate the work was (Author: phrocker): Maybe this will help me resolve my confusion. It sounds like allowable values cannot be used in cases where we have a defined set of choices? Why is that? > Implement listvalidators > > > Key: MINIFICPP-985 > URL: https://issues.apache.org/jira/browse/MINIFICPP-985 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > As [~nghiaxlee] pointed out in a change we don't have the functionality to > create list validators (in which case all elements of the input lists should > be validated using an encapsulated validator), so we can't validate multiple > choice properties for eg. > The change itself is quite easy, although we should keep in mind C2 > integration. Because of this [~phrocker] and [~kdoran] I would like to ask > for your feedback on adding this into agent manifest. Thanks in advance! > I scoped this for 0.7.0 as the MiNiFi impact would be small, however I can > accept rescheduling it to 0.8.0 in case C2 integration requires more effort > and testing. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (MINIFICPP-985) Implement listvalidators
[ https://issues.apache.org/jira/browse/MINIFICPP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault updated MINIFICPP-985: - Fix Version/s: (was: 0.7.0) > Implement listvalidators > > > Key: MINIFICPP-985 > URL: https://issues.apache.org/jira/browse/MINIFICPP-985 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > As [~nghiaxlee] pointed out in a change we don't have the functionality to > create list validators (in which case all elements of the input lists should > be validated using an encapsulated validator), so we can't validate multiple > choice properties for eg. > The change itself is quite easy, although we should keep in mind C2 > integration. Because of this [~phrocker] and [~kdoran] I would like to ask > for your feedback on adding this into agent manifest. Thanks in advance! > I scoped this for 0.7.0 as the MiNiFi impact would be small, however I can > accept rescheduling it to 0.8.0 in case C2 integration requires more effort > and testing. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MINIFICPP-985) Implement listvalidators
[ https://issues.apache.org/jira/browse/MINIFICPP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893797#comment-16893797 ] Mr TheSegfault commented on MINIFICPP-985: -- "What you mentioned (4 and 3.3) is something I definitely DON'T plan to support this way, only multiple values of one type, where the validation doesn't become complex: just tokenising the value and validating the tokens using one encapsulated validator. I think it's very low risk and low impact (~15 lines of new code + tests)," What you describe here is a list of allowable values. I'm very confused by everything else. > Implement listvalidators > > > Key: MINIFICPP-985 > URL: https://issues.apache.org/jira/browse/MINIFICPP-985 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 0.6.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Fix For: 0.7.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As [~nghiaxlee] pointed out in a change we don't have the functionality to > create list validators (in which case all elements of the input lists should > be validated using an encapsulated validator), so we can't validate multiple > choice properties for eg. > The change itself is quite easy, although we should keep in mind C2 > integration. Because of this [~phrocker] and [~kdoran] I would like to ask > for your feedback on adding this into agent manifest. Thanks in advance! > I scoped this for 0.7.0 as the MiNiFi impact would be small, however I can > accept rescheduling it to 0.8.0 in case C2 integration requires more effort > and testing. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
