Re: [NIFI-CPP] Issues with TIMER_DRIVEN and CRON_DRIVEN

2018-04-23 Thread Marc P. 
Iyán,

   Yes, both strategies seem to work.

   Here is my config file
   https://gist.github.com/phrocker/a209b5163044a5bb3c2e6ac4662ce143

   Below is the output form my pi. Due to the time it takes to create
a PNG and the scheduling strategy it usually ends up being about a
second between pictures. When I reduce my configuration to what you
have ( for the timing config options ) I maintain the same pattern of
taking a picture every 900 ms. Logs may help further diagnose the
issue.

[2018-04-23 11:11:17.484]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:18.690]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:19.850]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:20.998]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:22.145]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:23.284]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:24.427]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:25.573]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:26.718]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:27.860]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:28.998]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:30.143]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:31.286]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:32.429]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:33.573]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:34.717]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:35.875]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:36.999]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:38.129]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:39.255]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:40.384]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:41.513]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame
[2018-04-23 11:11:42.639]
[org::apache::nifi::minifi::processors::GetUSBCamera] [debug] Got
frame

On Mon, Apr 23, 2018 at 6:27 AM, Mendez Veiga, Iyan
 wrote:
> Ok, thanks Marc!
>
>
>
> And do they work for you both TIMER_DRIVEN and EVENT_DRIVEN in the
> raspberry? Can you send me a config.yml you have using the GetUSBCamera?
>
>
>
> Best regards,
>
> Iyán
>
>
>
>
>
> De: Marc [mailto:phroc...@apache.org]
> Enviado el: lunes, 23 de abril de 2018 0:22
> Para: users@nifi.apache.org
> Asunto: Re: [NIFI-CPP] Issues with TIMER_DRIVEN and CRON_DRIVEN
>
>
>
> **This Message originated from a Non-ArcelorMittal source**
>
> I wanted to follow up and say that timer driven and event driven are the
> only scheduling strategies available in minifi c++. Cron is not a supported
> strategy yet. Sorry for not mentioning this earlier
>
>
>
> On Sun, Apr 22, 2018, 4:05 PM Iyán Méndez Veiga  wrote:
>
> Hi,
>
> I haven't been able to configure MiNiFi to take pictures every 30 seconds
> using GetUSBCamera processor. I have the following in the config.yml
>
>   - name: take_picture
> class: GetUSBCamera
> max concurrent tasks: 1
> scheduling strategy: TIMER_DRIVEN
> scheduling period: 30 sec
> auto-terminated relationships list:
>   - failure
> Properties:
>   FPS: 1
>   Format: PNG
>   USB Vendor ID: 0x046d
>   USB Product ID: 0x082d
>
> The processor ignores the scheduling period and it takes pictures
> continuously. I tried to use the CRON_DRIVEN but any processor seems to
> work.
> Perhaps is that I didn't understand how to use it reading this:
> http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/
> crontrigger.html
>
>   - name: take_picture
> class: GetUSBCamera
> max concurrent tasks: 1
> scheduling strategy: CRON_DRIVEN
> scheduling period: 0/30 * * * * *
> auto-terminated relationships list:
>   - failure
> Properties:
>   FPS: 1
>   Format: PNG
>   USB Vendor ID: 0x046d
>   USB Product ID: 0x082d
>
> Hope anyone can help me. By the way, is there any place to check current
> open/
> known issues and bugs?
>
> Thanks,
> Iyán
>
> --
> Iyán Méndez Veiga | Physicist
> GPG: 0x422E3694311E5AC1
> Web: https://iyanmv.com
>
> ♫♪.ılılıll|̲̅̅●̲̅̅|̲̅̅=̲̅̅|̲̅̅●̲̅̅|llılılı.♫♪

Re: [NIFI-CPP] Issues with TIMER_DRIVEN and CRON_DRIVEN

2018-04-22 Thread Marc P. 
Iyán,
   Is there an error or session rollback in the log file?

   You can view issues here:
https://issues.apache.org/jira/projects/MINIFICPP/issues

Thanks,
Marc

On Sun, Apr 22, 2018 at 4:05 PM, Iyán Méndez Veiga  wrote:
> Hi,
>
> I haven't been able to configure MiNiFi to take pictures every 30 seconds
> using GetUSBCamera processor. I have the following in the config.yml
>
>   - name: take_picture
> class: GetUSBCamera
> max concurrent tasks: 1
> scheduling strategy: TIMER_DRIVEN
> scheduling period: 30 sec
> auto-terminated relationships list:
>   - failure
> Properties:
>   FPS: 1
>   Format: PNG
>   USB Vendor ID: 0x046d
>   USB Product ID: 0x082d
>
> The processor ignores the scheduling period and it takes pictures
> continuously. I tried to use the CRON_DRIVEN but any processor seems to work.
> Perhaps is that I didn't understand how to use it reading this:
> http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/
> crontrigger.html
>
>   - name: take_picture
> class: GetUSBCamera
> max concurrent tasks: 1
> scheduling strategy: CRON_DRIVEN
> scheduling period: 0/30 * * * * *
> auto-terminated relationships list:
>   - failure
> Properties:
>   FPS: 1
>   Format: PNG
>   USB Vendor ID: 0x046d
>   USB Product ID: 0x082d
>
> Hope anyone can help me. By the way, is there any place to check current open/
> known issues and bugs?
>
> Thanks,
> Iyán
>
> --
> Iyán Méndez Veiga | Physicist
> GPG: 0x422E3694311E5AC1
> Web: https://iyanmv.com
>
> ♫♪.ılılıll|̲̅̅●̲̅̅|̲̅̅=̲̅̅|̲̅̅●̲̅̅|llılılı.♫♪

Re: USB Camera support with MiNiFi 0.4.0 in Rasp3?

2018-04-09 Thread Marc P. 
Iyan,
  That one along with a few others required a bit more manual effort
at the time in bootstrap and thus need to be enabled manually.  Enable
that one manually with cmake -DENABLE_USB_CAMERA=1 ..
  In the next version it'll be added to the bootstrap with 0.5.0. I
was running GetUSBCamera on a Pi last night but I don't specifically
recall if there was much more to take into account. I have a branch
where USB Camera can be enabled on PI. I'll test that on a Pi and
submit a PR if it works.
  Thanks,
  Marc

On Mon, Apr 9, 2018 at 3:46 AM, Mendez Veiga, Iyan
 wrote:
> Hi,
>
> I am trying to compile MiNiFi C++ 0.4.0 in a raspberry pi 3 running Raspbian.
> I have installed all packages in the requirements (and some optional ones, 
> too):
> https://github.com/apache/nifi-minifi-cpp#system-requirements
>
> However, when I execute the bootstrap script I cannot enable the USB Camera 
> support (since I want to use GetUSBCamera processor) and the only information 
> I get is that "Extension cannot be installed due to version of cmake or other 
> software".
>
> I couldn't find any information on the Internet. Any one is facing a similar 
> issue?
>
> Regards,
> Iyán

Re: minifi secure connection

2018-02-13 Thread Marc P. 
Arne,

Thanks for the info. I'm running the same environment with the same
warnings produced -- and segfault -- aso I'll get back to you once I've
identified the issue.

TL;DR: Created a ticket (
https://issues.apache.org/jira/browse/MINIFICPP-400. ) to help insulate
users from this more.

More explanation:


  In regards to the different versions, there were a number of tickets on
Debian's bug report logs regarding the curl ABI. An example [1] was solved
by changing linked versions of libraries.

  The gist is that the SSL_CTX struct changes. I've validated with gdb that
the struct is being passed in on U16 ( and works ) AND Debian stretch (
does not work ). The structures are slightly different. I'm going to dive
deeper into this. I've created [2]
https://issues.apache.org/jira/browse/MINIFICPP-400.

   [2] has more recent conversation as the issue still exists. This will
require an soname change hence why they've likely been discussing this for
two years. I think our job will be to insulate users, so [3] will be our
efforts to do so. Thanks for identifying this. I'll continue to investigate
this to address it seamlessly through either our bootstrap script (
bootstrap.sh in the root ) or within CMAKE. Obviously we don't want to
alienate Debian users.


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844018
[2] https://issues.apache.org/jira/browse/MINIFICPP-400.
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858398

On Tue, Feb 13, 2018 at 8:28 AM, Arne Oslebo  wrote:

> Marc,
>
> I'm running it on Debian Stretch. libssl version might be the problem. I
> see that both libssl1.0.2 and libssl1.1 are installed. I took another look
> at the build log and found this:
> /usr/bin/ld: warning: libssl.so.1.0.2, needed by
> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may
> conflict with libssl.so.1.1
> /usr/bin/ld: warning: libcrypto.so.1.0.2, needed by
> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may
> conflict with libcrypto.so.1.1
>
> I'll see if can remove one of the versions to avoid possible conflict.
>
> The full backtrace log is attached and thanks for looking into this,
>
> Arne
>
>
>
> On 13/02/2018 12:55, Marc wrote:
>
> Arne,
>Sorry for your troubles! Can you give me some insight into your distro?
>
>I've been unable to replicate the issue on OSX, u16, and arch...but all
> are running a different version of OpenSSL. What version of OpenSSL are you
> running and on what distro? I'll be happy to try it to track this down.
>
>Additionally, do you have the log file you can pass on? The reason I
> ask is that the line specified in gdb is a log statement with usage of
> constructs that are inherent to libstdc++, so they won't cause a memory
> error. The log file should help identify this and give me insight into what
> occurred just before the error.
>
>Thanks,
>Marc
>
> On Tue, Feb 13, 2018 at 6:17 AM, Arne Oslebo 
> wrote:
>
>> Hello Mark,
>>
>> thanks for the update. I tried running master from github but
>> unfortunately I now get a segmentation fault:
>>
>> Thread 1 "minifi" received signal SIGSEGV, Segmentation fault.
>> 0x7777420a in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>> (gdb) bt full
>> #0  0x7777420a in ?? () from /usr/lib/x86_64-linux-gnu/libs
>> sl.so.1.1
>> No symbol table info available.
>> #1  0x77799681 in ?? () from /usr/lib/x86_64-linux-gnu/libs
>> sl.so.1.1
>> No symbol table info available.
>> #2  0x7777f2f6 in SSL_CTX_use_certificate () from
>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>> No symbol table info available.
>> #3  0x7777f6c0 in SSL_CTX_use_certificate_file () from
>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>> No symbol table info available.
>> #4  0x55ef91cb in org::apache::nifi::minifi::con
>> trollers::SSLContextService::configure_ssl_context (this=0x569948b0,
>> ctx=0x56a28430) at /usr/local/src/nifi-minifi-cpp
>> /extensions/http-curl/../../libminifi/include/controllers/SS
>> LContextService.h:165
>> retp = 1
>> #5  0x55ef9be4 in org::apache::nifi::minifi::uti
>> ls::HTTPClient::configure_ssl_context (curl=0x56a149e0,
>> ctx=0x56a28430, param=0x569948b0) at /usr/local/src/nifi-minifi-cpp
>> /extensions/http-curl/client/HTTPClient.h:177
>> ssl_context_service = 0x569948b0
>>
>> Any idea what the problem might be?
>>
>> My full config.yml:
>>
>> Flow Controller:
>>   name: MiNiFi Flow
>> Controller Services:
>> - name: SSLServiceName
>>   id: 2438e3c8-015a-1000-79ca-83af40ec1974
>>   class: SSLContextService
>>   Properties:
>>   Client Certificate: /opt/minifi/conf/client.pem
>>   Private Key: /opt/minifi/conf/client.key
>>   Passphrase: secret
>>   CA Certificate: /opt/minifi/conf/nifi-cert.pem
>> Processors:
>> - id: cecb1868-9e5a-3e6c--
>>   name: TailFile
>>   class:

Re: minifi-cpp s2s no protocol

2017-09-20 Thread Marc P. 
Andy,
   Can you try adding the RPG host and port in the properties for your
ports?


Remote Processing Groups:
- name: NiFi Flow
  id: 2438e3c8-015a-1000-79ca-83af40ec1998
  url: http://127.0.0.1:8080/nifi
  timeout: 30 secs
  yield period: 5 sec
  Input Ports:
  - id: 2438e3c8-015a-1000-79ca-83af40ec1999
name: fromnifi
max concurrent tasks: 1
Properties:
Port: 10443
Host Name: 127.0.0.1
  Output Ports:
  - id: ac82e521-015c-1000-2b21-41279516e19a
name: tominifi
max concurrent tasks: 1
Properties:
Port: 10443
Host Name: 127.0.0.1


On Wed, Sep 20, 2017 at 11:11 AM, Andy Christianson <
achristian...@hortonworks.com> wrote:

> ​This is on the phrocker/MINIFI-339 branch with some additional testing
> code I'm working on.
>
>
> If it helps, in the logs, the s2s connection fails several times as my
> testing nifi instance boots, then once nifi is ready, it gets this config
> JSON back & spits out the no protocol error.
>
>
> I'm running nifi master branch.
>
>
> -Andy I.C.
> --
> *From:* Marc 
> *Sent:* Wednesday, September 20, 2017 11:06 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: minifi-cpp s2s no protocol
>
> Andy,
>   What branch are you using?
>
>   I believe this is was one issue resolved under the guise of
> https://issues.apache.org/jira/browse/MINIFICPP-4 ?
>
> On Wed, Sep 20, 2017 at 11:03 AM, Andy Christianson <
> achristian...@hortonworks.com> wrote:
>
>> Hi All,
>>
>>
>> I have a minifi-cpp flow using HTTP s2s to send data to a nifi instance.
>> The minifi instance seems to be making an initial connection to nifi, but
>> it fails to send data, and I see a "no protocol" in the logs. Here's the
>> log output & flow yml:
>>
>>
>> [2017-09-20 14:37:35.634] [org::apache::nifi::minifi::utils::HTTPClient]
>> [info] Submitting to http://nifi:8080/nifi-api/controller
>> [2017-09-20 14:37:35.648] 
>> [org::apache::nifi::minifi::RemoteProcessorGroupPort]
>> [debug] controller config {"controller":{"id":"96120c9a-
>> b94b-434e-a439-42a4927b5d5e","runningCount":3,"stoppedCount"
>> :0,"invalidCount":0,"disabledCount":0,"inputPortCount":1,"ou
>> tputPortCount":0,"siteToSiteSecure":false,"instanceId":"
>> 9fb88d05-015e-1000-f6e9-688ce4c58047","inputPorts":[{"
>> id":"df466fb6-d4f4-4fb1-ac59-5ceb90fcf527","name":"from-
>> minifi","state":"RUNNING"}],"outputPorts":[]}}
>> [2017-09-20 14:37:35.648] 
>> [org::apache::nifi::minifi::RemoteProcessorGroupPort]
>> [info] process group remote site2site port -1, is secure 0
>> [2017-09-20 14:37:35.648] 
>> [org::apache::nifi::minifi::RemoteProcessorGroupPort]
>> [info] no protocol
>> ​
>> Connections:
>> - destination id: dcc4ebf7-e4ae-4e9e-bac5-e0fd11e08033
>>   name: 86f953f0-6200-448c-b078-3a7c9d082d84
>>   source id: ea5d3fff-7bc8-4fc7-a01a-7d10013fa883
>>   source relationship name: success
>> - destination id: df466fb6-d4f4-4fb1-ac59-5ceb90fcf527
>>   name: 18a94963-bd92-4043-82c9-1e9e49f2448e
>>   source id: dcc4ebf7-e4ae-4e9e-bac5-e0fd11e08033
>>   source relationship name: success
>> Controller Services: []
>> Flow Controller:
>>   name: MiNiFi Flow
>> Processors:
>> - Properties:
>> Input Directory: /tmp/input
>>   auto-terminated relationships list: []
>>   class: org.apache.nifi.processors.standard.GetFile
>>   id: ea5d3fff-7bc8-4fc7-a01a-7d10013fa883
>>   name: ea5d3fff-7bc8-4fc7-a01a-7d10013fa883
>>   penalization period: 30 sec
>>   run duration nanos: 0
>>   scheduling period: 0 sec
>>   scheduling strategy: EVENT_DRIVEN
>>   yield period: 1 sec
>> - Properties: {}
>>   auto-terminated relationships list: []
>>   class: org.apache.nifi.processors.standard.LogAttribute
>>   id: dcc4ebf7-e4ae-4e9e-bac5-e0fd11e08033
>>   name: dcc4ebf7-e4ae-4e9e-bac5-e0fd11e08033
>>   penalization period: 30 sec
>>   run duration nanos: 0
>>   scheduling period: 1 sec
>>   scheduling strategy: EVENT_DRIVEN
>>   yield period: 1 sec
>> Remote Processing Groups:
>> - Input Ports:
>>   - Properties: {}
>> id: df466fb6-d4f4-4fb1-ac59-5ceb90fcf527
>> max concurrent tasks: 1
>> name: from-minifi
>>   id: 3a94707d-7717-46fa-b641-57d1104bd927
>>   name: 3a94707d-7717-46fa-b641-57d1104bd927
>>   timeout: 30 sec
>>   url: http://nifi:8080/nifi
>>   yield period: 10 sec
>>
>> Ideas?
>>
>>
>> -Andy I.C.
>>
>
>