Arne,
  I took a break from the issue and came back and tried installing a
different version of openssl on top of the distro. When doing so it linked
properly and I'm able to send data through a secure Socket. Now that I have
a solution, I will move this discussion to the ticket.

  As a result of my testing, I will make updates to the bootstrap script
and build instructions to instruct users to install libssl1.0 on Debian
Stretch ( and perhaps Raspbian ). Any comments on the ticket will be
appreciated: https://issues.apache.org/jira/browse/MINIFICPP-400 . I will
have a fix once I finish testing across a few platforms.

  Thanks,
  Marc

On Tue, Feb 13, 2018 at 9:55 AM, Marc P. <marc.par...@gmail.com> wrote:

> Arne,
>
> Thanks for the info. I'm running the same environment with the same
> warnings produced -- and segfault -- aso I'll get back to you once I've
> identified the issue.
>
> TL;DR: Created a ticket ( https://issues.apache.org/
> jira/browse/MINIFICPP-400. ) to help insulate users from this more.
>
> More explanation:
>
>
>   In regards to the different versions, there were a number of tickets on
> Debian's bug report logs regarding the curl ABI. An example [1] was solved
> by changing linked versions of libraries.
>
>   The gist is that the SSL_CTX struct changes. I've validated with gdb
> that the struct is being passed in on U16 ( and works ) AND Debian stretch
> ( does not work ). The structures are slightly different. I'm going to dive
> deeper into this. I've created [2] https://issues.apache.org/
> jira/browse/MINIFICPP-400.
>
>    [2] has more recent conversation as the issue still exists. This will
> require an soname change hence why they've likely been discussing this for
> two years. I think our job will be to insulate users, so [3] will be our
> efforts to do so. Thanks for identifying this. I'll continue to investigate
> this to address it seamlessly through either our bootstrap script (
> bootstrap.sh in the root ) or within CMAKE. Obviously we don't want to
> alienate Debian users.
>
>
> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844018
> [2] https://issues.apache.org/jira/browse/MINIFICPP-400.
> [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858398
>
> On Tue, Feb 13, 2018 at 8:28 AM, Arne Oslebo <arne.osl...@uninett.no>
> wrote:
>
>> Marc,
>>
>> I'm running it on Debian Stretch. libssl version might be the problem. I
>> see that both libssl1.0.2 and libssl1.1 are installed. I took another look
>> at the build log and found this:
>> /usr/bin/ld: warning: libssl.so.1.0.2, needed by
>> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so,
>> may conflict with libssl.so.1.1
>> /usr/bin/ld: warning: libcrypto.so.1.0.2, needed by
>> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so,
>> may conflict with libcrypto.so.1.1
>>
>> I'll see if can remove one of the versions to avoid possible conflict.
>>
>> The full backtrace log is attached and thanks for looking into this,
>>
>> Arne
>>
>>
>>
>> On 13/02/2018 12:55, Marc wrote:
>>
>> Arne,
>>    Sorry for your troubles! Can you give me some insight into your distro?
>>
>>    I've been unable to replicate the issue on OSX, u16, and arch...but
>> all are running a different version of OpenSSL. What version of OpenSSL are
>> you running and on what distro? I'll be happy to try it to track this down.
>>
>>    Additionally, do you have the log file you can pass on? The reason I
>> ask is that the line specified in gdb is a log statement with usage of
>> constructs that are inherent to libstdc++, so they won't cause a memory
>> error. The log file should help identify this and give me insight into what
>> occurred just before the error.
>>
>>    Thanks,
>>    Marc
>>
>> On Tue, Feb 13, 2018 at 6:17 AM, Arne Oslebo <arne.osl...@uninett.no>
>> wrote:
>>
>>> Hello Mark,
>>>
>>> thanks for the update. I tried running master from github but
>>> unfortunately I now get a segmentation fault:
>>>
>>> Thread 1 "minifi" received signal SIGSEGV, Segmentation fault.
>>> 0x00007ffff777420a in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>>> (gdb) bt full
>>> #0  0x00007ffff777420a in ?? () from /usr/lib/x86_64-linux-gnu/libs
>>> sl.so.1.1
>>> No symbol table info available.
>>> #1  0x00007ffff7799681 in ?? () from /usr/lib/x86_64-linux-gnu/libs
>>> sl.so.1.1
>>> No symbol table info available.
>>> #2  0x00007ffff777f2f6 in SSL_CTX_use_certificate () from
>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>>> No symbol table info available.
>>> #3  0x00007ffff777f6c0 in SSL_CTX_use_certificate_file () from
>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>>> No symbol table info available.
>>> #4  0x0000555555ef91cb in org::apache::nifi::minifi::con
>>> trollers::SSLContextService::configure_ssl_context
>>> (this=0x5555569948b0, ctx=0x555556a28430) at /usr/local/src/nifi-minifi-cpp
>>> /extensions/http-curl/../../libminifi/include/controllers/SS
>>> LContextService.h:165
>>>         retp = 1
>>> #5  0x0000555555ef9be4 in org::apache::nifi::minifi::uti
>>> ls::HTTPClient::configure_ssl_context (curl=0x555556a149e0,
>>> ctx=0x555556a28430, param=0x5555569948b0) at /usr/local/src/nifi-minifi-cpp
>>> /extensions/http-curl/client/HTTPClient.h:177
>>>         ssl_context_service = 0x5555569948b0
>>>
>>> Any idea what the problem might be?
>>>
>>> My full config.yml:
>>>
>>> Flow Controller:
>>>   name: MiNiFi Flow
>>> Controller Services:
>>> - name: SSLServiceName
>>>   id: 2438e3c8-015a-1000-79ca-83af40ec1974
>>>   class: SSLContextService
>>>   Properties:
>>>       Client Certificate: /opt/minifi/conf/client.pem
>>>       Private Key: /opt/minifi/conf/client.key
>>>       Passphrase: secret
>>>       CA Certificate: /opt/minifi/conf/nifi-cert.pem
>>> Processors:
>>> - id: cecb1868-9e5a-3e6c-0000-000000000000
>>>   name: TailFile
>>>   class: org.apache.nifi.processors.standard.TailFile
>>>   max concurrent tasks: 1
>>>   scheduling strategy: TIMER_DRIVEN
>>>   scheduling period: 0 sec
>>>   penalization period: 30 sec
>>>   yield period: 1 sec
>>>   run duration nanos: 0
>>>   auto-terminated relationships list: []
>>>   Properties:
>>>     File Location: Local
>>>     File to Tail: /tmp/test.log
>>>     Initial Start Position: Beginning of File
>>>     Rolling Filename Pattern:
>>>     tail-base-directory:
>>>     tail-mode: Single file
>>>     tailfile-lookup-frequency: 10 minutes
>>>     tailfile-maximum-age: 24 hours
>>>     tailfile-recursive-lookup: 'false'
>>> Connections:
>>> - id: 76ad4bc4-6557-3e23-0000-000000000000
>>>   name: TailFile/success/56ae5abc-0161-1000-aa9e-c340d726e043
>>>   source id: cecb1868-9e5a-3e6c-0000-000000000000
>>>   source relationship names:
>>>   - success
>>>   destination id: 56ae5abc-0161-1000-aa9e-c340d726e043
>>>   max work queue size: 10000
>>>   max work queue data size: 1 GB
>>>   flowfile expiration: 0 sec
>>>   queue prioritizer class: ''
>>> Remote Processing Groups:
>>> - id: 3a25e1a3-c1b2-3e78-0000-000000000000
>>>   name: ''
>>>   url: https://w.x.y.z:8443/nifi
>>>   comment: ''
>>>   timeout: 30 sec
>>>   yield period: 10 sec
>>>   transport protocol: RAW
>>>   proxy host: ''
>>>   proxy port: ''
>>>   proxy user: ''
>>>   proxy password: ''
>>>   local network interface: ''
>>>   Input Ports:
>>>   - id: 56ae5abc-0161-1000-aa9e-c340d726e043
>>>     name: Minifi
>>>     comment: ''
>>>     max concurrent tasks: 1
>>>     use compression: false
>>>     Properties:
>>>       Port: 10443
>>>       SSL Context Service: SSLServiceName
>>>       Host Name: w.x.y.z
>>>   Output Ports: []
>>> Provenance Reporting:
>>>
>>>
>>>
>>> On 09/02/2018 20:18, Marc wrote:
>>>
>>> Arne,
>>>   I submitted a PR https://github.com/apache/nifi-minifi-cpp/pull/263
>>> to address these issues.
>>>
>>> On Fri, Feb 9, 2018 at 8:38 AM, Marc <phroc...@apache.org> wrote:
>>>
>>>> Arne,
>>>>    Evidently the HTTPClient relies on an SSL Context Service. Try the
>>>> following configuration in the config.yml file, where you define the
>>>> context service and reference it from the RPG. Let me know if that works
>>>> for you!
>>>>
>>>>   Additionally, I think you pointed out an inconsistency where we can
>>>> improve the configuration and documentation. I've created
>>>> https://issues.apache.org/jira/browse/MINIFICPP-396 and will take care
>>>> of that ASAP. Thanks
>>>>   very much for identifying this!
>>>>
>>>> Remote Processing Groups:
>>>>     - name: NiFi Flow
>>>>       id: 2438e3c8-015a-1000-79ca-83af40ec1998
>>>>       url: https://127.0.0.1:8383/nifi
>>>>       timeout: 30 secs
>>>>       yield period: 5 sec
>>>>       Input Ports:
>>>>           - id: 2438e3c8-015a-1000-79ca-83af40ec1999
>>>>             name: fromnifi
>>>>             max concurrent tasks: 1
>>>>             Properties:
>>>>                 Port: 10443
>>>>                 SSL Context Service: SSLMe
>>>>                 Host Name: 127.0.0.1
>>>>       Output Ports:
>>>>           - id: ac82e521-015c-1000-2b21-41279516e19a
>>>>             name: tominifi
>>>>             max concurrent tasks: 2
>>>>             Properties:
>>>>                 Port: 10443
>>>>                 SSL Context Service: SSLMe
>>>>                 Host Name: 127.0.0.1
>>>>
>>>> Controller Services:
>>>>     - name: SSLMe
>>>>       id: 2438e3c8-015a-1000-79ca-83af40ec1974
>>>>       class: SSLContextService
>>>>       Properties:
>>>>           Client Certificate: /opt/minifi/conf/client.pem
>>>>           Private Key: /opt/minifi/conf/client.key
>>>>           Passphrase: /opt/minifi/conf/password
>>>>           CA Certificate certificate: /opt/minifi/conf/nifi-cert.pem
>>>>
>>>> On Fri, Feb 9, 2018 at 5:54 AM, Arne Oslebo <arne.osl...@uninett.no>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I'm trying to set up secure communication between minifi-cpp 0.4.0 and
>>>>> nifi, but unfortunately it fails with the following errors:
>>>>>
>>>>> [org::apache::nifi::minifi::utils::HTTPClient] [error]
>>>>> curl_easy_perform() failed SSL connect error
>>>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error]
>>>>> ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed
>>>>>
>>>>> I looked quickly at the code and it seems the problem is that
>>>>> HTTPClient
>>>>> never calls configure_secure_connection and therefor never presents a
>>>>> client certificate to nifi.
>>>>>
>>>>> The config.yml file defines a TailFail that send data directly to a
>>>>> remote process group.
>>>>>
>>>>> My  minifi.properties file:
>>>>> nifi.version=0.1.0
>>>>> nifi.flow.configuration.file=/opt/minifi/conf/config.yml
>>>>> nifi.administrative.yield.duration=30 sec
>>>>> nifi.bored.yield.duration=10 millis
>>>>> nifi.provenance.repository.directory.default=/opt/minifi/pro
>>>>> venance_repository
>>>>> nifi.provenance.repository.max.storage.time=1 MIN
>>>>> nifi.provenance.repository.max.storage.size=1 MB
>>>>> nifi.remote.input.secure=true
>>>>> nifi.https.need.ClientAuth=true
>>>>> nifi.https.client.certificate=/opt/minifi/conf/client.pem
>>>>> nifi.https.client.private.key=/opt/minifi/conf/client.key
>>>>> nifi.https.client.pass.phrase=/opt/minifi/conf/password
>>>>> nifi.https.client.ca.certificate=/opt/minifi/conf/nifi-cert.pem
>>>>> controller.socket.host=localhost
>>>>> controller.socket.port=9998
>>>>>
>>>>> Certificates and key are correct and have been verified using curl from
>>>>> the command line. Are there any other things I need to do to get minifi
>>>>> to set up a secure connection? As far as I understand the "Security
>>>>> Properties:" in config.yml is only used by the java version of minifi?
>>>>>
>>>>> Thanks,
>>>>> Arne
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>

Reply via email to