Arne,

Thanks for the info. I'm running the same environment with the same
warnings produced -- and segfault -- aso I'll get back to you once I've
identified the issue.

TL;DR: Created a ticket (
https://issues.apache.org/jira/browse/MINIFICPP-400. ) to help insulate
users from this more.

More explanation:


  In regards to the different versions, there were a number of tickets on
Debian's bug report logs regarding the curl ABI. An example [1] was solved
by changing linked versions of libraries.

  The gist is that the SSL_CTX struct changes. I've validated with gdb that
the struct is being passed in on U16 ( and works ) AND Debian stretch (
does not work ). The structures are slightly different. I'm going to dive
deeper into this. I've created [2]
https://issues.apache.org/jira/browse/MINIFICPP-400.

   [2] has more recent conversation as the issue still exists. This will
require an soname change hence why they've likely been discussing this for
two years. I think our job will be to insulate users, so [3] will be our
efforts to do so. Thanks for identifying this. I'll continue to investigate
this to address it seamlessly through either our bootstrap script (
bootstrap.sh in the root ) or within CMAKE. Obviously we don't want to
alienate Debian users.


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844018
[2] https://issues.apache.org/jira/browse/MINIFICPP-400.
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858398

On Tue, Feb 13, 2018 at 8:28 AM, Arne Oslebo <arne.osl...@uninett.no> wrote:

> Marc,
>
> I'm running it on Debian Stretch. libssl version might be the problem. I
> see that both libssl1.0.2 and libssl1.1 are installed. I took another look
> at the build log and found this:
> /usr/bin/ld: warning: libssl.so.1.0.2, needed by
> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may
> conflict with libssl.so.1.1
> /usr/bin/ld: warning: libcrypto.so.1.0.2, needed by
> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may
> conflict with libcrypto.so.1.1
>
> I'll see if can remove one of the versions to avoid possible conflict.
>
> The full backtrace log is attached and thanks for looking into this,
>
> Arne
>
>
>
> On 13/02/2018 12:55, Marc wrote:
>
> Arne,
>    Sorry for your troubles! Can you give me some insight into your distro?
>
>    I've been unable to replicate the issue on OSX, u16, and arch...but all
> are running a different version of OpenSSL. What version of OpenSSL are you
> running and on what distro? I'll be happy to try it to track this down.
>
>    Additionally, do you have the log file you can pass on? The reason I
> ask is that the line specified in gdb is a log statement with usage of
> constructs that are inherent to libstdc++, so they won't cause a memory
> error. The log file should help identify this and give me insight into what
> occurred just before the error.
>
>    Thanks,
>    Marc
>
> On Tue, Feb 13, 2018 at 6:17 AM, Arne Oslebo <arne.osl...@uninett.no>
> wrote:
>
>> Hello Mark,
>>
>> thanks for the update. I tried running master from github but
>> unfortunately I now get a segmentation fault:
>>
>> Thread 1 "minifi" received signal SIGSEGV, Segmentation fault.
>> 0x00007ffff777420a in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>> (gdb) bt full
>> #0  0x00007ffff777420a in ?? () from /usr/lib/x86_64-linux-gnu/libs
>> sl.so.1.1
>> No symbol table info available.
>> #1  0x00007ffff7799681 in ?? () from /usr/lib/x86_64-linux-gnu/libs
>> sl.so.1.1
>> No symbol table info available.
>> #2  0x00007ffff777f2f6 in SSL_CTX_use_certificate () from
>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>> No symbol table info available.
>> #3  0x00007ffff777f6c0 in SSL_CTX_use_certificate_file () from
>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1
>> No symbol table info available.
>> #4  0x0000555555ef91cb in org::apache::nifi::minifi::con
>> trollers::SSLContextService::configure_ssl_context (this=0x5555569948b0,
>> ctx=0x555556a28430) at /usr/local/src/nifi-minifi-cpp
>> /extensions/http-curl/../../libminifi/include/controllers/SS
>> LContextService.h:165
>>         retp = 1
>> #5  0x0000555555ef9be4 in org::apache::nifi::minifi::uti
>> ls::HTTPClient::configure_ssl_context (curl=0x555556a149e0,
>> ctx=0x555556a28430, param=0x5555569948b0) at /usr/local/src/nifi-minifi-cpp
>> /extensions/http-curl/client/HTTPClient.h:177
>>         ssl_context_service = 0x5555569948b0
>>
>> Any idea what the problem might be?
>>
>> My full config.yml:
>>
>> Flow Controller:
>>   name: MiNiFi Flow
>> Controller Services:
>> - name: SSLServiceName
>>   id: 2438e3c8-015a-1000-79ca-83af40ec1974
>>   class: SSLContextService
>>   Properties:
>>       Client Certificate: /opt/minifi/conf/client.pem
>>       Private Key: /opt/minifi/conf/client.key
>>       Passphrase: secret
>>       CA Certificate: /opt/minifi/conf/nifi-cert.pem
>> Processors:
>> - id: cecb1868-9e5a-3e6c-0000-000000000000
>>   name: TailFile
>>   class: org.apache.nifi.processors.standard.TailFile
>>   max concurrent tasks: 1
>>   scheduling strategy: TIMER_DRIVEN
>>   scheduling period: 0 sec
>>   penalization period: 30 sec
>>   yield period: 1 sec
>>   run duration nanos: 0
>>   auto-terminated relationships list: []
>>   Properties:
>>     File Location: Local
>>     File to Tail: /tmp/test.log
>>     Initial Start Position: Beginning of File
>>     Rolling Filename Pattern:
>>     tail-base-directory:
>>     tail-mode: Single file
>>     tailfile-lookup-frequency: 10 minutes
>>     tailfile-maximum-age: 24 hours
>>     tailfile-recursive-lookup: 'false'
>> Connections:
>> - id: 76ad4bc4-6557-3e23-0000-000000000000
>>   name: TailFile/success/56ae5abc-0161-1000-aa9e-c340d726e043
>>   source id: cecb1868-9e5a-3e6c-0000-000000000000
>>   source relationship names:
>>   - success
>>   destination id: 56ae5abc-0161-1000-aa9e-c340d726e043
>>   max work queue size: 10000
>>   max work queue data size: 1 GB
>>   flowfile expiration: 0 sec
>>   queue prioritizer class: ''
>> Remote Processing Groups:
>> - id: 3a25e1a3-c1b2-3e78-0000-000000000000
>>   name: ''
>>   url: https://w.x.y.z:8443/nifi
>>   comment: ''
>>   timeout: 30 sec
>>   yield period: 10 sec
>>   transport protocol: RAW
>>   proxy host: ''
>>   proxy port: ''
>>   proxy user: ''
>>   proxy password: ''
>>   local network interface: ''
>>   Input Ports:
>>   - id: 56ae5abc-0161-1000-aa9e-c340d726e043
>>     name: Minifi
>>     comment: ''
>>     max concurrent tasks: 1
>>     use compression: false
>>     Properties:
>>       Port: 10443
>>       SSL Context Service: SSLServiceName
>>       Host Name: w.x.y.z
>>   Output Ports: []
>> Provenance Reporting:
>>
>>
>>
>> On 09/02/2018 20:18, Marc wrote:
>>
>> Arne,
>>   I submitted a PR https://github.com/apache/nifi-minifi-cpp/pull/263 to
>> address these issues.
>>
>> On Fri, Feb 9, 2018 at 8:38 AM, Marc <phroc...@apache.org> wrote:
>>
>>> Arne,
>>>    Evidently the HTTPClient relies on an SSL Context Service. Try the
>>> following configuration in the config.yml file, where you define the
>>> context service and reference it from the RPG. Let me know if that works
>>> for you!
>>>
>>>   Additionally, I think you pointed out an inconsistency where we can
>>> improve the configuration and documentation. I've created
>>> https://issues.apache.org/jira/browse/MINIFICPP-396 and will take care
>>> of that ASAP. Thanks
>>>   very much for identifying this!
>>>
>>> Remote Processing Groups:
>>>     - name: NiFi Flow
>>>       id: 2438e3c8-015a-1000-79ca-83af40ec1998
>>>       url: https://127.0.0.1:8383/nifi
>>>       timeout: 30 secs
>>>       yield period: 5 sec
>>>       Input Ports:
>>>           - id: 2438e3c8-015a-1000-79ca-83af40ec1999
>>>             name: fromnifi
>>>             max concurrent tasks: 1
>>>             Properties:
>>>                 Port: 10443
>>>                 SSL Context Service: SSLMe
>>>                 Host Name: 127.0.0.1
>>>       Output Ports:
>>>           - id: ac82e521-015c-1000-2b21-41279516e19a
>>>             name: tominifi
>>>             max concurrent tasks: 2
>>>             Properties:
>>>                 Port: 10443
>>>                 SSL Context Service: SSLMe
>>>                 Host Name: 127.0.0.1
>>>
>>> Controller Services:
>>>     - name: SSLMe
>>>       id: 2438e3c8-015a-1000-79ca-83af40ec1974
>>>       class: SSLContextService
>>>       Properties:
>>>           Client Certificate: /opt/minifi/conf/client.pem
>>>           Private Key: /opt/minifi/conf/client.key
>>>           Passphrase: /opt/minifi/conf/password
>>>           CA Certificate certificate: /opt/minifi/conf/nifi-cert.pem
>>>
>>> On Fri, Feb 9, 2018 at 5:54 AM, Arne Oslebo <arne.osl...@uninett.no>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I'm trying to set up secure communication between minifi-cpp 0.4.0 and
>>>> nifi, but unfortunately it fails with the following errors:
>>>>
>>>> [org::apache::nifi::minifi::utils::HTTPClient] [error]
>>>> curl_easy_perform() failed SSL connect error
>>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error]
>>>> ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed
>>>>
>>>> I looked quickly at the code and it seems the problem is that HTTPClient
>>>> never calls configure_secure_connection and therefor never presents a
>>>> client certificate to nifi.
>>>>
>>>> The config.yml file defines a TailFail that send data directly to a
>>>> remote process group.
>>>>
>>>> My  minifi.properties file:
>>>> nifi.version=0.1.0
>>>> nifi.flow.configuration.file=/opt/minifi/conf/config.yml
>>>> nifi.administrative.yield.duration=30 sec
>>>> nifi.bored.yield.duration=10 millis
>>>> nifi.provenance.repository.directory.default=/opt/minifi/pro
>>>> venance_repository
>>>> nifi.provenance.repository.max.storage.time=1 MIN
>>>> nifi.provenance.repository.max.storage.size=1 MB
>>>> nifi.remote.input.secure=true
>>>> nifi.https.need.ClientAuth=true
>>>> nifi.https.client.certificate=/opt/minifi/conf/client.pem
>>>> nifi.https.client.private.key=/opt/minifi/conf/client.key
>>>> nifi.https.client.pass.phrase=/opt/minifi/conf/password
>>>> nifi.https.client.ca.certificate=/opt/minifi/conf/nifi-cert.pem
>>>> controller.socket.host=localhost
>>>> controller.socket.port=9998
>>>>
>>>> Certificates and key are correct and have been verified using curl from
>>>> the command line. Are there any other things I need to do to get minifi
>>>> to set up a secure connection? As far as I understand the "Security
>>>> Properties:" in config.yml is only used by the java version of minifi?
>>>>
>>>> Thanks,
>>>> Arne
>>>>
>>>>
>>>
>>
>>
>
>

Reply via email to