Arne, Thanks for the info. I'm running the same environment with the same warnings produced -- and segfault -- aso I'll get back to you once I've identified the issue.
TL;DR: Created a ticket ( https://issues.apache.org/jira/browse/MINIFICPP-400. ) to help insulate users from this more. More explanation: In regards to the different versions, there were a number of tickets on Debian's bug report logs regarding the curl ABI. An example [1] was solved by changing linked versions of libraries. The gist is that the SSL_CTX struct changes. I've validated with gdb that the struct is being passed in on U16 ( and works ) AND Debian stretch ( does not work ). The structures are slightly different. I'm going to dive deeper into this. I've created [2] https://issues.apache.org/jira/browse/MINIFICPP-400. [2] has more recent conversation as the issue still exists. This will require an soname change hence why they've likely been discussing this for two years. I think our job will be to insulate users, so [3] will be our efforts to do so. Thanks for identifying this. I'll continue to investigate this to address it seamlessly through either our bootstrap script ( bootstrap.sh in the root ) or within CMAKE. Obviously we don't want to alienate Debian users. [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844018 [2] https://issues.apache.org/jira/browse/MINIFICPP-400. [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858398 On Tue, Feb 13, 2018 at 8:28 AM, Arne Oslebo <[email protected]> wrote: > Marc, > > I'm running it on Debian Stretch. libssl version might be the problem. I > see that both libssl1.0.2 and libssl1.1 are installed. I took another look > at the build log and found this: > /usr/bin/ld: warning: libssl.so.1.0.2, needed by > /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may > conflict with libssl.so.1.1 > /usr/bin/ld: warning: libcrypto.so.1.0.2, needed by > /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may > conflict with libcrypto.so.1.1 > > I'll see if can remove one of the versions to avoid possible conflict. > > The full backtrace log is attached and thanks for looking into this, > > Arne > > > > On 13/02/2018 12:55, Marc wrote: > > Arne, > Sorry for your troubles! Can you give me some insight into your distro? > > I've been unable to replicate the issue on OSX, u16, and arch...but all > are running a different version of OpenSSL. What version of OpenSSL are you > running and on what distro? I'll be happy to try it to track this down. > > Additionally, do you have the log file you can pass on? The reason I > ask is that the line specified in gdb is a log statement with usage of > constructs that are inherent to libstdc++, so they won't cause a memory > error. The log file should help identify this and give me insight into what > occurred just before the error. > > Thanks, > Marc > > On Tue, Feb 13, 2018 at 6:17 AM, Arne Oslebo <[email protected]> > wrote: > >> Hello Mark, >> >> thanks for the update. I tried running master from github but >> unfortunately I now get a segmentation fault: >> >> Thread 1 "minifi" received signal SIGSEGV, Segmentation fault. >> 0x00007ffff777420a in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >> (gdb) bt full >> #0 0x00007ffff777420a in ?? () from /usr/lib/x86_64-linux-gnu/libs >> sl.so.1.1 >> No symbol table info available. >> #1 0x00007ffff7799681 in ?? () from /usr/lib/x86_64-linux-gnu/libs >> sl.so.1.1 >> No symbol table info available. >> #2 0x00007ffff777f2f6 in SSL_CTX_use_certificate () from >> /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >> No symbol table info available. >> #3 0x00007ffff777f6c0 in SSL_CTX_use_certificate_file () from >> /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >> No symbol table info available. >> #4 0x0000555555ef91cb in org::apache::nifi::minifi::con >> trollers::SSLContextService::configure_ssl_context (this=0x5555569948b0, >> ctx=0x555556a28430) at /usr/local/src/nifi-minifi-cpp >> /extensions/http-curl/../../libminifi/include/controllers/SS >> LContextService.h:165 >> retp = 1 >> #5 0x0000555555ef9be4 in org::apache::nifi::minifi::uti >> ls::HTTPClient::configure_ssl_context (curl=0x555556a149e0, >> ctx=0x555556a28430, param=0x5555569948b0) at /usr/local/src/nifi-minifi-cpp >> /extensions/http-curl/client/HTTPClient.h:177 >> ssl_context_service = 0x5555569948b0 >> >> Any idea what the problem might be? >> >> My full config.yml: >> >> Flow Controller: >> name: MiNiFi Flow >> Controller Services: >> - name: SSLServiceName >> id: 2438e3c8-015a-1000-79ca-83af40ec1974 >> class: SSLContextService >> Properties: >> Client Certificate: /opt/minifi/conf/client.pem >> Private Key: /opt/minifi/conf/client.key >> Passphrase: secret >> CA Certificate: /opt/minifi/conf/nifi-cert.pem >> Processors: >> - id: cecb1868-9e5a-3e6c-0000-000000000000 >> name: TailFile >> class: org.apache.nifi.processors.standard.TailFile >> max concurrent tasks: 1 >> scheduling strategy: TIMER_DRIVEN >> scheduling period: 0 sec >> penalization period: 30 sec >> yield period: 1 sec >> run duration nanos: 0 >> auto-terminated relationships list: [] >> Properties: >> File Location: Local >> File to Tail: /tmp/test.log >> Initial Start Position: Beginning of File >> Rolling Filename Pattern: >> tail-base-directory: >> tail-mode: Single file >> tailfile-lookup-frequency: 10 minutes >> tailfile-maximum-age: 24 hours >> tailfile-recursive-lookup: 'false' >> Connections: >> - id: 76ad4bc4-6557-3e23-0000-000000000000 >> name: TailFile/success/56ae5abc-0161-1000-aa9e-c340d726e043 >> source id: cecb1868-9e5a-3e6c-0000-000000000000 >> source relationship names: >> - success >> destination id: 56ae5abc-0161-1000-aa9e-c340d726e043 >> max work queue size: 10000 >> max work queue data size: 1 GB >> flowfile expiration: 0 sec >> queue prioritizer class: '' >> Remote Processing Groups: >> - id: 3a25e1a3-c1b2-3e78-0000-000000000000 >> name: '' >> url: https://w.x.y.z:8443/nifi >> comment: '' >> timeout: 30 sec >> yield period: 10 sec >> transport protocol: RAW >> proxy host: '' >> proxy port: '' >> proxy user: '' >> proxy password: '' >> local network interface: '' >> Input Ports: >> - id: 56ae5abc-0161-1000-aa9e-c340d726e043 >> name: Minifi >> comment: '' >> max concurrent tasks: 1 >> use compression: false >> Properties: >> Port: 10443 >> SSL Context Service: SSLServiceName >> Host Name: w.x.y.z >> Output Ports: [] >> Provenance Reporting: >> >> >> >> On 09/02/2018 20:18, Marc wrote: >> >> Arne, >> I submitted a PR https://github.com/apache/nifi-minifi-cpp/pull/263 to >> address these issues. >> >> On Fri, Feb 9, 2018 at 8:38 AM, Marc <[email protected]> wrote: >> >>> Arne, >>> Evidently the HTTPClient relies on an SSL Context Service. Try the >>> following configuration in the config.yml file, where you define the >>> context service and reference it from the RPG. Let me know if that works >>> for you! >>> >>> Additionally, I think you pointed out an inconsistency where we can >>> improve the configuration and documentation. I've created >>> https://issues.apache.org/jira/browse/MINIFICPP-396 and will take care >>> of that ASAP. Thanks >>> very much for identifying this! >>> >>> Remote Processing Groups: >>> - name: NiFi Flow >>> id: 2438e3c8-015a-1000-79ca-83af40ec1998 >>> url: https://127.0.0.1:8383/nifi >>> timeout: 30 secs >>> yield period: 5 sec >>> Input Ports: >>> - id: 2438e3c8-015a-1000-79ca-83af40ec1999 >>> name: fromnifi >>> max concurrent tasks: 1 >>> Properties: >>> Port: 10443 >>> SSL Context Service: SSLMe >>> Host Name: 127.0.0.1 >>> Output Ports: >>> - id: ac82e521-015c-1000-2b21-41279516e19a >>> name: tominifi >>> max concurrent tasks: 2 >>> Properties: >>> Port: 10443 >>> SSL Context Service: SSLMe >>> Host Name: 127.0.0.1 >>> >>> Controller Services: >>> - name: SSLMe >>> id: 2438e3c8-015a-1000-79ca-83af40ec1974 >>> class: SSLContextService >>> Properties: >>> Client Certificate: /opt/minifi/conf/client.pem >>> Private Key: /opt/minifi/conf/client.key >>> Passphrase: /opt/minifi/conf/password >>> CA Certificate certificate: /opt/minifi/conf/nifi-cert.pem >>> >>> On Fri, Feb 9, 2018 at 5:54 AM, Arne Oslebo <[email protected]> >>> wrote: >>> >>>> Hello, >>>> >>>> I'm trying to set up secure communication between minifi-cpp 0.4.0 and >>>> nifi, but unfortunately it fails with the following errors: >>>> >>>> [org::apache::nifi::minifi::utils::HTTPClient] [error] >>>> curl_easy_perform() failed SSL connect error >>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error] >>>> ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed >>>> >>>> I looked quickly at the code and it seems the problem is that HTTPClient >>>> never calls configure_secure_connection and therefor never presents a >>>> client certificate to nifi. >>>> >>>> The config.yml file defines a TailFail that send data directly to a >>>> remote process group. >>>> >>>> My minifi.properties file: >>>> nifi.version=0.1.0 >>>> nifi.flow.configuration.file=/opt/minifi/conf/config.yml >>>> nifi.administrative.yield.duration=30 sec >>>> nifi.bored.yield.duration=10 millis >>>> nifi.provenance.repository.directory.default=/opt/minifi/pro >>>> venance_repository >>>> nifi.provenance.repository.max.storage.time=1 MIN >>>> nifi.provenance.repository.max.storage.size=1 MB >>>> nifi.remote.input.secure=true >>>> nifi.https.need.ClientAuth=true >>>> nifi.https.client.certificate=/opt/minifi/conf/client.pem >>>> nifi.https.client.private.key=/opt/minifi/conf/client.key >>>> nifi.https.client.pass.phrase=/opt/minifi/conf/password >>>> nifi.https.client.ca.certificate=/opt/minifi/conf/nifi-cert.pem >>>> controller.socket.host=localhost >>>> controller.socket.port=9998 >>>> >>>> Certificates and key are correct and have been verified using curl from >>>> the command line. Are there any other things I need to do to get minifi >>>> to set up a secure connection? As far as I understand the "Security >>>> Properties:" in config.yml is only used by the java version of minifi? >>>> >>>> Thanks, >>>> Arne >>>> >>>> >>> >> >> > >
