Arne, Yes, I do know what is wrong. I intended to follow up to this E-mail in a bit once a second PR was merged. I split the efforts into two PRs to make reviewing it easier. I posted that PR a few days ago and review was completed a little while ago and it has been merged.
The PR resolved a disparity between how HTTP site to site and Raw socket site to site have to be configured [1]. This would have led to tremendous confusion among users to configure them differently so I made it consistent, while also backwards compatible. Please try again, and sorry for poorly communicating the fact that there were two PRs to solve this. In an effort to make reviewing easier ( and write a better unit test ) I think I dropped the ball in communicating intent to you. Please me me know if you have any additional problems with master. [1] https://github.com/apache/nifi-minifi-cpp/pull/265 On Wed, Feb 21, 2018 at 7:02 AM, Arne Oslebo <[email protected]> wrote: > Hello Marc, > > thank you for committing a fix for the Debian issues. Minifi now compiles > without any warnings but unfortunately I'm still having some problems > getting things to work properly. My config.yml is now a copy of the example > in the readme file where you have a GetFile and a RPG. Using unsecured > communication everything works fine. I then add a SSLContextService and > reference it from the RPG. The full config.yml is: > > Flow Controller: > id: 471deef6-2a6e-4a7d-912a-81cc17e3a205 > name: MiNiFi Flow > Processors: > - name: GetFile > id: 471deef6-2a6e-4a7d-912a-81cc17e3a206 > class: org.apache.nifi.processors.standard.GetFile > max concurrent tasks: 1 > scheduling strategy: TIMER_DRIVEN > scheduling period: 1 sec > penalization period: 30 sec > yield period: 1 sec > run duration nanos: 0 > auto-terminated relationships list: > Properties: > Input Directory: /tmp/test > Keep Source File: false > Controller Services: > - name: SSLServiceName > id: 2438e3c8-015a-1000-79ca-83af40ec1974 > class: SSLContextService > Properties: > Client Certificate: /opt/minifi/conf/client.pem > Private Key: /opt/minifi/conf/key.pem > CA Certificate: /opt/minifi/conf/nifi-cert.pem > Connections: > - name: TransferFilesToRPG > id: 471deef6-2a6e-4a7d-912a-81cc17e3a207 > source name: GetFile > source id: 471deef6-2a6e-4a7d-912a-81cc17e3a206 > source relationship name: success > destination id: 8e7979f9-0161-1000-941e-3be83b4479b0 > max work queue size: 0 > max work queue data size: 1 MB > flowfile expiration: 60 sec > Remote Processing Groups: > - name: NiFi Flow > id: 471deef6-2a6e-4a7d-912a-81cc17e3a208 > url: https://***:8443/nifi > timeout: 30 secs > yield period: 10 sec > Input Ports: > - id: 8e7979f9-0161-1000-941e-3be83b4479b0 > name: Input > max concurrent tasks: 1 > Properties: > Port: 10433 > Host Name: *** > SSL Context Service: SSLServiceName > > From the nifi-user.log i see that minifi connects and authenticates > properly. The problem is that when I add a file to the /tmp/test directory I > get the following error from minifi: > [2018-02-21 12:56:22.943] > [org::apache::nifi::minifi::sitetosite::RawSiteToSiteClient] [error] > Site2Site Protocol Version Negotiation failed > [2018-02-21 12:56:22.943] > [org::apache::nifi::minifi::RemoteProcessorGroupPort] [info] Have 0 peers > [2018-02-21 12:56:22.943] > [org::apache::nifi::minifi::RemoteProcessorGroupPort] [info] no protocol, > yielding > > In nifi-app.log I get: > 2018-02-21 12:56:09,654 ERROR [Site-to-Site Worker Thread-0] > o.a.n.r.io.socket.ssl.SSLSocketChannel > org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel@21e3bcdc Failed to > connect due to {} > javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection? > at > sun.security.ssl.EngineInputRecord.bytesInCompletePacket(EngineInputRecord.java:156) > at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:868) > at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) > at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) > at > org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performHandshake(SSLSocketChannel.java:237) > at > org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.connect(SSLSocketChannel.java:163) > at > org.apache.nifi.remote.SocketRemoteSiteListener$1$1.run(SocketRemoteSiteListener.java:166) > at java.lang.Thread.run(Thread.java:748) > 2018-02-21 12:56:09,655 ERROR [Site-to-Site Worker Thread-0] > o.a.nifi.remote.SocketRemoteSiteListener RemoteSiteListener Unable to accept > connection from Socket[unconnected] due to javax.net.ssl.SSLException: > Inbound closed before receiving peer's close_notify: possible truncation > attack? > > I've tried both nifi-1.5.0 and nifi-1.6.0-SNAPSHOT. > > Any suggestions as to what might be wrong? > > Best regards, > Arne > > > > On 13/02/2018 18:53, Marc wrote: > > Arne, > I took a break from the issue and came back and tried installing a > different version of openssl on top of the distro. When doing so it linked > properly and I'm able to send data through a secure Socket. Now that I have > a solution, I will move this discussion to the ticket. > > As a result of my testing, I will make updates to the bootstrap script and > build instructions to instruct users to install libssl1.0 on Debian Stretch > ( and perhaps Raspbian ). Any comments on the ticket will be appreciated: > https://issues.apache.org/jira/browse/MINIFICPP-400 . I will have a fix once > I finish testing across a few platforms. > > Thanks, > Marc > > On Tue, Feb 13, 2018 at 9:55 AM, Marc P. <[email protected]> wrote: >> >> Arne, >> >> Thanks for the info. I'm running the same environment with the same >> warnings produced -- and segfault -- aso I'll get back to you once I've >> identified the issue. >> >> TL;DR: Created a ticket ( >> https://issues.apache.org/jira/browse/MINIFICPP-400. ) to help insulate >> users from this more. >> >> More explanation: >> >> >> In regards to the different versions, there were a number of tickets on >> Debian's bug report logs regarding the curl ABI. An example [1] was solved >> by changing linked versions of libraries. >> >> The gist is that the SSL_CTX struct changes. I've validated with gdb >> that the struct is being passed in on U16 ( and works ) AND Debian stretch ( >> does not work ). The structures are slightly different. I'm going to dive >> deeper into this. I've created [2] >> https://issues.apache.org/jira/browse/MINIFICPP-400. >> >> [2] has more recent conversation as the issue still exists. This will >> require an soname change hence why they've likely been discussing this for >> two years. I think our job will be to insulate users, so [3] will be our >> efforts to do so. Thanks for identifying this. I'll continue to investigate >> this to address it seamlessly through either our bootstrap script ( >> bootstrap.sh in the root ) or within CMAKE. Obviously we don't want to >> alienate Debian users. >> >> >> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844018 >> [2] https://issues.apache.org/jira/browse/MINIFICPP-400. >> [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858398 >> >> On Tue, Feb 13, 2018 at 8:28 AM, Arne Oslebo <[email protected]> >> wrote: >>> >>> Marc, >>> >>> I'm running it on Debian Stretch. libssl version might be the problem. I >>> see that both libssl1.0.2 and libssl1.1 are installed. I took another look >>> at the build log and found this: >>> /usr/bin/ld: warning: libssl.so.1.0.2, needed by >>> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may >>> conflict with libssl.so.1.1 >>> /usr/bin/ld: warning: libcrypto.so.1.0.2, needed by >>> /usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libcurl.so, may >>> conflict with libcrypto.so.1.1 >>> >>> I'll see if can remove one of the versions to avoid possible conflict. >>> >>> The full backtrace log is attached and thanks for looking into this, >>> >>> Arne >>> >>> >>> >>> On 13/02/2018 12:55, Marc wrote: >>> >>> Arne, >>> Sorry for your troubles! Can you give me some insight into your >>> distro? >>> >>> I've been unable to replicate the issue on OSX, u16, and arch...but >>> all are running a different version of OpenSSL. What version of OpenSSL are >>> you running and on what distro? I'll be happy to try it to track this down. >>> >>> Additionally, do you have the log file you can pass on? The reason I >>> ask is that the line specified in gdb is a log statement with usage of >>> constructs that are inherent to libstdc++, so they won't cause a memory >>> error. The log file should help identify this and give me insight into what >>> occurred just before the error. >>> >>> Thanks, >>> Marc >>> >>> On Tue, Feb 13, 2018 at 6:17 AM, Arne Oslebo <[email protected]> >>> wrote: >>>> >>>> Hello Mark, >>>> >>>> thanks for the update. I tried running master from github but >>>> unfortunately I now get a segmentation fault: >>>> >>>> Thread 1 "minifi" received signal SIGSEGV, Segmentation fault. >>>> 0x00007ffff777420a in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >>>> (gdb) bt full >>>> #0 0x00007ffff777420a in ?? () from >>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >>>> No symbol table info available. >>>> #1 0x00007ffff7799681 in ?? () from >>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >>>> No symbol table info available. >>>> #2 0x00007ffff777f2f6 in SSL_CTX_use_certificate () from >>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >>>> No symbol table info available. >>>> #3 0x00007ffff777f6c0 in SSL_CTX_use_certificate_file () from >>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.1 >>>> No symbol table info available. >>>> #4 0x0000555555ef91cb in >>>> org::apache::nifi::minifi::controllers::SSLContextService::configure_ssl_context >>>> (this=0x5555569948b0, ctx=0x555556a28430) at >>>> /usr/local/src/nifi-minifi-cpp/extensions/http-curl/../../libminifi/include/controllers/SSLContextService.h:165 >>>> retp = 1 >>>> #5 0x0000555555ef9be4 in >>>> org::apache::nifi::minifi::utils::HTTPClient::configure_ssl_context >>>> (curl=0x555556a149e0, ctx=0x555556a28430, param=0x5555569948b0) at >>>> /usr/local/src/nifi-minifi-cpp/extensions/http-curl/client/HTTPClient.h:177 >>>> ssl_context_service = 0x5555569948b0 >>>> >>>> Any idea what the problem might be? >>>> >>>> My full config.yml: >>>> >>>> Flow Controller: >>>> name: MiNiFi Flow >>>> Controller Services: >>>> - name: SSLServiceName >>>> id: 2438e3c8-015a-1000-79ca-83af40ec1974 >>>> class: SSLContextService >>>> Properties: >>>> Client Certificate: /opt/minifi/conf/client.pem >>>> Private Key: /opt/minifi/conf/client.key >>>> Passphrase: secret >>>> CA Certificate: /opt/minifi/conf/nifi-cert.pem >>>> Processors: >>>> - id: cecb1868-9e5a-3e6c-0000-000000000000 >>>> name: TailFile >>>> class: org.apache.nifi.processors.standard.TailFile >>>> max concurrent tasks: 1 >>>> scheduling strategy: TIMER_DRIVEN >>>> scheduling period: 0 sec >>>> penalization period: 30 sec >>>> yield period: 1 sec >>>> run duration nanos: 0 >>>> auto-terminated relationships list: [] >>>> Properties: >>>> File Location: Local >>>> File to Tail: /tmp/test.log >>>> Initial Start Position: Beginning of File >>>> Rolling Filename Pattern: >>>> tail-base-directory: >>>> tail-mode: Single file >>>> tailfile-lookup-frequency: 10 minutes >>>> tailfile-maximum-age: 24 hours >>>> tailfile-recursive-lookup: 'false' >>>> Connections: >>>> - id: 76ad4bc4-6557-3e23-0000-000000000000 >>>> name: TailFile/success/56ae5abc-0161-1000-aa9e-c340d726e043 >>>> source id: cecb1868-9e5a-3e6c-0000-000000000000 >>>> source relationship names: >>>> - success >>>> destination id: 56ae5abc-0161-1000-aa9e-c340d726e043 >>>> max work queue size: 10000 >>>> max work queue data size: 1 GB >>>> flowfile expiration: 0 sec >>>> queue prioritizer class: '' >>>> Remote Processing Groups: >>>> - id: 3a25e1a3-c1b2-3e78-0000-000000000000 >>>> name: '' >>>> url: https://w.x.y.z:8443/nifi >>>> comment: '' >>>> timeout: 30 sec >>>> yield period: 10 sec >>>> transport protocol: RAW >>>> proxy host: '' >>>> proxy port: '' >>>> proxy user: '' >>>> proxy password: '' >>>> local network interface: '' >>>> Input Ports: >>>> - id: 56ae5abc-0161-1000-aa9e-c340d726e043 >>>> name: Minifi >>>> comment: '' >>>> max concurrent tasks: 1 >>>> use compression: false >>>> Properties: >>>> Port: 10443 >>>> SSL Context Service: SSLServiceName >>>> Host Name: w.x.y.z >>>> Output Ports: [] >>>> Provenance Reporting: >>>> >>>> >>>> >>>> On 09/02/2018 20:18, Marc wrote: >>>> >>>> Arne, >>>> I submitted a PR https://github.com/apache/nifi-minifi-cpp/pull/263 to >>>> address these issues. >>>> >>>> On Fri, Feb 9, 2018 at 8:38 AM, Marc <[email protected]> wrote: >>>>> >>>>> Arne, >>>>> Evidently the HTTPClient relies on an SSL Context Service. Try the >>>>> following configuration in the config.yml file, where you define the >>>>> context >>>>> service and reference it from the RPG. Let me know if that works for you! >>>>> >>>>> Additionally, I think you pointed out an inconsistency where we can >>>>> improve the configuration and documentation. I've created >>>>> https://issues.apache.org/jira/browse/MINIFICPP-396 and will take care of >>>>> that ASAP. Thanks >>>>> very much for identifying this! >>>>> >>>>> Remote Processing Groups: >>>>> - name: NiFi Flow >>>>> id: 2438e3c8-015a-1000-79ca-83af40ec1998 >>>>> url: https://127.0.0.1:8383/nifi >>>>> timeout: 30 secs >>>>> yield period: 5 sec >>>>> Input Ports: >>>>> - id: 2438e3c8-015a-1000-79ca-83af40ec1999 >>>>> name: fromnifi >>>>> max concurrent tasks: 1 >>>>> Properties: >>>>> Port: 10443 >>>>> SSL Context Service: SSLMe >>>>> Host Name: 127.0.0.1 >>>>> Output Ports: >>>>> - id: ac82e521-015c-1000-2b21-41279516e19a >>>>> name: tominifi >>>>> max concurrent tasks: 2 >>>>> Properties: >>>>> Port: 10443 >>>>> SSL Context Service: SSLMe >>>>> Host Name: 127.0.0.1 >>>>> >>>>> Controller Services: >>>>> - name: SSLMe >>>>> id: 2438e3c8-015a-1000-79ca-83af40ec1974 >>>>> class: SSLContextService >>>>> Properties: >>>>> Client Certificate: /opt/minifi/conf/client.pem >>>>> Private Key: /opt/minifi/conf/client.key >>>>> Passphrase: /opt/minifi/conf/password >>>>> CA Certificate certificate: /opt/minifi/conf/nifi-cert.pem >>>>> >>>>> On Fri, Feb 9, 2018 at 5:54 AM, Arne Oslebo <[email protected]> >>>>> wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> I'm trying to set up secure communication between minifi-cpp 0.4.0 and >>>>>> nifi, but unfortunately it fails with the following errors: >>>>>> >>>>>> [org::apache::nifi::minifi::utils::HTTPClient] [error] >>>>>> curl_easy_perform() failed SSL connect error >>>>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error] >>>>>> ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed >>>>>> >>>>>> I looked quickly at the code and it seems the problem is that >>>>>> HTTPClient >>>>>> never calls configure_secure_connection and therefor never presents a >>>>>> client certificate to nifi. >>>>>> >>>>>> The config.yml file defines a TailFail that send data directly to a >>>>>> remote process group. >>>>>> >>>>>> My minifi.properties file: >>>>>> nifi.version=0.1.0 >>>>>> nifi.flow.configuration.file=/opt/minifi/conf/config.yml >>>>>> nifi.administrative.yield.duration=30 sec >>>>>> nifi.bored.yield.duration=10 millis >>>>>> >>>>>> nifi.provenance.repository.directory.default=/opt/minifi/provenance_repository >>>>>> nifi.provenance.repository.max.storage.time=1 MIN >>>>>> nifi.provenance.repository.max.storage.size=1 MB >>>>>> nifi.remote.input.secure=true >>>>>> nifi.https.need.ClientAuth=true >>>>>> nifi.https.client.certificate=/opt/minifi/conf/client.pem >>>>>> nifi.https.client.private.key=/opt/minifi/conf/client.key >>>>>> nifi.https.client.pass.phrase=/opt/minifi/conf/password >>>>>> nifi.https.client.ca.certificate=/opt/minifi/conf/nifi-cert.pem >>>>>> controller.socket.host=localhost >>>>>> controller.socket.port=9998 >>>>>> >>>>>> Certificates and key are correct and have been verified using curl >>>>>> from >>>>>> the command line. Are there any other things I need to do to get >>>>>> minifi >>>>>> to set up a secure connection? As far as I understand the "Security >>>>>> Properties:" in config.yml is only used by the java version of minifi? >>>>>> >>>>>> Thanks, >>>>>> Arne >>>>>> >>>>> >>>> >>>> >>> >>> >> > >
