Hi Arne,
Thanks for sending this. What I see in this example is that the file
listing is empty and there is no flow file produced. Per the log
statements below a call to get on the process session was null,
resulting in the RPG yielding. "protocol transmission failed" is a
catch all here. I see toward the end the log GetFile starts to capture
data, so perhaps the log ended a bit early, but in the case where "Get
is null for Minifi" resulting in "protocol transmission failed,
yielding"...this is an expected and valid warning.
[2018-05-07 08:24:20.675]
[org::apache::nifi::minifi::core::ProcessSession] [debug] Get is null
for Minifi
[2018-05-07 08:24:20.675]
[org::apache::nifi::minifi::core::logging::LoggerConfiguration]
[debug] org::apache::nifi::minifi::c2::C2Agent logger got sinks from
namespace org::apache::nifi::minifi and level debug from namespace
org::apache::nifi::minifi
[2018-05-07 08:24:20.675]
[org::apache::nifi::minifi::RemoteProcessorGroupPort] [warning]
protocol transmission failed, yielding
Though I did notice something when I inspected your config a little closer.
In your connection can you change:
Connections:
- name: TransferFilesToRPG
id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
source name: GetFile
source relationship name: success
destination id: 8e7979f9-0161-1000-941e-3be83b4479b0
max work queue size: 0
max work queue data size: 1 MB
flowfile expiration: 60 sec
to
Connections:
- name: TransferFilesToRPG
id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
source name: GetFile
source relationship name: success
destination id: 29fd32ba-0163-1000-c751-29aa2e8750c5
max work queue size: 0
max work queue data size: 1 MB
flowfile expiration: 60 sec
I changed the destination of the input port in the RPG. Since the RPG
could be a collection of input and output ports I changed this to
direct the data to the input port.
Let me know if this helps. Thanks,
Marc
On Mon, May 7, 2018 at 4:17 AM, Arne Oslebo <[email protected]> wrote:
> Hello Marc,
>
> here is what I think is the relevant part of the log file.
>
> Thank you for the help,
> Arne
>
> On 04/05/2018 15:48, Marc wrote:
>> Arne,
>> Great. Would you be able to change your minifi-log.properties to
>> log to debug? You should be able to change all instances of INFO to
>> DEBUG. It'll be chatty, but feel free to attach the log to the E-mail
>> if you can. s/INFO/DEBUG/
>>
>> At that level we get more useful information about why the connection
>> failed. My gut reaction is that this likely has something to do with
>> host verification ( I've seen that a number of times myself ) of the
>> server certificate's common name ( or subject alt name ). The logs
>> will help guide us to give you better information.
>> Thanks!
>> Marc
>>
>> On Fri, May 4, 2018 at 7:30 AM, Arne Oslebo <[email protected]> wrote:
>>>
>>> On 04/05/2018 12:54, Marc Parisi wrote:
>>>
>>> Arne,
>>> Do you have your certs defined in minifi.properties ? You either need
>>> those properties or an SSL context service.
>>>
>>>
>>> Yes, the certs are defined in minifi.properties:
>>> nifi.remote.input.http.enabled=true
>>> nifi.remote.input.secure=true
>>> nifi.security.need.ClientAuth=yes
>>> nifi.security.client.certificate=/opt/minifi/conf/cert.pem
>>> nifi.security.client.private.key=/opt/minifi/conf/key.pem
>>> nifi.security.client.ca.certificate=/opt/minifi/conf/nifi-cert.pem
>>>
>>> No password on the key.
>>>
>>> I've also tried using an SSL context service but get the same result: minifi
>>> authenticates with the nifi cluster but no data is transmitted. Same warning
>>> in the minifi-app.log:
>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [warning] protocol
>>> transmission failed, yielding
>>>
>>> Best regards,
>>> Arne
>>>
>>>
>>>
>>> On Fri, May 4, 2018, 6:47 AM Arne Oslebo <[email protected]> wrote:
>>>> Hello,
>>>>
>>>> I've pulled minifi cpp from master and I'm trying to send data to a
>>>> secure nifi cluster with it. The config.yml is simply a getfile
>>>> connected to a remote processing group:
>>>>
>>>> Flow Controller:
>>>> name: MiNiFi Flow
>>>>
>>>> Processors:
>>>> - name: GetFile
>>>> class: org.apache.nifi.processors.standard.GetFile
>>>> max concurrent tasks: 1
>>>> scheduling strategy: TIMER_DRIVEN
>>>> scheduling period: 1 sec
>>>> penalization period: 30 sec
>>>> yield period: 1 sec
>>>> run duration nanos: 0
>>>> auto-terminated relationships list:
>>>> Properties:
>>>> Input Directory: /tmp/getfile
>>>> Keep Source File: true
>>>>
>>>> Connections:
>>>> - name: TransferFilesToRPG
>>>> id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
>>>> source name: GetFile
>>>> source relationship name: success
>>>> destination id: 8e7979f9-0161-1000-941e-3be83b4479b0
>>>> max work queue size: 0
>>>> max work queue data size: 1 MB
>>>> flowfile expiration: 60 sec
>>>>
>>>> Remote Processing Groups:
>>>> - name: NiFi Flow
>>>> id: 8e7979f9-0161-1000-941e-3be83b4479b0
>>>> url: https://w.x.y.z:8443/nifi
>>>> timeout: 30 secs
>>>> yield period: 10 sec
>>>> Input Ports:
>>>> - id: 29fd32ba-0163-1000-c751-29aa2e8750c5
>>>> name: Minifi
>>>> max concurrent tasks: 1
>>>> transport protocol: HTTP
>>>>
>>>> Certificate and key is set in minifi.properties and minifi successfully
>>>> authenticates with the cluster:
>>>> 2018-05-04 11:23:12,105 INFO [NiFi Web Server-85]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=minifi,
>>>> OU=NIFI) GET https://w.x.y.z:8443/nifi-api/site-to-site (source ip:
>>>> w.x.y.z)
>>>> 2018-05-04 11:23:12,105 INFO [NiFi Web Server-85]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=minifi,
>>>> OU=NIFI
>>>> 2018-05-04 11:23:12,169 INFO [NiFi Web Server-84]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=minifi,
>>>> OU=NIFI) GET https://w.x.y.z:8443/nifi-api/site-to-site/peers (source
>>>> ip: w.x.y.z)
>>>> 2018-05-04 11:23:12,170 INFO [NiFi Web Server-84]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=minifi,
>>>> OU=NIFI
>>>>
>>>> The problem is that no data is sent to the cluster. In the
>>>> minifi-app.log I find a warning that protocol transmission failed:
>>>> [2018-05-04 11:23:21.945]
>>>> [org::apache::nifi::minifi::processors::GetFile] [info] GetFile process
>>>> /tmp/getfile/test
>>>> [2018-05-04 11:23:21.945]
>>>> [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring
>>>> ca111018-4f7c-11e8-88be-000c297ab98c from GetFile to relationship success
>>>> [2018-05-04 11:23:22.196]
>>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [warning] protocol
>>>> transmission failed, yielding
>>>>
>>>> I've tried both RAW and HTTP as transport protocol with the same result.
>>>>
>>>> Any idea what might be wrong?
>>>>
>>>> Thanks for any help,
>>>> Arne
>>>
>
