Hi all. The error was caused neither from NIFI or sftp server but on the
freeipa client side running on the sftp server.
thanks all
On Mon, Nov 5, 2018 at 9:11 AM Alfredo De Luca <alfredo.del...@gmail.com>
wrote:
> Hi Joe.. have you had any thought about this? I don't know what else to do.
> Cheers
>
> On Tue, Oct 30, 2018 at 8:18 PM Alfredo De Luca <alfredo.del...@gmail.com>
> wrote:
>
>> Any thought Joe? thank so much
>>
>> On Tue, Oct 30, 2018 at 2:28 PM Alfredo De Luca <alfredo.del...@gmail.com>
>> wrote:
>>
>>> I have attached the nifi properties. Also these are more info ...
>>>
>>> - ListSftp is running on primary node of the cluster and with 1
>>> concurrent tasks
>>> - nifi is in cluster with 3 nodes (pod in kubernetes)
>>> ---
>>> *Loading of nifi*
>>> how many threads does nifi show are running?
>>> *150 in average*
>>> ---
>>> How often does the problem happen? How often does the connection/flow
>>> work?
>>>
>>> *Often, some processors does not list any file on sftp server because of
>>> this problem. Other Processors could list file on server sftp but at
>>> certain time listing fails.*
>>> *In addition sometimes fails also the unique fetchSftp processor*
>>>
>>> SFTP Server
>>> 1 CPU
>>> 2Gb Ram
>>> Load avg. load average: 0.20, 0.27, 0.31
>>>
>>> Cheers
>>>
>>>
>>>
>>> On Mon, Oct 29, 2018 at 5:41 PM Alfredo De Luca <
>>> alfredo.del...@gmail.com> wrote:
>>>
>>>> Thanks Joe. Appreciated. I will get back to you asap with the info.
>>>>
>>>> Cheers
>>>>
>>>>
>>>> On Mon, Oct 29, 2018 at 5:26 PM Joe Witt <joe.w...@gmail.com> wrote:
>>>>
>>>>> Alfredo
>>>>>
>>>>> There are quite a few details that will be needed to help folks guess
>>>>> at possible issues.
>>>>>
>>>>> Configuration of the machine running nifi
>>>>> how many cores?, etc..
>>>>>
>>>>> Configuration of nifi
>>>>> you shared common properties from nifi.properties which is good.
>>>>> how many threads does the flow controller have?
>>>>> how many processors are on the graph?
>>>>> How many threads does ListSFTP have?
>>>>> Is this a NiFi cluster or single node?
>>>>>
>>>>> Loading of the system running nifi
>>>>> cpu load avg
>>>>> garbage collection behavior in the jVM?
>>>>>
>>>>> Loading of nifi
>>>>> how many threads does nifi show are running?
>>>>>
>>>>> Similar info for the SFTP server side
>>>>> System error 4 could mean a lot of things. Is that system taxed and
>>>>> unable to establish connections at times?
>>>>>
>>>>> How often does the problem happen? How often does the connection/flow
>>>>> work?
>>>>>
>>>>> Have you tried manually connecting the way nifi would by running
>>>>> commands like 'sftp u...@host.sftp.server -vvv' repeatedly to see if
>>>>> you can reproduce the issue?
>>>>>
>>>>> Does a file system get mounted for the sftp user and periodically is
>>>>> this failing?
>>>>>
>>>>> Unfortunately it could be a lot of things so we'll probably need to
>>>>> unpack a lot of info to get there.
>>>>>
>>>>> Thanks
>>>>> On Mon, Oct 29, 2018 at 12:14 PM Alfredo De Luca
>>>>> <alfredo.del...@gmail.com> wrote:
>>>>> >
>>>>> > hi all. any idea? I am trying everything..but not sure what cause
>>>>> this error.
>>>>> >
>>>>> > Cheers
>>>>> >
>>>>> > On Mon, Oct 29, 2018 at 12:34 PM Alfredo De Luca <
>>>>> alfredo.del...@gmail.com> wrote:
>>>>> >>
>>>>> >> nifi.swap.in.threads=1
>>>>> >> nifi.swap.out.threads=4
>>>>> >> nifi.cluster.node.protocol.threads=25
>>>>> >> nifi.cluster.node.protocol.max.threads=50
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Mon, Oct 29, 2018 at 12:28 PM Alfredo De Luca <
>>>>> alfredo.del...@gmail.com> wrote:
>>>>> >>>
>>>>> >>> anyway...this is the error
>>>>> >>> 2018-10-29 11:21:06,768 ERROR [Timer-Driven Process Thread-13]
>>>>> SimpleProcessLogger.java:254
>>>>> ListSFTP[id=766ac418-27ce-335a-97c9-8823f2cf5a96] Failed to perform
>>>>> listing
>>>>> on remote host due to java.io.IOException: Failed to obtain connection to
>>>>> remote host due to com.jcraft.jsch.JSchException: Auth fail: {}
>>>>> >>>
>>>>> >>>
>>>>> >>> On Mon, Oct 29, 2018 at 12:27 PM Alfredo De Luca <
>>>>> alfredo.del...@gmail.com> wrote:
>>>>> >>>>
>>>>> >>>> Hi Joe.
>>>>> >>>> Sftp server allows 200 connections... not sure about the nifi. I
>>>>> ll check
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> On Mon, Oct 29, 2018 at 12:21 PM Joe Witt <joe.w...@gmail.com>
>>>>> wrote:
>>>>> >>>>>
>>>>> >>>>> how many connections does your sftp server allow at once and how
>>>>> many threads is nifi given for communication?
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On Mon, Oct 29, 2018, 7:10 AM Alfredo De Luca <
>>>>> alfredo.del...@gmail.com> wrote:
>>>>> >>>>>>
>>>>> >>>>>> Hi all. We have some issue with our nifi (v. 1.6) where
>>>>> occasionally we have an error saying ...
>>>>> >>>>>> Auth fail.
>>>>> >>>>>>
>>>>> >>>>>> On the sftp server side we get
>>>>> >>>>>>
>>>>> >>>>>> Oct 29 11:48:06 sftp sshd[13845]: pam_sss(sshd:account): Access
>>>>> denied for user <sftp user>: 4 (System error)
>>>>> >>>>>>
>>>>> >>>>>> Any idea?
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> --
>>>>> >>>>>> Alfredo
>>>>> >>>>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> --
>>>>> >>>> Alfredo
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>> --
>>>>> >>> Alfredo
>>>>> >>>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Alfredo
>>>>> >>
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Alfredo
>>>>> >
>>>>>
>>>>
>>>>
>>>> --
>>>> *Alfredo*
>>>>
>>>>
>>>
>>> --
>>> *Alfredo*
>>>
>>>
>>
>> --
>> *Alfredo*
>>
>>
>
> --
> *Alfredo*
>
>
--
*Alfredo*
