[
https://issues.apache.org/jira/browse/CAMEL-11104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954732#comment-15954732
]
Ayush Dixit commented on CAMEL-11104:
-------------------------------------
Already put on user mailing list
> Performance issue in camel sftp
> -------------------------------
>
> Key: CAMEL-11104
> URL: https://issues.apache.org/jira/browse/CAMEL-11104
> Project: Camel
> Issue Type: Bug
> Components: camel-sftp
> Affects Versions: 2.15.2
> Environment: Performance Enviroment
> Reporter: Ayush Dixit
>
> We have implemented a camel route where we are having camel sftp producer to
> transfer a files to remote SFTP location but on performance testing on client
> environment and on our local environment we have observed degradation in the
> time for transferring files to remote SFTP location. Please find the detailed
> analysis below. The we tried the various test in our local environment. In
> each test we put around 22 files on camel file consumer and each file took
> below time to write the file. PFB details
> ========================================================================================
> When target directory having 20,000 files. Camel sftp producer took around 1
> minute 43 second to a transfer file
> ========================================================================================
> DEBUG 07:00:38 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.FileConsumer> About to process file:
> GenericFile[/data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1/ST30PERFORPMAJAR17020726.txt]
> using exchange: Exchange[ST30PERFORPMAJAR17020726.txt]
> INFO 07:00:38 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> com.sigma.samp.imp.virginmedia.bss.voice.camelroutes.iopRoute.IOPResponseFtpRouteBuilder>
> Picked IOP response file : ST30PERFORPMAJAR17020726.txt from IOP FTP
> directory
> INFO 07:00:38 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> com.sigma.samp.imp.virginmedia.bss.voice.camelroutes.iopRoute.IOPResponseFtpRouteBuilder>
> Sending IOP response file : ST30PERFORPMAJAR17020726.txt to ICOMS FTP
> directory
> DEBUG 07:00:38 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.processor.SendProcessor> >>>>
> Endpoint[sftp://10.100.150.190/icoms/1?download=false&maxMessagesPerPoll=10&password=xxxxxx&tempPrefix=Q&username=sigmauser]
> Exchange[ST30PERFORPMAJAR17020726.txt]
> DEBUG 07:02:19 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.GenericFileConverter> Read file
> /data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1/ST30PERFORPMAJAR17020726.txt
> (no charset)
> DEBUG 07:02:19 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.SftpOperations> About to store file:
> QST30PERFORPMAJAR17020726.txt using stream:
> java.io.BufferedInputStream@54a89ff5
> DEBUG 07:02:19 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.SftpOperations> Took 0.658 seconds
> (658 millis) to store file: QST30PERFORPMAJAR17020726.txt and FTP client
> returned: true
> DEBUG 07:02:20 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.RemoteFileProducer> Wrote
> [icoms/1/QST30PERFORPMAJAR17020726.txt] to
> [Endpoint[sftp://10.100.150.190/icoms/1?download=false&maxMessagesPerPoll=10&password=xxxxxx&tempPrefix=Q&username=sigmauser]]
> DEBUG 07:02:20 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.SftpOperations> Renaming file:
> icoms/1/QST30PERFORPMAJAR17020726.txt to: icoms/1/ST30PERFORPMAJAR17020726.txt
> INFO 07:02:20 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> com.sigma.samp.imp.virginmedia.bss.voice.camelroutes.iopRoute.IOPResponseFtpRouteBuilder>
> IOP response file : ST30PERFORPMAJAR17020726.txt successfully sent to ICOMS
> FTP directory
> =======================================================================================
> When target directory having 40,000 files. Camel sftp producer took around 3
> minute 17 second to transfer file
> =======================================================================================
> DEBUG 07:47:23 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.FileConsumer> About to process file:
> GenericFile[/data/users/slvm02/smp53/domains/v
>
> mb/Icoms/iop_responses/1/SX30RPMAJAR1702483756.txt] using exchange:
> Exchange[SX30RPMAJAR1702483756.txt]
> INFO 07:47:23 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> com.sigma.samp.imp.virginmedia.bss.voice.camelroutes.iopRoute.IOPResponseFtpRouteBuilder>
> Picked IOP response file :
> SX30RPMAJAR1702483756.txt from IOP FTP
> directory
> INFO 07:47:23 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> com.sigma.samp.imp.virginmedia.bss.voice.camelroutes.iopRoute.IOPResponseFtpRouteBuilder>
> Sending IOP response file
> : SX30RPMAJAR1702483756.txt to ICOMS FTP
> directory
> DEBUG 07:47:23 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.processor.SendProcessor> >>>>
> Endpoint[sftp://10.100.150.190/icoms/1?download=false&maxMessagesPer
>
> Poll=10&password=xxxxxx&tempPrefix=Q&username=sigmauser]
> Exchange[SX30RPMAJAR1702483756.txt]
> DEBUG 07:50:40 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.GenericFileConverter> Read file
> /data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1/SX30RPMAJAR1702483756.txt
> (no charset)
> DEBUG 07:50:40 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.SftpOperations> About to store file:
> QSX30RPMAJAR1702483756.txt using stream: java.io.BufferedInputStream@7954cfa8
> DEBUG 07:50:40 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.SftpOperations> Took 0.659 seconds
> (659 millis) to store file: QSX30RPMAJAR1702483756.txt and FTP client
> returned: true
> DEBUG 07:50:41 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.RemoteFileProducer> Wrote
> [icoms/1/QSX30RPMAJAR1702483756.txt] to
> [Endpoint[sftp://10.100.150.190/icoms/1?download=false&maxMessagesPerPoll=10&password=xxxxxx&tempPrefix=Q&username=sigmauser]]
> DEBUG 07:50:41 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> org.apache.camel.component.file.remote.SftpOperations> Renaming file:
> icoms/1/QSX30RPMAJAR1702483756.txt to: icoms/1/SX30RPMAJAR1702483756.txt
> INFO 07:50:41 (Camel (cfgMgrCamelContext) thread #6 -
> file:///data/users/slvm02/smp53/domains/vmb/Icoms/iop_responses/1)
> com.sigma.samp.imp.virginmedia.bss.voice.camelroutes.iopRoute.IOPResponseFtpRouteBuilder>
> IOP response file : SX30RPMAJAR1702483756.txt successfully sent to ICOMS FTP
> directory
> ====================================================================================================
> Similarly when we achieved the files from target directory. It took around 6
> sec.It seems like there is a performance issue with camel sftp component.
> Does it list the files in target directory which is taking time.
> ===================================================================================================
> PFB the producer route which we set up
> sftp://10.100.150.190/icoms/1?download=false&maxMessagesPerPoll=10&password=xxxxxx&tempPrefix=Q&username=sigmauser
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
