SV: Upper file limit in SMB protocol ?

Tue, 11 Mar 2025 10:18:09 -0700

Brilliant. Thank you very much Claus. 

Venlig hilsen / Best regards

Rolf Sommer Poulsen
Senior Software Developer, FRI Integration 
+45 6012 5444 | +45 7924 2811
r...@bankdata.dk
BANKDATA | Development Centre Silkeborg | Stagehøjvej 30 | 8600 ;Silkeborg -----Oprindelig meddelelse----- Fra: Claus Ibsen <claus.ib...@gmail.com> Sendt: 6. marts 2025 09:28 
Til: users@camel.apache.org
Emne: Re: Upper file limit in SMB protocol ?

[Du får ikke ofte mails fra claus.ib...@gmail.com. Få mere at vide om, hvorfor 
dette er vigtigt, på https://aka.ms/LearnAboutSenderIdentification ]

Hi

Ah okay read about stream caching
https://camel.apache.org/manual/stream-caching.html

You can turn that off or enable its spool to disk option

On Thu, Mar 6, 2025 at 8:17 AM Rolf Sommer Poulsen <r...@bankdata.dk.invalid>
wrote:


Hi

I am currently using the 4.10.1 release and get the following stacktrace:
java.lang.OutOfMemoryError: Required array length 2147483639 + 9 is too large 
        at
java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.jav
a:649)
~[na:na]
        at
java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java
:642)
~[na:na]
at java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutput 
Stream.java:100)
~[na:na]
at java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.ja 
va:130)
~[na:na]
        at
org.apache.camel.converter.stream.CachedOutputStream.write(CachedOutpu
tStream.java:108)
~[camel-support-4.10.1.jar:4.10.1]
at java.base/java.io.InputStream.transferTo(InputStream.java:783) 
~[na:na]
        at org.apache.camel.util.IOHelper.copy(IOHelper.java:165)
~[camel-util-4.10.1.jar:4.10.1]
        at
org.apache.camel.util.IOHelper.copyAndCloseInput(IOHelper.java:283)
~[camel-util-4.10.1.jar:4.10.1]
        at
org.apache.camel.converter.stream.StreamCacheConverter.convertToStream
Cache(StreamCacheConverter.java:54)
~[camel-support-4.10.1.jar:4.10.1]
        at
org.apache.camel.converter.stream.StreamCacheBulkConverterLoader.doCon
vertTo(StreamCacheBulkConverterLoader.java:83)
~[camel-support-4.10.1.jar:4.10.1]
        at
org.apache.camel.converter.stream.StreamCacheBulkConverterLoader.conve
rtTo(StreamCacheBulkConverterLoader.java:56)
~[camel-support-4.10.1.jar:4.10.1]
        at
org.apache.camel.spi.BulkTypeConverters.convertTo(BulkTypeConverters.j
ava:122)
~[camel-api-4.10.1.jar:4.10.1]
        at
org.apache.camel.component.file.GenericFileConverter.convertTo(Generic
FileConverter.java:106)
~[camel-file-4.10.1.jar:4.10.1]
        at
org.apache.camel.component.file.GenericFileConverterLoader.lambda$regi
sterFallbackConverters$4(GenericFileConverterLoader.java:63)
~[camel-file-4.10.1.jar:4.10.1]
        at
org.apache.camel.support.SimpleTypeConverter.convertTo(SimpleTypeConve
rter.java:101)
~[camel-support-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.doConvert(Co
reTypeConverterRegistry.java:463)
~[camel-base-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.tryFallback(
CoreTypeConverterRegistry.java:432)
~[camel-base-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.doConvertTo(
CoreTypeConverterRegistry.java:381)
~[camel-base-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.doConvertToA
ndStat(CoreTypeConverterRegistry.java:269)
~[camel-base-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.convertTo(Co
reTypeConverterRegistry.java:164)
~[camel-base-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.DefaultStreamCachingStrategy.doCache(Defa
ultStreamCachingStrategy.java:278)
~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.DefaultStreamCachingStrategy.cache(Defaul
tStreamCachingStrategy.java:255) ~[camel-base-engine-4.10.1.jar:4.10.1] 
        at
org.apache.camel.impl.engine.StreamCachingHelper.tryStreamCache(Stream
CachingHelper.java:67) ~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.StreamCachingHelper.convertToStreamCache(
StreamCachingHelper.java:54) ~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.CamelInternalProcessor$StreamCachingAdvic
e.before(CamelInternalProcessor.java:1010)
~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.CamelInternalProcessor$StreamCachingAdvic
e.before(CamelInternalProcessor.java:1000)
~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInter
nalProcessor.java:310) ~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.processor.Pipeline$PipelineTask.run(Pipeline.java:102
) ~[camel-core-processor-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.doRun(Defa
ultReactiveExecutor.java:199) ~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.executeRea
ctiveWork(DefaultReactiveExecutor.java:189)
~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.tryExecute
ReactiveWork(DefaultReactiveExecutor.java:166)
~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(D
efaultReactiveExecutor.java:148) ~[camel-base-engine-4.10.1.jar:4.10.1] 
        at
org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(Defa
ultReactiveExecutor.java:59) ~[camel-base-engine-4.10.1.jar:4.10.1]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:163) 
~[camel-core-processor-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.CamelInternalProcessor.processNonTransact
ed(CamelInternalProcessor.java:347)
~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInter
nalProcessor.java:323) ~[camel-base-engine-4.10.1.jar:4.10.1]
        at
org.apache.camel.component.file.GenericFileConsumer.processExchange(Ge
nericFileConsumer.java:459)
~[camel-file-4.10.1.jar:4.10.1]
        at
org.apache.camel.component.smb.SmbConsumer.processExchange(SmbConsumer
.java:309)
~[camel-smb-4.10.1.jar:4.10.1]
        at
org.apache.camel.component.file.GenericFileConsumer.processBatch(Gener
icFileConsumer.java:245)
~[camel-file-4.10.1.jar:4.10.1]
        at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileCo
nsumer.java:206)
~[camel-file-4.10.1.jar:4.10.1]
        at
org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollCons
umer.java:208)
~[camel-support-4.10.1.jar:4.10.1]
        at
org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsum
er.java:119)
~[camel-support-4.10.1.jar:4.10.1]
        at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable
.run(DelegatingErrorHandlingRunnable.java:54)
~[spring-context-6.2.2.jar:6.2.2]
        at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(Res
chedulingRunnable.java:96)
~[spring-context-6.2.2.jar:6.2.2]
        at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executor
s.java:539)
~[na:na]
        at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
        at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
tureTask.run(ScheduledThreadPoolExecutor.java:304)
~[na:na]
        at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1136)
~[na:na]
        at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:635)
~[na:na]
        at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]


Venlig hilsen / Best regards

Rolf Sommer Poulsen
Senior Software Developer,
FRI Integration
+45 6012 5444 | +45 7924 2811
r...@bankdata.dk
BANKDATA | Development Centre Silkeborg | Stagehøjvej 30 | 8600 ;Silkeborg -----Oprindelig meddelelse----- 
Fra: Claus Ibsen <claus.ib...@gmail.com>
Sendt: 6. marts 2025 08:09
Til: users@camel.apache.org
Emne: Re: Upper file limit in SMB protocol ?
[Du får ikke ofte mails fra claus.ib...@gmail.com. Få mere at vide om, hvorfor dette er vigtigt, på https://aka.ms/LearnAboutSenderIdentification 
]

Hi
I am not aware of a limit. But can you try with latest 4.10.1 release and also if you see any exception then report the stacktrace
On Thu, Mar 6, 2025 at 8:00 AM Rolf Sommer Poulsen <r...@bankdata.dk.invalid 
>
wrote:

> Hi
>
> I was wondering if there is a upper bound on the file size the > camel-smb component I capable of handling. I have set up a route, > that streamdownload a file from a windows server using the > smb-component, and it works fine with normal sized files, however > when testing on a very large file (~8gb) I get and error about an > array that has grown to large even before my route is being called. 
>
> That made me wonder if there is a upper bound on the size of the > files that the smb-component can handle and if that size is around > INTEGER.MAX_VALUE bytes ? 
>
>
> Venlig hilsen / Best regards
>
> Rolf Sommer Poulsen
>
>
>
>

--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www/. 
manning.com%2Fibsen2&data=05%7C02%7Crsp%40bankdata.dk%7Cec9462e067c845
efda5d08dd5c88fbcd%7C1177168698fc4f1b8e0076c52159e8a6%7C0%7C0%7C638768
465582935182%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL
jAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%
7C&sdata=vQOPORJJ%2BZnXv6262%2FTVHbpoMFQrM%2FlWeCYh%2B%2Fsc30k%3D&rese
rved=0




--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Reply via email to