I was looking at running an external program (wget or curl) to do this, but I need to extract some information from the HTTP header and I have not yet seen how I can return some of it.
The OS Process Sampler was a good idea (I missed that one) because I can at least match the return code from the external process, but I still need the header as well. I was thinking about making my own "web server" from Python or Ruby to call and have it do the' HTTP get, because then I could return the header as a response to the HTTP Request Sampler in JMeter. I can see in the code (HTTPSamplerBase.java) that the readResponse function does store all of the received data if MD5 is not used and that it drops the data if MD5 is used. Since I am a complete newbie in the JMeter code I do not yet know the structure, but if I could get the file name from a child Save Response to a file Listener it would be easy to add a few lines to have readResponse write to received data to a file as well. I will see if I find the time to look at this. Thank you for your help so far. Carsten. -----Oprindelig meddelelse----- Fra: sebb [mailto:[email protected]] Sendt: fr 08-06-2012 10:32 Til: JMeter Users List Emne: Re: Saving large (>500Mb) http response to a file. On 8 June 2012 08:12, Carsten Petersen <[email protected]> wrote: > Hi Sebb. > > That was what I was afraid of. > > I have not looked at the JMeter code yet, but it looks like the sampler > stores the entire > response internally before making a copy for the listener that could store > the data. Yes. > For this to work I guess the sampler must store the data itself, but that > could hurt the > modularity of JMeter. Yes, the sampler must store the data directly. Won't necessarily affect JMeter modularity. > Because of the time I have left, I might not have the time dig into this in > details through. As already mentioned, a workround would be to use the OS Process Sampler with curl or wget. Or you could use a BeanShell Sampler to load and store the file. > Thank you, > Carsten. > > > -----Oprindelig meddelelse----- > Fra: sebb [mailto:[email protected]] > Sendt: to 07-06-2012 17:41 > Til: JMeter Users List > Emne: Re: Saving large (>500Mb) http response to a file. > > On 7 June 2012 14:04, Carsten Petersen <[email protected]> wrote: >> Hi. >> >> I am trying to verify a large (>500Mb) response from a HTTP request, but >> whenever I run my TestPlan >> the thread fails after running out of memory. >> >> I have added a "Save Responses to a file" Listener, but it looks like it is >> the HTTP Request sampler that >> fails when handling all the data in memory after the reception completes - >> see error ourput below. >> >> Console window output: >> java.lang.OutOfMemoryError: Java heap space >> Dumping heap to java_pid1912.hprof ... >> Heap dump file created [558234258 bytes in 6.291 secs] >> >> JMeter log file output: >> 2012/06/07 14:51:53 ERROR - jmeter.threads.JMeterThread: Test failed! >> java.lang.OutOfMemoryError: Java heap space >> at java.util.Arrays.copyOf(Unknown Source) >> at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) >> at >> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.readResponse(HTTPSamplerBase.java:1635) >> at >> org.apache.jmeter.protocol.http.sampler.HTTPAbstractImpl.readResponse(HTTPAbstractImpl.java:236) >> at >> org.apache.jmeter.protocol.http.sampler.HTTPJavaImpl.readResponse(HTTPJavaImpl.java:288) >> at >> org.apache.jmeter.protocol.http.sampler.HTTPJavaImpl.sample(HTTPJavaImpl.java:518) >> at >> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:62) >> at >> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1060) >> at >> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1049) >> at >> org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.java:442) >> at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:271) >> at java.lang.Thread.run(Unknown Source) >> >> I have already increased the heap allocation to its max on my hardware >> (-Xmx1G), but without success. I guess I could >> add memory, but since I have to run multiple downloads at the same time, it >> does not look realistic. >> >> I can enable the "Save response as MD5 hash" option and have my TestPlan >> succeed, but then the output file from the >> run only contain the MD5 hash and since my response data is somewhat dynamic >> it is of no real use. >> >> I can see that JMeter allows a large file to be uploaded by reading it as >> needed during transmission, so I guess I need >> the reverse of this i.e. store the response to a file while receiving it >> without keeping it in memory. I guess something >> like having the "Save response as MD5 hash" option dump the data to a file >> after its calculation are done. >> >> Does anybody know how I can do this ? > > I think this will need a change to the JMeter HttpSampler code. > Listeners can only save what the sampler stores in the sample result. > > I suggest you file an enhancement request in Bugzilla. > >> Thanks in advance, >> Carsten. >> >> P.S. I am using the following JMeter: >> 2012/06/07 14:20:45 INFO - jmeter.JMeter: Version 2.7 r1342410 >> 2012/06/07 14:20:45 INFO - jmeter.JMeter: java.version=1.7.0_04 >> 2012/06/07 14:20:45 INFO - jmeter.JMeter: java.vm.name=Java HotSpot(TM) >> Client VM >> 2012/06/07 14:20:45 INFO - jmeter.JMeter: os.name=Windows 7 >> 2012/06/07 14:20:45 INFO - jmeter.JMeter: os.arch=x86 >> 2012/06/07 14:20:45 INFO - jmeter.JMeter: os.version=6.1 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
