Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
Some people may need to analyze the returned data. I am in a situation where that is not necessary, so the MD5 hash is perfect. But, if I had to more robust analysis of the data, the wget via an OS sampler seems like a viable approach. I do feel like you need to be extremely thoughtful regarding resource usage if your requirement is to scan hundreds or thousands of threads streaming umpteen GB of data per test run. I might recommend maybe rewording the control though. The reason I didn't check it initially (other than missing the line in the docs) was that I thought it provided both the MD5 hash and the saved data, not one or the other. Maybe just make it a radio button "Save: O Response data O Hash of response data" or something. On Thu, Jan 15, 2015 at 6:12 PM, sebb wrote: > On 15 January 2015 at 22:23, Sergio Boso wrote: > > In my experience, > > > > the main problem is that Jmeter tries to keep all the response in > memory, in > > order to support content validation (e..g. reg exp matching etc.). > > This obviously doesn't work for very large file like yours is. > > Agreed. > > Which is why HTTP samplers have the option to "Save response as MD5 hash?" > > http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request > > > The only work around I found was using an OS sampler + wget command. > > Not the simplest thing, not always applicable, but in my case it worked > out. > > > ciao > > > > > > Il 13/01/2015 17.35, Colin Freas ha scritto: > > > >> Oh man. I swear I read the HTTP request docs, or I thought I did. But > >> there it is:"Save response as MD5 hash? | If this is selected, then the > >> response is not stored in the sample result. Instead, the 32 character > MD5 > >> hash of the data is calculated and stored instead. This is intended for > >> testing large amounts of data." > >> > >> Thanks so much! > >> > >> -Colin > >> > >> On Tue, Jan 13, 2015 at 11:19 AM, sebb wrote: > >> > >>> On 13 January 2015 at 03:00, Colin Freas wrote: > > I'm testing a REST call that streams data back in a response. JMeter > >>> > >>> works > > fine with small files, but when I stream a 4gb file, it just chokes, > >>> > >>> every > > time: "ERROR - jmeter.threads.JMeterThread: Test failed! > java.lang.OutOfMemoryError: Requested array size exceeds VM limit" > > Some troubleshooting I have already tried: > * modified the heap to 6gb (running on a MBP with 16gb) > * running headless > * no listeners > > These tests are just for throughput and performance. I don't need a > >>> > >>> single > > byte from the response. My first thought is to just tell JMeter to > >>> > >>> discard > > it. Is there a way to do that? > >>> > >>> Yes, select "Save response as MD5 hash?" > >>> > >>> > http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request > >>> > If there's a different approach here, I'm open to ideas. Really any > suggestions appreciated! > > Thanks, > Colin > >>> > >>> - > >>> To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org > >>> For additional commands, e-mail: user-h...@jmeter.apache.org > > > > -- > > > > Ing. Sergio Boso > > > > > > > > > > > > - > > To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org > > For additional commands, e-mail: user-h...@jmeter.apache.org > > > > - > To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org > For additional commands, e-mail: user-h...@jmeter.apache.org > >
Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
On 15 January 2015 at 22:23, Sergio Boso wrote: > In my experience, > > the main problem is that Jmeter tries to keep all the response in memory, in > order to support content validation (e..g. reg exp matching etc.). > This obviously doesn't work for very large file like yours is. Agreed. Which is why HTTP samplers have the option to "Save response as MD5 hash?" http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request > The only work around I found was using an OS sampler + wget command. > Not the simplest thing, not always applicable, but in my case it worked out. > ciao > > > Il 13/01/2015 17.35, Colin Freas ha scritto: > >> Oh man. I swear I read the HTTP request docs, or I thought I did. But >> there it is:"Save response as MD5 hash? | If this is selected, then the >> response is not stored in the sample result. Instead, the 32 character MD5 >> hash of the data is calculated and stored instead. This is intended for >> testing large amounts of data." >> >> Thanks so much! >> >> -Colin >> >> On Tue, Jan 13, 2015 at 11:19 AM, sebb wrote: >> >>> On 13 January 2015 at 03:00, Colin Freas wrote: I'm testing a REST call that streams data back in a response. JMeter >>> >>> works fine with small files, but when I stream a 4gb file, it just chokes, >>> >>> every time: "ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit" Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a >>> >>> single byte from the response. My first thought is to just tell JMeter to >>> >>> discard it. Is there a way to do that? >>> >>> Yes, select "Save response as MD5 hash?" >>> >>> http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request >>> If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin >>> >>> - >>> To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org >>> For additional commands, e-mail: user-h...@jmeter.apache.org > > -- > > Ing. Sergio Boso > > > > > > - > To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org > For additional commands, e-mail: user-h...@jmeter.apache.org > - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
In my experience, the main problem is that Jmeter tries to keep all the response in memory, in order to support content validation (e..g. reg exp matching etc.). This obviously doesn't work for very large file like yours is. The only work around I found was using an OS sampler + wget command. Not the simplest thing, not always applicable, but in my case it worked out. ciao Il 13/01/2015 17.35, Colin Freas ha scritto: Oh man. I swear I read the HTTP request docs, or I thought I did. But there it is:"Save response as MD5 hash? | If this is selected, then the response is not stored in the sample result. Instead, the 32 character MD5 hash of the data is calculated and stored instead. This is intended for testing large amounts of data." Thanks so much! -Colin On Tue, Jan 13, 2015 at 11:19 AM, sebb wrote: On 13 January 2015 at 03:00, Colin Freas wrote: I'm testing a REST call that streams data back in a response. JMeter works fine with small files, but when I stream a 4gb file, it just chokes, every time: "ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit" Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a single byte from the response. My first thought is to just tell JMeter to discard it. Is there a way to do that? Yes, select "Save response as MD5 hash?" http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org -- Ing. Sergio Boso - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
Oh man. I swear I read the HTTP request docs, or I thought I did. But there it is:"Save response as MD5 hash? | If this is selected, then the response is not stored in the sample result. Instead, the 32 character MD5 hash of the data is calculated and stored instead. This is intended for testing large amounts of data." Thanks so much! -Colin On Tue, Jan 13, 2015 at 11:19 AM, sebb wrote: > On 13 January 2015 at 03:00, Colin Freas wrote: > > I'm testing a REST call that streams data back in a response. JMeter > works > > fine with small files, but when I stream a 4gb file, it just chokes, > every > > time: "ERROR - jmeter.threads.JMeterThread: Test failed! > > java.lang.OutOfMemoryError: Requested array size exceeds VM limit" > > > > Some troubleshooting I have already tried: > > * modified the heap to 6gb (running on a MBP with 16gb) > > * running headless > > * no listeners > > > > These tests are just for throughput and performance. I don't need a > single > > byte from the response. My first thought is to just tell JMeter to > discard > > it. Is there a way to do that? > > Yes, select "Save response as MD5 hash?" > > http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request > > > If there's a different approach here, I'm open to ideas. Really any > > suggestions appreciated! > > > > Thanks, > > Colin > > - > To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org > For additional commands, e-mail: user-h...@jmeter.apache.org > >
Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
On 13 January 2015 at 03:00, Colin Freas wrote: > I'm testing a REST call that streams data back in a response. JMeter works > fine with small files, but when I stream a 4gb file, it just chokes, every > time: "ERROR - jmeter.threads.JMeterThread: Test failed! > java.lang.OutOfMemoryError: Requested array size exceeds VM limit" > > Some troubleshooting I have already tried: > * modified the heap to 6gb (running on a MBP with 16gb) > * running headless > * no listeners > > These tests are just for throughput and performance. I don't need a single > byte from the response. My first thought is to just tell JMeter to discard > it. Is there a way to do that? Yes, select "Save response as MD5 hash?" http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request > If there's a different approach here, I'm open to ideas. Really any > suggestions appreciated! > > Thanks, > Colin - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
jmeter out of memory on 4gb streaming response (1 request, 1 thread)
I'm testing a REST call that streams data back in a response. JMeter works fine with small files, but when I stream a 4gb file, it just chokes, every time: "ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit" Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a single byte from the response. My first thought is to just tell JMeter to discard it. Is there a way to do that? If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin
