Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
Some people may need to analyze the returned data. I am in a situation where that is not necessary, so the MD5 hash is perfect. But, if I had to more robust analysis of the data, the wget via an OS sampler seems like a viable approach. I do feel like you need to be extremely thoughtful regarding resource usage if your requirement is to scan hundreds or thousands of threads streaming umpteen GB of data per test run. I might recommend maybe rewording the control though. The reason I didn't check it initially (other than missing the line in the docs) was that I thought it provided both the MD5 hash and the saved data, not one or the other. Maybe just make it a radio button Save: O Response data O Hash of response data or something. On Thu, Jan 15, 2015 at 6:12 PM, sebb seb...@gmail.com wrote: On 15 January 2015 at 22:23, Sergio Boso ser...@bosoconsulting.it wrote: In my experience, the main problem is that Jmeter tries to keep all the response in memory, in order to support content validation (e..g. reg exp matching etc.). This obviously doesn't work for very large file like yours is. Agreed. Which is why HTTP samplers have the option to Save response as MD5 hash? http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request The only work around I found was using an OS sampler + wget command. Not the simplest thing, not always applicable, but in my case it worked out. ciao Il 13/01/2015 17.35, Colin Freas ha scritto: Oh man. I swear I read the HTTP request docs, or I thought I did. But there it is:Save response as MD5 hash? | If this is selected, then the response is not stored in the sample result. Instead, the 32 character MD5 hash of the data is calculated and stored instead. This is intended for testing large amounts of data. Thanks so much! -Colin On Tue, Jan 13, 2015 at 11:19 AM, sebb seb...@gmail.com wrote: On 13 January 2015 at 03:00, Colin Freas colinfr...@gmail.com wrote: I'm testing a REST call that streams data back in a response. JMeter works fine with small files, but when I stream a 4gb file, it just chokes, every time: ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a single byte from the response. My first thought is to just tell JMeter to discard it. Is there a way to do that? Yes, select Save response as MD5 hash? http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org -- Ing. Sergio Boso - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
On 15 January 2015 at 22:23, Sergio Boso ser...@bosoconsulting.it wrote: In my experience, the main problem is that Jmeter tries to keep all the response in memory, in order to support content validation (e..g. reg exp matching etc.). This obviously doesn't work for very large file like yours is. Agreed. Which is why HTTP samplers have the option to Save response as MD5 hash? http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request The only work around I found was using an OS sampler + wget command. Not the simplest thing, not always applicable, but in my case it worked out. ciao Il 13/01/2015 17.35, Colin Freas ha scritto: Oh man. I swear I read the HTTP request docs, or I thought I did. But there it is:Save response as MD5 hash? | If this is selected, then the response is not stored in the sample result. Instead, the 32 character MD5 hash of the data is calculated and stored instead. This is intended for testing large amounts of data. Thanks so much! -Colin On Tue, Jan 13, 2015 at 11:19 AM, sebb seb...@gmail.com wrote: On 13 January 2015 at 03:00, Colin Freas colinfr...@gmail.com wrote: I'm testing a REST call that streams data back in a response. JMeter works fine with small files, but when I stream a 4gb file, it just chokes, every time: ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a single byte from the response. My first thought is to just tell JMeter to discard it. Is there a way to do that? Yes, select Save response as MD5 hash? http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org -- Ing. Sergio Boso - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
On 13 January 2015 at 03:00, Colin Freas colinfr...@gmail.com wrote: I'm testing a REST call that streams data back in a response. JMeter works fine with small files, but when I stream a 4gb file, it just chokes, every time: ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a single byte from the response. My first thought is to just tell JMeter to discard it. Is there a way to do that? Yes, select Save response as MD5 hash? http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
Re: jmeter out of memory on 4gb streaming response (1 request, 1 thread)
Oh man. I swear I read the HTTP request docs, or I thought I did. But there it is:Save response as MD5 hash? | If this is selected, then the response is not stored in the sample result. Instead, the 32 character MD5 hash of the data is calculated and stored instead. This is intended for testing large amounts of data. Thanks so much! -Colin On Tue, Jan 13, 2015 at 11:19 AM, sebb seb...@gmail.com wrote: On 13 January 2015 at 03:00, Colin Freas colinfr...@gmail.com wrote: I'm testing a REST call that streams data back in a response. JMeter works fine with small files, but when I stream a 4gb file, it just chokes, every time: ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a single byte from the response. My first thought is to just tell JMeter to discard it. Is there a way to do that? Yes, select Save response as MD5 hash? http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
jmeter out of memory on 4gb streaming response (1 request, 1 thread)
I'm testing a REST call that streams data back in a response. JMeter works fine with small files, but when I stream a 4gb file, it just chokes, every time: ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError: Requested array size exceeds VM limit Some troubleshooting I have already tried: * modified the heap to 6gb (running on a MBP with 16gb) * running headless * no listeners These tests are just for throughput and performance. I don't need a single byte from the response. My first thought is to just tell JMeter to discard it. Is there a way to do that? If there's a different approach here, I'm open to ideas. Really any suggestions appreciated! Thanks, Colin
JMeter out of memory
Hi All, I am executing the JMeter for a script with 100 user threads for 2 hours duration and loop count set to Forever. Ramp up and ramp down in of total 100 users in one minute. The min and max Heap size of Jmeter.bat is set as 1GB. It cannot be increase to more. Whether any other way to increase the heap size. We currently have listeners for Summary Report, View Results in Table and View Results in Tree. I am getting out of memory error after 30minutes. Could you please let me know how to fix this. Thank You Regards Shenoy -- - - DISCLAIMER: This e-mail (including any attachments) is for the addressee(s) only and may contain confidential information. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message. MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e- mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law.
Re: JMeter out of memory
View Results in Table and View Results in Tree listeners consumes lot's of memory. Remove this two listeners and re-run the test again. If you are still facing the same issue then use distributed testing. On Wed, Jan 9, 2013 at 2:24 PM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: Hi All, I am executing the JMeter for a script with 100 user threads for 2 hours duration and loop count set to Forever. Ramp up and ramp down in of total 100 users in one minute. The min and max Heap size of Jmeter.bat is set as 1GB. It cannot be increase to more. Whether any other way to increase the heap size. We currently have listeners for Summary Report, View Results in Table and View Results in Tree. I am getting out of memory error after 30minutes. Could you please let me know how to fix this. Thank You Regards Shenoy -- - - DISCLAIMER: This e-mail (including any attachments) is for the addressee(s) only and may contain confidential information. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message. MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e- mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law.
RE: JMeter out of memory
I am already using distributed testing one master and 3 slaves. We need these two listeners for analysis. Thank You Regards Shenoy -Original Message- From: Niraj [mailto:niraj.khatm...@gmail.com] Sent: Wednesday, 9 January, 2013 5:02 PM To: JMeter Users List Cc: jmeter-u...@jakarta.apache.org Subject: Re: JMeter out of memory View Results in Table and View Results in Tree listeners consumes lot's of memory. Remove this two listeners and re-run the test again. If you are still facing the same issue then use distributed testing. On Wed, Jan 9, 2013 at 2:24 PM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: Hi All, I am executing the JMeter for a script with 100 user threads for 2 hours duration and loop count set to Forever. Ramp up and ramp down in of total 100 users in one minute. The min and max Heap size of Jmeter.bat is set as 1GB. It cannot be increase to more. Whether any other way to increase the heap size. We currently have listeners for Summary Report, View Results in Table and View Results in Tree. I am getting out of memory error after 30minutes. Could you please let me know how to fix this. Thank You Regards Shenoy -- - - DISCLAIMER: This e-mail (including any attachments) is for the addressee(s) only and may contain confidential information. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message. MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e- mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law. - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
Re: JMeter out of memory
That doesn't make any sense, honestly. If you want an answer to your question, you need to repeat the test in the conditions already mentioned to see if you don't have a real problem at root to this. However if the memory problem is caused by you storing thousands of requests in memory, than there is no problem and you need to search for alternatives, as there are plenty to what you are doing right now. Regards, Adrian On Wed, Jan 9, 2013 at 11:04 AM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: I am already using distributed testing one master and 3 slaves. We need these two listeners for analysis. Thank You Regards Shenoy -Original Message- From: Niraj [mailto:niraj.khatm...@gmail.com] Sent: Wednesday, 9 January, 2013 5:02 PM To: JMeter Users List Cc: jmeter-u...@jakarta.apache.org Subject: Re: JMeter out of memory View Results in Table and View Results in Tree listeners consumes lot's of memory. Remove this two listeners and re-run the test again. If you are still facing the same issue then use distributed testing. On Wed, Jan 9, 2013 at 2:24 PM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: Hi All, I am executing the JMeter for a script with 100 user threads for 2 hours duration and loop count set to Forever. Ramp up and ramp down in of total 100 users in one minute. The min and max Heap size of Jmeter.bat is set as 1GB. It cannot be increase to more. Whether any other way to increase the heap size. We currently have listeners for Summary Report, View Results in Table and View Results in Tree. I am getting out of memory error after 30minutes. Could you please let me know how to fix this. Thank You Regards Shenoy -- - - DISCLAIMER: This e-mail (including any attachments) is for the addressee(s) only and may contain confidential information. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message. MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e- mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law. - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
Re: JMeter out of memory
Hello, No you don't need these 2 listeners for analysis with 100 threads and 2 hours run. These 2 listeners are to be used during scripting debug with 1 to 5 users max and with few seconds to very few minutes of run, once your script is debugged and OK, you must remove them and absolutely not use them during load test. With 100 threads, I doubt you really need distributed testing Read this: - http://jmeter.apache.org/usermanual/best-practices.html - http://www.ubik-ingenierie.com/blog/jmeter_performance_tuning_tips/ RegardsPhilippe M. On Wed, Jan 9, 2013 at 10:10 AM, Niraj niraj.khatm...@gmail.com wrote: Then use non gui mode. But having these two listeners in your test plan will consumes lot's of memory. I am not sure why do you need both the listeners? Are you analyzing all the requests responses for 100 users? On Wed, Jan 9, 2013 at 2:34 PM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: I am already using distributed testing one master and 3 slaves. We need these two listeners for analysis. Thank You Regards Shenoy -Original Message- From: Niraj [mailto:niraj.khatm...@gmail.com] Sent: Wednesday, 9 January, 2013 5:02 PM To: JMeter Users List Cc: jmeter-u...@jakarta.apache.org Subject: Re: JMeter out of memory View Results in Table and View Results in Tree listeners consumes lot's of memory. Remove this two listeners and re-run the test again. If you are still facing the same issue then use distributed testing. On Wed, Jan 9, 2013 at 2:24 PM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: Hi All, I am executing the JMeter for a script with 100 user threads for 2 hours duration and loop count set to Forever. Ramp up and ramp down in of total 100 users in one minute. The min and max Heap size of Jmeter.bat is set as 1GB. It cannot be increase to more. Whether any other way to increase the heap size. We currently have listeners for Summary Report, View Results in Table and View Results in Tree. I am getting out of memory error after 30minutes. Could you please let me know how to fix this. Thank You Regards Shenoy -- - - DISCLAIMER: This e-mail (including any attachments) is for the addressee(s) only and may contain confidential information. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message. MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e- mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law. - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org -- Cordialement. Philippe Mouawad.
Re: JMeter out of memory
An alternative is to use those two listeners, but only capture Errors. I can't imagine any value (for analysis) of reviewing thousands of successful request/responses (can you say data overload?)... But if you check the little box on the listener to just capture errors, then those listeners are quite useful. Of course, you have to make sure that your test returns an error when something goes awry. You can use Assertions for this. In our tests, each request has at least one assert associated with it, looking in the response for some string that should be present if the request was successful, but absent if it fails. This assures we get an error logged in the Tree listener when a request does do what we expected it to do. -- Robin D. Wilson VOICE: 512-777-1861 On Jan 9, 2013, at 3:11 AM, Philippe Mouawad philippe.moua...@gmail.com wrote: Hello, No you don't need these 2 listeners for analysis with 100 threads and 2 hours run. These 2 listeners are to be used during scripting debug with 1 to 5 users max and with few seconds to very few minutes of run, once your script is debugged and OK, you must remove them and absolutely not use them during load test. With 100 threads, I doubt you really need distributed testing Read this: - http://jmeter.apache.org/usermanual/best-practices.html - http://www.ubik-ingenierie.com/blog/jmeter_performance_tuning_tips/ RegardsPhilippe M. On Wed, Jan 9, 2013 at 10:10 AM, Niraj niraj.khatm...@gmail.com wrote: Then use non gui mode. But having these two listeners in your test plan will consumes lot's of memory. I am not sure why do you need both the listeners? Are you analyzing all the requests responses for 100 users? On Wed, Jan 9, 2013 at 2:34 PM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: I am already using distributed testing one master and 3 slaves. We need these two listeners for analysis. Thank You Regards Shenoy -Original Message- From: Niraj [mailto:niraj.khatm...@gmail.com] Sent: Wednesday, 9 January, 2013 5:02 PM To: JMeter Users List Cc: jmeter-u...@jakarta.apache.org Subject: Re: JMeter out of memory View Results in Table and View Results in Tree listeners consumes lot's of memory. Remove this two listeners and re-run the test again. If you are still facing the same issue then use distributed testing. On Wed, Jan 9, 2013 at 2:24 PM, Srinivasa Tekkatte Shenoy tsrinivas.she...@mimos.my wrote: Hi All, I am executing the JMeter for a script with 100 user threads for 2 hours duration and loop count set to Forever. Ramp up and ramp down in of total 100 users in one minute. The min and max Heap size of Jmeter.bat is set as 1GB. It cannot be increase to more. Whether any other way to increase the heap size. We currently have listeners for Summary Report, View Results in Table and View Results in Tree. I am getting out of memory error after 30minutes. Could you please let me know how to fix this. Thank You Regards Shenoy -- - - DISCLAIMER: This e-mail (including any attachments) is for the addressee(s) only and may contain confidential information. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message. MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e- mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law. - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org -- Cordialement. Philippe Mouawad. - To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
