Re: [matplotlib-devel] Memory leaks in Basemap
Aalok kapoor wrote:
Thanks for reply,
This was just an example for mem leak in my application. Actually i
want maps with different data to be loaded each time and the map
objects taken once changees after ploting so i have to take different
object each time. After around 20 maps generation it gives memory
error to me.
thanks,
-Aalok
*/Jeff Whitaker [EMAIL PROTECTED]/* wrote:
Aalok kapoor wrote:
Hi all,
I am using matplotlib-0.87.7, Basemap-0.9.3, Numpy-1.1 and agg
backend.
I am facing memory leak problems after running following script -
import os, sys, time
import cStringIO
import Image
import matplotlib
matplotlib.use('Agg')
import matplotlib.pylab as p
import matplotlib.numerix as nx
from matplotlib.toolkits.basemap import Basemap
def get_image_bytes(width_fig, height_fig, str_img):
Returns the bytes representing the image generated
The createImage method should be called before calling this method.
:Returns:
Byte data.
# Get the actual image data
f = cStringIO.StringIO()
dims = (width_fig, height_fig)
im = Image.fromstring(RGB, dims, str_img)
im.save(f, JPEG)
return f.getvalue()
def report_memory(i):
pid = os.getpid()
a2 = os.popen('ps -p %d -o rss,vsz,%%mem' % pid).readlines()
print i, ' ', a2[1],
return int(a2[1].split()[1])
# take a memory snapshot on indStart and compare it with indEnd
indStart, indEnd = 30, 150
print ' rss vsz %mem'
for i in range(indEnd):
p.figure(figsize=(float(640)/80.0,float(480)/80.0),
facecolor='w', edgecolor='w')
p.axes([0.0, 0.0, 1.0, 1.0])
map1 = Basemap(projection='cyl', lat_0=50, lon_0=-100,
area_thresh=1000.)
# draw coastlines, country boundaries, fill continents.
map1.drawcoastlines(linewidth=.5)
map1.drawcountries()
map1.fillcontinents(color=#CEFFCE)
# draw the edge of the map projection region (the projection limb)
map1.drawmapboundary()
# compute the native map projection coordinates for cities.
lats, lons = ([18.728], [20.890])
x,y = map1(lons,lats)
# plot filled circles at the locations of the cities.
map1.plot(x,y,'bo')
canvas = p.get_current_fig_manager().canvas
agg = canvas.switch_backends \
(matplotlib.backends.backend_agg.FigureCanvasAgg)
agg.draw() # Takes ~.9 seconds
# get the image data
str_img = agg.tostring_rgb()
width_fig, height_fig = map(int, agg.figure.bbox.get_bounds()[2:])
p.cla()
p.close()
bytes = get_image_bytes(width_fig, height_fig, str_img)
val = report_memory(i)
# wait a few cycles for memory usage to stabilize
if i==indStart: start = val
end = val
print 'Average memory consumed per loop: %1.4fk bytes' % \
((end-start)/float(indEnd-indStart))
Here is the O/P
$python memory_leak_map.py
rss vsz %mem
0 47272 50632 9.7
1 74412 77700 15.3
2 93960 97380 19.3
3 113308 116776 23.3
4 132824 136416 27.3
5 152352 155828 31.3
6 171860 175216 35.3
7 191372 194868 39.3
8 210872 214248 43.3
9 230336 233916 47.3
10 249732 253284 51.3
11 269252 272692 55.3
12 288680 292336 59.3
13 308108 311724 63.2
14 305160 331112 62.6
15 301096 350764 61.8
16 304884 370160 62.6
17 298276 389804 61.2
18 305876 409184 62.8
19 298316 428596 61.2
20 307856 448224 63.2
21 308004 467640 63.2
22 308844 487016 63.4
23 306260 506656 62.9
24 300612 526052 61.7
Traceback (most recent call last):
File memory_leak_map.py, line 41, in ?
area_thresh=1000.)
File
/usr/local/lib/python2.4/site-packages/matplotlib/toolkits/basemap/basemap.py,
line 815, in __init__
self.riversegs = segments+segments2+segments3
MemoryError
Killed: 9
Can someone please help me out solving this problem?
Thanks
-Aalok
Aaolk: All your plots use the same map projection, yet you are
initializing a new Basemap instance every time through the loop. Try
moving the line
map1 = Basemap(projection='cyl', lat_0=50, lon_0=-100,
area_thresh=1000.)
outside the loop.
-Jeff
Aaolk: Here's what I get with matplotlib 0.87.7, basemap 0.9.4, numpy
1.0 on MacOS 10.4.9 with python 2.5 (python 2.4 gives the same result):
[mac28:~/matplotlib/toolkits/basemap] jsw% python memleak.py
rss vsz %mem
028248 153200 -1.3
134028 157316 -1.6
234908 158084 -1.7
335688 158852 -1.7
436200 159364 -1.7
536712 159876 -1.8
636456 159620 -1.7
736712 159876 -1.8
836712
Re: [matplotlib-devel] Memory leaks in Basemap
After using numpy-1.0 the results are more bad. After 15 maps it is reaching to
71% memory usage. I am working on freeBSD box.
python memory_leak_map.py
rss vsz%mem
047940 50636 9.8
175080 77700 15.4
294636 97380 19.4
3114156 116776 23.4
4133672 136416 27.4
5153204 155832 31.4
6172708 175216 35.5
7192224 194872 39.5
8211720 214248 43.5
9231244 233916 47.5
10250732 253280 51.5
11270252 272688 55.5
12289756 292332 59.5
13309272 311728 63.5
14328764 331108 67.5
15347804 350760 71.4
thanks,
-Aalok
Aalok kapoor [EMAIL PROTECTED] wrote: Hi,
Thanks for this direction. I am using Numpy-1.1. Do I not supposed to use this
version? I will try to use older version and see what happens. I hope this will
solve the problem.
thanks
-Aalok
Jeff Whitaker [EMAIL PROTECTED] wrote: Aalok kapoor wrote:
Thanks for reply,
This was just an example for mem leak in my application. Actually i
want maps with different data to be loaded each time and the map
objects taken once changees after ploting so i have to take different
object each time. After around 20 maps generation it gives memory
error to me.
thanks,
-Aalok
*/Jeff Whitaker /* wrote:
Aalok kapoor wrote:
Hi all,
I am using matplotlib-0.87.7, Basemap-0.9.3, Numpy-1.1 and agg
backend.
I am facing memory leak problems after running following script -
import os, sys, time
import cStringIO
import Image
import matplotlib
matplotlib.use('Agg')
import matplotlib.pylab as p
import matplotlib.numerix as nx
from matplotlib.toolkits.basemap import Basemap
def get_image_bytes(width_fig, height_fig, str_img):
Returns the bytes representing the image generated
The createImage method should be called before calling this method.
:Returns:
Byte data.
# Get the actual image data
f = cStringIO.StringIO()
dims = (width_fig, height_fig)
im = Image.fromstring(RGB, dims, str_img)
im.save(f, JPEG)
return f.getvalue()
def report_memory(i):
pid = os.getpid()
a2 = os.popen('ps -p %d -o rss,vsz,%%mem' % pid).readlines()
print i, ' ', a2[1],
return int(a2[1].split()[1])
# take a memory snapshot on indStart and compare it with indEnd
indStart, indEnd = 30, 150
print ' rss vsz %mem'
for i in range(indEnd):
p.figure(figsize=(float(640)/80.0,float(480)/80.0),
facecolor='w', edgecolor='w')
p.axes([0.0, 0.0, 1.0, 1.0])
map1 = Basemap(projection='cyl', lat_0=50, lon_0=-100,
area_thresh=1000.)
# draw coastlines, country boundaries, fill continents.
map1.drawcoastlines(linewidth=.5)
map1.drawcountries()
map1.fillcontinents(color=#CEFFCE)
# draw the edge of the map projection region (the projection limb)
map1.drawmapboundary()
# compute the native map projection coordinates for cities.
lats, lons = ([18.728], [20.890])
x,y = map1(lons,lats)
# plot filled circles at the locations of the cities.
map1.plot(x,y,'bo')
canvas = p.get_current_fig_manager().canvas
agg = canvas.switch_backends \
(matplotlib.backends.backend_agg.FigureCanvasAgg)
agg.draw() # Takes ~.9 seconds
# get the image data
str_img = agg.tostring_rgb()
width_fig, height_fig = map(int, agg.figure.bbox.get_bounds()[2:])
p.cla()
p.close()
bytes = get_image_bytes(width_fig, height_fig, str_img)
val = report_memory(i)
# wait a few cycles for memory usage to stabilize
if i==indStart: start = val
end = val
print 'Average memory consumed per loop: %1.4fk bytes' % \
((end-start)/float(indEnd-indStart))
Here is the O/P
$python memory_leak_map.py
rss vsz %mem
0 47272 50632 9.7
1 74412 77700 15.3
2 93960 97380 19.3
3 113308 116776 23.3
4 132824 136416 27.3
5 152352 155828 31.3
6 171860 175216 35.3
7 191372 194868 39.3
8 210872 214248 43.3
9 230336 233916 47.3
10 249732 253284 51.3
11 269252 272692 55.3
12 288680 292336 59.3
13 308108 311724 63.2
14 305160 331112 62.6
15 301096 350764 61.8
16 304884 370160 62.6
17 298276 389804 61.2
18 305876 409184 62.8
19 298316 428596 61.2
20 307856 448224 63.2
21 308004 467640 63.2
22 308844 487016 63.4
23 306260 506656 62.9
24 300612 526052 61.7
Traceback (most recent call last):
File memory_leak_map.py, line 41, in ?
area_thresh=1000.)
File
/usr/local/lib/python2.4/site-packages/matplotlib/toolkits/basemap/basemap.py,
line 815, in __init__
self.riversegs = segments+segments2+segments3
MemoryError
Re: [matplotlib-devel] Memory leaks in Basemap
Jeff and Aalok,
I am in central MO and it doesn't bother me but, for some unknown reason I'm
recieving correspondense between you two. You might want to start a new email
instead of replying to each other. Have a nice day. Curt Bridges
Aalok kapoor [EMAIL PROTECTED] wrote:
After using numpy-1.0 the results are more bad. After 15 maps it is reaching
to 71% memory usage. I am working on freeBSD box.
python memory_leak_map.py
rss vsz%mem
047940 50636 9.8
175080 77700 15.4
294636 97380 19.4
3114156 116776 23.4
4133672 136416 27.4
5153204 155832 31.4
6172708 175216 35.5
7192224 194872 39.5
8211720 214248 43.5
9231244 233916 47.5
10250732 253280 51.5
11270252 272688 55.5
12289756 292332 59.5
13309272 311728 63.5
14328764 331108 67.5
15347804 350760 71.4
thanks,
-Aalok
Aalok kapoor [EMAIL PROTECTED] wrote: Hi,
Thanks for this direction. I am using Numpy-1.1. Do I not supposed to use this
version? I will try to use older version and see what happens. I hope this will
solve the problem.
thanks
-Aalok
Jeff Whitaker [EMAIL PROTECTED] wrote: Aalok kapoor wrote:
Thanks for reply,
This was just an example for mem leak in my application. Actually i
want maps with different data to be loaded each time and the map
objects taken once changees after ploting so i have to take different
object each time. After around 20 maps generation it gives memory
error to me.
thanks,
-Aalok
*/Jeff Whitaker /* wrote:
Aalok kapoor wrote:
Hi all,
I am using matplotlib-0.87.7, Basemap-0.9.3, Numpy-1.1 and agg
backend.
I am facing memory leak problems after running following script -
import os, sys, time
import cStringIO
import Image
import matplotlib
matplotlib.use('Agg')
import matplotlib.pylab as p
import matplotlib.numerix as nx
from matplotlib.toolkits.basemap import Basemap
def get_image_bytes(width_fig, height_fig, str_img):
Returns the bytes representing the image generated
The createImage method should be called before calling this method.
:Returns:
Byte data.
# Get the actual image data
f = cStringIO.StringIO()
dims = (width_fig, height_fig)
im = Image.fromstring(RGB, dims, str_img)
im.save(f, JPEG)
return f.getvalue()
def report_memory(i):
pid = os.getpid()
a2 = os.popen('ps -p %d -o rss,vsz,%%mem' % pid).readlines()
print i, ' ', a2[1],
return int(a2[1].split()[1])
# take a memory snapshot on indStart and compare it with indEnd
indStart, indEnd = 30, 150
print ' rss vsz %mem'
for i in range(indEnd):
p.figure(figsize=(float(640)/80.0,float(480)/80.0),
facecolor='w', edgecolor='w')
p.axes([0.0, 0.0, 1.0, 1.0])
map1 = Basemap(projection='cyl', lat_0=50, lon_0=-100,
area_thresh=1000.)
# draw coastlines, country boundaries, fill continents.
map1.drawcoastlines(linewidth=.5)
map1.drawcountries()
map1.fillcontinents(color=#CEFFCE)
# draw the edge of the map projection region (the projection limb)
map1.drawmapboundary()
# compute the native map projection coordinates for cities.
lats, lons = ([18.728], [20.890])
x,y = map1(lons,lats)
# plot filled circles at the locations of the cities.
map1.plot(x,y,'bo')
canvas = p.get_current_fig_manager().canvas
agg = canvas.switch_backends \
(matplotlib.backends.backend_agg.FigureCanvasAgg)
agg.draw() # Takes ~.9 seconds
# get the image data
str_img = agg.tostring_rgb()
width_fig, height_fig = map(int, agg.figure.bbox.get_bounds()[2:])
p.cla()
p.close()
bytes = get_image_bytes(width_fig, height_fig, str_img)
val = report_memory(i)
# wait a few cycles for memory usage to stabilize
if i==indStart: start = val
end = val
print 'Average memory consumed per loop: %1.4fk bytes' % \
((end-start)/float(indEnd-indStart))
Here is the O/P
$python memory_leak_map.py
rss vsz %mem
0 47272 50632 9.7
1 74412 77700 15.3
2 93960 97380 19.3
3 113308 116776 23.3
4 132824 136416 27.3
5 152352 155828 31.3
6 171860 175216 35.3
7 191372 194868 39.3
8 210872 214248 43.3
9 230336 233916 47.3
10 249732 253284 51.3
11 269252 272692 55.3
12 288680 292336 59.3
13 308108 311724 63.2
14 305160 331112 62.6
15 301096 350764 61.8
16 304884 370160 62.6
17 298276 389804 61.2
18 305876 409184 62.8
19 298316 428596 61.2
20 307856 448224 63.2
21 308004 467640 63.2
22 308844 487016 63.4
23 306260 506656 62.9
24 300612 526052 61.7
Traceback (most recent call last):
File memory_leak_map.py, line 41, in ?
area_thresh=1000.)
File
/usr/local/lib/python2.4/site-packages/matplotlib/toolkits/basemap/basemap.py,
line 815, in __init__
self.riversegs = segments+segments2+segments3
MemoryError
Killed: 9
Can someone please help me out solving this problem?
Thanks
-Aalok
Aaolk: All your plots use the same map projection, yet you are
initializing a new Basemap instance every time
