Re: [ovs-dev] [PATCH] [python] Avoid alloc of an attr dict/row+column

2021-11-08 Thread Ilya Maximets 
On 10/13/21 16:34, Timothy Redaelli wrote:
> On Tue, 12 Oct 2021 14:13:31 -0500
> Terry Wilson  wrote:
> 
>> Python objects normally have a dictionary named __dict__ allocated
>> for handling dynamically assigned attributes. Depending on
>> architecture and Python version, that empty dict may be between
>> 64 and 280 bytes.
>>
>> Seeing as Atom and Datum objects do not need dynamic attribute
>> support and there can be millions of rows in a database, avoiding
>> this allocation with __slots__ can save 100s of MBs of memory per
>> Idl process.
>>
>> Signed-off-by: Terry Wilson 
> 
> It reduce the memory usage and it's also faster:
> 
> print(min(timeit.repeat(lambda:
> data.Atom.default(ovs.db.types.ATOMIC_TYPES[1]
> 
> 0.7460950060049072 (w/o patch) vs 0.6909653190232348 (with patch)
> 
> 
> [data.Atom.default(ovs.db.types.ATOMIC_TYPES[1]) for x in
> range(100)]
> 
> print(guppy.hpy().heap())
> 
> 322605613 bytes (w/o patch) vs 114605277 bytes (with patch)
> 
> 
> Acked-by: Timothy Redaelli 
> Tested-by: Timothy Redaelli 

Applied.  Thanks!

Best regards, Ilya Maximets.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] [python] Avoid alloc of an attr dict/row+column

2021-10-13 Thread Timothy Redaelli 
On Tue, 12 Oct 2021 14:13:31 -0500
Terry Wilson  wrote:

> Python objects normally have a dictionary named __dict__ allocated
> for handling dynamically assigned attributes. Depending on
> architecture and Python version, that empty dict may be between
> 64 and 280 bytes.
> 
> Seeing as Atom and Datum objects do not need dynamic attribute
> support and there can be millions of rows in a database, avoiding
> this allocation with __slots__ can save 100s of MBs of memory per
> Idl process.
> 
> Signed-off-by: Terry Wilson 

It reduce the memory usage and it's also faster:

print(min(timeit.repeat(lambda:
data.Atom.default(ovs.db.types.ATOMIC_TYPES[1]

0.7460950060049072 (w/o patch) vs 0.6909653190232348 (with patch)


[data.Atom.default(ovs.db.types.ATOMIC_TYPES[1]) for x in
range(100)]

print(guppy.hpy().heap())

322605613 bytes (w/o patch) vs 114605277 bytes (with patch)


Acked-by: Timothy Redaelli 
Tested-by: Timothy Redaelli 

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] [python] Avoid alloc of an attr dict/row+column

2021-10-12 Thread Terry Wilson 
Python objects normally have a dictionary named __dict__ allocated
for handling dynamically assigned attributes. Depending on
architecture and Python version, that empty dict may be between
64 and 280 bytes.

Seeing as Atom and Datum objects do not need dynamic attribute
support and there can be millions of rows in a database, avoiding
this allocation with __slots__ can save 100s of MBs of memory per
Idl process.

Signed-off-by: Terry Wilson 
---
 python/ovs/db/data.py | 4 
 1 file changed, 4 insertions(+)

diff --git a/python/ovs/db/data.py b/python/ovs/db/data.py
index 99bf80ed6..8db21b837 100644
--- a/python/ovs/db/data.py
+++ b/python/ovs/db/data.py
@@ -64,6 +64,8 @@ def returnUnchanged(x):
 
 @functools.total_ordering
 class Atom(object):
+__slots__ = ('value', 'type')
+
 def __init__(self, type_, value=None):
 self.type = type_
 if value is not None:
@@ -266,6 +268,8 @@ class Atom(object):
 
 @functools.total_ordering
 class Datum(object):
+__slots__ = ('type', 'values')
+
 def __init__(self, type_, values={}):
 self.type = type_
 self.values = values
-- 
2.31.1

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev