On Fri, Jan 20, 2017 at 8:52 PM, Ivan Levkivskyi <levkivs...@gmail.com> wrote: > On 20 January 2017 at 11:49, INADA Naoki <songofaca...@gmail.com> wrote: >> >> * typing may increase memory footprint, through functions >> __attributes__ and abc. >> * Can we add option to remove or lazy evaluate __attributes__ ? > > > This idea already appeared few times. I proposed to introduce a flag (e.g. > -OOO) to ignore function and variable annotations in compile.c > It was decide to postpone this, but maybe we can get back to this idea. > > In 3.6, typing is already (quite heavily) optimized for both speed and > space. > I remember doing an experiment comparing a memory footprint with and without > annotations, the difference was few percent. > Do you have such comparison (with and without annotations) for your app? > It would be nice to have a realistic number to estimate what would the > additional optimization flag save. > > -- > Ivan > >
Hi, Ivan. I investigated why our app has so many WeakSet today. We have dozen or hundreds of annotations like Iterable[User] or List[User]. (User is one example of application's domain object. There are hundreds of classes). On the other hand, SQLAlchemy calls isinstance(obj, collections.Iterable) many times, in [sqlalchemy.util._collections.to_list](https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/util/_collections.py#L795-L804) method. So there are (# of iterable subclasses) weaksets for negative cache, and each weakset contains (# of column types) entries. That's why WeakSet ate much RAM. It may be slowdown application startup too, because thousands of __subclasscheck_ is called. I gave advice to use 'List[User]' instead of List[User] to the team of the project, if the team think RAM usage or boot speed is important. FWIW, stacktrace is like this: File "/Users/inada-n/local/py37dbg/lib/python3.7/_weakrefset.py", line 84 self.data.add(ref(item, self._remove)) File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 233 cls._abc_negative_cache.add(subclass) File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 226 if issubclass(subclass, scls): File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 226 if issubclass(subclass, scls): File "/Users/inada-n/local/py37dbg/lib/python3.7/abc.py", line 191 return cls.__subclasscheck__(subclass) File "venv/lib/python3.7/site-packages/sqlalchemy/util/_collections.py", line 803 or not isinstance(x, collections.Iterable): File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 1680 columns = util.to_list(prop) File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 1575 prop = self._property_from_column(key, prop) File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 1371 setparent=True) File "venv/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 675 self._configure_properties()<PasteEnd> Regards, _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
