Re: [Python-Dev] "Global freepool"

I thought pymalloc is SLAB allocator. What is the difference between SLAB and pymalloc allocator? On Thu, Jun 1, 2017 at 6:20 PM, Victor Stinner wrote: > 2017-06-01 10:40 GMT+02:00 Antoine Pitrou : >> This is already exactly how PyObject_Malloc()

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

x86's hugepage is 2MB. And some Linux enables "Transparent Huge Page" feature. Maybe, 2MB arena size is better for TLB efficiency. Especially, for servers having massive memory. On Thu, Jun 1, 2017 at 4:38 PM, Larry Hastings wrote: > > > When CPython's small block allocator

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

arena, or stop returning arena to system, it can be problem. Regards, On Thu, Jun 1, 2017 at 6:05 PM, Siddhesh Poyarekar <siddh...@gotplt.org> wrote: > On Thursday 01 June 2017 01:53 PM, INADA Naoki wrote: >> * On Linux, madvice(..., MADV_DONTNEED) can be used. > > madvise does

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

> * On Linux, madvice(..., MADV_DONTNEED) can be used. Recent Linux has MADV_FREE. It is faster than MADV_DONTNEED, https://lwn.net/Articles/591214/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev

Re: [Python-Dev] "Global freepool"

Hi, As you said, I think PyObject_Malloc() is fast enough. But PyObject_Free() is somewhat complex. Actually, there are some freelists (e.g. tuple, dict, frame) and they improve performance significantly. My "global unified freelist" idea is unify them. The merit is: * Unify

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

Hello. AFAIK, allocating arena doesn't eat real (physical) memory. * On Windows, VirtualAlloc is used for arena. Real memory page is assigned when the page is used first time. * On Linux and some other *nix, anonymous mmap is used. Real page is assigned when first touch, like Windows.

Re: [Python-Dev] PEP 544: Protocols - second round

Hi, I'm interested in startup time too, not only execution time. Here is very rough test: with open('with_abc.py', 'w') as f: print("import abc", file=f) for i in range(1, 1001): print(f"class A{i}(metaclass=abc.ABCMeta): pass", file=f) with open('without_abc.py', 'w') as f:

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

and envvar option to disable it for people who want to continue to use C locale explicitly. Congrats, Nick! On Sat, May 27, 2017 at 4:19 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > On 24 May 2017 at 02:34, Nick Coghlan <ncogh...@gmail.com> wrote: >> On 23 May 2017 at 18:38, I

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

Hi, Nick. I read again and I think PEP 538 is mostly ready for accepted, without waiting PEP 540. One remaining my concern is setting LANG. > Setting LANG to C.UTF-8 ensures that even components that only check the LANG > fallback for their locale settings will still use C.UTF-8 .

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

>>> On platforms where they would have no effect (e.g. Mac OS X, iOS, Android, >>> Windows) these preprocessor variables would always be undefined. >> >> Why ``--with[out]-c-locale-coercion`` have no effect on macOS, iOS and >> Android? > > On these three, we know the system encoding is UTF-8, so

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

de warning by setting PYTHONUTF8=1 too. On Fri, May 5, 2017 at 10:21 PM, INADA Naoki <songofaca...@gmail.com> wrote: > On Fri, May 5, 2017 at 1:25 AM, Antoine Pitrou <solip...@pitrou.net> wrote: >> On Thu, 4 May 2017 11:24:27 +0900 >> INADA Naoki <songofaca...@gma

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Fri, May 5, 2017 at 1:25 AM, Antoine Pitrou <solip...@pitrou.net> wrote: > On Thu, 4 May 2017 11:24:27 +0900 > INADA Naoki <songofaca...@gmail.com> wrote: >> Hi, Nick and all core devs who are interested in this PEP. >> >> I'm reviewing PEP 538 a

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

Hi, Nick and all core devs who are interested in this PEP. I'm reviewing PEP 538 and I want to accept it in this month. It will reduces much UnicodeError pains which server-side OPs facing. Thank you Nick for working on this PEP. If you have something worrying about this PEP, please post a

Re: [Python-Dev] Misc/NEWS entries for Python 3.7a1

On Tue, Mar 28, 2017 at 11:07 PM, Ned Deily <n...@python.org> wrote: > On Mar 28, 2017, at 08:49, INADA Naoki <songofaca...@gmail.com> wrote: >> Currently, changelog of Python 3.7a1 [1] contains changes between >> 3.6b1 and 3.7a1. >> So lot's of bugfixes are

[Python-Dev] Misc/NEWS entries for Python 3.7a1

Hi. Currently, changelog of Python 3.7a1 [1] contains changes between 3.6b1 and 3.7a1. So lot's of bugfixes are listed twice or more in changelog. For example, "bpo-28258: Fixed build with Estonian locale..." are listed under 3.5.3rc1, 3.6.0b2 and 3.7.0a1. [1]

Re: [Python-Dev] 3.5 unittest does not support namespace packages for discovering

hes normal directly, it may walk deep into very large tree containing millions of directories. I don't like it. On Fri, Mar 24, 2017 at 12:53 AM, INADA Naoki <songofaca...@gmail.com> wrote: > There is already http://bugs.python.org/issue29642 > > On Thu, Mar 23, 2017 at 11:04 PM, Ilya K

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Mon, Mar 13, 2017 at 10:31 PM, Random832 <random...@fastmail.com> wrote: > On Mon, Mar 13, 2017, at 04:37, INADA Naoki wrote: >> But locale coercing works nice on platforms like android. >> So how about simplified version of PEP 538? Just adding configure >> option f

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

On Mon, Mar 13, 2017 at 8:01 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > On 13 March 2017 at 18:37, INADA Naoki <songofaca...@gmail.com> wrote: >> >> But locale coercing works nice on platforms like android. >> So how about simplified version of PEP 538?

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

> It seems to based on an assumption that the C locale is always some kind of > pathology. Admittedly, it sometimes is a result of misconfiguration or a > mistake. (But I don't see why it's the interpreter's job to correct such > mistakes.) However, in some cases the C locale is a normal

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

LGTM and I love this PEP and PEP 540. Some comments: ... > * PEP 540 proposes to entirely decouple CPython's default text encoding from > the C locale system in that case, allowing text handling inconsistencies > to > arise between CPython and other C/C++ components running in the same >

Re: [Python-Dev] Help requested with Python 2.7 performance regression

On Thu, Mar 2, 2017 at 4:07 AM, Antoine Pitrou wrote: > On Wed, 1 Mar 2017 19:58:14 +0100 > Matthias Klose wrote: >> On 01.03.2017 18:51, Antoine Pitrou wrote: >> > As for the high level: what if the training set used for PGO in Xenial >> > has become skewed

Re: [Python-Dev] Translated Python documentation

Yes, right place to discussion is one of important things what I want. I haven't know about i18n-sig. Is it better than docs-sig? Another thing I want is agreement to use project name looks like (semi)official project. We used "python-doc-jp" name on Transifex at first. But since some people

Re: [Python-Dev] Translated Python documentation

On Fri, Feb 24, 2017 at 9:20 PM, Berker Peksağ wrote: > On Thu, Feb 23, 2017 at 12:01 AM, Victor Stinner > wrote: >> 2017-02-22 19:04 GMT+01:00 Serhiy Storchaka : >>> What percent of lines is changed between bugfix releases?

[Python-Dev] Can I commit a hack for translation? (Was: Translated Python documentation

Thanks, Victor and all. This thread is very encouraging for me. Currently, I have a suspended pull request: https://github.com/python/cpython/pull/195 This pull request (1) fixes "CPython implementation detail:" label is disappear when it's body is translated, and (2) make the label

Re: [Python-Dev] Translated Python documentation

> Where should these translated docs live and how does one make it clear to > users reading them that doc bugs shouldn't be submitted to the main bug > tracker/github repo? > We setup github page. See https://python-doc-ja.github.io/py36/ (note that version switcher won't work because this html

Re: [Python-Dev] Deterministic builds of the interpreter

On Fri, Feb 10, 2017 at 7:58 PM, Freddy Rietdijk wrote: > For Python 3.5 PYTHONHASHSEED doesn't seem to be sufficient, these items > still seem indeterministic. > To be sure, I ran `PYTHONHASHSEED=1 $out/bin/python -m compileall -f $out` > where $out is the path where I

Re: [Python-Dev] Deterministic builds of the interpreter

> > These all seem to be sets. Maybe, PYTHONHASHSEED help you. https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHASHSEED > > On Thu, Feb 9, 2017 at 6:04 PM, INADA Naoki <songofaca...@gmail.com> wrote: >> >> As reading [4], mtime is not 0. >> >> dat

Re: [Python-Dev] Deterministic builds of the interpreter

As reading [4], mtime is not 0. data = bytearray(MAGIC_NUMBER) data.extend(_w_long(mtime)) data.extend(_w_long(source_size)) data.extend(marshal.dumps(code)) First 4 bytes are magic. Next 4 bytes are mtime. │ │ │ │ -: 160d 0d0a 6b2e 9c58 6c21 e300

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

; -- > Ivan > > > On 23 January 2017 at 12:25, INADA Naoki <songofaca...@gmail.com> wrote: >> >> On Fri, Jan 20, 2017 at 8:52 PM, Ivan Levkivskyi <levkivs...@gmail.com> >> wrote: >> > On 20 January 2017 at 11:49, INADA Naoki <songofaca.

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

On Thu, Jan 26, 2017 at 2:33 AM, Antoine Pitrou <solip...@pitrou.net> wrote: > On Wed, 25 Jan 2017 20:54:02 +0900 > INADA Naoki <songofaca...@gmail.com> wrote: >> >> ## Stripped annotation + without pydebug > > Does this mean the other measurements were

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

More detailed information: ## With annotations === tracemalloc stat === traced: (46969277, 46983753) 18,048,888 / 181112 File "", line 488 File "", line 780 File "", line 675 === size by types === dict 9,083,816 (8,870.91KB) / 21846 = 415.811bytes (21.38%) tuple

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

On Tue, Jan 24, 2017 at 11:08 PM, Victor Stinner <victor.stin...@gmail.com> wrote: > 2017-01-24 15:00 GMT+01:00 INADA Naoki <songofaca...@gmail.com>: >> And here are top 3 tracebacks from tracemalloc: >> >> 15109615 (/180598) >> File "", line

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

FWIW, I tried to skip compiler_visit_annotations() in Python/compile.c a) default: 41278060 b) remove annotations: 37140094 c) (b) + const merge: 35933436 (a-b)/a = 10% (a-c)/a = 13% And here are top 3 tracebacks from tracemalloc: 15109615 (/180598) File "", line 488 File

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

> 3. I am -1 on ignoring annotations altogether. Sometimes they could be > helpful at runtime: typing.NamedTuple and mypy_extensions.TypedDict are two > examples. ignoring annotations doesn't mean ignoring typing at all. You can use typing.NamedTuple even when functions doesn't have

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

>> So basically the equivalent of -OO for docstrings? Maybe this can be the >> final motivator for some of us to come up with a design to generalize -O or >> something as it keeps coming up. > Yes, please. We've talked about generalizing this for years now. FWIW, I know > of projects that run

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

On Mon, Jan 23, 2017 at 8:33 PM, Victor Stinner <victor.stin...@gmail.com> wrote: > 2017-01-23 12:25 GMT+01:00 INADA Naoki <songofaca...@gmail.com>: >> I gave advice to use 'List[User]' instead of List[User] to the team of >> the project, >> if the team think RAM

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

On Fri, Jan 20, 2017 at 8:52 PM, Ivan Levkivskyi <levkivs...@gmail.com> wrote: > On 20 January 2017 at 11:49, INADA Naoki <songofaca...@gmail.com> wrote: >> >> * typing may increase memory footprint, through functions >> __attributes__ and abc. >>* Can w

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

I've filed an issue about merging tuples: http://bugs.python.org/issue29336 I'll try the patch with my company's codebase again in next week. But could someone try the patch with realworld large application too? Or if you know OSS large application easy to install, could you share

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

> > Moving the refcount out of the PyObject will probably make increfs / > decrefs more costly, and there are a lot of them. We'd have to see > actual measurements if a patch is written, but my intuition is that the > net result won't be positive. > > Regards > > Antoine. I agree with you. But

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

On Fri, Jan 20, 2017 at 8:17 PM, Victor Stinner <victor.stin...@gmail.com> wrote: > 2017-01-20 11:49 GMT+01:00 INADA Naoki <songofaca...@gmail.com>: >> Report is here >> https://gist.github.com/methane/ce723adb9a4d32d32dc7525b738d3c31 > > Very interesting

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

On Fri, Jan 20, 2017 at 8:52 PM, Ivan Levkivskyi <levkivs...@gmail.com> wrote: > On 20 January 2017 at 11:49, INADA Naoki <songofaca...@gmail.com> wrote: >> >> * typing may increase memory footprint, through functions >> __attributes__ and abc. >>* Can w

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

> > "this script counts static memory usage. It doesn’t care about dynamic > memory usage of processing real request" > > You may be trying to optimize something which is only a very small > fraction of your actual memory footprint. That said, the marshal > module could certainly try to intern

[Python-Dev] Investigating Python memory footprint of one real Web application

Hi, all. After reading Instagram's blog article [1], I’m thinking about how Python can reduce memory usage of Web applications. My company creating API server with Flask, SQLAlchemy and typing. (sorry, it's closed source). So I can get some data from it's codebase. [1]:

Re: [Python-Dev] Can we use "designated initializer" widely in core modules?

On Wed, Jan 18, 2017 at 9:00 AM, Erik <pyt...@lucidity.plus.com> wrote: > On 17/01/17 06:32, INADA Naoki wrote: >> >> With designated initializer, it becomes: >> >> 0, /* tp_free */ >> +.tp_fastcall = (fastternary

Re: [Python-Dev] Can we use "designated initializer" widely in coremodules?

On Wed, Jan 18, 2017 at 8:48 AM, Larry Hastings wrote: > > On 01/17/2017 12:02 PM, Steve Dower wrote: > > Avoiding header files would be my only request. As Brett says, the C99 > requirement should not be enforced on all embedders or extenders, so we > should try and keep the

[Python-Dev] Can we use "designated initializer" widely in core modules?

Hi. --- This discussion is started in http://bugs.python.org/issue29259, and there is separated issue at http://bugs.python.org/issue29260 . But I want to see more comments, before issue 29259 is committed. --- Since Python 3.6, some of C99 features are accepted in PEP 7. "designated

Re: [Python-Dev] Context manager lifetime rules Was: Re: Adding bytes.frombuffer() constructor to PEP 467

>> > with memoryview(x) as m: >> > b = bytes(m) >> > # or b = m.tobytes() >> >> Thinking more about this, and after looking at my own code in asyncpg >> and uvloop, I'm now in favor of adding bytes.frombuffer() with the >> proposed signature: ``bytes.frombuffer(byteslike, length=-1,

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467

> > Thinking more about this, and after looking at my own code in asyncpg and > uvloop, I'm now in favor of adding bytes.frombuffer() with the proposed > signature: ``bytes.frombuffer(byteslike, length=-1, offset=0)`` > Do you prefer a signature proposed first? I thought it from asyncio usage

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467

I submit an issue about it. See https://bugs.python.org/issue29178 On Fri, Jan 6, 2017 at 7:38 PM, INADA Naoki <songofaca...@gmail.com> wrote: >> >> Thinking more about this, and after looking at my own code in asyncpg and >> uvloop, I'm now in favor of

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467

> > bytes.frombuffer(x) is bytes(memoryview(x)) or memoryview(x).tobytes(). > There is pitfall: memoryview should be closed. So b = bytes.frombuffer(x) is: with memoryview(x) as m: b = bytes(m) # or b = m.tobytes() ___ Python-Dev mailing list

[Python-Dev] IRC logs via BotBot.me

Hi. IRC #python-dev channel is nice place to know what happens recently. But I can't log in always because I use only laptop PC. I found BotBot.me seems nice IRC log service and used by some major channels. https://botbot.me/ I wonder if #python-dev is logged by BotBot.me. I'm sorry if it had

Re: [Python-Dev] Making sure dictionary adds/deletes during iteration always raise exception

gt; http://bugs.python.org/issue19332 > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/songofacan

Re: [Python-Dev] Bus error in Python 3.6.0beta

3 02:46 libpython3.5m.so.1.0* ... $ chmod u+w lib/libpython3.5m.so.1.0 $ bin/python3.5 Python 3.5.2+ (3.5:0ee76f3afd70, Nov 23 2016, 02:39:08) [GCC 6.2.0 20161005] on linux Type "help", "copyright", "credits" or "license" for more information. >>> f =

Re: [Python-Dev] [Python-ideas] Show more info when `python -vV`

> > I agree with Barry. I'm open to adding this feature to 3.6.0b3 but > first we need an issue and a final patch to review. > Here is the issue and patch. Could someone review it? http://bugs.python.org/issue28532 -- INADA Naoki <songofa

Re: [Python-Dev] [Python-ideas] Show more info when `python -vV`

(Added python-dev in CC list, because there are enough +1 already). On Mon, Oct 17, 2016 at 3:06 PM, Chris Angelico <ros...@gmail.com> wrote: > On Mon, Oct 17, 2016 at 5:02 PM, INADA Naoki <songofaca...@gmail.com> wrote: >> $ ./python.exe -V >> Python 3.6.0b2+ >>

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

sue tracker. https://bitbucket.org/pypy/pypy/issues/2367/del-bytearray-n-should-be-optimized In case of micropython, it doesn't support deletion for now. https://github.com/micropython/micropython/blob/b9672bcbe8dd29b61326af8eb026df4d10a8f0ce/py/objarray.c#L371-L378 -- INADA Naoki <s

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

nvert when needed. > > -- > Terry Jan Reedy > Because it's public module API. While bytearray is mostly API compatible (passes duck typing), isinstance(b, bytes) is False when b is bytearray. So, I feel changing return type from bytes to bytearray is last option. I want to return bytes if

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

rop Python 2.7 support, they can use bytearray, and iostream can be more simple and fast. So I hope "amortized O(1) deletion from the front" is language spec, at least for Python 3.5+ -- INADA Naoki <songofaca...@gmail.com> ___ Pytho

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

On Wed, Oct 12, 2016 at 2:32 PM, Serhiy Storchaka <storch...@gmail.com> wrote: > On 12.10.16 07:08, INADA Naoki wrote: >> >> Sample code: >> >> def read_line(buf: bytearray) -> bytes: >> try: >> n = buf.index(b'\r\n') >&

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

matic due to the way > it differs from range() & slice(), but I don't think it makes sense to > get into that kind of detail before discussing the larger question of > adding a new helper module for working efficiently with memory buffers > vs further widening the method API for the builti

[Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

ore constructor to bytes: # when length=-1 (default), use until end of *byteslike*. bytes.frombuffer(byteslike, length=-1, offset=0) With ths API with memoryview(buf) as m: line = bytes(m[:n]) becomes line = bytes.frombuffer(buf, n) Thanks, -- INADA Naoki <songofa

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

On Tue, Sep 20, 2016 at 7:02 PM, INADA Naoki <songofaca...@gmail.com> wrote: > On Tue, Sep 20, 2016 at 6:56 PM, Dima Tisnek <dim...@gmail.com> wrote: >> Totally random thought: >> >> Can lru_cache be simplified to use an ordered dict instead of dict + >> lin

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

On Thu, Sep 15, 2016 at 5:57 PM Victor Stinner <victor.stin...@gmail.com> wrote: > 2016-09-15 10:02 GMT+02:00 INADA Naoki <songofaca...@gmail.com>: > > In my environ: > > > > ~/local/python-master/bin/python3 -m timeit -s "d = > > dict.fromkeys(range(

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

> > > Note that this is made at the expense of the 20% slowing down an iteration. > > $ ./python -m timeit -s "d = dict.fromkeys(range(10**6))" -- "list(d)" > Python 3.5: 66.1 msec per loop > Python 3.6: 82.5 msec per loop > > Are two Pythons built with same options? In my environ:

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

> > I mean using a compact representation, if not an ordered one. > > I have no particular usecase in mind. As far as I understand the compact > implementation, sets can do it just as well. The original discussion > proposed trying to implement it for sets first. > > Like dict, they would

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

> From what I understood, Python 3.6 dict got two *different* changes: > > * modify the dict structure to use two tables instead of only one: an > "index" table (the hash table) and a second key/value table > * tune the dict implementation to only append to the key/value table > > The second

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

On Tue, Sep 13, 2016 at 1:35 AM, Guido van Rossum wrote: > Couldn't we use the order in the actual hash table (which IIUC now > contains just indexes into the ordered vector of key/value/hash > structs)? That would probably simulate the pre-3.6 order quite > effectively. Maybe,

Re: [Python-Dev] Review request: issue 27350, compact ordered dict

Thank you for all core devs! I'll polish the implementation until 3.6b2. On Fri, Sep 9, 2016 at 6:42 AM, Guido van Rossum <gu...@python.org> wrote: > It's in! Congrats, and thanks for your great work! See longer post by Victor. > > On Sun, Aug 28, 2016 at 12:16 AM, INADA N

Re: [Python-Dev] Review request: issue 27350, compact ordered dict

int. Thank you! > > -- > --Guido van Rossum (python.org/~guido) -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.

Re: [Python-Dev] Review request: issue 27350, compact ordered dict

Last call. There are only two weeks until 3.6 beta. Please review it if possible. On Tue, Aug 9, 2016 at 10:12 PM, INADA Naoki <songofaca...@gmail.com> wrote: > Hi, devs. > > I've implemented compact and ordered dictionary [1], which PyPy > implemented in 2015 [2]. > > Si

Re: [Python-Dev] Review request: issue 27350, compact ordered dict

>> >> I've implemented compact and ordered dictionary [1], which PyPy >> implemented in 2015 [2]. > > > Does this mean that keyword arguments will become ordered? > > Yury > Yes, regardless it will be language spec or just an implementation detail like PyPy

Re: [Python-Dev] Review request: issue 27350, compact ordered dict

API) > * there are nice speedups > * the C version of OrderedDict can be killed > * it saves memory, on 64bit by quite a bit (not everyone stores more > than 4bln items in a dictionary) > * it solves the problem of tests relying on order in dictionaries > > In short, it has no dow

[Python-Dev] Review request: issue 27350, compact ordered dict

://morepypy.blogspot.jp/2015/01/faster-more-memory-efficient-and-more.html -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/o

Re: [Python-Dev] C99

>> >> * inline function >> static inline function can be used instead of may macros. >> It is more readable, and type safe. > > My experience from a few months ago with some cross-platform code is > that clang, GCC and MSVC have different ideas about how inline > functions in C work. Are they

Re: [Python-Dev] C99

d with C99 adoption but it does point out that there might be more >> ramifications to this decision. What may be more difficult is to judge the >> impact on other platforms that don't get as much attention from most of us. >> For this to move forward, we need to be able to state wha

Re: [Python-Dev] Status of Python 3.6 PEPs?

> > "PEP 520 -- Preserving Class Attribute Definition Order" > https://www.python.org/dev/peps/pep-0520/ > => accepted -- what is the status of its implementation? > ... > > > I also see some discussions for even more compact dict implementation. > Here is implementation of the compact dict

Re: [Python-Dev] Compact ordered dict is not ordered for split table. (was: PEP XXX: Compact ordered dict

se so > much thanks to compact dict. I did it. issue27350 is now ordered for key sharing dict, too. -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python

Re: [Python-Dev] Compact ordered dict is not ordered for split table. (was: PEP XXX: Compact ordered dict

On Sun, Jun 26, 2016 at 8:40 AM, Franklin? Lee <leewangzhong+pyt...@gmail.com> wrote: > On Jun 21, 2016 11:12 AM, "INADA Naoki" <songofaca...@gmail.com> wrote: >> >> I'm sorry, but I hadn't realized which compact ordered dict is >> not ordered for s

Re: [Python-Dev] Idea: more compact, interned string key only dict for namespace.

com/methane/cpython/tree/interned-dict (cb0a125c79 passes most tests, except tests using sys.getsizeof()). -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev U

Re: [Python-Dev] Compact ordered dict is not ordered for split table. (was: PEP XXX: Compact ordered dict

ered dict is more efficient than key-sharing dict in case of Sphinx. It means, instance __dict__ is not dominance. I'll implement POC of my new idea and compare it with Sphinx. If you know another good *real application*, which is easy to benchmark, please tell me it. -- INADA Naoki <songofa

Re: [Python-Dev] PEP XXX: Compact ordered dict

On Fri, Jun 24, 2016 at 12:03 AM, Eric Snow <ericsnowcurren...@gmail.com> wrote: > On Mon, Jun 20, 2016 at 11:02 PM, INADA Naoki <songofaca...@gmail.com> wrote: >> On Tue, Jun 21, 2016 at 12:17 PM, Oleg Broytman <p...@phdru.name> wrote: >>> (if a PEP is neede

Re: [Python-Dev] Idea: more compact, interned string key only dict for namespace.

:12.16elapsed 99%CPU (0avgtext+0avgdata 165884maxresident)k 480inputs+200792outputs (0major+56947minor)pagefaults 0swaps 71.84user 0.27system 1:12.13elapsed 99%CPU (0avgtext+0avgdata 166888maxresident)k 640inputs+200792outputs (5major+56834minor)pagefaults 0swaps -- INADA Naoki <songof

Re: [Python-Dev] Idea: more compact, interned string key only dict for namespace.

If "orderd, except key sharing dict" is acceptable, no problem. Key sharing compact dict is smaller than current key sharing dict of Python 3.5 for most cases. https://docs.google.com/spreadsheets/d/1nN5y6IsiJGdNxD7L7KBXmhdUyXjuRAQR_WbrS8zf6mA/edit#gid=0 Regards, -- INADA Naoki <songo

Re: [Python-Dev] Idea: more compact, interned string key only dict for namespace.

- > > Interned key only dict is still larger than key-shared dict. > > But it can be used for more purpose. It can be used for interning string > for example. It can be used to kwargs dict when all keys are interned > already. > > If we provide _PyDict_NewForNamespace

[Python-Dev] Idea: more compact, interned string key only dict for namespace.

already. If we provide _PyDict_NewForNamespace to extension modules, json decoder can have option to use this, too. -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/

Re: [Python-Dev] PEP XXX: Compact ordered dict

FYI, Here is calculated size of each dict by len(d). https://docs.google.com/spreadsheets/d/1nN5y6IsiJGdNxD7L7KBXmhdUyXjuRAQR_WbrS8zf6mA/edit?usp=sharing On Tue, Jun 21, 2016 at 12:17 PM, Oleg Broytman <p...@phdru.name> wrote: > Hi! > > On Tue, Jun 21, 2016 at 11:14:39AM +09

Re: [Python-Dev] Compact ordered dict is not ordered for split table. (was: PEP XXX: Compact ordered dict

ess change may cause sudden memory usage increase. (__slots__ is more predicable). On Wed, Jun 22, 2016 at 12:10 AM, INADA Naoki <songofaca...@gmail.com> wrote: > I'm sorry, but I hadn't realized which compact ordered dict is > not ordered for split ta

Re: [Python-Dev] PEP 487: Simpler customization of class creation

On Wed, Jun 22, 2016 at 2:50 AM, Raymond Hettinger wrote: > >> On Jun 21, 2016, at 10:18 AM, Guido van Rossum wrote: >> >> Judging from Inada's message there seems to be some confusion about how well >> the compact dict preserves order (personally

[Python-Dev] Compact ordered dict is not ordered for split table. (was: PEP XXX: Compact ordered dict

ems() dict_items([('a', 1), ('b', 2)]) >>> b.__dict__.items() dict_items([('a', 4), ('b', 3)]) This doesn't affects to **kwargs and class namespace. But if we change the language spec to dict preserves insertion order, this should be addressed. On Tue, Jun 21, 2016 at 2:02 PM, INADA Naoki

Re: [Python-Dev] PEP XXX: Compact ordered dict

On Tue, Jun 21, 2016 at 12:17 PM, Oleg Broytman <p...@phdru.name> wrote: > Hi! > > On Tue, Jun 21, 2016 at 11:14:39AM +0900, INADA Naoki > <songofaca...@gmail.com> wrote: >> Here is my draft, but I haven't >> posted it yet since >> my English is much

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

>> >> Finally, it seems someone is working on making all dicts ordered. Does that >> mean this will soon be obsolete? > > Nope. Having an ordered definition namespace by default does not give > us __definition_order__ for free. Furthermore, the compact dict under > consideration isn't strictly

Re: [Python-Dev] Compact dict implementations (was: PEP 468

I've sent my patch to issue tracker, since I can't fix some remains TODOs by myself. http://bugs.python.org/issue27350 On Fri, Jun 17, 2016 at 6:15 PM, INADA Naoki <songofaca...@gmail.com> wrote: > Hi, developers. > > I'm trying to implement compact dict. > https://github.c

Re: [Python-Dev] Compact dict implementations (was: PEP 468

. In case of very small dict, index hash (8byte) and first two entries (24*2=48byte) can be on one cache line. * Easy to implement "split dictionary" (aka. key sharing dictionary). You can see what I implemented in here. https://github.com/methane/cpy

Re: [Python-Dev] Compact dict implementations (was: PEP 468

> > pybench: https://gist.github.com/methane/cfad1427d87ceff9310350e78a214880 > benchmark: https://gist.github.com/methane/5eb11fdd93863813b222e795ca0bfc1f > > Is it acceptable? latest result is here https://gist.github.com/methane/22cf5d1dadb62bc87a15e9244a9d0ab8 -- INADA Na

Re: [Python-Dev] Compact dict implementations (was: PEP 468

em in core-mentor ML or bugs.python.org. Thanks -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-de

[Python-Dev] Compact dict implementations (was: PEP 468

implementation, I want to see comments and tests from core developers. Please come to core-mentorship ML or pull request and try it if you interested in. Regards, -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()

Sorry, I've forgot to use "Reply All". On Tue, Apr 12, 2016 at 9:49 AM, INADA Naoki <songofaca...@gmail.com> wrote: > IHMO it's safer to get an encoding error rather than no error when you >> concatenate two byte strings encoded to two different encodings (mojibake).

Re: [Python-Dev] Defining a path protocol

FYI, Ruby's Pathname class doesn't inherit String. http://ruby-doc.org/stdlib-2.1.0/libdoc/pathname/rdoc/Pathname.html Ruby has two "convert to string" method. `.to_s` is like `__str__`. `.to_str` is like `__index__` but for str. It is used for implicit conversion. File.open accepts any object

Re: [Python-Dev] Defining a path protocol

gineering. So I'm -0.5 on adding __fspath__. I'm +1 on adding general protocol for *coerce to string* like __index__. +0.5 on inherit from str (and drop byte path support). -- INADA Naoki <songofaca...@gmail.com> ___ Python-Dev mailing l

Re: [Python-Dev] FAT Python (lack of) performance

r from our priorities is almost always going to be > irritating rather than helpful. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > -- INADA Naoki <songofaca...@gmail.com>

<    1   2   3   4   5   6   >