[Python-Dev] Discussion about crash tolerance feature for gdbm module
Hi folks,
>From gdbm 1.21, gdbm supports the crash tolerance feature.
see: https://www.gnu.org.ua/software/gdbm/manual/Crash-Tolerance.html
I would like to introduce this feature since python standard library is the
only gdbm binding library that is available for end-users.
And this is also effort for end-users to provide the latest new features.
If this feature is added following methods will be added.
- 'x' flag will be added for extended gdbm format.
- gdbm.gdbm_failure_atomic('path1', 'path2') API will be added for
snapshot paths.
- gdbm.gdbm_latest_snapshot('path1', 'path2') API will be added for
getting latest valid snapshot path.
The above APIs will be used for people who need to recover their gdbm file
if disaster situations happen.
However, this feature is not yet decided to land to CPython.
and we have already discussed this issue at
https://bugs.python.org/issue45452
but I would like to hear other devs opinions.
I cc the original authors of this feature and Serhiy who thankfully already
participated in this discussion too :)
Warm Regards,
Dong-hee
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/RFICESJKJW3KZSHYSU4R7UEPUUAJL2B3/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Is anyone using 15-bit PyLong digits (PYLONG_BITS_IN_DIGIT=15)?
[Barry Scott and Steve Dower share tips for convincing Visual Studio to show assembler without recompiling the file] Thanks, fellows! That mostly ;-) workedl. Problem remaining is that breakpoints just didn't work. They showed up "visually", and in the table of set breakpoints, but code went whizzing right by them every time. I didn't investigate. It's possible, e.g., that the connection between C source and the generated PGO code was so obscure that VS just gave up - or just blew it. Instead I wrote a Python loop to run a division of interest "forever". That way I hoped I'd be likely to land in the loop of interest by luck when I broke into the process. Which worked! So here's the body of the main loop: 7FFE451D2760 mov eax,dword ptr [rcx-4] 7FFE451D2763 lea rcx,[rcx-4] 7FFE451D2767 shl r9,1Eh 7FFE451D276B or r9,rax 7FFE451D276E cmp r8,0Ah 7FFE451D2772 jne long_div+25Bh (07FFE451D27CBh) 7FFE451D2774 mov rax,rdi 7FFE451D2777 mul rax,r9 7FFE451D277A mov rax,rdx 7FFE451D277D shr rax,3 7FFE451D2781 mov dword ptr [r10+rcx],eax 7FFE451D2785 mov eax,eax 7FFE451D2787 imulrax,r8 7FFE451D278B sub r9,rax 7FFE451D278E sub rbx,1 7FFE451D2792 jns long_div+1F0h (07FFE451D2760h) And above the loop is this line, which you'll recognize as loading the same scaled reciprocal of 10 as the gcc code Mark posted earlier. The code above moves %rdi into %rax before the mul instruction: 7FFE451D2747 mov rdi,0CCCDh Note an odd decision here:the MS code compares the divisor to 10 on _every_ iteration. There are not two, "10 or not 10?", loop; bodies. Instead, if the divisor isn't 10, "jne long_div+25Bh" jumps to code not shown here, a few instructions that use hardware division, and then jump back into the tail end of the loop above to finish computing the remainder (etc). So they not only optimized division by 10, they added a useless test and two branches to every iteration of the loop when we're not dividing by 10 ;-) ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DGRMIVHOIAYQEQJD32ED4MCBOPFE5SIT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Is anyone using 15-bit PyLong digits (PYLONG_BITS_IN_DIGIT=15)?
On 1/17/2022 8:47 PM, Barry Scott wrote: On 17 Jan 2022, at 06:35, Tim Peters wrote: [Guido] I don't think there's a way to do a PGO build from Visual Studio; but a command prompt in the repo can do it using `PCbuild\build.bat --pgo`. Just be patient with it. Thanks! That worked, and was easy, and gave me an executable that runs "// 10" at supernatural speed. Alas, Visual Studio will not show a disassembly window unless the debugger is running, and there appears to be no way to convince it to run its debugger without it first recompiling the source file you're staring at. Which it recomplies without benefit of PGO. The trick you need is to close the project you use to build python from source then you can open the python.exe and run that under the debugger. Because it can find the python.pdb it will source/disasm as you want. Or you can Debug/Attach to the process if you start it running outside of Visual Studio. You should also be able to just select the "PGUpdate" configuration (either from the dropdown in the toolbar, on Build/Configuration Manager). Provided you've done a PGO build from the command line, it will have the profile, and should be able to fairly quickly rebuild and reapply it - even with code changes - and start debugging. Or you can try and enable the preprocessed assembler output from the compiler. I'm not sure how that handles PGO though, since the assembly comes from the compiler but it's the linker that does most of the optimisation. Attaching to a running process is what I normally do. Cheers, Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UFGYPAOZ5G7MN2QYYGUDAGJ3X6OCYP6F/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Is anyone using 15-bit PyLong digits (PYLONG_BITS_IN_DIGIT=15)?
> On 17 Jan 2022, at 06:35, Tim Peters wrote: > > [Guido] >> I don't think there's a way to do a PGO build from Visual Studio; but >> a command prompt in the repo can do it using `PCbuild\build.bat --pgo`. >> Just be patient with it. > > Thanks! That worked, and was easy, and gave me an executable that runs > "// 10" at supernatural speed. > > Alas, Visual Studio will not show a disassembly window unless the > debugger is running, and there appears to be no way to convince it to > run its debugger without it first recompiling the source file you're > staring at. Which it recomplies without benefit of PGO. The trick you need is to close the project you use to build python from source then you can open the python.exe and run that under the debugger. Because it can find the python.pdb it will source/disasm as you want. Barry > > So, in the end, it was an elaborate way to reproduce the ;non-PGO > optimized machine code I already saw :-) > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/D47HHYYASRHS56AZL6SEK4Y7K5FNSJOP/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HYSQAPPREFZPYSNMEJCWY5EKICX2R4RV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Fwd: PEP 646 (Variadic Generics): final call for comments
> Even less, actually. > The PEP doesn't make a very clear distinction between invalid Python > syntax vs. invalid type annotation, so I wanted to check if we're on the > same page here: the newly valid syntax will be subject to PEP 387. > We clearly are on the same page, and I don't think you need to update > the PEP. Ok, fair enough. > When I asked my curious question, I thought I misread a piece of text, > not that it's a detail that went unnoticed, and could delay the PEP. > I can't speak for the whole SC, but on the Monday meeting I'll suggest > accepting the PEP with a note that > - index assignment is also affected, and > - the details around multiple unpackings in a type expression aren't > specified precisely. This gives individual type checkers some leeway, > but can be tightened in future PEPs. Cool. Thanks, Petr! ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VZHGKB7SKI45GFP7BI7FDTO6ENOL4NL6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on PEP 673: Self Type
El lun, 17 ene 2022 a las 6:25, Petr Viktorin () escribió: > On Wed, Nov 17, 2021 at 8:31 AM Pradeep Kumar Srinivasan > wrote: > > > > This PEP [1] introduces a simple and intuitive way to annotate methods > and classmethods that return an instance of their class. Such methods and > classmethods occur quite frequently, but the existing way to annotate them > correctly is quite arcane and error-prone. The PEP introduces a special > type `Self` to represent the type of the `self` parameter, similar to the > `this` type in TypeScript and the `Self` type in Rust. We have > implementations for mypy and pyright. The PEP does not affect CPython > directly except for the addition of one special form (Self) to typing.py > [2]. > > > > Since we have reached consensus on the PEP in typing-sig [3], we wanted > to get your comments and suggestions before submitting to the Steering > Council. > > > > Thanks, > > Pradeep Kumar Srinivasan > > James Hilton-Balfe > > > > [1]: https://www.python.org/dev/peps/pep-0673/ > > [2]: Adding `Self` to typing_extensions.py: > https://github.com/python/typing/pull/933 > > [3]: See the comments from typing-sig members on the Google doc: > https://docs.google.com/document/d/1ujuSMXDmSIOJpiZyV7mvBEC8P-y55AgSzXcvhrZciuI/edit?usp=sharing > > Hello, and thanks for the PEP! > Sorry I'm late, but I have two curious questions about the PEP. > I don't think they should hold back accepting the PEP, but I'm > interested in the answers. > > The PEP uses `reveal_type`, a function that's appeared in a few PEPs > already, but was never described. Is it a standard function in typing > tools, something specific to mypy, or pseudocode? > It's a function that doesn't exist at runtime, but when a type checker sees a call, it emits the inferred type of the argument. It originated with mypy but I believe has been adopted by all type checkers. There's been some talk of adding it to the `typing` module, but that hasn't happened so far. I opened https://bugs.python.org/issue46414 to suggest adding it. > > The PEP says "we reject Self in metaclasses." > "Metaclass" can mean "subclass of `type`", or it can refer to how a > value is used -- for example, you can write `class > Foo(metaclass=print): ...`. > In the PEP's example, is MyMetaclass rejected because: > - it's used as a metaclass in a class statement, or > - it's a subclass of `type` (so it's rejected even if unused), or > - it becomes a class of a class? > Or is the exact interpretation best left to the type checker? > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/DVOQK3U6MOYXPRXF5OVLVLJBPEJRUIM5/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6OZWGEQYZI4XR6GKRPKYEBOKXANSNGXE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on PEP 673: Self Type
On Wed, Nov 17, 2021 at 8:31 AM Pradeep Kumar Srinivasan wrote: > > This PEP [1] introduces a simple and intuitive way to annotate methods and > classmethods that return an instance of their class. Such methods and > classmethods occur quite frequently, but the existing way to annotate them > correctly is quite arcane and error-prone. The PEP introduces a special type > `Self` to represent the type of the `self` parameter, similar to the `this` > type in TypeScript and the `Self` type in Rust. We have implementations for > mypy and pyright. The PEP does not affect CPython directly except for the > addition of one special form (Self) to typing.py [2]. > > Since we have reached consensus on the PEP in typing-sig [3], we wanted to > get your comments and suggestions before submitting to the Steering Council. > > Thanks, > Pradeep Kumar Srinivasan > James Hilton-Balfe > > [1]: https://www.python.org/dev/peps/pep-0673/ > [2]: Adding `Self` to typing_extensions.py: > https://github.com/python/typing/pull/933 > [3]: See the comments from typing-sig members on the Google doc: > https://docs.google.com/document/d/1ujuSMXDmSIOJpiZyV7mvBEC8P-y55AgSzXcvhrZciuI/edit?usp=sharing Hello, and thanks for the PEP! Sorry I'm late, but I have two curious questions about the PEP. I don't think they should hold back accepting the PEP, but I'm interested in the answers. The PEP uses `reveal_type`, a function that's appeared in a few PEPs already, but was never described. Is it a standard function in typing tools, something specific to mypy, or pseudocode? The PEP says "we reject Self in metaclasses." "Metaclass" can mean "subclass of `type`", or it can refer to how a value is used -- for example, you can write `class Foo(metaclass=print): ...`. In the PEP's example, is MyMetaclass rejected because: - it's used as a metaclass in a class statement, or - it's a subclass of `type` (so it's rejected even if unused), or - it becomes a class of a class? Or is the exact interpretation best left to the type checker? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DVOQK3U6MOYXPRXF5OVLVLJBPEJRUIM5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: How about using modern C++ in development of CPython ?
On Mon, 17 Jan 2022 at 06:52, Denis Kotov wrote: > > And that's why you need to do more work than arguing that in principle > > C++ is just a better language than C. We've been hearing that for 4 > > decades now (at least we greybeards have), and we've discovered that > > for many existing applications, C++ may be better but the cost of > > converting large swaths of C code to equivalent C++ that passes all > > tests is too big. Python may very well be one of them. > > So if you're not going to do the work to demonstrate big wins from > > using C++ instead of C in actual Python implementation code, I think > > you're wasting your posts. > > I thought about it, but will CPython accept the PR for this changes if it > would show benefits ? We can't say that, no. The point here is that if you *don't* write such a PR and demonstration, nothing will change. If you do, then *maybe* it will get accepted, it depends if the benefits demonstrated by the PR are sufficient. The question is whether just having a *chance* of getting it in is sufficient to persuade you to produce such a PR. Paul. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QNUXQ66OKNET5AF6PNMZ5T6ZJWUTU7HV/ Code of Conduct: http://python.org/psf/codeofconduct/
