Re: [Python-Dev] just a friendly developer
ty armour writes: > I am looking for tutorials on basically how to write the python > language. Newcomers to Python are especially invited to the Python ‘tutor’ forum https://mail.python.org/mailman/listinfo/tutor>, for collaborative tutoring in a friendly environment. -- \ “The internet's completely over.… Anyway, all these computers | `\and digital gadgets are no good. They just fill your head with | _o__) numbers and that can't be good for you.” —Prince, 2010-07-05 | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
On Sun, Jan 31, 2016 at 08:23:00PM +, Brett Cannon wrote: > So freezing the stdlib helps on UNIX and not on OS X (if my old testing is > still accurate). I guess the next question is what it does on Windows and > if we would want to ever consider freezing the stdlib as part of the build > process (and if we would want to change the order of importers on > sys.meta_path so frozen modules came after file-based ones). I find that being able to easily open stdlib .py files in a text editor to read the source is extremely valuable. I've learned much more from reading the source than from (e.g.) StackOverflow. Likewise, it's often handy to do a grep over the stdlib. When you talk about freezing the stdlib, what exactly does that mean? - will the source files still be there? - how will this affect people writing patches for bugs? -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
On Sun, 31 Jan 2016, 15:36 Terry Reedy wrote: > On 1/31/2016 12:09 PM, Antoine Pitrou wrote: > > > The following documentation leaves me absolutely clueless: > > > > """This class only works with loaders that define exec_module() as > control > > over what module type is used for the module is required. > > No wonder. I cannot parse it as an English sentence. It needs rewriting. > Feel free to open an issue to clarify the wording. -Brett > > For those same > > reasons, the loader’s create_module() method will be ignored (i.e., the > > loader’s method should only return None). Finally, modules which > substitute > > the object placed into sys.modules will not work as there is no way to > > properly replace the module references throughout the interpreter safely; > > ValueError is raised if such a substitution is detected.""" > > > > (reference: > > > https://docs.python.org/3/library/importlib.html#importlib.util.LazyLoader > ) > > -- > Terry Jan Reedy > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] just a friendly developer
On 01/31/2016 12:16 PM, ty armour wrote: > I am looking for tutorials on basically how to write the python > language. Try asking on the python-list [1] mailing list, as that is for general discussion of Python. This list is for developing Python itself. -- ~Ethan~ [1] https://mail.python.org/mailman/listinfo/python-list ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
On 1/31/2016 12:09 PM, Antoine Pitrou wrote: The following documentation leaves me absolutely clueless: """This class only works with loaders that define exec_module() as control over what module type is used for the module is required. No wonder. I cannot parse it as an English sentence. It needs rewriting. For those same reasons, the loader’s create_module() method will be ignored (i.e., the loader’s method should only return None). Finally, modules which substitute the object placed into sys.modules will not work as there is no way to properly replace the module references throughout the interpreter safely; ValueError is raised if such a substitution is detected.""" (reference: https://docs.python.org/3/library/importlib.html#importlib.util.LazyLoader) -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] just a friendly developer
I am looking for tutorials on basically how to write the python language. As well I want to know how to write wrappers for things like alsa and directx and coreaudio and jackd and pulseaudio. I would be looking to write wrappers for coreaudio in mac and coreaudio for windows as well as alsa. i think it would benefit you to have this information. I am looking to aid in development of computers and languages and tutorials on how to write the actual python language would be super useful. but yeah im looking to write applications like guitarix rakarrack blender ardour lmms. I am also going to build my own computers so these tutorials would help a lot. anyhow thanks for your time ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
So freezing the stdlib helps on UNIX and not on OS X (if my old testing is still accurate). I guess the next question is what it does on Windows and if we would want to ever consider freezing the stdlib as part of the build process (and if we would want to change the order of importers on sys.meta_path so frozen modules came after file-based ones). On Sun, 31 Jan 2016, 10:43 M.-A. Lemburg wrote: > On 30.01.2016 20:15, Steve Dower wrote: > > Brett tried freezing the entire stdlib at one point (as we do for parts > of importlib) and reported no significant improvement. Since that rules out > code compilation as well as the OS calls, it'd seem the priority is to > execute less code on startup. > > > > Details of that work were posted to python-dev about twelve months ago, > IIRC. Maybe a little longer. > > Freezing the entire stdlib does improve the startup time, > simply because it removes stat calls, which dominate the startup > time at least on Unix. > > It also allows sharing the stdlib byte code in memory, since it gets > stored in static C structs which the OS will happily mmap into > multiple processes for you without any additional effort. > > Our eGenix PyRun does exactly that. Even though the original > motivation is a different one, the gained improvement in > startup time is a nice side effect: > > http://www.egenix.com/products/python/PyRun/ > > Aside: The encodings don't really make much difference here. The > dictionaries aren't all that big, so generating them on the fly doesn't > really create much overhead. The trade off in terms of > maintainability/speed > definitely leans toward maintainability. For the larger encoding > tables we already have C implementations with appropriate data > structures to make lookup speed vs. storage needs efficient. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Experts (#1, Jan 31 2016) > >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ > >>> Python Database Interfaces ... http://products.egenix.com/ > >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ > > > ::: We implement business ideas - efficiently in both time and costs ::: > >eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >Registered at Amtsgericht Duesseldorf: HRB 46611 >http://www.egenix.com/company/contact/ > http://www.malemburg.com/ > > > > Top-posted from my Windows Phone > > > > -Original Message- > > From: "Serhiy Storchaka" > > Sent: 1/30/2016 10:22 > > To: "python-dev@python.org" > > Subject: Re: [Python-Dev] More optimisation ideas > > > > On 30.01.16 18:31, Steve Dower wrote: > >> On 30Jan2016 0645, Serhiy Storchaka wrote: > >>> $ ./python -m timeit -s "import codecs; from encodings.cp437 import > >>> decoding_table" -- "codecs.charmap_build(decoding_table)" > >>> 10 loops, best of 3: 4.36 usec per loop > >>> > >>> Getting rid from charmap_build() would save you at most 4.4 > microseconds > >>> per encoding. 0.0005 seconds if you have imported *all* standard > >>> encodings! > >> > >> Just as happy to be proven wrong. Perhaps I misinterpreted my original > >> profiling and then, embarrassingly, ran with the result for a long time > >> without retesting. > > > > AFAIK the most time is spent in system calls like stat or open. > > Archiving the stdlib into the ZIP file and using zipimport can decrease > > Python startup time (perhaps there is an open issue about this). > > > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org > > > > > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mal%40egenix.com > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
There are no example docs for it yet, but enough people have asked this week about how to set up a custom importer that I will write up a generic example case which will make sense for a lazy loader (need to file the issue before I forget). On Sun, 31 Jan 2016, 09:11 Donald Stufft wrote: > > On Jan 31, 2016, at 12:02 PM, Brett Cannon wrote: > > A lazy importer was added in Python 3.5 > > > Is there any docs on how to actually use the LazyLoader in 3.5? I can’t > seem to find any but I don’t really know the import system that well. > > - > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
> On Jan 31, 2016, at 12:02 PM, Brett Cannon wrote: > > A lazy importer was added in Python 3.5 Is there any docs on how to actually use the LazyLoader in 3.5? I can’t seem to find any but I don’t really know the import system that well. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
Brett Cannon python.org> writes: > > > A lazy importer was added in Python 3.5 and it was not possible > without the module spec refactoring. Wow... Thank you, I didn't know about that. Now for the next question: how am I supposed to use it? The following documentation leaves me absolutely clueless: """This class only works with loaders that define exec_module() as control over what module type is used for the module is required. For those same reasons, the loader’s create_module() method will be ignored (i.e., the loader’s method should only return None). Finally, modules which substitute the object placed into sys.modules will not work as there is no way to properly replace the module references throughout the interpreter safely; ValueError is raised if such a substitution is detected.""" (reference: https://docs.python.org/3/library/importlib.html#importlib.util.LazyLoader) I want to import lazily the modules from package "foobar.*", but not other modules as other libraries may depend on import side effects. How do I do that? The quoted snippet doesn't really help. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
A lazy importer was added in Python 3.5 and it was not possible without the module spec refactoring. On Sun, 31 Jan 2016, 08:57 Antoine Pitrou wrote: > > Hi, > > If you want to make startup time faster for a broad range of applications, > please consider adding a lazy import facility in the stdlib. > I recently tried to write a lazy import mechanism using import hooks > (to make it portable from 2.6 to 3.5), it seems nearly impossible to do > so (or, at least, for an average Python programmer like me). > > This would be much more useful (for actual users, not for architecture > astronauts) than refactoring the importlib APIs in each feature version... > > Thanks in advance > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
Hi, If you want to make startup time faster for a broad range of applications, please consider adding a lazy import facility in the stdlib. I recently tried to write a lazy import mechanism using import hooks (to make it portable from 2.6 to 3.5), it seems nearly impossible to do so (or, at least, for an average Python programmer like me). This would be much more useful (for actual users, not for architecture astronauts) than refactoring the importlib APIs in each feature version... Thanks in advance Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
On 30.01.2016 20:15, Steve Dower wrote: > Brett tried freezing the entire stdlib at one point (as we do for parts of > importlib) and reported no significant improvement. Since that rules out code > compilation as well as the OS calls, it'd seem the priority is to execute > less code on startup. > > Details of that work were posted to python-dev about twelve months ago, IIRC. > Maybe a little longer. Freezing the entire stdlib does improve the startup time, simply because it removes stat calls, which dominate the startup time at least on Unix. It also allows sharing the stdlib byte code in memory, since it gets stored in static C structs which the OS will happily mmap into multiple processes for you without any additional effort. Our eGenix PyRun does exactly that. Even though the original motivation is a different one, the gained improvement in startup time is a nice side effect: http://www.egenix.com/products/python/PyRun/ Aside: The encodings don't really make much difference here. The dictionaries aren't all that big, so generating them on the fly doesn't really create much overhead. The trade off in terms of maintainability/speed definitely leans toward maintainability. For the larger encoding tables we already have C implementations with appropriate data structures to make lookup speed vs. storage needs efficient. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jan 31 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ > Top-posted from my Windows Phone > > -Original Message- > From: "Serhiy Storchaka" > Sent: 1/30/2016 10:22 > To: "python-dev@python.org" > Subject: Re: [Python-Dev] More optimisation ideas > > On 30.01.16 18:31, Steve Dower wrote: >> On 30Jan2016 0645, Serhiy Storchaka wrote: >>> $ ./python -m timeit -s "import codecs; from encodings.cp437 import >>> decoding_table" -- "codecs.charmap_build(decoding_table)" >>> 10 loops, best of 3: 4.36 usec per loop >>> >>> Getting rid from charmap_build() would save you at most 4.4 microseconds >>> per encoding. 0.0005 seconds if you have imported *all* standard >>> encodings! >> >> Just as happy to be proven wrong. Perhaps I misinterpreted my original >> profiling and then, embarrassingly, ran with the result for a long time >> without retesting. > > AFAIK the most time is spent in system calls like stat or open. > Archiving the stdlib into the ZIP file and using zipimport can decrease > Python startup time (perhaps there is an open issue about this). > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mal%40egenix.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More optimisation ideas
I have opened http://bugs.python.org/issue26252 to track writing the example (and before ppl go playing with the lazy loader, be aware of http://bugs.python.org/issue26186). On Sun, 31 Jan 2016 at 09:26 Brett Cannon wrote: > There are no example docs for it yet, but enough people have asked this > week about how to set up a custom importer that I will write up a generic > example case which will make sense for a lazy loader (need to file the > issue before I forget). > > On Sun, 31 Jan 2016, 09:11 Donald Stufft wrote: > >> >> On Jan 31, 2016, at 12:02 PM, Brett Cannon wrote: >> >> A lazy importer was added in Python 3.5 >> >> >> Is there any docs on how to actually use the LazyLoader in 3.5? I can’t >> seem to find any but I don’t really know the import system that well. >> >> - >> Donald Stufft >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 >> DCFA >> >> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fun with ancient unsupported platforms
On 01/28/2016 06:57 AM, Larry Hastings wrote: It's currently 2016. Perhaps it's time to remove all vestiges of these unsupported operating systems nobody's cared about since a year that started with a '1'? We dropped support for Irix in 2.3. We dropped support for Irix threads in 3.2. All our supported platforms have Thread Local Storage (TLS) support. Maybe we can drop our 250-line portable TLS library from Python/thread.c? //arry/ p.s. Derpy code in Python/thread_nt.h. It literally looks like this: /* use native Windows TLS functions */ #define Py_HAVE_NATIVE_TLS #ifdef Py_HAVE_NATIVE_TLS It seems this developer had the short-term memory of a goldfish. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
