On Thu, 20 Jul 2017 at 22:11 Chris Jerdonek <chris.jerdo...@gmail.com> wrote:
> On Thu, Jul 20, 2017 at 8:49 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > > ... > > * Lazy loading can have a significant impact on startup time, as it > > means you don't have to pay for the cost of finding and loading > > modules that you don't actually end up using on that particular run > It should be mentioned that I have started designing an API to make using lazy loading much easier in Python 3.7 (i.e. "calling a single function" easier), but I still have to write the tests and such before I propose a patch and it will still be mainly for apps that know what they are doing since lazy loading makes debugging import errors harder. > > > > We've historically resisted adopting these techniques for the standard > > library because they *do* make things more complicated *and* harder to > > debug relative to plain old eagerly imported dynamic Python code. > > However, if we're going to recommend them as good practices for 3rd > > party developers looking to optimise the startup time of their Python > > applications, then it makes sense for us to embrace them for the > > standard library as well, rather than having our first reaction be to > > write more hand-crafted C code. > > Are there any good write-ups of best practices and techniques in this > area for applications (other than obvious things like avoiding > unnecessary imports)? I'm thinking of things like how to structure > your project, things to look for, developer tools that might help, and > perhaps third-party runtime libraries? > Nothing beyond "profile your application" and "don't do stuff during import as a side-effect" that I'm aware of. -Brett > > --Chris > > > > > > > On that last point, it's also worth keeping in mind that we have a > > much harder time finding new C-level contributors than we do new > > Python-level ones, and have every reason to expect that problem to get > > worse over time rather than better (since writing and maintaining > > handcrafted C code is likely to go the way of writing and maintaining > > handcrafted assembly code as a skillset: while it will still be > > genuinely necessary in some contexts, it will also be an increasingly > > niche technical specialty). > > > > Starting to migrate to using Cython for our acceleration modules > > instead of plain C should thus prove to be a win for everyone: > > > > - Cython structurally avoids a lot of typical bugs that arise in > > hand-coded extensions (e.g. refcount bugs) > > - by design, it's much easier to mentally switch between Python & > > Cython than it is between Python & C > > - Cython accelerated modules are easier to adapt to other interpeter > > implementations than handcrafted C modules > > - keeping Python modules and their C accelerated counterparts in sync > > will be easier, as they'll mostly be using the same code > > - we'd be able to start writing C API test cases in Cython rather than > > in handcrafted C (which currently mostly translates to only testing > > them indirectly) > > - CPython's own test suite would naturally help test Cython > > compatibility with any C API updates > > - we'd have an inherent incentive to help enhance Cython to take > > advantage of new C API features > > > > The are some genuine downsides in increasing the complexity of > > bootstrapping CPython when all you're starting with is a VCS clone and > > a C compiler, but those complications are ultimately no worse than > > those we already have with Argument Clinic, and hence amenable to the > > same solution: if we need to, we can check in the generated C files in > > order to make bootstrapping easier. > > > > Cheers, > > Nick. > > > > -- > > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
