Re: [Python-Dev] Completing the email6 API changes.
> Glenn writes: > > Steve writes: >> OTOH, if the message is structured >> >> multipart/related >> multipart/alternative >> text/plain >> text/html >> image/png >> image/png >> >> the receiver can infer that the images are related to both text/* >> parts and DTRT for each. >With the images being treated as attachments. Or is there a syntax to >allow the text/html to embed the images and the text/plain to see them >as attachments? I believe the above is that syntax. But the standard doesn't say anything about this. The standard for multipart/alternative is RFC 2046, which doesn't know about multipart/related. RFC 2387 doesn't update RFC 2046, so it doesn't say anything about multipart/alternative within multipart/related, either. >I think the text/html wants to refer to things within its containing >multipart/related, but am not sure if that allows the intervening >multipart/alternative. I don't see why not. But it would depend on the implementations, which we'll have to test before recommending the structure I (theoretically :-) prefer.e ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: Redesigning extension modules
Le Sun, 1 Sep 2013 02:19:48 +1000, Nick Coghlan a écrit : > On 1 Sep 2013 00:10, "Stefan Behnel" wrote: > > > > *bump* > > > > Does this sound like a viable solution? > > This isn't likely to progress until we have Eric's PEP 451 to a point > where it's ready for python-dev discussion and pronouncement. > > However, the revised loader API is being designed to allow for the > loader returning arbitrary objects, so something along these lines > should work. There will likely be some adjustments to the API > signature to allow extension modules to optionally support reloading > if they so desire. I think the biggest challenge here is to propose an API that's simple and easy to use (i.e. that doesn't make extension module writing more complicated than it currently is). The basic concept of putting custom module objects in sys.modules is sound, IMHO. As for "extension module as a wrapper", though, it shounds like the kind of complication I would personally prefer to stay away from. Also, it would make extension modules less like Python modules, rather than more. Regards Antoine. > > Cheers, > Nick. > > > > > Stefan > > > > > > Stefan Behnel, 25.08.2013 14:36: > > > Hi, > > > > > > thanks for bringing this up. It clearly shows that there is more > > > to this problem than I initially thought. > > > > > > Let me just add one idea that your post gave me. > > > > > > PJ Eby, 25.08.2013 06:12: > > >> My "Importing" package offers lazy imports by creating module > > >> objects in sys.modules that are a subtype of ModuleType, and use > > >> a __getattribute__ hook so that trying to use them fires off a > > >> reload() of the module. > > > > > > I wonder if this wouldn't be an approach to fix the reloading > > > problem in general. What if extension module loading, at least > > > with the new scheme, didn't return the module object itself and > > > put it into sys.modules but created a wrapper that redirects its > > > __getattr__ and __setattr__ to the actual module object? That > > > would have a tiny performance impact on attribute access, but I'd > > > expect that to be negligible given that the > usual > > > reason for the extension module to exist is that it does > > > non-trivial > stuff > > > in whatever its API provides. Reloading could then really create a > > > completely new module object and replace the reference inside of > > > the > wrapper. > > > > > > That way, code that currently uses "from extmodule import xyz" > > > would continue to see the original version of the module as of > > > the time of its import, and code that just did "import extmodule" > > > and then used > attribute > > > access at need would always see the current content of the module > > > as it > was > > > last loaded. I think that, together with keeping module global > > > state in > the > > > module object itself, would nicely fix both cases. > > > > > > Stefan > > > > > > ___ > > Python-Dev mailing list > > [email protected] > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a new tracemalloc module to trace memory allocations
Le Sun, 1 Sep 2013 11:36:51 +1000, Nick Coghlan a écrit : > +1 from me for both tracemalloc and failmalloc in the same vein as > faulthandler (and using similar API concepts to faulthandler). > > While I like the flat top level naming structure, we should clearly > document these as implementation dependent runtime services. Other > implementations may not provide them at all, and even if they do, many > details will likely be different. So, it sounds like Victor may write a new PEP soon :-) +1 from me on a flat namespace. Non-designed naming hierarchies are just a pain to work with. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SEEK_* constants in io and os
Le Sun, 1 Sep 2013 18:02:30 -0700, Eli Bendersky a écrit : > Hello, > > I was looking at the possibility of replacing the SEEK_* constants by > IntEnums, and the first thing that catches attention is that these > constants are defined in both Lib/os.py and Lib/io.py; both places > also recently started supporting SEEK_HOLE and SEEK_DATA (though here > io refers to os.SEEK_HOLE and os.SEEK_DATA). What is the runtime cost of doing so? os is a fundamental module that is imported by almost every Python program. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: Redesigning extension modules
On 2 September 2013 18:16, Antoine Pitrou wrote: > Le Sun, 1 Sep 2013 02:19:48 +1000, > Nick Coghlan a écrit : >> On 1 Sep 2013 00:10, "Stefan Behnel" wrote: >> > >> > *bump* >> > >> > Does this sound like a viable solution? >> >> This isn't likely to progress until we have Eric's PEP 451 to a point >> where it's ready for python-dev discussion and pronouncement. >> >> However, the revised loader API is being designed to allow for the >> loader returning arbitrary objects, so something along these lines >> should work. There will likely be some adjustments to the API >> signature to allow extension modules to optionally support reloading >> if they so desire. > > I think the biggest challenge here is to propose an API that's simple > and easy to use (i.e. that doesn't make extension module writing more > complicated than it currently is). > > The basic concept of putting custom module objects in sys.modules is > sound, IMHO. The hook API I currently have in mind is a two step initialisation: PyImport_PrepareNAME (optional) PyImport_ExecNAME If you don't define prepare, the import system takes care of creating a module object for you, and passing it in to the exec hook. The return result from that is just an integer indicating success or failure (on failure, an exception should be set). If you *do* define the prepare hook, then it's similar to the existing init hook, but receives a PEP 451 module spec object with info about the module being imported (see PEP 451 for the draft details) and is permitted to return an arbitrary PyObject reference. The main open questions I have are how to deal with clearly indicating whether modules support in-place reloading, unloading, loading in subinterpreters and/or loading a second copy in the same interpreter. That's actually more a question for PEP 451 though, so I'll post some more detailed thoughts on that over on import-sig, which may eventually make their way into a PEP 451 draft. > As for "extension module as a wrapper", though, it shounds like the > kind of complication I would personally prefer to stay away from. Also, > it would make extension modules less like Python modules, rather than > more. Yeah, that will be allowed (since we'll probably permit returning arbitrary objects), but definitely not required. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SEEK_* constants in io and os
On Mon, Sep 2, 2013 at 1:24 AM, Antoine Pitrou wrote: > Le Sun, 1 Sep 2013 18:02:30 -0700, > Eli Bendersky a écrit : > > Hello, > > > > I was looking at the possibility of replacing the SEEK_* constants by > > IntEnums, and the first thing that catches attention is that these > > constants are defined in both Lib/os.py and Lib/io.py; both places > > also recently started supporting SEEK_HOLE and SEEK_DATA (though here > > io refers to os.SEEK_HOLE and os.SEEK_DATA). > > What is the runtime cost of doing so? os is a fundamental module that is > imported by almost every Python program. > Theoretically, it should be very low given that we just need to add an import and define one class. os already does a number of things in its toplevel (mostly a few imports which transitively do other things). Compounded with import caching, since this is done just once per run, doesn't seem like a problem. Empirically, I tried measuring it but I can't discern a difference with/without translating SEEK_* to enums. There's a fluctuation of ~1usec which I can't distinguish from noise. Let me know if you have a good methodology of benchmarking these things Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SEEK_* constants in io and os
Le Mon, 2 Sep 2013 06:18:31 -0700, Eli Bendersky a écrit : > On Mon, Sep 2, 2013 at 1:24 AM, Antoine Pitrou > wrote: > > > Le Sun, 1 Sep 2013 18:02:30 -0700, > > Eli Bendersky a écrit : > > > Hello, > > > > > > I was looking at the possibility of replacing the SEEK_* > > > constants by IntEnums, and the first thing that catches attention > > > is that these constants are defined in both Lib/os.py and > > > Lib/io.py; both places also recently started supporting SEEK_HOLE > > > and SEEK_DATA (though here io refers to os.SEEK_HOLE and > > > os.SEEK_DATA). > > > > What is the runtime cost of doing so? os is a fundamental module > > that is imported by almost every Python program. > > > > Theoretically, it should be very low given that we just need to add an > import and define one class. os already does a number of things in its > toplevel (mostly a few imports which transitively do other things). > Compounded with import caching, since this is done just once per run, > doesn't seem like a problem. > > Empirically, I tried measuring it but I can't discern a difference > with/without translating SEEK_* to enums. There's a fluctuation of > ~1usec which I can't distinguish from noise. Let me know if you have > a good methodology of benchmarking these things How did you get that result? You have to remove to "os" from sys.modules before importing it again, otherwise "import os" will simply return the already imported module. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SEEK_* constants in io and os
Le Mon, 2 Sep 2013 15:45:22 +0200, Antoine Pitrou a écrit : > Le Mon, 2 Sep 2013 06:18:31 -0700, > Eli Bendersky a écrit : > > On Mon, Sep 2, 2013 at 1:24 AM, Antoine Pitrou > > wrote: > > > > > Le Sun, 1 Sep 2013 18:02:30 -0700, > > > Eli Bendersky a écrit : > > > > Hello, > > > > > > > > I was looking at the possibility of replacing the SEEK_* > > > > constants by IntEnums, and the first thing that catches > > > > attention is that these constants are defined in both Lib/os.py > > > > and Lib/io.py; both places also recently started supporting > > > > SEEK_HOLE and SEEK_DATA (though here io refers to os.SEEK_HOLE > > > > and os.SEEK_DATA). > > > > > > What is the runtime cost of doing so? os is a fundamental module > > > that is imported by almost every Python program. > > > > > > > Theoretically, it should be very low given that we just need to add > > an import and define one class. os already does a number of things > > in its toplevel (mostly a few imports which transitively do other > > things). Compounded with import caching, since this is done just > > once per run, doesn't seem like a problem. > > > > Empirically, I tried measuring it but I can't discern a difference > > with/without translating SEEK_* to enums. There's a fluctuation of > > ~1usec which I can't distinguish from noise. Let me know if you have > > a good methodology of benchmarking these things > > How did you get that result? You have to remove to "os" from > sys.modules before importing it again, otherwise "import os" will > simply return the already imported module. Oh and remove "enum" too... Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SEEK_* constants in io and os
On Mon, Sep 2, 2013 at 6:51 AM, Antoine Pitrou wrote: > Le Mon, 2 Sep 2013 15:45:22 +0200, > Antoine Pitrou a écrit : > > Le Mon, 2 Sep 2013 06:18:31 -0700, > > Eli Bendersky a écrit : > > > On Mon, Sep 2, 2013 at 1:24 AM, Antoine Pitrou > > > wrote: > > > > > > > Le Sun, 1 Sep 2013 18:02:30 -0700, > > > > Eli Bendersky a écrit : > > > > > Hello, > > > > > > > > > > I was looking at the possibility of replacing the SEEK_* > > > > > constants by IntEnums, and the first thing that catches > > > > > attention is that these constants are defined in both Lib/os.py > > > > > and Lib/io.py; both places also recently started supporting > > > > > SEEK_HOLE and SEEK_DATA (though here io refers to os.SEEK_HOLE > > > > > and os.SEEK_DATA). > > > > > > > > What is the runtime cost of doing so? os is a fundamental module > > > > that is imported by almost every Python program. > > > > > > > > > > Theoretically, it should be very low given that we just need to add > > > an import and define one class. os already does a number of things > > > in its toplevel (mostly a few imports which transitively do other > > > things). Compounded with import caching, since this is done just > > > once per run, doesn't seem like a problem. > > > > > > Empirically, I tried measuring it but I can't discern a difference > > > with/without translating SEEK_* to enums. There's a fluctuation of > > > ~1usec which I can't distinguish from noise. Let me know if you have > > > a good methodology of benchmarking these things > > > > How did you get that result? You have to remove to "os" from > > sys.modules before importing it again, otherwise "import os" will > > simply return the already imported module. > > Oh and remove "enum" too... > Yes, now I see a 500 usec difference timed within the Python script. When timing the whole execution of Python: $ sudo nice -n -20 perf stat -r 100 python -c 'import os' It still gets lost in the noise, though. I usually get around 34 ms per run with 1 - 1.5% jitter. Since 1.5% of 34 ms is ~0.5 ms, it's difficult to distinguish the runs clearly. That 0.5 ms seems to be a one-time penalty for the importing of enum (most time goes to importing, not defining the new class based on IntEnum), no matter where/how it's used in the stdlib and the user code. Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SEEK_* constants in io and os
2013/9/2 Eli Bendersky : > Yes, now I see a 500 usec difference timed within the Python script. When > timing the whole execution of Python: (...) Can you please provide the list of imported modules by: python -c 'import sys; print(sys.modules)' For python with default options and for python with -S (no site module) options? And also with your patch? Python should be fast to write "hello world" (or "python -c pass), it's a dummy but common benchmark (to compare Python to other VM / other programming languages). Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SEEK_* constants in io and os
On Mon, Sep 2, 2013 at 8:48 AM, Victor Stinner wrote: > 2013/9/2 Eli Bendersky : > > Yes, now I see a 500 usec difference timed within the Python script. When > > timing the whole execution of Python: (...) > > Can you please provide the list of imported modules by: > python -c 'import sys; print(sys.modules)' > > For python with default options and for python with -S (no site > module) options? And also with your patch? > > Python should be fast to write "hello world" (or "python -c pass), > it's a dummy but common benchmark (to compare Python to other VM / > other programming languages). > The sorted list for both default and -S (they're identical) is http://pastebin.com/4vzSMCu7 - there are 55 entries there, including things like itertools, heapq, functools and collections. With my patch, enum is also added (which makes sense since os is in the list). So the 0.5ms increase for the 34ms runtime kind-of makes sense. Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Completing the email6 API changes.
On Mon, 02 Sep 2013 16:06:53 +0900, "Stephen J. Turnbull" wrote: > > Glenn writes: > > > Steve writes: > > >> OTOH, if the message is structured > >> > >> multipart/related > >> multipart/alternative > >> text/plain > >> text/html > >> image/png > >> image/png > >> > >> the receiver can infer that the images are related to both text/* > >> parts and DTRT for each. > > >With the images being treated as attachments. Or is there a syntax to > >allow the text/html to embed the images and the text/plain to see them > >as attachments? > > I believe the above is that syntax. But the standard doesn't say > anything about this. The standard for multipart/alternative is RFC > 2046, which doesn't know about multipart/related. RFC 2387 doesn't > update RFC 2046, so it doesn't say anything about > multipart/alternative within multipart/related, either. > > >I think the text/html wants to refer to things within its containing > >multipart/related, but am not sure if that allows the intervening > >multipart/alternative. > > I don't see why not. But it would depend on the implementations, > which we'll have to test before recommending the structure I > (theoretically :-) prefer.e I'm still not understanding how the text/plain part *refers* to the related parts. I can understand the structure Glen found in Applemail: a series of text/plain parts interspersed with image/jpg, with all parts after the first being marked 'Contentent-Disposition: inline'. Any MUA that can display text and images *ought* to handle that correctly and produce the expected result. But that isn't what your structure above would produce. If you did: multipart/related multipart/alternative text/html text/plain image/png text/plain image/png text/plain and only referred to the png parts in the text/html part and marked all the parts as 'inline' (even though that is irrelevant in the text/html related case), an MUA that *knew* about this technique *could* display it "correctly", but an MUA that is just following the standards most likely won't. I don't see any way short of duplicating the image parts to make it likely that a typical MUA would display images for both a text/plain sequence and a text/html related part. On the other hand, my experience with MUAs is actually quite limited :) Unless there is some standard for referring to related parts in a text/plain part? I'm not aware of any, but you have much more experience with this stuff than I do. (Even text/enriched (RFC 1896) doesn't seem to have one, though of course there could be "extensions" that define both that and the font support you used as an example.) --David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Completing the email6 API changes.
On 9/2/2013 2:40 PM, R. David Murray wrote: I'm still not understanding how the text/plain part*refers* to the related parts. I don't think the text/plain part can refer to the related parts, but, like you, am willing to be educated if there is a way; but while the text/html may be able to if things like cid: URIs can reach up a level in a given MUA, the text/plain would be left with the additional parts being attachments, methinks. This is less interesting than the technique Apple Mail uses, but more interesting than not even seeing the attached pictures. MUAs tend to be able to display what they produce themselves, but I have situations where they don't handle what other MUAs produce. One nice thing about this email6 toolkit might be the ability to produce, more easily than before, a variety of MIME combinations to exercise and test a variety of MUAs. While perhaps most of them have been tested with some obviously standard MIME combinations, I suspect most of them will produce strange results with combinations that are out of the ordinary. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Completing the email6 API changes.
On Mon, 02 Sep 2013 15:52:59 -0700, Glenn Linderman wrote: > MUAs tend to be able to display what they produce themselves, but I have > situations where they don't handle what other MUAs produce. > > One nice thing about this email6 toolkit might be the ability to > produce, more easily than before, a variety of MIME combinations to > exercise and test a variety of MUAs. While perhaps most of them have > been tested with some obviously standard MIME combinations, I suspect > most of them will produce strange results with combinations that are out > of the ordinary. Yeah, RFC compliance and other types of testing is something I want this package to be good for. The API under discussion here, though, is oriented toward people using the library for easily generating emails from their application and/or easily accessing the information from emails sent to their application. --David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Completing the email6 API changes.
R. David Murray writes: > I'm still not understanding how the text/plain part *refers* to the > related parts. Like this: "Check out this picture of my dog!" Or this: "The terms of the contract are found in the attached PDF. Please print it and sign it, then return it by carrier pigeon (attached)." With this structure multipart/alternative text/plain multipart/related text/html application/pdf application/rfc6214-transport the rendering of the text/plain part will not show evidence of the PDF at all (eg, a view/download button), at least in some of the MUAs I've tested. And it *should* not, in an RFC-conforming MUA. > I can understand the structure Glen found in Applemail: > a series of text/plain parts interspersed with image/jpg, with all parts > after the first being marked 'Contentent-Disposition: inline'. Any MUA > that can display text and images *ought* to handle that correctly and > produce the expected result. But that isn't what your structure above > would produce. If you did: > > multipart/related > multipart/alternative > text/html > text/plain > image/png > text/plain > image/png > text/plain > > and only referred to the png parts in the text/html part and marked all > the parts as 'inline' (even though that is irrelevant in the text/html > related case), an MUA that *knew* about this technique *could* display it > "correctly", but an MUA that is just following the standards most > likely won't. OK, I see that now. It requires non-MIME information about the treatment of the root entity by the implementation. On the other hand, it shouldn't *hurt*. RFC 2387 explicitly specifies that at least some parts of a contained multipart/related part should be able to refer to entities related via the containing multipart/related. Since it does not mention *any* restrictions on contained root entities, I take it that it implicitly specifies that any contained multipart may make such references. But I suspect it's not implemented by most MUAs. I'll have to test. > I don't see any way short of duplicating the image parts to make it > likely that a typical MUA would display images for both a text/plain > sequence and a text/html related part. On the other hand, my experience > with MUAs is actually quite limited :) > > Unless there is some standard for referring to related parts in a > text/plain part? No, the whole point is that we MUA implementers *know* that there is no machine-parsable way to refer to the related parts in text/plain, and therefore the only way to communicate even the *presence* of the attachment in multipart/related text/plain image/jpeg; name="dog-photo.jpg" to the receiving user is to make an exception in the implementation and treat it as multipart/mixed.[1] It *does* make sense, i.e., doesn't require any information not already available to the implementation. I wonder if use of external bodies could avoid the duplication in current implementations. Probably too fragile, though. Footnotes: [1] This is conformant to the RFC, as the mechanism of "relation" is explicitly application-dependent. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
