[Python-Dev] datetime nanosecond support (ctd?)
newbie first post on this list, if what follows is of context ... Hi all, I'm struggling with issue per the subject, read different threads and issue http://bugs.python.org/issue15443 that started 2012 still opened as of today. Isn't there a legitimate case for nanosecond support? it's all over the place in 'struct timespec' and maybe wrongly I always found python and C were best neighbors. That's for the notional aspect. More practically, aren't we close enough yet with current hardware, PTP and the likes, this deserves more consideration? Maybe this has been mentioned before but the limiting factor isn't just getting nanoseconds, but anything sub-microseconds wont work with the current format. OpcUA that I was looking right now has 10-th us resolution, so really cares about 100ns, but the datetime 1us simply wont cut it. Regards, Matthieu ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support (ctd?)
Thanks Stephen elaborating on the process. and apologies, I was dismissing the last point only half jokingly. I read the comment for strftime / strptime in the report as meant to remember to implement it. It seems picking a new format letter (or keep using "%f" if acceptable) that would accept/produce up to 9 characters instead of 6 for nanoseconds would do most of the trick. Maybe there's no issue or I don't understand it. That completes my chant to awaken the Elderers! Regards, Matthieu On 12/10/14 9:10 PM, Stephen J. Turnbull wrote: mdcb808 writes: > These are typically discussed on this list or using the bug > tracker? I think this discussion belongs on python-dev because the requirement is clear, but a full specification involves backward compatibility with older interfaces, and clearly different people place different values on the various aspects of the problem. It makes sense to go straight to tracker when the design is done or obvious, or backward compatibility is clearly not involved. The tracker is also the place to record objective progress (patches, tests, bug reports). Python-Dev is where minds meet. What Nick is saying is that more design needs to be done to resolve differences of opinion on the best way to move forward. > maybe YNGTNI applied, Evidently not. If a senior developer really thought it's a YAGNI, the issue would have been closed WONTFIX. It seems the need is believable. > not clear why it's not there after 2 eyars. There's only one reason you need to worry about: nobody wrote a patch that meets the concerns of the senior developers (one of which is that concerns raised by anybody remain unresolved; they don't always have strong opinions themselves).[1] > - not sure what's at stake with the strp/ftime() but cant imagine > it's a biggie If you want something done, you don't necessarily need to supply a patch. But you have to do more to move things forward that just say "I can't imagine why anybody worries about that." You have to find out what their worries are, and explain that their worries won't be realized in the case of the obvious design (eg, the one you presented), or provide a design that avoids realizing those worries. Or you can get the senior developers to overrule the worriers, but you need a relatively important use case to make that fly. Or you can get somebody else to do some of the above, but that also requires presenting an important use case (to that somebody). Footnotes: [1] That's not 100% accurate: there is a shortage of senior developer time for reviewing patches. If it's simply that nobody has looked at the issue, simply bringing it up may be sufficient to get attention and then action. But Nick's response makes it clear that doesn't apply to this issue; people have looked at the issue and have unresolved concerns. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support (ctd?)
Agreed with Antoine, strftime/strptime are somewhat different concerns. Doesn't mean thay cannot be fixed at the same time but it's a bit a separate. I'm not sure this thread / discussion has reached critical mass yet, meanwhile I was looking at what was involved and came up with this half-baked patch - by no way meant to be complete or correct, but rather get a feel to it. I can't help thinking this is much more involved than I first expected, and what I try to achieve should be reasonably simple. Stepping back a little, I wonder if the datetime module is really the right location, that has constructor(year, month, day, ..., second, microsecond) - with 0quite ugly, in fact nothing looks quite right. datetime seems to be dealing with 'calendar time' stuff - and to some extent, maybe, shouldn't concern itself so much with fraction of seconds. I'm pondering now if the 'time' module that seems to be dealing with 'computer clock time' wouldn't be a better target to introduce struct_timespec and the like. I'd be curious to hear back from this list, if this deserves more investigation or what might be a better way to go. Regards, Matthieu On 12/11/2014 11:46 AM, Antoine Pitrou wrote: On Thu, 11 Dec 2014 13:43:05 -0600 Skip Montanaro wrote: On Thu, Dec 11, 2014 at 1:23 PM, Antoine Pitrou wrote: I think strftime / strptime support is a low-priority concern on this topic, and can probably be discussed independently of the core nanosecond support. Might be low-priority, but with %f support as a template, supporting something to specify nanoseconds should be pretty trivial. The hardest question will be to convince ourselves that we aren't choosing a format code which some other strftime/strptime implementation is already using. In addition, ISTR that one of the use cases was analysis of datetime data generated by other applications which has nanosecond resolution. One of the use cases is to deal with OS-generated timestamps without losing information. As long as you don't need to represent or parse those timestamps, strptime / strftime don't come into the picture. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mdcb808%40gmail.com -- Matthieu BecGMTO Corp cell : +1 626 425 7923 251 S Lake Ave, Suite 300 phone: +1 626 204 0527 Pasadena, CA 91101 diff -r 5754f069b123 Include/datetime.h --- a/Include/datetime.h Sat Nov 22 12:54:57 2014 -0800 +++ b/Include/datetime.h Tue Dec 16 09:04:35 2014 -0800 @@ -18,18 +18,34 @@ * 5 minute 1 byte, 0-59 * 6 second 1 byte, 0-59 * 7 usecond 3 bytes, 0-99 - * 10 + * 10 nsecond 2 bytes, 0-999 + * 12 */ /* # of bytes for year, month, and day. */ #define _PyDateTime_DATE_DATASIZE 4 -/* # of bytes for hour, minute, second, and usecond. */ -#define _PyDateTime_TIME_DATASIZE 6 +/* # of bytes for hour, minute, second, usecond and nsecond. */ +#define _PyDateTime_TIME_DATASIZE 8 +#define _PyDateTime_TIME_DATASIZE_V0 6 /* older version used for backward compatibility access */ -/* # of bytes for year, month, day, hour, minute, second, and usecond. */ -#define _PyDateTime_DATETIME_DATASIZE 10 +/* # of bytes for year, month, day, hour, minute, second, usecond and nsecond. */ +#define _PyDateTime_DATETIME_DATASIZE 12 +#define _PyDateTime_DATETIME_DATASIZE_V0 10 /* older version used for backward compatibility access */ +/* max usecond is hex(1e6) = 0x0f4240, hence the 4 upper bits that are not used to encode the + * number of milliseconds can be used for pickle versioning. + */ +#define _PyDateTime_USECOND_MSB_VALUE_MASK 0x0f +#define _PyDateTime_USECOND_MSB_PICKLEVERSION_MASK 0xf0 +/* up to 15 pickle versions are available + * version 0 is vanilla + * version 1 adds nanosecond (this version) + * version 2..15 are available for future upgrades + */ +#define _PyDateTime_PICKLE_V0_VANILLA 0 +#define _PyDateTime_PICKLE_V1_NANOSECOND 1 +#define _PyDateTime_PICKLE_CURRENT_VERSION _PyDateTime_PICKLE_V1_NANOSECOND typedef struct { @@ -38,6 +54,7 @@ int days; /* -MAX_DELTA_DAYS <= days <= MAX_DELTA_DAYS */ int seconds;/* 0 <= seconds < 24*3600 is invariant */ int microseconds; /* 0 <= microseconds < 100 is invariant */ +int nanoseconds;/* 0 <= nanoseconds < 1000 is invariant */ } PyDateTime_Delta; typedef struct @@ -122,18 +139,30 @@ #define PyDateTime_DATE_GET_MINUTE(o) (((PyDateTime_DateTime*)o)->data[5]) #define PyDateTime_DATE_GET_SECOND(o) (((PyDateTime_DateTime*)o)->data[6]) #define PyDateTime_DATE_GET_MICROSECOND(o) \ -PyDateTime_DateTime*)
Re: [Python-Dev] datetime nanosecond support (ctd?)
yes that was mentioned in this thread, %nN looks quite reasonable. still, I'd like to steer the conversation back to the other aspect - where should something like struct_timespec land in the first place, is datetime really the best to capture that? Most of the conversation has been revolving around strftime/strptime. That seems to validate Antoine's point in the first place. Let's see what people say but maybe this thread should end to restart as separate topics? Regards, Matthieu On 12/16/14 11:08 AM, Skip Montanaro wrote: On Tue, Dec 16, 2014 at 11:10 AM, matthieu bec mailto:[email protected]>> wrote: > Agreed with Antoine, strftime/strptime are somewhat different concerns. > Doesn't mean thay cannot be fixed at the same time but it's a bit a > separate. Which reminds me... Somewhere else (maybe elsewhere in this thread? maybe on a bug tracker issue?) someone mentioned that Ruby uses %N for fractions of a second (and %L specifically for milliseconds). Here's the bit from the Ruby strftime doc: %L - Millisecond of the second (000..999) %N - Fractional seconds digits, default is 9 digits (nanosecond) %3N millisecond (3 digits) %6N microsecond (6 digits) %9N nanosecond (9 digits) %12N picosecond (12 digits) There's no obvious reason I can see to reinvent this particular wheel, at least the %N spoke. The only question might be whether to modify Python's existing %f format to accept a precision (defaulting to 6), or add %N in a manner similar (or identical) to Ruby's semantics. Skip ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mdcb808%40gmail.com -- Matthieu BecGMTO Corp cell : +1 626 425 7923 251 S Lake Ave, Suite 300 phone: +1 626 204 0527 Pasadena, CA 91101 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support (ctd?)
Maybe what I meant with `nothing looks quite right': seconds as float, microseconds as float, nanosecond as 0..999, nanoseconds as 0..9 with mandatory keyword that precludes microseconds - all can be made to work, none seems completely satisfying. In fact, I don't really have a use for it from python - but something would be needed in C for the implementation of datetime.from_timespec and time.from_timespec that calls the constructor PyObjectCall_CallFunction(clas,"...",...) - can this happen and remain hidden from the python layer? Regards, Matthieu On 12/16/14 12:45 PM, Alexander Belopolsky wrote: On Tue, Dec 16, 2014 at 12:10 PM, matthieu bec mailto:[email protected]>> wrote: I wonder if the datetime module is really the right location, that has constructor(year, month, day, ..., second, microsecond) - with 0 01:02:03.123456789 and time(1, 2, 3, 4, nanosecond=123456789) -> error Users will probably be encouraged to avoid positional form when specifying time to subsecond precision. I would say time(1, 2, 3, microsecond=4) is clearer than time(1, 2, 3, 4) anyways. Another option is to allow float for the "second" argument: time(1, 2, 3.123456789) -> 01:02:03.123456789 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mdcb808%40gmail.com -- Matthieu BecGMTO Corp cell : +1 626 425 7923 251 S Lake Ave, Suite 300 phone: +1 626 204 0527 Pasadena, CA 91101 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support (ctd?)
On 12/16/14 3:28 PM, Matthieu Bec wrote: Maybe what I meant with `nothing looks quite right': seconds as float, microseconds as float, nanosecond as 0..999, nanoseconds as 0..9 with mandatory keyword that precludes microseconds - all can be made to work, none seems completely satisfying. In fact, I don't really have a use for it from python - but something would be needed in C for the implementation of datetime.from_timespec and time.from_timespec that calls the constructor that's the datetime.time.from_timespec btw. PyObjectCall_CallFunction(clas,"...",...) - can this happen and remain hidden from the python layer? Regards, Matthieu On 12/16/14 12:45 PM, Alexander Belopolsky wrote: On Tue, Dec 16, 2014 at 12:10 PM, matthieu bec mailto:[email protected]>> wrote: I wonder if the datetime module is really the right location, that has constructor(year, month, day, ..., second, microsecond) - with 0 01:02:03.123456789 and time(1, 2, 3, 4, nanosecond=123456789) -> error Users will probably be encouraged to avoid positional form when specifying time to subsecond precision. I would say time(1, 2, 3, microsecond=4) is clearer than time(1, 2, 3, 4) anyways. Another option is to allow float for the "second" argument: time(1, 2, 3.123456789) -> 01:02:03.123456789 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mdcb808%40gmail.com -- Matthieu BecGMTO Corp cell : +1 626 425 7923 251 S Lake Ave, Suite 300 phone: +1 626 204 0527 Pasadena, CA 91101 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support (ctd?)
python-n (for next) - just poking fun at the other thread On 12/16/14 4:17 PM, Guido van Rossum wrote: "Nothing looks quite right" is a common phenomenon when you're constrained by backward compatibility. The perfect solution would throw away compatibility, but that has its own downsides. So just go for what looks the least wrong. On Tue, Dec 16, 2014 at 3:28 PM, Matthieu Bec mailto:[email protected]>> wrote: Maybe what I meant with `nothing looks quite right': seconds as float, microseconds as float, nanosecond as 0..999, nanoseconds as 0..9 with mandatory keyword that precludes microseconds - all can be made to work, none seems completely satisfying. In fact, I don't really have a use for it from python - but something would be needed in C for the implementation of datetime.from_timespec and time.from_timespec that calls the constructor PyObjectCall_CallFunction(__clas,"...",...) - can this happen and remain hidden from the python layer? Regards, Matthieu On 12/16/14 12:45 PM, Alexander Belopolsky wrote: On Tue, Dec 16, 2014 at 12:10 PM, matthieu bec mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>> wrote: I wonder if the datetime module is really the right location, that has constructor(year, month, day, ..., second, microsecond) - with 0 01:02:03.123456789 and time(1, 2, 3, 4, nanosecond=123456789) -> error Users will probably be encouraged to avoid positional form when specifying time to subsecond precision. I would say time(1, 2, 3, microsecond=4) is clearer than time(1, 2, 3, 4) anyways. Another option is to allow float for the "second" argument: time(1, 2, 3.123456789 ) -> 01:02:03.123456789 _ Python-Dev mailing list [email protected] <mailto:[email protected]> https://mail.python.org/__mailman/listinfo/python-dev <https://mail.python.org/mailman/listinfo/python-dev> Unsubscribe: https://mail.python.org/__mailman/options/python-dev/__mdcb808%40gmail.com <https://mail.python.org/mailman/options/python-dev/mdcb808%40gmail.com> -- Matthieu BecGMTO Corp cell : +1 626 425 7923 251 S Lake Ave, Suite 300 phone: +1 626 204 0527 Pasadena, CA 91101 _ Python-Dev mailing list [email protected] <mailto:[email protected]> https://mail.python.org/__mailman/listinfo/python-dev <https://mail.python.org/mailman/listinfo/python-dev> Unsubscribe: https://mail.python.org/__mailman/options/python-dev/__guido%40python.org <https://mail.python.org/mailman/options/python-dev/guido%40python.org> -- --Guido van Rossum (python.org/~guido <http://python.org/~guido>) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mdcb808%40gmail.com -- Matthieu BecGMTO Corp cell : +1 626 425 7923 251 S Lake Ave, Suite 300 phone: +1 626 204 0527 Pasadena, CA 91101 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support (ctd?)
On 12/16/14 3:31 PM, Matthieu Bec wrote: On 12/16/14 3:28 PM, Matthieu Bec wrote: Maybe what I meant with `nothing looks quite right': seconds as float, microseconds as float, nanosecond as 0..999, nanoseconds as 0..9 with mandatory keyword that precludes microseconds - all can be made to work, none seems completely satisfying. In fact, I don't really have a use for it from python - but something would be needed in C for the implementation of datetime.from_timespec and time.from_timespec that calls the constructor that's the datetime.time.from_timespec btw. datetime.time.from_timespec actually makes no sense. PyObjectCall_CallFunction(clas,"...",...) - can this happen and remain hidden from the python layer? ... occured to me I might simply create a us datetime object and set its nanofield after. I'll try wrap up a recap proposal later. Regards, Matthieu ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] datetime nanosecond support (ctd?)
Attached patch defines a new type struct_timespec for the time module. A
new capsule exports the type along with to/from converters - opening a
bridge for C, and for example the datetime module.
Your comments welcomed. If people feel this is worth the effort and
going the right direction, I should be able to finish doco, unit-tests,
whatever else is missing with a bit of guidance and move on other
datetime aspects.
Regards,
Matthieu
diff -r 5754f069b123 Include/timemodule.h
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/Include/timemodule.h Wed Dec 17 18:21:28 2014 -0800
@@ -0,0 +1,56 @@
+#ifndef Py_TIMEMODULE_H
+#define Py_TIMEMODULE_H
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#ifdef HAVE_CLOCK_GETTIME
+typedef struct timespec _PyTime_timespec;
+#else
+typedef struct {
+time_t tv_sec; /* seconds */
+long tv_nsec; /* nanoseconds */
+} _PyTime_timespec;
+#endif
+
+
+#define PyTime_CAPSULE_NAME "time._C_API"
+
+typedef struct {
+PyTypeObject *StrucTimespecType;
+PyObject *(*PyStructTimespecFromCStructTimespec)(_PyTime_timespec *);
+int (*PyStructTimespecToCStructTimespec)(PyObject*, _PyTime_timespec *);
+} PyTime_C_API;
+
+
+#ifndef TIME_MODULE
+
+static PyTime_C_API *api = NULL;
+
+#define PyStructTimespecFromCStructTimespec \
+ api->PyStructTimespecFromCStructTimespec
+
+#define PyStructTimespecToCStructTimespec \
+ api->PyStructTimespecToCStructTimespec
+
+#define StructTimespec_Check(op) \
+ PyObject_TypeCheck(op, api->StrucTimespecType)
+
+#define StructTimespec_CheckExact(op) \
+ (Py_TYPE(op) == api->StrucTimespecType)
+
+static int
+import_time(void)
+{
+api = (PyTime_C_API*)PyCapsule_Import(PyTime_CAPSULE_NAME, 0);
+return (api != NULL) ? 0 : -1;
+}
+
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* !defined(Py_TIMEMODULE_H) */
+
diff -r 5754f069b123 Modules/timemodule.c
--- a/Modules/timemodule.c Sat Nov 22 12:54:57 2014 -0800
+++ b/Modules/timemodule.c Wed Dec 17 18:21:28 2014 -0800
@@ -30,6 +30,9 @@
#endif /* MS_WINDOWS */
#endif /* !__WATCOMC__ || __QNX__ */
+#define TIME_MODULE
+#include "timemodule.h"
+
/* Forward declarations */
static int floatsleep(double);
static PyObject* floattime(_Py_clock_info_t *info);
@@ -270,8 +273,25 @@
9,
};
+static PyStructSequence_Field struct_timespec_type_fields[] = {
+{"tv_sec", "seconds"},
+{"tv_nsec", "nanoseconds"},
+{0}
+};
+
+static PyStructSequence_Desc struct_timespec_type_desc = {
+"time.struct_timespec",
+"POSIX.1b structure for a time value.",
+struct_timespec_type_fields,
+2,
+};
+
+
static int initialized;
static PyTypeObject StructTimeType;
+static PyTypeObject StrucTimespecType;
+#define StructTimespec_Check(op) PyObject_TypeCheck(op, &StrucTimespecType)
+#define StructTimespec_CheckExact(op) (Py_TYPE(op) == &StrucTimespecType)
static PyObject *
@@ -1284,13 +1304,13 @@
PyDoc_STRVAR(module_doc,
"This module provides various functions to manipulate time values.\n\
\n\
-There are two standard representations of time. One is the number\n\
+There are three standard representations of time. One is the number\n\
of seconds since the Epoch, in UTC (a.k.a. GMT). It may be an integer\n\
or a floating point number (to represent fractions of seconds).\n\
The Epoch is system-defined; on Unix, it is generally January 1st, 1970.\n\
The actual value can be retrieved by calling gmtime(0).\n\
\n\
-The other representation is a tuple of 9 integers giving local time.\n\
+Another representation is a tuple of 9 integers giving local time.\n\
The tuple items are:\n\
year (including century, e.g. 1998)\n\
month (1-12)\n\
@@ -1305,6 +1325,11 @@
if it is 1, the time is given in the DST time zone;\n\
if it is -1, mktime() should guess based on the date and time.\n\
\n\
+The other representation is a tuple of 2 integers similar to Posix.1b\n\
+struct timespec. The tuple items are:\n\
+ seconds\n\
+ nanoseconds (0-9)\n\
+\n\
Variables:\n\
\n\
timezone -- difference in seconds between UTC and local standard time\n\
@@ -1340,10 +1365,51 @@
NULL
};
+/*
+ * C API capsule
+ */
+
+static PyObject *
+PyStructTimespecFromCStructTimespec(_PyTime_timespec * ts)
+{
+PyObject *v = PyStructSequence_New(&StrucTimespecType);
+if (v == NULL)
+return NULL;
+
+#define SET(i,val) PyStructSequence_SET_ITEM(v, i, PyLong_FromLong((long) val))
+SET(0, ts->tv_sec);
+SET(1, ts->tv_nsec);
+#undef SET
+if (PyErr_Occurred()) {
+Py_XDECREF(v);
+return NULL;
+}
+
+return v;
+}
+
+int PyStructTimespecToCStructTimespec(PyObject* obj, _PyTime_timespec * ts)
+{
+
+if (! StructTimespec_Check(obj))
+return -1;
+
+ts->tv_sec = PyLong_AsLong(PyStructSequence_GET_ITEM(obj,0));
+ts->tv_nsec = PyLong_AsLong(PyStructSequence_GET_ITEM(obj,1));
+
+return 0;
+}
+
+static PyTime_C_API api = {
+&StrucTimespecType,
+&PyStructTimespecFromCStructTimespec,
+&PyStructTi
Re: [Python-Dev] datetime nanosecond support (ctd?)
FWIW issue23084 was withdrawn as it appears to be touching and PEP sized, and/or anyways nanosec is defacto just an int. another patch went to http://bugs.python.org/issue15443 seemingly under the radar, I'm not arguing the quality of the patch, but bringing it up here in case with the holidays season it didn't get noticed. I wont bring much anything more so you may rest otherwise. Happy new year! On 12/18/2014 12:47 PM, mdcb808 wrote: done - http://bugs.python.org/issue23084 On 12/17/14 8:20 PM, Eric Snow wrote: On Wed, Dec 17, 2014 at 7:52 PM, Matthieu Bec wrote: Attached patch defines a new type struct_timespec for the time module. A new capsule exports the type along with to/from converters - opening a bridge for C, and for example the datetime module. I'd recommend opening a new issue in the bug tracker (bugs.python.org) and attach the patch there. Attaching it to an email is a good way for it to get lost and forgotten. :) -eric -- Matthieu BecGMTO Corp cell : +1 626 425 7923 251 S Lake Ave, Suite 300 phone: +1 626 204 0527 Pasadena, CA 91101 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] parallelizing
There are times when you deal with completely independent input/output 'pipes' - where parallelizing would really help speed things up. Can't there be a way to capture that idiom and multi thread it in the language itself? Example: loop: read an XML produce a JSON like Regards, MB ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] parallelizing
Thank you, I'll take your advice. Regarding your example, I think it gives the illusion to work because sleep() is GIL aware under the hood. I don't think it works for process() that mainly runs bytecode, because of the GIL. Sorry if I wrongly thought that was a language level discussion. Regards, Matthieu On 9/13/17 10:49 AM, Chris Barker wrote: This really isn't the place to ask this kind of question. If you want to know how to do something with python, try python-users , stack overflow, etc. If you have an idea about a new feature you think python could have, then the python-ideas list is the place for that. But if you want anyone to take it seriously, it should be a better formed idea before you post there. But: On Tue, Sep 12, 2017 at 4:43 PM, Matthieu Bec <mailto:[email protected]>> wrote: There are times when you deal with completely independent input/output 'pipes' - where parallelizing would really help speed things up. Can't there be a way to capture that idiom and multi thread it in the language itself? Example: loop: read an XML produce a JSON like Regular old threading works fine for this: import time import random import threading def process(infile, outfile): "fake function to simulate a process that takes a random amount of time" time.sleep(random.random()) print("processing: {} to make {}".format(infile, outfile)) for i in range(10): threading.Thread(target=process, args=("file%i.xml" % i, "file%i.xml" % i)).start() It gets complicated if you need to pass information back and forth, or worry about race conditions, or manage a queue, or But just running a nice self-contained thread safe function in another thread is pretty straightforward. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [email protected] <mailto:[email protected]> ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
