Re: [Numpy-discussion] Unused private functions
On Sun, Jul 13, 2008 at 6:26 PM, Alan McIntyre [EMAIL PROTECTED] wrote: Does anybody know whether there's any reason to keep the following functions? They don't appear to be used anywhere. linalg/linalg.py:95 _castCopyAndTranspose lib/function_base.py:659 _asarray1d __ +1 to remove them if they aren't used. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Schedule for 1.1.1
On Mon, Jul 14, 2008 at 7:50 AM, Charles R Harris [EMAIL PROTECTED] wrote: ` On Tue, Jul 8, 2008 at 6:21 PM, Sebastian Haase [EMAIL PROTECTED] wrote: Hi, I haven't checked out a recent numpy (( N.__version__ '1.0.3.1')) But could someone please check if the division has been changed from '/' to '//' in these places: C:\Priithon_25_win\numpy\core\numerictypes.py:142: DeprecationWarning: classic int division bytes = bits / 8 C:\Priithon_25_win\numpy\core\numerictypes.py:182: DeprecationWarning: classic int division na_name = '%s%d' % (base.capitalize(), bit/2) C:\Priithon_25_win\numpy\core\numerictypes.py:212: DeprecationWarning: classic int division charname = 'i%d' % (bits/8,) C:\Priithon_25_win\numpy\core\numerictypes.py:213: DeprecationWarning: classic int division ucharname = 'u%d' % (bits/8,) C:\Priithon_25_win\numpy\core\numerictypes.py:409: DeprecationWarning: classic int division nbytes[obj] = val[2] / 8 I don't believe we have made any changes to '/'. It is going to be tricky making the transition to 3.0, a lot of code is going to break, and we haven't started down that path. Maybe next summer... Chuck Hi, thanks for the reply. Could you maybe please check in these changes, and replace the '/' with '//' at those places !? The '//' operator is supported by Python 2.4 (if not before). For a matlab / IDL - like interactive environment I'm using numpy already for some 5+ month with the -Qnew option. And everything seems to work smoothly - so those minimal changes would advance a small step further towards the already started true division support. Thanks, Sebastian Haase ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] RFC: A proposal for implement ing some date/time types in NumPy
A Friday 11 July 2008, Christopher Barker escrigué: print example [Jul-2008 Aug-2008 Sep-2008 Oct-2008 Nov-2008 Dec-2008] I like this -- seeing the integers for the times makes me wonder what that point is -- we've all been using numbers for time for years already -- what would a datetime array give us other than auto-conversion from datetime objects, if it doesn't include nicer display, timedelta, etc. I see your point and I think that it would be a great addition to the date/time types to support additional resolution meta-information in order to offer the most proper string representation. And I'm starting to see the merit of a timedelta type too. Now that I think about this, wouldn't be better if, after the eventual introduction of the new datetime types in NumPy, the matplotlib would use any of these three and throw away their current datetime class? yes, that would be better, but what to do during the transition? Well, John Hunter has already agreed to adapt mpl to the NumPy date/time as soon as they are in, so I suppose they will have to decide the path for the transition. I'm also imaging some extra utility functions/method that would be nice: aDateTimeArray.hours(dtype=float) to convert to hours (and days, and seconds, etc). And maybe some that would create a DateTimeArray from various time units. I often have to read/write data files that have time in various units like that -- it would be nice to use array operations to work with them. I think that a datetime type with a resolution property can help here. I hope to post soon the new proposal about this. Cheers, -- Francesc Alted ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] RFC: A proposal for implementing some date/time types in NumPy
A Saturday 12 July 2008, Jon Wright escrigué: Charles R Harris wrote: On Fri, Jul 11, 2008 at 12:37 PM, Francesc Alted A Friday 11 July 2008, Francesc Alted escrigué: A Friday 11 July 2008, Jon Wright escrigué: Nice idea - please can you make it work with matplotlib's time/date Hmmm, following the matplotlib docstrings: datetime objects are converted to floating point numbers which represent the number of days since 0001-01-01 UTC ... this more carefully, but I suppose that if there is interest enough that could be implemented, yes. Now that I think about this, wouldn't be better if, after the eventual introduction of the new datetime types in NumPy, the matplotlib would use any of these three and throw away their current datetime class? [Unless they have good reasons for keeping their epoch and/or scale] Especially as there was a ten day adjustment made with the adoption of the Gregorian calender on Oct 4, 1582; early dates can be hard to interpret. Curiously, IIRC, 01/01/0001 was a Monday. So I think I will just want to plot timeseries without (ever please) caring about date formats again. If you're proposing a new format then I'm assuming you want me to once again care that: 1) my temperature logger is recording data in Romance Standard Time, but not saying so, just day/month/year : time. 2) When we read that data we cannot tell which time zone it was recorded in, even if we think we remember where the logger was when it logged. 3) That the program I am running could currently be in any time zone 4) Whether the program is plotting compared to now in the current time zone or then that the data were recorded. None of these problems are new, or indeed unique, I think we only want a to_ and from_ converter to what we mean that we can plot, using matplotlib. Well, timezones are a hairy issue. From the docstrings of the Python's ``datetime`` module: Supporting timezones at whatever level of detail is required is up to the application. The rules for time adjustment across the world are more political than rational, and there is no standard suitable for every application. In fact, the approach for the ``datetime`` module is that the user has to provide a concrete class (derived from the abstract ``tzinfo`` class) with *user-defined* methods that are supposed to do the necessary computations to convert UTC -- 'localtime' timestamps. However, doing the same in NumPy itself would be prohibitive in terms of computational effort. So, our intention was to ignore this part of the reality (timezones) by always working internally with UTC time and let the user do the necessary conversions by using the machinery (together with user's own methods) that provides the ``datetime`` module. We really think that this is a sensible way to proceed with timezones in the NumPy context. Timezones are a heck of a problem if you want to be accurate. You are talking about nanosecond resolutions, however, atomic clocks in orbit apparently suffer from relativistic corrections of the order 38000 nanoseconds per day [1]. What will you do about data recorded on the international space station? Getting into time formats at this level seems to be rather complicated - there is no absolute time you can reference to - it is all relative :-) Wow, this is really a much larger jitter than I expected. I suppose that this reinforces the decision of deprecate the use of the TAI standard just for improving the precision in large time spans. Thanks, -- Francesc Alted ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] error importing a f2py compiled module.
Le lundi 23 juin 2008 à 14:10 +0200, Fabrice Silva a écrit : I don't have ideas what is causing this import error. Try the instructions above, may be it is due to some compile object conflicts. The only posts on mailing lists I've read mention security policies (SElinux) and /tmp execution limitations... Another point is that the working directory has been created by a subversion checkout command but has proper permissions drwxr-xr-x 4 fab fab 4096 jui 14 13:29 lib as the fortran and the shared object files : -rwxr-xr-x 1 fab fab 6753 jui 9 14:14 systeme.f -rwxr-xr-x 1 fab fab 85746 jui 14 13:21 systeme.so Moving these files in another directory (and adding this latter to path) suppress the import problem... -- Fabrice Silva [EMAIL PROTECTED] LMA UPR CNRS 7051 - équipe S2M ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] RFC: A proposal for implement ing some date/time types in NumPy
A Saturday 12 July 2008, Matt Knox escrigué: Christopher Barker Chris.Barker at noaa.gov writes: I'm also imaging some extra utility functions/method that would be nice: aDateTimeArray.hours(dtype=float) to convert to hours (and days, and seconds, etc). And maybe some that would create a DateTimeArray from various time units. The DateArray class in the timeseries scikits can do part of what you want. Observe... import scikits.timeseries as ts a = ts.date_array(start_date=ts.now('hourly'), length=15) a DateArray([12-Jul-2008 11:00, 12-Jul-2008 12:00, 12-Jul-2008 13:00, 12-Jul-2008 14:00, 12-Jul-2008 15:00, 12-Jul-2008 16:00, 12-Jul-2008 17:00, 12-Jul-2008 18:00, 12-Jul-2008 19:00, 12-Jul-2008 20:00, 12-Jul-2008 21:00, 12-Jul-2008 22:00, 12-Jul-2008 23:00, 13-Jul-2008 00:00, 13-Jul-2008 01:00], freq='H') Mmh, I like very much your notion of 'frequency' as meta-information of your DateArray class. I was in fact thinking in something similar for the (more general) date/time in NumPy, but based on the notion of 'resolution' instead of 'frequency'. I'll expand more about this in our next proposal. a.year array([2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008]) a.hour array([11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1]) a.day array([12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13]) Well, while I see the merits of the '.year', '.hour' and so on properties, I'm not sure whether this would be useful for a general date/time type. I'd prefer what was suggested by Chris before, i.e. something like: a.hours(dtype=float) to convert to hours (and days, and seconds, etc). I would encourage you to take a look at the wiki (http://scipy.org/scipy/scikits/wiki/TimeSeries) as you may find some surprises in there that prove useful. I've had a look at it, and it is clear that you guys have put a lot of thought on it. We will be sure to have your implementation in mind. I often have to read/write data files that have time in various units like that -- it would be nice to use array operations to work with them. If peak performance is not a concern, parsing of most date formats can be done automatically using the built in parser in the timeseries module (borrowed from mx.DateTime). Observe... dlist = ['14-jan-2001 14:34:33', '16-jan-2001 10:09:11'] a = ts.date_array(dlist, freq='secondly') a DateArray([14-Jan-2001 14:34:33, 16-Jan-2001 10:09:11], freq='S') That's great. However we only planned to import/export dates from the ``datetime`` module for the time being, mainly because of efficency but also simplicity. Would many people be interested in seeing this kind of string date parsing integrated in the native NumPy types? Thanks, -- Francesc Alted ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] NumPy date/time types and the resolution concept
Hi, Before giving more thought to the new proposal of the date/time types for NumPy based in the concept of 'resolution', I'd like to gather more feedback on your opinions about this. After pondering about the opinions about the first proposal, the idea we are incubating is to complement the ``datetime64`` with a 'resolution' metainfo. The ``datetime64`` will still be based on a int64 type, but the meaning of the 'ticks' would depend on a 'resolution' property. This is best seen with an example: In [21]: numpy.arange(3, dtype=numpy.dtype('datetime64', 'sec')) Out [21]: [1970-01-01T00:00:00, 1970-01-01T00:00:01, 1970-01-01T00:00:02] In [22]: numpy.arange(3, dtype=numpy.dtype('datetime64', 'hour')) Out [22]: [1970-01-01T00, 1970-01-01T01, 1970-01-01T02] i.e. the 'resolution' gives the actual meaning to the 'int64' counter. The advantage of this abstraction is that the user can easily choose the scale of resolution that better fits his need. I'm thinking in providing the next resolutions: [femtosec, picosec, nanosec, microsec, millisec, sec, min, hour, month, year] Also, together with the absolute ``datetime64`` one can have a relative counterpart, say, ``timedelta64`` that also supports the notion of 'resolution'. Between both one would cover the needs for most uses, while providing the user with a lot of flexibility, IMO. We very much prefer this new approach than the stated in our first proposal. Now, it comes the tricky part: how to integrate the notion of 'resolution' with the 'dtype' data type factory of NumPy? Well, we have thought a couple of possibilities. 1) Using the NumPy 'dtype' factory: nanoabs = numpy.dtype('datetime64', resolution=nanosec) nanorel = numpy.dtype('timedelta64', resolution=nanosec) 2) Extending the string notation by using the '[]' square brackets: nanoabs = numpy.dtype('datetime64[nanosec]') # long notation nanoabs = numpy.dtype('T[nanosec]') # short notation nanorel = numpy.dtype('timedelta64[nanosec]') # long notation nanorel = numpy.dtype('t[nanosec]') # short notation With these building blocks, one may obtain more complex dtype structures easily. Now, the question is: would that proposal enter in conflict with the spirit of the current 'dtype' factory? And another important one, would that complicate the implementation too much? If the answer to the both previous questions is 'no', then we will study this more and provide another proposal based on this. BTW, I suppose that the best candidate to answer these would be Travis O., but if anybody feels brave enough ;-) please go ahead and give your advice. Cheers, -- Francesc Alted ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] RFC: A proposal for implementing some date/time types in NumPy
2008/7/14 Francesc Alted [EMAIL PROTECTED]: [...] DateArray([14-Jan-2001 14:34:33, 16-Jan-2001 10:09:11], freq='S') That's great. However we only planned to import/export dates from the ``datetime`` module for the time being, mainly because of efficency but also simplicity. Would many people be interested in seeing this kind of string date parsing integrated in the native NumPy types? It's useful to have a complete string representation to write dates to a file and be able to retrieve them later on. In this sense, a strftime-like write/read method would be appreciated (where the date format is specified by the user or set by convention). On the other hand, trying to second guess the format the date is formatted in can quickly turn into a regular expression nightmare (look at the mx.datetime module that does this). I'd hate to see you waste time on this. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy date/time types and the resolution concept
On Monday 14 July 2008 09:07:47 Francesc Alted wrote: The advantage of this abstraction is that the user can easily choose the scale of resolution that better fits his need. I'm thinking in providing the next resolutions: [femtosec, picosec, nanosec, microsec, millisec, sec, min, hour, month, year] In TimeSeries, we don't have anything less than a second, but we have 'daily', 'business daily', 'weekly' and 'quarterly' resolutions. A very useful point that Matt Knox had coded is the possibility to specify starting points for switching from one resolution to another. For example, you can have a series with a 'ANN_MAR' frequency, that corresponds to 1 point a year, the year starting in April. When switching back to a monthly resolution, the points from January to March of the first year will be masked. Another useful point would be allow the user to define his/her own resolution (every 15min, every 12h...). Right now it's a bit clunky in TimeSeries, we have to use the lowest resolution of the series (min, hour) and leave a lot of blanks (TimeSeries don't have to be regularly spaced, but it helps...) Now, it comes the tricky part: how to integrate the notion of 'resolution' with the 'dtype' data type factory of NumPy? In TimeSeries, the frequency is stored as an integer. For example, a daily frequency is stored as 6000, an annual frequency as 1000, a 'ANN_MAR' frequency as 1003... ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy date/time types and the resolution concept
On Mon, 14 Jul 2008, Francesc Alted apparently wrote: Before giving more thought to the new proposal of the date/time types for NumPy based in the concept of 'resolution', I'd like to gather more feedback on your opinions about this. It might be a good idea to run the proposal(s) past Marc-Andre Lemburg mal (at) egenix (dot) com Cheers, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy date/time types and the resolution concept
2008/7/14 Francesc Alted [EMAIL PROTECTED]: After pondering about the opinions about the first proposal, the idea we are incubating is to complement the ``datetime64`` with a 'resolution' metainfo. The ``datetime64`` will still be based on a int64 type, but the meaning of the 'ticks' would depend on a 'resolution' property. This is an interesting idea. To be useful, though, you would also need a flexible offset defining the zero of time. After all, the reason not to just always use (say) femtosecond accuracy is that 2**64 femtoseconds is only about five hours. So if you're going to use femtosecond steps, you really want to choose your start point carefully. (It's also worth noting that there is little need for more time accuracy than atomic clocks can provide, since anyone looking for more than that is going to be doing some tricky metrology anyway.) One might take guidance from the FITS format, which represents (arrays of) quantities as (usually) fixed-point numbers, but has a global scale and offset parameter for each array. This allows one to accurately represent many common arrays with relatively few bits. The FITS libraries transparently convert these quantities. Of course, this isn't so convenient if you don't have basic machine datatypes with enough precision to handle all the quantities of interest. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Schedule for 1.1.1
The NumPy 1.1.1 release date (7/31/08) is rapidly approaching and we need everyone's help. Chuck Harris has volunteered to take the lead on coordinating this release. As a reminder here is the schedule for 1.1.1: - 7/20/08 tag the 1.1.1rc1 release and prepare packages - 7/27/08 tag the 1.1.1 release and prepare packages - 7/31/08 announce release This release should include only bug-fixes and improved documentation. We need to follow this schedule as closely as possible because we will need to start focusing on the upcoming NumPy 1.2.0 release as soon as 1.1.1 is released. As a reminder, the trunk is for 1.2.0 development; 1.1.1 will be tagged off the 1.1.x branch: svn co http://svn.scipy.org/svn/numpy/branches/1.1.xnumpy-1.1.x If you have any fixes that you haven't back-ported yet, please do so ASAP. According to our release schedule we will be tagging the release candidate for 1.1.1 next Sunday (7/20). We will be asking for wide-spread testing of the release candidate during the week of the 20th. If you want to open a ticket specifically for this bug-fix release, please use the NumPy 1.1.1 milestone: http://scipy.org/scipy/numpy/milestone/1.1.1 Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy date/time types and the resolution concept
A Monday 14 July 2008, Alan G Isaac escrigué: On Mon, 14 Jul 2008, Francesc Alted apparently wrote: Before giving more thought to the new proposal of the date/time types for NumPy based in the concept of 'resolution', I'd like to gather more feedback on your opinions about this. It might be a good idea to run the proposal(s) past Marc-Andre Lemburg mal (at) egenix (dot) com Sure. And maybe also to Fred Drake, the original autor of the ``datetime`` module. However, I'd prefer to send them something in a more advanced state of refinement than it is now. Thanks for the suggestion, -- Francesc Alted ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] permissions on tests in numpy and scipy
I have a rather unconventional install pipeline at work and owner only read permissions on a number of the tests are causing me minor problems. It appears the permissions on the tests are set rather inconsistently in numpy and python -- is there any reason not to make these all 644? [EMAIL PROTECTED]:site-packages find numpy -name test_*.py|xargs ls -l -rw--- 1 johnhresearch2769 Jul 14 12:01 numpy/core/tests/test_defchararray.py -rw--- 1 johnhresearch8283 Jul 14 12:01 numpy/core/tests/test_defmatrix.py -rw--- 1 johnhresearch1769 Jun 25 10:00 numpy/core/tests/test_errstate.py -rw--- 1 johnhresearch1508 Jun 25 10:00 numpy/core/tests/test_memmap.py -rw--- 1 johnhresearch 4 Jul 14 12:01 numpy/core/tests/test_multiarray.py -rw--- 1 johnhresearch 26695 Jul 14 12:01 numpy/core/tests/test_numeric.py -rw--- 1 johnhresearch 13781 Jul 14 12:01 numpy/core/tests/test_numerictypes.py -rw--- 1 johnhresearch1113 Jul 14 12:01 numpy/core/tests/test_print.py -rw--- 1 johnhresearch4290 Jul 14 12:01 numpy/core/tests/test_records.py -rw--- 1 johnhresearch 39370 Jul 14 12:01 numpy/core/tests/test_regression.py -rw--- 1 johnhresearch4097 Jul 14 12:01 numpy/core/tests/test_scalarmath.py -rw--- 1 johnhresearch9330 Jun 25 10:00 numpy/core/tests/test_ufunc.py -rw--- 1 johnhresearch7583 Jul 14 12:01 numpy/core/tests/test_umath.py -rw--- 1 johnhresearch 11264 Jul 14 12:01 numpy/core/tests/test_unicode.py -rw--- 1 johnhresearch 222 Jul 14 12:00 numpy/distutils/tests/f2py_ext/tests/test_fib2.py -rw--- 1 johnhresearch 220 Jul 14 12:00 numpy/distutils/tests/f2py_f90_ext/tests/test_foo.py -rw--- 1 johnhresearch 220 Jul 14 12:00 numpy/distutils/tests/gen_ext/tests/test_fib3.py -rw--- 1 johnhresearch 277 Jul 14 12:00 numpy/distutils/tests/pyrex_ext/tests/test_primes.py -rw--- 1 johnhresearch 387 Jul 14 12:00 numpy/distutils/tests/swig_ext/tests/test_example.py -rw--- 1 johnhresearch 276 Jul 14 12:00 numpy/distutils/tests/swig_ext/tests/test_example2.py -rw--- 1 johnhresearch1800 Jul 14 12:00 numpy/distutils/tests/test_fcompiler_gnu.py -rw--- 1 johnhresearch2421 Jun 25 09:59 numpy/distutils/tests/test_misc_util.py -rw-r--r-- 1 johnhresearch 64338 Jun 25 09:59 numpy/f2py/lib/parser/test_Fortran2003.py -rw-r--r-- 1 johnhresearch 24785 Jun 25 09:59 numpy/f2py/lib/parser/test_parser.py -rw--- 1 johnhresearch 574 Jul 14 12:00 numpy/fft/tests/test_fftpack.py -rw--- 1 johnhresearch1256 Jul 14 12:00 numpy/fft/tests/test_helper.py -rw--- 1 johnhresearch9948 Jul 14 12:01 numpy/lib/tests/test__datasource.py -rw--- 1 johnhresearch4088 Jul 14 12:01 numpy/lib/tests/test_arraysetops.py -rw--- 1 johnhresearch 809 Jul 14 12:01 numpy/lib/tests/test_financial.py -rw--- 1 johnhresearch 23794 Jul 14 12:01 numpy/lib/tests/test_format.py -rw--- 1 johnhresearch 30772 Jul 14 12:01 numpy/lib/tests/test_function_base.py -rw--- 1 johnhresearch1587 Jun 25 10:00 numpy/lib/tests/test_getlimits.py -rw--- 1 johnhresearch2111 Jul 14 12:01 numpy/lib/tests/test_index_tricks.py -rw--- 1 johnhresearch7691 Jul 14 12:01 numpy/lib/tests/test_io.py -rw--- 1 johnhresearch 999 Jun 25 10:00 numpy/lib/tests/test_machar.py -rw--- 1 johnhresearch2431 Jun 25 10:00 numpy/lib/tests/test_polynomial.py -rw--- 1 johnhresearch1409 Jul 14 12:01 numpy/lib/tests/test_regression.py -rw--- 1 johnhresearch 14353 Jul 14 12:01 numpy/lib/tests/test_shape_base.py -rw--- 1 johnhresearch6958 Jul 14 12:01 numpy/lib/tests/test_stride_tricks.py -rw--- 1 johnhresearch6735 Jul 14 12:01 numpy/lib/tests/test_twodim_base.py -rw--- 1 johnhresearch9579 Jul 14 12:01 numpy/lib/tests/test_type_check.py -rw--- 1 johnhresearch1771 Jun 25 10:00 numpy/lib/tests/test_ufunclike.py -rw--- 1 johnhresearch8279 Jul 14 12:01 numpy/linalg/tests/test_linalg.py -rw--- 1 johnhresearch1739 Jul 14 12:01 numpy/linalg/tests/test_regression.py -rw--- 1 johnhresearch 78819 Jun 25 10:00 numpy/ma/tests/test_core.py -rw--- 1 johnhresearch 15744 Jul 14 12:01 numpy/ma/tests/test_extras.py -rw--- 1 johnhresearch 17759 Jun 25 10:00 numpy/ma/tests/test_mrecords.py -rw--- 1 johnhresearch 33009 Jun 25 10:00 numpy/ma/tests/test_old_ma.py -rw--- 1 johnhresearch5956 Jun 25 10:00 numpy/ma/tests/test_subclassing.py -rw--- 1 johnhresearch3173 Jul 14 12:01 numpy/oldnumeric/tests/test_oldnumeric.py -rw--- 1 johnhresearch2043 Jun 25 09:59 numpy/random/tests/test_random.py -rw--- 1
Re: [Numpy-discussion] permissions on tests in numpy and scipy
On Mon, Jul 14, 2008 at 12:22, John Hunter [EMAIL PROTECTED] wrote: I have a rather unconventional install pipeline at work and owner only read permissions on a number of the tests are causing me minor problems. It appears the permissions on the tests are set rather inconsistently in numpy and python -- is there any reason not to make these all 644? We're not doing anything special, here. When I install using sudo python install.py on OS X, all of the permissions are 644. I think the problem may be in your pipeline. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] mrecarray
On Monday 14 July 2008 13:53:09 Michael Droettboom wrote: I'm running into a couple of small problems with mrecarray. I'm not sure if they're bugs or a usage error. Bugs are my bet. I'll check that. The first one might be problematic, as it probable comes from ma.core. The second one is most likely due to the negligence of the author ;), thanks a lot for the patch and the feedback. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy date/time types and the resolution concept
A Monday 14 July 2008, Pierre GM escrigué: On Monday 14 July 2008 12:50:21 Francesc Alted wrote: A very useful point that Matt Knox had coded is the possibility to specify starting points for switching from one resolution to another. For example, you can have a series with a 'ANN_MAR' frequency, that corresponds to 1 point a year, the year starting in April. When switching back to a monthly resolution, the points from January to March of the first year will be masked. Ok. Ann was also suggesting that the origin of time would be configurable, but then, you are talking about *masking* values. Mmm, I don't think we should try to incorporate masking capabilities in the NumPy date/time types. Francesc, In scikits.timeseries, we have 2 different objects: * DateArray, which is basically a ndarray of integers with a given 'frequency' attribute. * TimeSeries, which is basically the combination of a MaskedArray (the data part) and a DateArray (which keeps track of the date corresponding to each data point. TimeSeries object have the resolution/origin of the companion DateArray, and when they're converted from one resolution to another, some masking may occur. My understanding is that you intend to define an object similar to DateArray. You want to define a new dtype (datetime64 or other), we used yet another class instead, Date. A dtype would be easier to manipulate, but as neither Matt nor I were particularly experienced with that at the time, we followed the simpler approach of a class... Well, what we are after is precisely this: a new dtype type. After integrating it in NumPy, I suppose that your DateArray would be similar than a NumPy array with a dtype ``datetime64`` (bar the conceptual differences between your 'frequency' behind DateArray and the 'resolution' behind the datetime64 dtype). [N]timeunit where ``timeunit`` can take the values in: ['y', 'm', 'd', 'h', 'm', 's', 'ms', 'us', 'ns', 'fs'] so, for example, '14d' means a resolution of 14 days, or '10ms' means a resolution of 1 hundreth of second. Sounds good to me. What other people think? Sounds pretty cool and intuitive to use. However, writing the conversion rules from one to another will be a lot of fun. Take weekly, for example: that's a period of 7 days, but when does it start ? On a monday ? Then, 12/31/2007 was the start of the first week of 2008... OK, we can leave that problem for the moment... It would start when the origin tells that it should start. It is important to note that our proposal will not force a '7d' (seven days) 'tick' to start on monday, or a '1m' (one month) to start the 1st day of a calendar month, but rather where the user decides to set its origin. Cheers, -- Francesc Alted ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy date/time types and the resolution concept
On Monday 14 July 2008 14:17:18 Francesc Alted wrote: Well, what we are after is precisely this: a new dtype type. After integrating it in NumPy, I suppose that your DateArray would be similar than a NumPy array with a dtype ``datetime64`` (bar the conceptual differences between your 'frequency' behind DateArray and the 'resolution' behind the datetime64 dtype). Well, you're losing me on this one: could you explain the difference between the two concepts ? It might only be a problem of vocabulary... It would start when the origin tells that it should start. It is important to note that our proposal will not force a '7d' (seven days) 'tick' to start on monday, or a '1m' (one month) to start the 1st day of a calendar month, but rather where the user decides to set its origin. OK, so we need 2 flags, one for the resolution, one for the origin. Because there won't be that many resolution possible, an int8 should be sufficient. What do you have in mind for the origin ? When using a resolution coarser than 1d (7d, 1m, 3m, 1a), an origin in day is OK. What about less than a day ? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] RFC: A proposal for implementing some date/time types in NumPy
A Monday 14 July 2008, Christopher Barker escrigué: Matt Knox wrote: The DateArray class in the timeseries scikits can do part of what you want. Observe... a.year array([2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008]) a.hour array([11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1]) a.day array([12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13]) This is great for what I often need: to output data in a format with columns of: year, month, day, hour, min, sec I see. However, the more I think about this, the more I see the need to split the date/time functionalities and duties in two parts: * the first one implementing a date/time dtype with the basic functionality for timestamping and/or time-interval measuring. * the second part would be a specific array container of date/time types (which maybe perfectly a porting of the DateArray of the scikits.timeseries that would be based on the date/time type) where one can implement all of the functionality (like the one that you are proposing above) that escapes to a humble date/time dtype. Definitely, having this two-layer approach is going to allow a more powerful and flexible approach in the long term, IMO. But I also often need to be able to convert a TimeDelta to a particular unit, for example (using the lib datetime): td = datetime.datetime(2008, 7, 14, 12) - datetime.datetime(2008, 7, 13, 10) td datetime.timedelta(1, 7200) so we have a timedelta of one day, 7200 seconds. I'd like: td.as_hours Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'datetime.timedelta' object has no attribute 'as_hours' which doesn't exist in the datetime module, so I do: hours = td.days*24 + td.seconds/3600. hours 26.0 I find myself writing this code al ot, so I'd love to have it built in. Hmm, I don't know if having a conversor for every time unit would be too much. I'd prefer the next: td.as_timeunit('hour') where you can specify the time unit as the parameter. Will take note of this. Which brings up an issue: The reason it isn't built in is that the philosophy behind the datetime module is that it provides the building blocks with which to build more feature-full packages. Personally, I really wish it had a bit more built in, but what can we do? As for the numpy datetime types, we need to decide how much to build in. I think the kind of functionality described here is pretty basic, and should be included, but if we inculde everyone's idea of basic, it could get pretty bloated! Completely agree. This is why I'm proposing the two-layer approach: have the basic date/time functionality implemented as a dtype (i.e. in C space), and put the other niceties into a sort of ``DateArray`` (perhaps in Python space). the idea we are incubating is to complement the ``datetime64`` with a 'resolution' metainfo. The ``datetime64`` will still be based on a int64 type, but the meaning of the 'ticks' would depend on a 'resolution' property. I like this! Would there be conversion between different resolutions available? I wonder what that syntax for that should be? Well, what about the .as_timeunit() stated above for the date/time scalar and another similar for the ``DateArray`` layer?. However, be aware that, as we are proposing integer arithmetic for the date/time types (and not fixed-point of floating-point arithmetic) you *will* loose precision when changing resolution from a fine-grained time unit to another more coarse-grained (and inversely, you may risk to overflow when changing resolution from a coarse-grained to another more fine-grained unit), and this may not be what you want. And definitely, offset would be similar to origin. So yes, we will try to introduce both concepts. yup -- origin is critical! What resolution (and numerical format) do you use to express the origin? Even if you data is ini days, you may want to specify the origin with more precision, so as not to have confusion about what 0 days means in some higher resolution unit. Also, if you want picosecond resolution, then the origin needs to be picosecond resolution as well. Good point. I'm afraid that we will only support the specification of the origin with a fixed resolution of microseconds, and between the year 1 and (mainly for ``datetime`` compatibility, but also to avoid the 'egg and the chicken' effect that you noticed ;-). Cheers, -- Francesc Alted ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] RFC: A proposal for implementing some date/time types in NumPy
On Monday 14 July 2008 15:12:18 Francesc Alted wrote: I see. However, the more I think about this, the more I see the need to split the date/time functionalities and duties in two parts: * the first one implementing a date/time dtype with the basic functionality for timestamping and/or time-interval measuring. That would be our Date class * the second part would be a specific array container of date/time types (which maybe perfectly a porting of the DateArray of the scikits.timeseries that would be based on the date/time type) where one can implement all of the functionality (like the one that you are proposing above) that escapes to a humble date/time dtype. That would be our DateArray class indeed... Francesc, Chris, may I suggest you to try TimeSeries if you didn't already ? That way you could see what kind of features are missing and which ones should be improved with the new dtype ? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Schedule for 1.1.1
The NumPy 1.1.1 release date (7/31/08) is rapidly approaching and we need everyone's help. Chuck Harris has volunteered to take the lead on coordinating this release. As a reminder here is the schedule for 1.1.1: - 7/20/08 tag the 1.1.1rc1 release and prepare packages - 7/27/08 tag the 1.1.1 release and prepare packages - 7/31/08 announce release This release should include only bug-fixes and improved documentation. We need to follow this schedule as closely as possible because we will need to start focusing on the upcoming NumPy 1.2.0 release as soon as 1.1.1 is released. As a reminder, the trunk is for 1.2.0 development; 1.1.1 will be tagged off the 1.1.x branch: svn co http://svn.scipy.org/svn/numpy/branches/1.1.xnumpy-1.1.x If you have any fixes that you haven't back-ported yet, please do so ASAP. According to our release schedule we will be tagging the release candidate for 1.1.1 next Sunday (7/20). We will be asking for wide-spread testing of the release candidate during the week of the 20th. If you want to open a ticket specifically for this bug-fix release, please use the NumPy 1.1.1 milestone: http://scipy.org/scipy/numpy/milestone/1.1.1 Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Doc on dtypes
All, Anybody could point me to some docs on dtypes ? Michael Droettboom's recent question made me realize that things were far more complex than I thought. For example, how can I find the shape and type of a field (without using dtype.descr). Thanks a lot in advance P. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Doc on dtypes
On Mon, Jul 14, 2008 at 17:06, Pierre GM [EMAIL PROTECTED] wrote: All, Anybody could point me to some docs on dtypes ? Michael Droettboom's recent question made me realize that things were far more complex than I thought. For example, how can I find the shape and type of a field (without using dtype.descr). dtype.fields is a dict-like object containing the same information, but accessible by field name. Chapter 7 of _The Guide to Numpy_ has more content, and the equivalent will be wending its way into the numpy documentation marathon sooner or later, but with the insight about dtype.fields, I think you can get pretty far on your own. If you have any other specific questions, feel free to ask. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Doc on dtypes
On Monday 14 July 2008 18:53:41 Robert Kern wrote: dtype.fields is a dict-like object containing the same information, but accessible by field name. But as it's a dictionary, I can't use iteritems() without risking having the wrong order of fields, right ? Or do dictproxies behave differently ? Chapter 7 of _The Guide to Numpy_ has more content, Oh, of course... My version is just 2 years old, has there been much change about dtypes in the meantime ? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Doc on dtypes
On Mon, Jul 14, 2008 at 18:24, Pierre GM [EMAIL PROTECTED] wrote: On Monday 14 July 2008 18:53:41 Robert Kern wrote: dtype.fields is a dict-like object containing the same information, but accessible by field name. But as it's a dictionary, I can't use iteritems() without risking having the wrong order of fields, right ? Or do dictproxies behave differently ? Right. If you want order, use dtype.descr, or sort on the last item in the tuple. We can probably reimplement the dictproxy to guarantee order. Chapter 7 of _The Guide to Numpy_ has more content, Oh, of course... My version is just 2 years old, has there been much change about dtypes in the meantime ? No. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Run np.test() twice, get message.
On Sun, Jul 13, 2008 at 10:59 PM, Alan McIntyre [EMAIL PROTECTED] wrote: On Mon, Jul 14, 2008 at 1:31 AM, Charles R Harris [EMAIL PROTECTED] wrote: Any idea what this is: *** DocTestRunner.merge: '__main__' in both testers; summing outcomes. Hmm..that's coming from nose. I'll see what it's about tomorrow. It's actually coming from doctest. Hardcode import doctest doctest.master = None in the code that runs the tests. This resets a module-global that's the cause of that message. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Release 1.1.1
All, The rc release of numpy-1.1.1 is due out next Sunday. I have gone through the commits made to the trunk since the 1.1.x branch to pull out backport candidates. If you find your name here could you make the backport or say why you think it inappropriate. David, I know that these are mostly build fixes and you have backported many of them, but I don't know the current build state of 1.1.x cdavid r5236 r5240 r5266 r5267 r5268 r5269 r5270 r5271 r5272 r5273 r5274 r5275 r5276 r5277 r5278 r5279 r5280 r5281 r5282 r5283 r5302 r5304 r5355 r5365 r5366 r5367 r5368 These are mine and I'll take care of them. charris r5259 r5312 r5322 r5324 r5392 r5394 r5399 r5406 r5407 dhuard r5254 fperez r5298 r5301 r5303 jarrod r5285 oliphant r5245 r5255 Pierre, I know you have been working diligently to get masked arrays up to speed and have made numerous fixes in the 1.1.x branch. All the tests pass for me. Is there more that needs to be done? pierregm r5242 r5244 r5248 r5249 r5251 r5253 r5256 r5260 r5263 r5264 r5284 r5292 r5314 r5329 r5332 ptvirtan r5261 rkern r5296 r5297 r5342 r5349 r5357 Stefan, these are mostly documentation related. IIRC, you planned to update the documentation in 1.1.1, which probably also needs ptvirtan's commit above. What is the current status of this project? stefan r5290 r5293 r5294 r5299 r5360 r5371 r5372 Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Release 1.1.1
On Mon, Jul 14, 2008 at 5:05 PM, Charles R Harris [EMAIL PROTECTED] wrote: The rc release of numpy-1.1.1 is due out next Sunday. I have gone through the commits made to the trunk since the 1.1.x branch to pull out backport candidates. If you find your name here could you make the backport or say why you think it inappropriate. Thanks for putting this together! jarrod r5285 I was just testing subversion (I inserted a blank line). -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Release 1.1.1
On Mon, Jul 14, 2008 at 19:05, Charles R Harris [EMAIL PROTECTED] wrote: All, The rc release of numpy-1.1.1 is due out next Sunday. I have gone through the commits made to the trunk since the 1.1.x branch to pull out backport candidates. If you find your name here could you make the backport or say why you think it inappropriate. rkern r5296 r5297 The latter is the backport of the former. r5342 r5349 r5357 Sure, I'll take these. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Release 1.1.1
Pierre, I know you have been working diligently to get masked arrays up to speed and have made numerous fixes in the 1.1.x branch. All the tests pass for me. Is there more that needs to be done? Charles, I did as much as I could to ensure compatibility with Python 2.3, but I can't test it myself (can't install Python 2.3 on my machine). It'd be great if somebody could check it works with that version, otherwise I'm all go (the recent pbs with mrecords and exotic flexible types are for 1.2). ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Doc on dtypes
On Monday 14 July 2008 19:39:39 Robert Kern wrote: Right. If you want order, use dtype.descr, or sort on the last item in the tuple. We can probably reimplement the dictproxy to guarantee order. Oh, with dtype.names and dtype.fields I can work. The Guide mentions a key [-1] in dtype.fields that should store the names in order: that would be quite useful but it doesn't work (KeyError). ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Doc on dtypes
On Mon, Jul 14, 2008 at 19:41, Pierre GM [EMAIL PROTECTED] wrote: On Monday 14 July 2008 19:39:39 Robert Kern wrote: Right. If you want order, use dtype.descr, or sort on the last item in the tuple. We can probably reimplement the dictproxy to guarantee order. Oh, with dtype.names and dtype.fields I can work. The Guide mentions a key [-1] in dtype.fields that should store the names in order: that would be quite useful but it doesn't work (KeyError). Where do you see this mention? -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Doc on dtypes
On Monday 14 July 2008 21:56:47 Robert Kern wrote: Oh, with dtype.names and dtype.fields I can work. The Guide mentions a key [-1] in dtype.fields that should store the names in order: that would be quite useful but it doesn't work (KeyError). Where do you see this mention? Page 116 of my version (once again, 07/17/2006) An ordered (by offset) list of field names is also stored in the fields dictionary under the key -1. This can be used to walk through all of the named fields in offset order. Notice that the defined fields do not have to “cover” the record, but the itemsize of the container data-type object must always be at least as large as the itemsizes of the data-type objects in the defined fields. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Doc on dtypes
On Mon, Jul 14, 2008 at 21:15, Pierre GM [EMAIL PROTECTED] wrote: On Monday 14 July 2008 21:56:47 Robert Kern wrote: Oh, with dtype.names and dtype.fields I can work. The Guide mentions a key [-1] in dtype.fields that should store the names in order: that would be quite useful but it doesn't work (KeyError). Where do you see this mention? Page 116 of my version (once again, 07/17/2006) An ordered (by offset) list of field names is also stored in the fields dictionary under the key -1. This can be used to walk through all of the named fields in offset order. Notice that the defined fields do not have to cover the record, but the itemsize of the container data-type object must always be at least as large as the itemsizes of the data-type objects in the defined fields. I have a slightly newer version of the book. The majority of this text appears verbatim under the description of dtype.names, so I assume dtype.fields[-1] got removed in favor of dtype.names. As well it should. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Run np.test() twice, get message.
On Mon, Jul 14, 2008 at 7:58 PM, Fernando Perez [EMAIL PROTECTED] wrote: It's actually coming from doctest. Hardcode import doctest doctest.master = None in the code that runs the tests. This resets a module-global that's the cause of that message. Thanks, Fernando, I added that and it prevents the message. This change is checked in. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Release 1.1.1
On Mon, Jul 14, 2008 at 8:21 PM, Pierre GM [EMAIL PROTECTED] wrote: I did as much as I could to ensure compatibility with Python 2.3, but I can't test it myself (can't install Python 2.3 on my machine). It'd be great if somebody could check it works with that version, otherwise I'm all go (the recent pbs with mrecords and exotic flexible types are for 1.2). For what it's worth, I ran the full test suite of NumPy 1.1.1 with Python 2.3.7 (both from svn) on a Linux machine (Gentoo 2.6.24, gcc 4.1.2) and everything passed. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Schedule for 1.1.1
On Sun, Jul 13, 2008 at 01:49:18AM -0700, Jarrod Millman wrote: The NumPy 1.1.1 release date (7/31/08) is rapidly approaching and we need everyone's help. Chuck Harris has volunteered to take the lead on coordinating this release. Anybody has an idea what the status is on #844? ( http://scipy.org/scipy/numpy/ticket/844 ) Cheers, Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion