Re: [Numpy-discussion] Moving to gcc 4.* for win32 installers ?
On Tue, Dec 13, 2011 at 3:43 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sun, Oct 30, 2011 at 12:18 PM, David Cournapeau courn...@gmail.com wrote: On Thu, Oct 27, 2011 at 5:19 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: Hi David, On Thu, Oct 27, 2011 at 3:02 PM, David Cournapeau courn...@gmail.com wrote: Hi, I was wondering if we could finally move to a more recent version of compilers for official win32 installers. This would of course concern the next release cycle, not the ones where beta/rc are already in progress. Basically, the pros: - we will have to move at some point - gcc 4.* seem less buggy, especially C++ and fortran. - no need to maintain msvcr90 vodoo The cons: - it will most likely break the ABI - we need to recompile atlas (but I can take care of it) - the biggest: it is difficult to combine gfortran with visual studio (more exactly you cannot link gfortran runtime to a visual studio executable). The only solution I could think of would be to recompile the gfortran runtime with Visual Studio, which for some reason does not sound very appealing :) To get the datetime changes to work with MinGW, we already concluded that building with 4.x is more or less required (without recognizing some of the points you list above). Changes to mingw32ccompiler to fix compilation with 4.x went in in https://github.com/numpy/numpy/pull/156. It would be good if you could check those. I will look into it more carefully, but overall, it seems that building atlas 3.8.4, numpy and scipy with gcc 4.x works quite well. The main issue is that gcc 4.* adds some dependencies on mingw dlls. There are two options: - adding the dlls in the installers - statically linking those, which seems to be a bad idea (generalizing the dll boundaries problem to exception and things we would rather not care about: http://cygwin.com/ml/cygwin/2007-06/msg00332.html). It probably makes sense make this move for numpy 1.7. If this breaks the ABI then it would be easiest to make numpy 1.7 the minimum required version for scipy 0.11. My thinking as well. Hi David, what is the current status of this issue? I kind of forgot this is a prerequisite for the next release when starting the 1.7.0 release thread. The only issue at this point is the distribution of mingw dlls. I have not found a way to do it nicely (where nicely means something that is distributed within numpy package). Given that those dlls are actually versioned and seem to have a strong versioning policy, maybe we can just install them inside the python installation ? cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy.mean problems
Hi Eraldo, Indeed, Pandas is a really really nice module! If it is going to take part of numpy, that's even better. Thanks for the suggestion. All the Best, Fred Eraldo Pomponi wrote: Hi Fred, Pandas has a nice interface to PyTable if you still need it: http://pandas.sourceforge.net/io.html#hdf5-pytables However, my intention was just to point you to pandas because it is really a powerful tool if you need to deal with tabular heterogenic data. It is also important to notice that there are plans in the numpy community to include/port part of this package directly in the codebase. This says a lot about how good it is... Best, Eraldo -- View this message in context: http://old.nabble.com/numpy.mean-problems-tp32945124p32975342.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fast Reading of ASCII files
On Wed, Dec 14, 2011 at 4:11 PM, Bruce Southey bsout...@gmail.com wrote: ** On 12/14/2011 01:03 AM, Chris Barker wrote: On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: genfromtxt sure looks close for an API This I don't agree with. It has a huge amount of keywords that just confuse or intimidate a beginning user. There should be a dead simple interface, even the loadtxt API is on the heavy side. well, yes, though it does do a lot -- do you have a smpler one in mind? Just looking at what I normally wouldn't need for simple data files and/or what a beginning user won't understand at once, the `unpack` and `ndmin` keywords could certainly be left out. `converters` is also questionable. That's probably as simple as it can get. Note that I don't think this should be changed now, that's not worth the trouble. But anyway, the really simple cases, are reallly simle, even with genfromtxt. I guess it's a matter of debate about what is a better API: a few functions, each adding a layer of sophistication or one function, with layers of sophistication added with an array of keyword arguments. There's always a trade-off, but looking at the docstring for genfromtxt should make it an easy call in this case. In either case, though I wish the multiple functionality built on the same, well optimized core code. I wish that too, but I'm fairly certain that you can't write that core code with the ability to handle missing and irregular data and make it close to the same speed as an optimized reader for regular data. I am not sure that you can even create a simple API here as even Python's csv module is rather complex especially when it just reads data as strings. It also 'hides' many arguments in the Dialect class although these are just the collection of 7 'fmtparam' arguments. It also provides the Sniffer class that tries to find correct format that can then be passed to the reader function. Then you still have to convert the data into the required types - another set of arguments as well as yet another pass through the data. In comparison, genfromtxt can perform sniffing I assume you mean the ``dtype=None`` example in the docstring? That works to some extent, but you still need to specify the delimiter. I commented on that on the loadtable PR. and both genfromtxt and loadtxt can read and convert the data. These also add some useful features like skipping rows (start, end and commented) and columns. However, it could be possible to create a sniffer function and a single data reader function leading to a 'simple' reader function but that probably would not change the API of the underlying data reader function. Better auto-detection of things like delimiters would indeed be quite useful. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fast Reading of ASCII files
On Wed, Dec 14, 2011 at 1:22 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Wed, Dec 14, 2011 at 4:11 PM, Bruce Southey bsout...@gmail.com wrote: ** On 12/14/2011 01:03 AM, Chris Barker wrote: On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: genfromtxt sure looks close for an API This I don't agree with. It has a huge amount of keywords that just confuse or intimidate a beginning user. There should be a dead simple interface, even the loadtxt API is on the heavy side. well, yes, though it does do a lot -- do you have a smpler one in mind? Just looking at what I normally wouldn't need for simple data files and/or what a beginning user won't understand at once, the `unpack` and `ndmin` keywords could certainly be left out. `converters` is also questionable. That's probably as simple as it can get. Just my two cents (and I was one of those who championed its inclusion), the ndmin feature is designed to prevent unexpected results that users (particularly beginners) may encounter with their datasets. Now, maybe it might be difficult to tell a beginner *why* they might need to be aware of it, but it is very easy to describe *how* to use. How many dimensions is your data? Two? Ok, just set ndmin=2 and you are good to go! Cheers! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fast Reading of ASCII files
On Wed, Dec 14, 2011 at 11:36 AM, Benjamin Root ben.r...@ou.edu wrote: well, yes, though it does do a lot -- do you have a smpler one in mind? Just looking at what I normally wouldn't need for simple data files and/or what a beginning user won't understand at once, the `unpack` and `ndmin` keywords could certainly be left out. `converters` is also questionable. That's probably as simple as it can get. this may be a function of a well written doc string -- if it is clear to the newbie that all the rest of this you don't need unless you have a wierd data file, then extra keyword arguments don't really hurt. A few examples of the basic use-cases go a long way. And yes, the core reader for the complex cases isn't going to e fast (it's going to be complex C code...). but we could still have a core reader that handled most cases. Anyway, I think it's time write code, and see if it can be rolled in somehow... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fast Reading of ASCII files
On Wed, Dec 14, 2011 at 9:54 PM, Chris Barker chris.bar...@noaa.gov wrote: On Wed, Dec 14, 2011 at 11:36 AM, Benjamin Root ben.r...@ou.edu wrote: well, yes, though it does do a lot -- do you have a smpler one in mind? Just looking at what I normally wouldn't need for simple data files and/or what a beginning user won't understand at once, the `unpack` and `ndmin` keywords could certainly be left out. `converters` is also questionable. That's probably as simple as it can get. this may be a function of a well written doc string -- if it is clear to the newbie that all the rest of this you don't need unless you have a wierd data file, then extra keyword arguments don't really hurt. A few examples of the basic use-cases go a long way. And yes, the core reader for the complex cases isn't going to e fast (it's going to be complex C code...). but we could still have a core reader that handled most cases. Okay, now we're on the same page I think. Anyway, I think it's time write code, and see if it can be rolled in somehow... Agreed. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] scipy installation problem
I'm using Eclipse (PyDev) on MacOS. I downloaded scipy010, installed it and added path to .mpkg file to PYTHONPATH and scipy to forced built-in. Nothing worked, I keep getting 'module scipy not found'. I then removed the link to the .mpkg and still nothing works. Strange enough, numpy works just fine. What should I do? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] scipy installation problem
On Wed, Dec 14, 2011 at 10:38 PM, Alex Ter-Sarkissov ater1...@gmail.comwrote: I'm using Eclipse (PyDev) on MacOS. I downloaded scipy010, installed it and added path to .mpkg file to PYTHONPATH and scipy to forced built-in. Nothing worked, I keep getting 'module scipy not found'. I then removed the link to the .mpkg and still nothing works. Strange enough, numpy works just fine. What should I do? Not sure what you mean by install here, but you're supposed to double-click the mpkg installer to run it, not put it on your PYTHONPATH. Note that to use the provided dmg installer, you have to also use the matching python from python.org Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] scipy installation problem
yes, that's exactly what i did. I'm using Python2.6 both in PyDev and Scipy. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] scipy installation problem
On Wed, Dec 14, 2011 at 10:56 PM, Alex Ter-Sarkissov ater1...@gmail.comwrote: yes, that's exactly what i did. I'm using Python2.6 both in PyDev and Scipy. Then you don't need to put anything on your pythonpath, since scipy gets installed to the normal site-packages dir. You'll have to provide more details about exactly which installers you used (incl. of python itself), and any possible PyDev oddities, for us to be able to help you here. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] scipy installation problem
yeah, I've already removed it, still doesn't work. I'm running Python 2.6 and SciPy version I'm trying to install is scipy-0.10.0-py2.6-python.org-macosx10.3., pydev version is 2.2.4 I've had no trouble running numpy or Tkinter for example. Also none of the other modules I'm using have been added to PythonPath The only problem I've had when installing the interpreter is the error message that sdlib was either not found or without .py files. I didn't add these files yet, but everything seems to be runnning smoothly except this issue with scipy. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] scipy installation problem
Dear Alexe, I'm not sure I understood what you mean by install like Ralf. However, I would also suggest, if you are using Eclipse and PyDev, (after installing new modules) to remove the current python interpreter (from Eclipse options) and then re-add it so that the whole pythonpath will be re-scanned and you will not see any red underline (with the msg: module not found) in your python editor. Cheers, Eraldo On Wed, Dec 14, 2011 at 10:52 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Wed, Dec 14, 2011 at 10:38 PM, Alex Ter-Sarkissov ater1...@gmail.comwrote: I'm using Eclipse (PyDev) on MacOS. I downloaded scipy010, installed it and added path to .mpkg file to PYTHONPATH and scipy to forced built-in. Nothing worked, I keep getting 'module scipy not found'. I then removed the link to the .mpkg and still nothing works. Strange enough, numpy works just fine. What should I do? Not sure what you mean by install here, but you're supposed to double-click the mpkg installer to run it, not put it on your PYTHONPATH. Note that to use the provided dmg installer, you have to also use the matching python from python.org Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] scipy installation problem
On Wed, Dec 14, 2011 at 11:12 PM, Alex Ter-Sarkissov ater1...@gmail.comwrote: yeah, I've already removed it, still doesn't work. I'm running Python 2.6 and SciPy version I'm trying to install is scipy-0.10.0-py2.6-python.org-macosx10.3., pydev version is 2.2.4 I've had no trouble running numpy or Tkinter for example. Also none of the other modules I'm using have been added to PythonPath The only problem I've had when installing the interpreter is the error message that sdlib was either not found or without .py files. I didn't add these files yet, but everything seems to be runnning smoothly except this issue with scipy. This seems to be a PyDev issue, google turns up http://stackoverflow.com/questions/5595276/pydev-eclipse-python-interpreters-error-stdlib-not-found If that doesn't help, you should ask on a PyDev mailing list. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] scipy installation problem
OK thanks guys, reinstalling the interpreter did the trick. I'm quite sure I did it before though, without any effect. More interestingly, I have two interpreters running, one for 2.6 and the other the auto. So the latter one still tells me the module isn't found, the former works just fine. Mystery? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
