Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
Matthieu Brucher wrote: Hi, I'm trying to test my code on several platforms, Windows and Linux, and I'm using some data files that where saved with a tofile(sep=' ') under Linux. Those files can be loaded without a problem under Linux, but under Windows with the latest numpy, these data cannot be loaded, some numbers are not considered - eg +inf or -inf). tofile is using pickling, right ? If you dump to a text file, there may be a problem because of end of line ? Is this a known behaviour ? How could I load these correctly under both platforms (I don't want to save them in binary form, I'm using the files for other purpose - Personally, I always use pytables: the file format (hdf5) is binary, but the file format has a standard C api (and C++/java as well), which means you can access those files pretty much anywhere, and is designed to be cross platform. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
Hi Matthieu On Fri, May 04, 2007 at 09:16:34AM +0200, Matthieu Brucher wrote: I'm trying to test my code on several platforms, Windows and Linux, and I'm using some data files that where saved with a tofile(sep=' ') under Linux. Those files can be loaded without a problem under Linux, but under Windows with the latest numpy, these data cannot be loaded, some numbers are not considered - eg +inf or -inf). Is this a known behaviour ? How could I load these correctly under both platforms (I don't want to save them in binary form, I'm using the files for other purpose - Please file a ticket at http://projects.scipy.org/scipy/numpy/newticket along with a short code snippet to reproduce the problem. That way we won't forget about it. Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
In short, I don't know that this is a bug. It is a missing feature, but It may be hard to get someone to write the code to account for the limited fscanf() in fromfile(). That's what I was thinking :( Matthieu ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
On Fri, May 04, 2007 at 09:44:02AM -0700, Christopher Barker wrote: Matthieu Brucher wrote: Example of the first line of my data file : 0.0 inf 13.9040914426 14.7406669444 inf 4.41783247603 inf inf 6.05071515635 inf inf inf 15.6925185021 inf inf inf inf inf inf inf I'm pretty sure fromfile() is using the standard C fscanf(). That means that whether in understands inf depends on the C lib. I'm guessing that the MS libc doesn't understand the same spelling of inf that the gcc one does. There may indeed be no literal for the IEEE Inf. It would be interesting to see how Inf and NaN (vs. inf and nan) are interpreted under Windows. Are there any free fscanf implementations out there that we can include with numpy? Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
Stefan van der Walt wrote: It would be interesting to see how Inf and NaN (vs. inf and nan) are interpreted under Windows. Neither works. I suspect there is no literal that works with the lib the python.org python is built with. (by the way, I'm testing with 2.4, but I think that's the same compiler as 2.5) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
On Sat, May 05, 2007 at 12:34:30AM +0200, Stefan van der Walt wrote: On Fri, May 04, 2007 at 09:44:02AM -0700, Christopher Barker wrote: Matthieu Brucher wrote: Example of the first line of my data file : 0.0 inf 13.9040914426 14.7406669444 inf 4.41783247603 inf inf 6.05071515635 inf inf inf 15.6925185021 inf inf inf inf inf inf inf I'm pretty sure fromfile() is using the standard C fscanf(). That means that whether in understands inf depends on the C lib. I'm guessing that the MS libc doesn't understand the same spelling of inf that the gcc one does. There may indeed be no literal for the IEEE Inf. It would be interesting to see how Inf and NaN (vs. inf and nan) are interpreted under Windows. Are there any free fscanf implementations out there that we can include with numpy? There's no need; all that fscanf is being used for is with the single format string %d (and variants for each type). So that's easily replaced with type-specific functions (strtol, strtod, etc.). For the floating-point types, checking first if the string matches inf or nan patterns would be sufficient. There's a bug in fromfile anyways: because it passes the separator directly to fscanf to skip over it, using a % in your separator will not work. -- ||\/| /--\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |[EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
Robert Kern wrote: 1.#INF and 1.#QNAN might be accepted though since that's what ftoa() gives for those quantities. not in Python: float(1.#INF) gives a value error. That may or may not reflect what *scanf does, but I suspect it does. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Difference in the number of elements in a fromfile() between Windows and Linux
On Fri, May 04, 2007 at 05:43:00PM -0500, Robert Kern wrote: Stefan van der Walt wrote: On Fri, May 04, 2007 at 09:44:02AM -0700, Christopher Barker wrote: Matthieu Brucher wrote: Example of the first line of my data file : 0.0 inf 13.9040914426 14.7406669444 inf 4.41783247603 inf inf 6.05071515635 inf inf inf 15.6925185021 inf inf inf inf inf inf inf I'm pretty sure fromfile() is using the standard C fscanf(). That means that whether in understands inf depends on the C lib. I'm guessing that the MS libc doesn't understand the same spelling of inf that the gcc one does. There may indeed be no literal for the IEEE Inf. It would be interesting to see how Inf and NaN (vs. inf and nan) are interpreted under Windows. I'm pretty sure that they are also rejected. 1.#INF and 1.#QNAN might be accepted though since that's what ftoa() gives for those quantities. So, from some googling, here's the special strings for floats, as regular expressions. The case of the letters doesn't seem to matter. positive infinity: [+]?inf [+]?Infinity 1\.#INF negative infinity: -Inf -1.#INF -Infinity not a number: s?NaN[0-9]+ (The 's' is for signalling NaNs, the digits are for diagnostic information. See the decimal spec at http://www2.hursley.ibm.com/decimal/daconvs.html) -1\.#IND 1\.#QNAN (Windows quiet NaN?) There may be more. If we wish to support these, then writing our own parser for them is probably the only way. I'll do it, I just need a complete list of what we want to accept :-) On the other side of the coin, I'd argue the string representations of our float scalars should also be platform-agnostic (standardising on Inf and NaN would be best, I think). -- ||\/| /--\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |[EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] matlab vs. python question
On 4/29/07, Benjamin Thyreau [EMAIL PROTECTED] wrote: Le Samedi 28 Avril 2007 20:03, Simon Berube a écrit: (...) On the other hand, if you are more interested in small projects where speed of development is more important than long term sustainability of the code Matlab is probably better. This is usually the case in research environment, at least until you figure out exactly what you want to do, then switch to python. I hope this help, (...) I guess the point of the question was, in what way should numpy/scipy be improved in order to get the same usability at those cases, according to you ? It's not as obvious as you make it sound, and there also are for sure some advantages in the ipython/scipy stack over matlab's prompt, usability-wise. Any experiences which helps in improving it is thus useful I like ipython's post-fix ? help. I was back in Matlab a couple of days ago and that's one of the few things I missed about my Numpy setup. I often find I'm typing a command and then I realize I don't recall what the arguments are so then I can just type one extra '?' to get the help. Of course even better would be if ipython were in a GUI, and docstrings could just be proactively prefectched and displayed in another pane while I type, or argument lists could be popped up like in PyCrust. Sadly, other than that I can't think of a whole lot about the numpy environment that I like better than the Matlab environment. (just talking about the environment here -- not the language). Matlab's debugger is very nicely integrated with the prompt, and plots just work, always, and are fast without having to think about ion/ioff stuff or show(). Never need any prayers to the god of GUI event loops to keep plots from freezing things up with Matlab. It's also refreshing to just be able to type zeros and rand and not have to worry about namespace prefixes. On the other hand I think I yelled a profanity out loud when Matlab gave me the can't define functions in a script error. I'd completely forgotten that particular bit of Matlab stupidity. There are probably ways to make the numpy environment more like the above. But there needs to be a 'button' that sets everything up like that or most people just won't find it. For instance maybe ipython could install a Numpy Interactive link that uses the -pylab flag and also pulls the numpy namespace into the global one. For debugger integration, is there any way to set up ipython so that on errors it will pop up a GUI debugger that shows the line of source code where the error occured and let you set break points? --bb ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
