Re: [Matplotlib-users] Best way to use Excel Data

2010-10-15 Thread Christopher Barker
On 10/14/10 9:52 PM, Alessio Civ wrote: > Let' put things this way: if you have to work with many records, it is > better if you have a database. pyTables is worth a look, too" http://www.pytables.org/moin -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/N

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-14 Thread Alessio Civ
Let' put things this way: if you have to work with many records, it is better if you have a database. With a database you can query what you need and only this is worthed the effort of using a DB. butterw wrote: > > Hi Alessio, > > Thank you for the sqlite code example. > > What have been t

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-11 Thread Peter Butterworth
Hi Alessio, Thank you for the sqlite code example. What have been the key advantages of using a Database over a structured array for your applications ? http://docs.python.org/library/sqlite3.html : SQLite is a C library that provides a lightweight disk-based database that doesn’t require a sepa

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-10 Thread Alessio Civ
Hi Butterw, first of all, if you are working on data a lot, we could get in contact. I need to work better on my scripts and we could help each other. I've uploaded my script that imports from csv to sqlite. It's not a totally clean script, as I am working on something else now I couldn't clean

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-10 Thread Peter Butterworth
Hi, To load csv data, I use a modified version of csv2rec for which the data type of each column is specified explicitly in the data file. By removing the dtype guessing you get a speedup and you also avoid potential mess-ups. Alessio: sadly you right about it not being possible to trust Excel

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-10 Thread Alessio Civ
Hi, a strong advice from someone who is using excel format with tons of data is to save them in csv and then import in Sqlite. Excel messes up the data types and gives a lot of troubles with numbers. Sqlite is fast and data are secure. The power of this system is that you can query your data an

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-07 Thread John Hunter
On Thu, Oct 7, 2010 at 12:27 PM, Michael Droettboom wrote: >  You may want to look at this as well: > > http://matplotlib.sourceforge.net/api/mlab_api.html?highlight=csv#matplotlib.mlab.csv2rec And these examples: http://matplotlib.sourceforge.net/search.html?q=codex+csv2rec JDH

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-07 Thread Michael Droettboom
You may want to look at this as well: http://matplotlib.sourceforge.net/api/mlab_api.html?highlight=csv#matplotlib.mlab.csv2rec Mike On 10/07/2010 01:04 PM, João Luís Silva wrote: > On 10/07/2010 05:11 PM, Jahan Mohiuddin wrote: >> Hi, >> >> I am an novice-intermediate user of python (I took a

Re: [Matplotlib-users] Best way to use Excel Data

2010-10-07 Thread João Luís Silva
On 10/07/2010 05:11 PM, Jahan Mohiuddin wrote: > Hi, > > I am an novice-intermediate user of python (I took a 1 semester course > in scientific programming with python). I wanted to know what the best > way is to manipulate, analyze, and plot Microsoft Excel data. The > methods I've looked into: >

[Matplotlib-users] Best way to use Excel Data

2010-10-07 Thread Jahan Mohiuddin
Hi, I am an novice-intermediate user of python (I took a 1 semester course in scientific programming with python). I wanted to know what the best way is to manipulate, analyze, and plot Microsoft Excel data. The methods I've looked into: 1. Save data in CSV file and use csv.dictreader