On Fri, Nov 16, 2007 at 07:50:07PM -0700, Rahul Garg wrote: > It would be awesome if you guys could respond to some of the following > questions :
OK, I'll take a bit of time to do this. > a) Can you guys tell me briefly about the kind of problems you are > tackling with numpy and scipy? My day-to-day use is data processing of an atomic physics experiment (Bose Einstein condensation). The data created by the experiment is retrived from a bunch of instruments (scopes, cameras, and sensors) and stored in an HDF5 file for each run, along with the parameters of the run. This is done in Matlab for historical reasons, and MatLab is an absolute pain for this. I have implemented a similar system in Python for another experiment (I wrote an article sumerizing the lessons learnt and the patterns to follow, hell this was the fourth experiment control system I built, each time using a different platform www.gael-varoquaux.info/computers/agile_computer_control_of_an_experiment.pdf ). The data is first stored on a windows box that runs the experiment, then it is downloaded by our Linux server that keeps a hash table of all the experiments ran and some figures of merit. The data processing is done on the Linux box (quite often connected remotely using Nomachine's NX). The data is accessed in a database-like way through a custom Python module (1000 LOC). Recently I have switched to a object-relationnal-mapping-like loading of the HDF5-files (hierarchical data trees) through a database-like interface. It rocks, IMHO. We do simple fitting, blurring, filter, averaging, fft, ... on the data, and plot the results. Another of my uses is for simple toy models. Recently I have been elaborating a statistical model of an experiment, so I was running Monte Carlo simulations, but I have ran dynamical model integrations, and linear algebra calculations always revolving around the physics of cold atomic gases. > b) Have you ever felt that numpy/scipy was slow and had to switch to > C/C++/Fortran? Never. However I really like the fact that I can if I need to. When controling instruments in Python I often have to use their C SDK. In which case I use Pyrex or Ctypes. > c) Do you use any form of parallel processing? Multicores? SMPs? > Clusters? If yes how did u utilize them? No. Most often I am limited by IO. I have the feeling that unlike most people on the list, my main work is not on a computer, but on an experiment. Gaël _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
