Hi, A suggestion, even if it is not directly related : generally the "io" module is used, with streams, as high level file objects. There you have two solutions : - either use paraview. I heard they had a Python API. Why not blend Spyder and Paraview, then... - Or use HDF5 format and Python bindings. PyTables, as told on their site, has the reputation to be "blind fast" and efficient at loading while having metadata looking like pandas' dataframes. But there it is up to you at the moment to refill your data into hdf stores, I suppose ? Personnally, the hdf5 format reminds me both hyperspy library and TDMS Labview's technical data format... Le 28 oct. 2014 15:20, "Charles Vellutini" <[email protected]> a écrit :
> Hi, > The addition of DataFrames as objects that can be viewed (and edited) in > Spyder's Variable Explorer is a fantastic development. Viewing data is > extremely important in serious data analysis and related debugging. With > this addition Spyder approaches the convenience and workability of > dedicated, mature statistical packages such as Stata -- all with the > performance and malleability of python. In my view, a true game changer. > > Now, I have noticed that the feature does not work well (yet) on large > data sets. On my system (python 3.4, 8 Go RAM), attempting to use the > Variable Explorer with a df with more than say 100,000 rows freezes Spyder > altogether. More work is needed is optimize viewing (load rows/columns only > as they are viewed, or a similar strategy?). Also, I would like to suggest > that viewing is much more important than editing -- in case it helps to > optimize the feature? Editing data through a browser is not something you > normally do - viewing data on the other hand, you do all the time. > > Again congratulations on this, I truly believe that this is important for > the python data analysis community. > > > > -- > You received this message because you are subscribed to the Google Groups > "spyder" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/spyderlib. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "spyder" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/spyderlib. For more options, visit https://groups.google.com/d/optout.
