A Monday 24 September 2007, Francesc Altet escrigué:
> A Monday 24 September 2007, Michael Hoffman escrigué:
> > Hi Francesc. Thanks for the explanation.
> >
> > Francesc Altet wrote:
> > >> Additionally, what would you think about allowing the user to
> > >> specify a depth-first iteration in the walk functions? That is
> > >> how I will work around the problem unless you think a fix to
> > >> this issue will be forthcoming very soon.
> > >
> > > Well, unfortunately, and for the reasons I exposed above, I don't
> > > think you can significantly accelerate the File.walkNodes() by
> > > just doing software improvements.
> >
> > Perhaps it would still take the same time over the whole file, but
> > I think that by doing a depth-first search I can get to the first
> > Leaf more quickly.
> >
> > This is important in two scenarios. First, sometimes I just need to
> > get a single array, and it doesn't matter which one. Secondly, it
> > is more reassuring to users (and even myself) to be able to print
> > progress reports as the file is loading.
>
> Mmm, I don't know if I understand you well, but perhaps using a
> combination of File.getNode and File.{iterNodes, listNodes} (or its
> Group.{_f_iterNodes, _f_iterNodes} counterparts) could help you avoid
> reinventing the wheel?Thinking a bit more into this problem, perhaps you can get the access to node metainfo faster if you create a table with several fields (pathname, node_class and others) and populate it. Then, for accessing the metainfo you are interested in, you only have to read this table, which is a very fast operation. Of course, you have to be careful and synchronize the table data with the actual metadata of nodes each time it changes on disk; however, if your data is mostly read-only then this is not a relevant issue. Hope that helps, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-" ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
