A Thursday 31 January 2008, Darren Dale escrigué: > > Ummm, I will ask to the HDF5 mailing list if it would be possible > > to get a unique identifier for all the links pointing to same data > > area. If HDF5 can provide such a identifier, the next step should > > be to rethink the structure of the metadata cache in PyTables and > > implement a new one based on the 'inode' concept, instead of the > > 'node path' one, which certainly is not a trivial task (to say the > > least). > > I saw your post, hopefully there is some support in HDF5 on which to > build.
As you can see in the message from Elena Pourmal (that I forwarded to the list), it seems that such a 'inode' identifier does exist. So, after discussing this new possibility with Ivan, we think that, although implementation of hard links in PyTables is doable, it will probably require the addition of a new layer between the node representation in-memory and the node in disk, with an additional conversion of all the objects inheriting from Node to proxies (the links) to a new object (the object node on disk) that is responsible to do the actual I/O. Also, the LRU cache for nodes in PyTables will need to be adapted to this new layer. All in all, this implies an almost complete rewrite of most of the Python code (as well as some Pyrex) in current PyTables, and more importantly, an important change in its internal structure. I want to be clear here: this is a formidable task for a person that is not familiar with the internals of PyTables, and even in the improbable case that you achieve what you want, you will probably end doing this very differently than we would do it, so the chances to include your implementation in PyTables mainstream would be rather remote (but you can always fork PyTables if that fulfill your needs). I don't like to seem money-grubbing but, given the complexity of the problem, I think that your best choice for getting this implemented is to sponsor us. Normally I'm not so direct, but after thinking a lot, it is the best suggestion I can do to you. Cheers, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-" ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
