On 25/08/11 16:53, Anders Logg wrote: > On Thu, Aug 25, 2011 at 09:59:44AM -0400, Garth N. Wells wrote: > >>>>> How about using DOM everywherme and reserve the use of SAX for an >>>>> XML->HDF5 converter? >>>>> >>>> >>>> That could be OK, but if we have the to implement a SAX parser it's >>>> probably easiest to have it in DOLFIN anyway. I don't see the advantage >>>> over having the SAX parser with the io code. >>> >>> I agree we should keep it in DOLFIN, but if the only thing it needs to >>> do is extract data and spit out HDF5, I imagine it can be simpler than >>> the current parser since it doesn't need to be parallel. (?) >>> >> >> To make things clearer, I've just renamed the LocalMeshData parsers to >> >> XMLLocalMeshDOM (was XMLLocalMeshData) >> >> and >> >> XMLLocalMeshSAX (was XMLLocalMeshDataDistributed) > > That's good. > >> When XMLLocalMeshSAX is complete, it may be desirable to remove >> XMLLocalMeshDOM. > > Either way is fine for me, as long as we decide which one to use. I > initially wanted to use SAX (as before) but the DOM looks easier and > may be enough if we plan to use HDF5 for large-scale problems anyway. > Or is it the case that DOM is a limitation even for medium sized > problems? >
It works for 'medium' (very arbitrary) size problems. >> I don't know what you mean by parallel - the XMLLocalMeshSAX works in >> the same way as the old parser (each process reading a chunk). I don't >> see how it can be made simpler by reading a XML file and then converting >> to HDF5. The steps that are there now will all still be required to read >> the XML mesh before writing a HDF file. > > I don't know HDF, but I imagine one could write one single file and > HDF will handle parallel parsing of that file later. Then the > conversion script we write does not need to do anything parallel, just > read line by line and convert from one format to another. > It may not be possible to do it line-by-line (I don't know, but I wouldn't want to bank on it). Even if line-by-line is technically possible, it could turn out to be terribly slow. We should support that a mesh can be read into memory (distributed), and written to HDF5. Since we'll have support for writing HDF5 meshes, if we can read a large XML mesh then we can re-use the HDF5 output code to make the conversion. I've removed the DOM-based LocalMeshData parser - there is no point to it since we can just read the mesh on one process using XMLMesh and use it to construct a dolfin::XMLLocalMeshData object. Garth > So we would not need to worry about the splitting of data into chunks, > skipping lines etc in the SAX-based XML --> HDF converter. And so we > would have a minimal amount of simpe SAX-based code. > > -- > Anders _______________________________________________ Mailing list: https://launchpad.net/~dolfin Post to : dolfin@lists.launchpad.net Unsubscribe : https://launchpad.net/~dolfin More help : https://help.launchpad.net/ListHelp
