Am Samstag, den 06.01.2007, 09:08 +0000 schrieb Dieter: > > We need a file format for the data files. > > > > magic string > > format version number > > data, including user specified strings for signal names > > checksum > > allow for comments (e.g '#' ) > > allow for demand paging (how?) > > Here is one way to do it: > > /ogp_pci_analyzer/data/testrun23/part10000 > > # OGP PCI ANALYZER DATASET FORMAT 0.1 > # CHECKSUM 123456789 > # sample number, clock, data0, data1, data2, ... > 10000,0,0,0,1,0,1,1,1, ... > 10001,1,0,0,1,0,1,1,1, ... > 10002,0,0,0,1,0,1,0,1, ... > 10003,1,0,0,1,0,1,1,1, ...
Ouch. Text format is ok most of the time, but not if it's 16 times bigger than the simpler alternative: just store the raw data in one file, and the metadata in another file, preferably XML. The two files would be linkey by base filename, not by reference as with cue/bin. This is to allow copying of metadata templates without having to modify them beyond a standard checksum recalculation. (Why do we need a checksum on disk anyway?) Advantages over the ASCII format: * Over an order of magnitude smaller files. * Only a single copy of metadata to keep up to date. * Only two files instead of a number proportional to the data size. * Even simpler because no conversion to ASCII and eventually back is necessary. * Human readability is just a hexdump away. It can even decode the address/data lines as hex for you, as long as they are byte-aligned (I'd put them always first as a convention). * Actually handles demand paging by delegating the work to the OS when mmapped. No need to switch between multiple data files while scrolling. * Actually possible to process files by piping them. No need to go back to the start of the file and insert the correct checksum. * Actually possible to cat files together. No need to filter out headers first. * No need to completely rewrite all data files because the length of the header changed (e. g. renamed a signal). * Lexic syntax is defined by a DTD for metadata and an endianness definition for the data. Final metadata parser can be written using any XML library instead of flex or a bunch of regular expressions. Regular expressions are still possible since the XML won't be recursive. My proposal is only for the simplest possible format, and only if we're going that way. If there is a suitable standard like whatever VCD is, then let's use that, because we instantly get graphic viewers. Yet another idea: Maybe we can patch GTKWave to be embeddable via GTK's plugin architecture and/or accept whatever format we choose via shared memory/pipe/other IPC. - Viktor Pracht _______________________________________________ Open-graphics mailing list [email protected] http://lists.duskglow.com/mailman/listinfo/open-graphics List service provided by Duskglow Consulting, LLC (www.duskglow.com)
