Well I posted about this in Qt-Interest yesterday, seems someone made their own
library. Please see my copied section below.
No, it wasn't SAX or DOM, rather more like the XML stream reader stuff you
have.
And the fact that your response is focusing on XML is telling me I wasn't
clear,
because that is a trivial feature that falls out of the core implementation.
--- BEGIN COPY
I used Qt to parse DICOM files, in which the metadata is crucially important. I
actually came up with a tag "scanner" and "collector" paradigm. DICOM has
nested
metadata, and can be several KB worth. I also had streaming requirements (no
ability to load whole file into memory).
So I created a tag scanner that emitted a "tag" and "value" The tags are
defined
by DICOM, and are identifiers for discrete metadata elements like height,
width,
etc. The value could be various types, but each tag has a defined type (date,
time, integer, etc) so QVariant was a natural choice. The tags were defined as
unsigned ints, since they took two 16-bit unsigned numbers (first is "group",
second is "element", where group is like "patient" or "image" to which the
element applies).
Anyway, the simplified scanner class (we're not talking nested data yet) has 3
signals, 1 function:
signal begin() - emitted after successful file open
signal tag(uint id, QVariant value) - emitted as a tag is found
signal end() - emitted when all tags are complete
function read() - starts reading
The simplified collector class has three slots:
begin()
tag(int uid, qvariant value)
end()
Now, in order to use the collector, you subclass it and override tag(..)
Here, you can "collect" them in a simple public QMap<uint, QVariant>, or do
something much more interesting.
For instance, I had to look at a directory of images, and update a database
with
a filename and specific tag value. (Actually, it was a set of tags)
Creating a scanner, I then connected is signals to the slot of my sub-classed
collector, where in the tag(...) slot had the following:
if (captureList.contains(id)){
...
query.insert(filename, id, value.toString());
}
This approach has a couple advantages:
The scanner takes up a constant amount of memory.
The collector only collects the tags you want. (Or, to avoid re-scanning, all,
but that is your choice) so you get to handle the memory usage.
The scanner can be connected to multiple collectors, say a PrintCollector,
which
lists it o the screen)
When hooked to an QIODevice, you get tags emitted/collected/processed as they
are available.
Then, probably my biggest benefit is that you can also connect them to a
TagWriter class, which can be used to write QMaps, XML, SQL, etc... and then
you
have a file writer, or when coupled with a scanner, a format converter. I
personally used a QMap as internal storage format, but output in DICOM or XML
as
needed. I was also able to interface to the offis DICOM toolkit and read/write
their XML format. Overall the effort was a huge success.
Now, I've simplified this a bit for not handling nested data. With the addition
of two signals/slots and a little code to handle a stack, you get nested data.
push(int id)
pop()
Then you can use recursive QMaps or XML (XQuery/Path) to query/modify data.
Example int main(){
QFile input("file.dcm");
TagScannerDICOM scanner(&input); //QIODevice
TagCollectorPrinter printer(&scanner); // overridden constructor
auto-connects
TagColllctorWriterXml xmlwriter(&scanner, "out.xml"); // overridden
constructor auto-connects
return scanner.read(); // both printer and xmlwriter do their thing.
}
The really nice thing (and yet slightly difficult) is coming up with the global
image tag dictionary. We should use a master (GUID?) dictionary or the standard
DICOM dictionary to start with (since it defines the most tags by far) and
either provide a mapping from the master to the file type. DICOM allows
"private" tags, so any tag not already found in DICOM (ie. geo tags) could map
to a private tag. The DICOM dictionary speifies the uint tag, data type, and
name. So that if you wanted a more formal interface you could do:
image->tag("Width")->toInt(); // Where "Width" is looked up in a dictionary
and
mapped to a uint.
Which would ultimately be translated into (assuming all tags have been
collected):
imageTagMap[0x00340032].toInt();
While I mention DICOM several times, it is important to note that I am only
referencing it because it has the most complicated metadata structure, and the
largest tag dictionary.
In this way, to support a new tag, assuming the tag seralization is the same,
we
only need to add an entry to a dictionary, we do not need an interface to be
updated. ie. image->height() (however any such interface is a trivial mapping)
If you got this far thanks for reading, and I look forward to your feedback.
--- END COPY
DICOM
Continuing on, a DICOM implicit syntax is formatted as such:
struct {
ushort group;
ushort element;
uint length; // in bytes, rounded up to be even
uchar[x]; // where x is the serialized length+( length &1 ? 1: 0)
}
The metadata tags are (group, element) tuples, that are pre-defined. The
datatype for tags are also pre-defined. Pixel data is just tag = (7FFF,FFFF) (A
"tag" is (uint of group << 16 | element) )
The DICOM reader would then open the file, read the tags, each time doing: emit
tag(tag, data).
JPEGS, GIFs, PNGs, TIFFs, etc.
These other formats would of course not use a DICOM read serializer, but they
would have an EXIF serializer, or whatever is appropriate. These formats would
also do emit tag(tag, data). And I suggest we use DICOM tags, else we'll need
a
tag-mapping layer.
None of this has anything to do with XML.
Where XML does come in, is when you make a XML serializer. Which is nothing
more
than changing out the struct for a XML-formatted tag. e.g. <item tag="7FFFFFFF"
value="..."/>
Now, with an XML output serializer, you can then use XQuery on the output. Or,
if you write an XML input serializer, you can read XML and create an image.
XML is not needed at all.
At a minimum we only really need input serializers for our supported media
formats. Output serializers gives you the ability to convert and create images
with metadata, Which is somethign you really, really want to do for movies, and
images captured off cell phones since Android and iPhone now GPS stamp them.
Does that help? Does that change your understanding and impressions?
----- Original Message ----
From: "[email protected]" <[email protected]>
To: [email protected]; [email protected]
Sent: Wed, January 26, 2011 11:53:52 PM
Subject: RE: [Qt-mobility-feedback] Metadata handling
Hi Jason,
Sorry that we didn't get back to you sooner. We did review your proposal.
We think you were describing an API for accessing image meta-data as though it
were an XML document. Were you proposing a SAX like API? Here is some review
details from our dev.
"SAX from what I little understand is pretty heavy weight API, and I think the
QXmlStream classes are better suited to most people's requirements. I imagine
what the proposed API would also be overkill for most people.
XML isn't a bad analogue for meta-data (XMP for example is XML) but it only
addresses the problem of parsing the meta-data, interpreting the data may still
require knowledge of the underlying format."
It seems your proposal to extend this API to include actual decoding of an
image
in a pieces or streaming of the decoded data would be beneficial by avoiding
memory allocation for an entire image if it's not needed.
At this point, our conclusion is that your needs are reasonable but perhaps not
appropriate for Qt without additional functionality. It doesn't make much
sense
to us to re-invent image loading to be more suitable for image processing
without having any image processing functionality. Meta-data may warrant two
API one for well understood data (tags, gps data) and another for the more
generic access you were describing.
Regards,
Min
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On
Behalf Of ext Jason H
Sent: Wednesday, 26 January 2011 3:49 AM
To: [email protected]
Subject: [Qt-mobility-feedback] Metadata handling
Well a few months ago, I posted my ideas on how to handle metadata.
Has anyone given it any thought? Were my ideas rejected? Did they seem
interesting but need more explanation?
_______________________________________________
Qt-mobility-feedback mailing list
[email protected]
http://lists.qt.nokia.com/mailman/listinfo/qt-mobility-feedback
_______________________________________________
Qt-mobility-feedback mailing list
[email protected]
http://lists.qt.nokia.com/mailman/listinfo/qt-mobility-feedback
