Hi Bert,
I'm not arguing that you can represent most data in XML. I'm just concerned that mangling high volume or specialized data like for example sensor data, genom data and geo-spatial data into a document format might not work too well. Also, when the ER-diagram of non-openEHR data is fairly complex, producing a meaningful XSD and XML documents might not be that quick and easy (at least I don't know of a industry-strength tool that can help with this task. However, I may be wrong about this and I'd be happy to learn).
Regarding performance, we did some tests on SQL Server 2012 last year. As I have only experience with this particular database, it might well be that my critique does not apply to Oracle or Marklogic!
Just a minute ago I compared a simple SQL Query with an XQuery on our data repository. I simply wanted to get all validated blood pressure values and their corresponding datetimes of a pediatric icu. Using the plain relational representation of the data (we automatically map data from compositions to tables), it takes under 1 second to get all 329.273 rows. Having a full index on the blood pressure fragment of the composition (this is needed to get the internal tabular representation of the data) and a secondary index on the paths, querying of the same rows still takes 30 seconds (without, it would be 2 minutes. No surprise). Additionally, the size of the data increases from 10MB to 270MB.
This is the reality we face in out system, therefore, I consider XQuery and XML not an option for us to do analysis in this database layer. As said, this might not apply to a better implementation of XML by other vendors but I'd love to see some real-world numbers.
Just some thoughts and experiences, I'm not a dedicated database expert, therefore, I would not be sad if I'm proven wrong :)
Cheers,
Birger
Bert Verhees <[email protected]> hat am 13. Februar 2016 um 18:51 geschrieben:On 13-02-16 18:13, Birger Haarbrandt wrote:Even as a research associate or independent developer, its sometimes good to take a break from work :) I hope I didn't miss a point in your e-mail and hope that I understood corretly: when you store openEHR-data using XML technologies, you will have to use XML technologies to query that data. To my understanding, also Oracle will need xpath/xquery when you store XML data using the XML datatyped column. If you want to combine this data with other data that isn't based on XML, you will have to convert the external data to XML or mix query languages in the mentioned data bases.
I don't understand the point you make. The leafnodes in an archetype are always constraints on primitives, and XML can represent all types of primitives.
Everything that is not a leafnode is a structure, the structure is object oriented and easy to convert to XML, where the tag-names are the attribute-names, and the type-names are the typenames from the RM-XSD. The structure which is represented in XML is also easy to bring back to an object oriented structure.
It is just a notation, nothing else.
There are even standard java-classes (I thought in JAXB) which support this kind of things.
AQL is easy to convert to XQuery, but maybe I would prefer XQuery, and it is, for a non technical person possible to use a drag and drop interface which represents the archetypepaths.
I am missing your point, or better sad, we are not on the same track of understanding
Best regards
Bert
In short: It makes sense to not use the openEHR CDR for such queries but export it to something more suited for such purpose. This does not need to be a classical Data Warehouse (it has only limited use for the questions clinicians have) but certainly XML would not be my first choice for the analytics layer.Birger
Von meinem iPad gesendetThanks,
I always forget it is weekend, as independent developer, there ain't no such thing as a day of.
I agree that mixing query-languages is ugly, and I don't see the necessity.
But maybe, you will explain that next week
Bert
On 13-02-16 17:46, Birger Haarbrandt wrote:Hi Bert,I will post some more thoughts on these things after the weekend :) To just give a quick answer: imo it's important to have a flexible data format like the one I2B2 uses (roughly said EAV) to mix openEHR data with non-openEHR data. Making analysis on XML documents/databaes might prevent integration of other sources (and even if its possible like in SQL Server or Oracle, queries that mix XQuery and SQL might become pretty ugly. And I see no good way to provide a drag & drop interface to researchers/physicians to make queries).Cheers,Birger
Von meinem iPad gesendetNo comments, on the other hand it is Saturday
I had left out some necessary technical details.
I will possible build it and then have possible the fastest two-level-modeling engine in the world, which will, of course, also support OpenEHR.
So it is not really bad that this happens.
When it is ready, I will make an announcement.
Best regards and enjoy the weekend.
Bert Verhees
On 12-02-16 20:49, Bert Verhees wrote:On 12-02-16 18:26, Erik Sundvall wrote:if you are experimenting with open source native XML DBs for openEHR, it preformed well for "clinical" patient-specific querying even though all xml databases we tested were not suitable for ad hoc epidemiological population queries (without query specific indexing)
A very interesting paper. I have some first opinions on that. But first I need to explain what I think about the matter.
I have not prepared the story below, so there may be things which I write to fast. See it as provisional view, not as a hard opinion.
---------
There are relational database-configurations for OLAP and for OLTP. The combination is hard to find. There are reasons. This is a classic problem.
You need specific indexes for>_______________________________________________
openEHR-technical mailing list
[email protected]
http://lists.openehr.org/mailman/listinfo/openehr-technical_lists.openehr.org
_______________________________________________ openEHR-technical mailing list [email protected] http://lists.openehr.org/mailman/listinfo/openehr-technical_lists.openehr.org_______________________________________________
openEHR-technical mailing list
[email protected]
http://lists.openehr.org/mailman/listinfo/openehr-technical_lists.openehr.org
_______________________________________________ openEHR-technical mailing list [email protected] http://lists.openehr.org/mailman/listinfo/openehr-technical_lists.openehr.org
_______________________________________________
openEHR-technical mailing list
[email protected]
http://lists.openehr.org/mailman/listinfo/openehr-technical_lists.openehr.org
_______________________________________________ openEHR-technical mailing list [email protected] http://lists.openehr.org/mailman/listinfo/openehr-technical_lists.openehr.org
