Re: [Dhis2-devs] Dynamic attributes
Hi, Den 14. sep. 2011 kl. 16.13 skrev Bob Jolliffe: OK. Lets try Friday morning 09h30 my time - that's 10h30 Oslo time Dial-in number: +1-424-203-8000 or Skype call to: freeconferencing.8000 Code: 553472 I know you are under pressure to finish this stuff so the intent is not to slow things up. Just keep us all pointing in same direction :-) just thought I'd give a quick recap of the meeting, but haven't really got time to write it up :) Morten's approach seems reasonable, once we cleared up some misunderstandings around the consistency of the model. Only specific meta objects (indicators, org units, data elements..) will have attributes, and the values are stored in a per-object-type way. This makes it consistent, and fine as long as not too many types need attributes. If it turns out that we want these attributes for many types, this apporach will not scale too well. So in that case we should revisit this design and abstract it into a more general implementation. A decent recap, Bob and Morten? Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
A decent recap, Bob and Morten? Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Could we also add users to the objects with dynamic attributes? We need some more fields for alternative telephone numbers, fax, etc etc. Thanks, Jason On Fri, Sep 16, 2011 at 12:23 PM, Morten Olav Hansen morte...@gmail.com wrote: A decent recap, Bob and Morten? Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
On 16 September 2011 11:23, Morten Olav Hansen morte...@gmail.com wrote: A decent recap, Bob and Morten? Yes thanks for calling in and good recap. Scope creep on which objects can have extensible attributes could render it cumbersome eventually, but while the number of such objects is few it should work just fine. One other thing we also discussed was how these attributes would be serialized to xml. Given that there is no constraint on the attribute name which would ensure that it is a 'good' xml element/attribute name, we can't simply do the natural thing of: orgunit nameBobs Clinic/name ... no_of_beds0no_of_beds /orginit So we will have to do: orgunit nameBobs Clinic/name ... extensionAttributes attribute name=number of beds0/attribute attribute name=../attribute /extensionAttributes /orginit I think this *might* be ok. As long as it doesn't lead us into the old mess of having number of beds, no of beds etc as synonyms for the same thing. But that's a background worry I have which is more to do with that we are essentially using name as identifier again, but maybe I'm worrying too much. Attribute is an identifiable object so we could technically improve upon this in time. Good discussion. 30 minutes. I hope people didn't think it was a waste of time. I'm happy to host another one next week at 09h30 if anyone has a pressing design issue which might benefit from a run through. Cheers Bob Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
On 16 September 2011 11:54, Jason Pickering jason.p.picker...@gmail.com wrote: Could we also add users to the objects with dynamic attributes? We need some more fields for alternative telephone numbers, fax, etc etc. and so it begins ... Sounds reasonable to me :-) Or use ldap. Bob Thanks, Jason On Fri, Sep 16, 2011 at 12:23 PM, Morten Olav Hansen morte...@gmail.com wrote: A decent recap, Bob and Morten? Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Fine with me if you want to integrate LDAP while you are at it. :) On Fri, Sep 16, 2011 at 1:40 PM, Bob Jolliffe bobjolli...@gmail.com wrote: On 16 September 2011 11:54, Jason Pickering jason.p.picker...@gmail.com wrote: Could we also add users to the objects with dynamic attributes? We need some more fields for alternative telephone numbers, fax, etc etc. and so it begins ... Sounds reasonable to me :-) Or use ldap. Bob Thanks, Jason On Fri, Sep 16, 2011 at 12:23 PM, Morten Olav Hansen morte...@gmail.com wrote: A decent recap, Bob and Morten? Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
On 16 September 2011 13:07, Jason Pickering jason.p.picker...@gmail.com wrote: Fine with me if you want to integrate LDAP while you are at it. :) Yes I see there's even a blueprint for this (I wonder who from?). I actually did this once - ie setup dhis so you can autthenticate against ldap. Worked pretty fine against openldap, apache-ds and fedora directory sercice (now 389). Probably would against ad as well, but I didn't have that. The trouble is I got as far as authenticating but didn't understand the dhis currentuser service well enough to get further. I suspect its a small step. Bob On Fri, Sep 16, 2011 at 1:40 PM, Bob Jolliffe bobjolli...@gmail.com wrote: On 16 September 2011 11:54, Jason Pickering jason.p.picker...@gmail.com wrote: Could we also add users to the objects with dynamic attributes? We need some more fields for alternative telephone numbers, fax, etc etc. and so it begins ... Sounds reasonable to me :-) Or use ldap. Bob Thanks, Jason On Fri, Sep 16, 2011 at 12:23 PM, Morten Olav Hansen morte...@gmail.com wrote: A decent recap, Bob and Morten? Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
LDAP is really overkill for what we need at this point. We just have a few more fields like alternate telephone number which we manage through a seperate Excel sheet. Not ideal really. No real requirement for LDAP just to get another number in. :) On Fri, Sep 16, 2011 at 2:21 PM, Bob Jolliffe bobjolli...@gmail.com wrote: On 16 September 2011 13:07, Jason Pickering jason.p.picker...@gmail.com wrote: Fine with me if you want to integrate LDAP while you are at it. :) Yes I see there's even a blueprint for this (I wonder who from?). I actually did this once - ie setup dhis so you can autthenticate against ldap. Worked pretty fine against openldap, apache-ds and fedora directory sercice (now 389). Probably would against ad as well, but I didn't have that. The trouble is I got as far as authenticating but didn't understand the dhis currentuser service well enough to get further. I suspect its a small step. Bob On Fri, Sep 16, 2011 at 1:40 PM, Bob Jolliffe bobjolli...@gmail.com wrote: On 16 September 2011 11:54, Jason Pickering jason.p.picker...@gmail.com wrote: Could we also add users to the objects with dynamic attributes? We need some more fields for alternative telephone numbers, fax, etc etc. and so it begins ... Sounds reasonable to me :-) Or use ldap. Bob Thanks, Jason On Fri, Sep 16, 2011 at 12:23 PM, Morten Olav Hansen morte...@gmail.com wrote: A decent recap, Bob and Morten? Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Hi I also agree that User would benefit from dynamic attributes. Right now, I will concentrate on DE/I/OU only, but when that is finished, it should be easy enough to add support for Users also (but no reporting for now). -- Morten On Fri, Sep 16, 2011 at 2:32 PM, Jason Pickering jason.p.picker...@gmail.com wrote: LDAP is really overkill for what we need at this point. We just have a few more fields like alternate telephone number which we manage through a seperate Excel sheet. Not ideal really. No real requirement for LDAP just to get another number in. :) On Fri, Sep 16, 2011 at 2:21 PM, Bob Jolliffe bobjolli...@gmail.com wrote: On 16 September 2011 13:07, Jason Pickering jason.p.picker...@gmail.com wrote: Fine with me if you want to integrate LDAP while you are at it. :) Yes I see there's even a blueprint for this (I wonder who from?). I actually did this once - ie setup dhis so you can autthenticate against ldap. Worked pretty fine against openldap, apache-ds and fedora directory sercice (now 389). Probably would against ad as well, but I didn't have that. The trouble is I got as far as authenticating but didn't understand the dhis currentuser service well enough to get further. I suspect its a small step. Bob On Fri, Sep 16, 2011 at 1:40 PM, Bob Jolliffe bobjolli...@gmail.com wrote: On 16 September 2011 11:54, Jason Pickering jason.p.picker...@gmail.com wrote: Could we also add users to the objects with dynamic attributes? We need some more fields for alternative telephone numbers, fax, etc etc. and so it begins ... Sounds reasonable to me :-) Or use ldap. Bob Thanks, Jason On Fri, Sep 16, 2011 at 12:23 PM, Morten Olav Hansen morte...@gmail.com wrote: A decent recap, Bob and Morten? Yes, that seems to be a good recap. Its good enough for now ;) -- Morten Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Breaking out from our current hard coded set of object attributes is certainly a great step forward. There is also some advantage in trying to think a bit generally about what we are doing. We are seeing lots of progress in the right direction. I was about to start a discussion about names earlier, but maybe it makes sense to consider some of these things together. In general an entity within the dhis model universe has: (i) a set of identifiers (ii) a set of names (iii) a set of attributes (iv) a set of relations with other attributes One could think of names as attributes, but there are reasonable semantic reasons to consider them separately. On top of this generic view we have classes or categories of entity. Such as DataElement, OrgUnit etc, where the class of the entity determines the set of attributes and the type of relations between them. I think from an xml metadata representation perspective, this is probably how it should be modelled. The fact that some of the attributes may be fixed database fields and some dynamically configured attributes is incidental and purely historical. If you had the benefit of hindsight and were designing a datadictionary from scratch, you might never have had the fixed fields in the database. Or you might have come up with different ones. On import/export I think the distinction should simply be masked by the API. Fixed or non-fixed is an internal arrangement of little interest outside the system. Bob PS. One of the possible uses of attributes which I am seeing is the in fact the possibility of dynamically adding new identifiers for objects. On 8 September 2011 15:55, Jo Størset stor...@gmail.com wrote: Thanks for the responses. So I guess this might very well be a start of a general evolution in the meta model? It might be worth already now thinking a little through the possible consequences of this direction going forward. I think this is something we need :) As we work through how to represent the meta model in external formats (xml and so on), this might have implications on how generic a model we should create for these formats (and I guess the attribute definitions themselves have to represented in some way). In that context it might make sense to generalize both the objects these attributes apply to and the way we represent atrributes more generally. And also at least look at and specify likely future harmonization now? Changing the external formats once they are created is much more difficult than changing implementation details, so it might be worth it to think a little bit broader already now. A minor point - when I saw the term dynamic attributes, I was immediately thinking free-form attributes (more like tags you could put on individual org units). But I guess this is about predefined attribute types applied to (specific) meta objects(?) Maybe just calling it attributes might be good enough, if that is what it is? Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Hi Morten On a more concrete note than before ... :-) I've finally had a chance to look more closely at what you are doing. Can I make a small suggestion? Currently you are replicating a lot of code as you implement attributes across dataelements, indicators and orgunits. Currently these all extend AbstractNameable object. What about creating AbstractExtendableObject which extends AbstractNameableObject? Then include your SetAttributeValue attributeValues and related methods in there. Then DataElements, Indicators or what have you become extendable by simply extending that class? Bob On 14 September 2011 12:27, Bob Jolliffe bobjolli...@gmail.com wrote: Breaking out from our current hard coded set of object attributes is certainly a great step forward. There is also some advantage in trying to think a bit generally about what we are doing. We are seeing lots of progress in the right direction. I was about to start a discussion about names earlier, but maybe it makes sense to consider some of these things together. In general an entity within the dhis model universe has: (i) a set of identifiers (ii) a set of names (iii) a set of attributes (iv) a set of relations with other attributes One could think of names as attributes, but there are reasonable semantic reasons to consider them separately. On top of this generic view we have classes or categories of entity. Such as DataElement, OrgUnit etc, where the class of the entity determines the set of attributes and the type of relations between them. I think from an xml metadata representation perspective, this is probably how it should be modelled. The fact that some of the attributes may be fixed database fields and some dynamically configured attributes is incidental and purely historical. If you had the benefit of hindsight and were designing a datadictionary from scratch, you might never have had the fixed fields in the database. Or you might have come up with different ones. On import/export I think the distinction should simply be masked by the API. Fixed or non-fixed is an internal arrangement of little interest outside the system. Bob PS. One of the possible uses of attributes which I am seeing is the in fact the possibility of dynamically adding new identifiers for objects. On 8 September 2011 15:55, Jo Størset stor...@gmail.com wrote: Thanks for the responses. So I guess this might very well be a start of a general evolution in the meta model? It might be worth already now thinking a little through the possible consequences of this direction going forward. I think this is something we need :) As we work through how to represent the meta model in external formats (xml and so on), this might have implications on how generic a model we should create for these formats (and I guess the attribute definitions themselves have to represented in some way). In that context it might make sense to generalize both the objects these attributes apply to and the way we represent atrributes more generally. And also at least look at and specify likely future harmonization now? Changing the external formats once they are created is much more difficult than changing implementation details, so it might be worth it to think a little bit broader already now. A minor point - when I saw the term dynamic attributes, I was immediately thinking free-form attributes (more like tags you could put on individual org units). But I guess this is about predefined attribute types applied to (specific) meta objects(?) Maybe just calling it attributes might be good enough, if that is what it is? Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Hi Bob I've finally had a chance to look more closely at what you are doing. Can I make a small suggestion? Currently you are replicating a lot of code as you implement attributes across dataelements, indicators and orgunits. Currently these all extend AbstractNameable object. Currently, theres not too much duplication... just one added one field called attributeValues (with getter/setter). If you're talking about the newly committed code for making attributes/attributeMap available (three duplications there), that code is not finished, I just had to commit it since I'm working on something else now. Yes, that functionality should be made more generic, as its basically doing the same thing three times (and more times, if we also add these to datasets, etc) What about creating AbstractExtendableObject which extends AbstractNameableObject? Then include your SetAttributeValue attributeValues and related methods in there. Then DataElements, Indicators or what have you become extendable by simply extending that class? Yes, I don't really want to mess too much with the original domain classes, but making it a bit more generic is on the agenda. This is very much WIP and more refactoring could/should happen. :) -- Morten Bob On 14 September 2011 12:27, Bob Jolliffe bobjolli...@gmail.com wrote: Breaking out from our current hard coded set of object attributes is certainly a great step forward. There is also some advantage in trying to think a bit generally about what we are doing. We are seeing lots of progress in the right direction. I was about to start a discussion about names earlier, but maybe it makes sense to consider some of these things together. In general an entity within the dhis model universe has: (i) a set of identifiers (ii) a set of names (iii) a set of attributes (iv) a set of relations with other attributes One could think of names as attributes, but there are reasonable semantic reasons to consider them separately. On top of this generic view we have classes or categories of entity. Such as DataElement, OrgUnit etc, where the class of the entity determines the set of attributes and the type of relations between them. I think from an xml metadata representation perspective, this is probably how it should be modelled. The fact that some of the attributes may be fixed database fields and some dynamically configured attributes is incidental and purely historical. If you had the benefit of hindsight and were designing a datadictionary from scratch, you might never have had the fixed fields in the database. Or you might have come up with different ones. On import/export I think the distinction should simply be masked by the API. Fixed or non-fixed is an internal arrangement of little interest outside the system. Bob PS. One of the possible uses of attributes which I am seeing is the in fact the possibility of dynamically adding new identifiers for objects. On 8 September 2011 15:55, Jo Størset stor...@gmail.com wrote: Thanks for the responses. So I guess this might very well be a start of a general evolution in the meta model? It might be worth already now thinking a little through the possible consequences of this direction going forward. I think this is something we need :) As we work through how to represent the meta model in external formats (xml and so on), this might have implications on how generic a model we should create for these formats (and I guess the attribute definitions themselves have to represented in some way). In that context it might make sense to generalize both the objects these attributes apply to and the way we represent atrributes more generally. And also at least look at and specify likely future harmonization now? Changing the external formats once they are created is much more difficult than changing implementation details, so it might be worth it to think a little bit broader already now. A minor point - when I saw the term dynamic attributes, I was immediately thinking free-form attributes (more like tags you could put on individual org units). But I guess this is about predefined attribute types applied to (specific) meta objects(?) Maybe just calling it attributes might be good enough, if that is what it is? Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
On 14 September 2011 14:05, Morten Olav Hansen morte...@gmail.com wrote: Hi Bob I've finally had a chance to look more closely at what you are doing. Can I make a small suggestion? Currently you are replicating a lot of code as you implement attributes across dataelements, indicators and orgunits. Currently these all extend AbstractNameable object. Currently, theres not too much duplication... just one added one field called attributeValues (with getter/setter). If you're talking about the newly committed code for making attributes/attributeMap available (three duplications there), that code is not finished, I just had to commit it since I'm working on something else now. Yes, that functionality should be made more generic, as its basically doing the same thing three times (and more times, if we also add these to datasets, etc) What about creating AbstractExtendableObject which extends AbstractNameableObject? Then include your SetAttributeValue attributeValues and related methods in there. Then DataElements, Indicators or what have you become extendable by simply extending that class? Yes, I don't really want to mess too much with the original domain classes, but making it a bit more generic is on the agenda. This is very much WIP and more refactoring could/should happen. :) Sure. I know. I'm just making sure I keep in touch because I'm also in the process of working out how best to represent these things for the purpose of of importing/exporting them. So what you do here has some significant bearing on what I do. The few bits of duplication you have currently will fan out further when we serialize/deserailize to xml. Can I suggest we fit in a brief design call this week to touch base? I have a freeconference number we can use. Maybe tomorrow or Friday? Lars are you around and/or available as well? Regards Bob -- Morten Bob On 14 September 2011 12:27, Bob Jolliffe bobjolli...@gmail.com wrote: Breaking out from our current hard coded set of object attributes is certainly a great step forward. There is also some advantage in trying to think a bit generally about what we are doing. We are seeing lots of progress in the right direction. I was about to start a discussion about names earlier, but maybe it makes sense to consider some of these things together. In general an entity within the dhis model universe has: (i) a set of identifiers (ii) a set of names (iii) a set of attributes (iv) a set of relations with other attributes One could think of names as attributes, but there are reasonable semantic reasons to consider them separately. On top of this generic view we have classes or categories of entity. Such as DataElement, OrgUnit etc, where the class of the entity determines the set of attributes and the type of relations between them. I think from an xml metadata representation perspective, this is probably how it should be modelled. The fact that some of the attributes may be fixed database fields and some dynamically configured attributes is incidental and purely historical. If you had the benefit of hindsight and were designing a datadictionary from scratch, you might never have had the fixed fields in the database. Or you might have come up with different ones. On import/export I think the distinction should simply be masked by the API. Fixed or non-fixed is an internal arrangement of little interest outside the system. Bob PS. One of the possible uses of attributes which I am seeing is the in fact the possibility of dynamically adding new identifiers for objects. On 8 September 2011 15:55, Jo Størset stor...@gmail.com wrote: Thanks for the responses. So I guess this might very well be a start of a general evolution in the meta model? It might be worth already now thinking a little through the possible consequences of this direction going forward. I think this is something we need :) As we work through how to represent the meta model in external formats (xml and so on), this might have implications on how generic a model we should create for these formats (and I guess the attribute definitions themselves have to represented in some way). In that context it might make sense to generalize both the objects these attributes apply to and the way we represent atrributes more generally. And also at least look at and specify likely future harmonization now? Changing the external formats once they are created is much more difficult than changing implementation details, so it might be worth it to think a little bit broader already now. A minor point - when I saw the term dynamic attributes, I was immediately thinking free-form attributes (more like tags you could put on individual org units). But I guess this is about predefined attribute types applied to (specific) meta objects(?) Maybe just calling it attributes might be good enough, if that is what it is? Jo
Re: [Dhis2-devs] Dynamic attributes
Can I suggest we fit in a brief design call this week to touch base? I have a freeconference number we can use. Maybe tomorrow or Friday? Lars are you around and/or available as well? Sure. I can tomorrow or Friday. Lars is in Uganda, but I'm sure he'll respond later if he has time for a conference call. -- Morten Regards Bob -- Morten Bob On 14 September 2011 12:27, Bob Jolliffe bobjolli...@gmail.com wrote: Breaking out from our current hard coded set of object attributes is certainly a great step forward. There is also some advantage in trying to think a bit generally about what we are doing. We are seeing lots of progress in the right direction. I was about to start a discussion about names earlier, but maybe it makes sense to consider some of these things together. In general an entity within the dhis model universe has: (i) a set of identifiers (ii) a set of names (iii) a set of attributes (iv) a set of relations with other attributes One could think of names as attributes, but there are reasonable semantic reasons to consider them separately. On top of this generic view we have classes or categories of entity. Such as DataElement, OrgUnit etc, where the class of the entity determines the set of attributes and the type of relations between them. I think from an xml metadata representation perspective, this is probably how it should be modelled. The fact that some of the attributes may be fixed database fields and some dynamically configured attributes is incidental and purely historical. If you had the benefit of hindsight and were designing a datadictionary from scratch, you might never have had the fixed fields in the database. Or you might have come up with different ones. On import/export I think the distinction should simply be masked by the API. Fixed or non-fixed is an internal arrangement of little interest outside the system. Bob PS. One of the possible uses of attributes which I am seeing is the in fact the possibility of dynamically adding new identifiers for objects. On 8 September 2011 15:55, Jo Størset stor...@gmail.com wrote: Thanks for the responses. So I guess this might very well be a start of a general evolution in the meta model? It might be worth already now thinking a little through the possible consequences of this direction going forward. I think this is something we need :) As we work through how to represent the meta model in external formats (xml and so on), this might have implications on how generic a model we should create for these formats (and I guess the attribute definitions themselves have to represented in some way). In that context it might make sense to generalize both the objects these attributes apply to and the way we represent atrributes more generally. And also at least look at and specify likely future harmonization now? Changing the external formats once they are created is much more difficult than changing implementation details, so it might be worth it to think a little bit broader already now. A minor point - when I saw the term dynamic attributes, I was immediately thinking free-form attributes (more like tags you could put on individual org units). But I guess this is about predefined attribute types applied to (specific) meta objects(?) Maybe just calling it attributes might be good enough, if that is what it is? Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Den 14. sep. 2011 kl. 15.32 skrev Morten Olav Hansen: Can I suggest we fit in a brief design call this week to touch base? I have a freeconference number we can use. Maybe tomorrow or Friday? Lars are you around and/or available as well? Sure. I can tomorrow or Friday. Lars is in Uganda, but I'm sure he'll respond later if he has time for a conference call. I should also be able to make it.. Just give the time in advance :) Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
OK. Lets try Friday morning 09h30 my time - that's 10h30 Oslo time Dial-in number: +1-424-203-8000 or Skype call to: freeconferencing.8000 Code: 553472 I know you are under pressure to finish this stuff so the intent is not to slow things up. Just keep us all pointing in same direction :-) Cheers Bob On 14 September 2011 14:40, Jo Størset stor...@gmail.com wrote: Den 14. sep. 2011 kl. 15.32 skrev Morten Olav Hansen: Can I suggest we fit in a brief design call this week to touch base? I have a freeconference number we can use. Maybe tomorrow or Friday? Lars are you around and/or available as well? Sure. I can tomorrow or Friday. Lars is in Uganda, but I'm sure he'll respond later if he has time for a conference call. I should also be able to make it.. Just give the time in advance :) Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Hi All, I was just watching this thread and wanted to share our possibly similar experiences from iHRIS. Admittedly I have not looked at the details of how DHIS is implementing dynamic attributes in the database, but I would suspect the design pattern is similar to how iHRIS has done its form and fields. (I have also seen this design pattern in openMRS). Although this flexibility has been great, it did have a significant impact when it came to reporting time. Even with proper indices, MySQL couldn't handle multiple joins and the reports quickly became essentially unusable. As a solution we had to implement a middle layer (form caching) which sits between the data storage system and the reporting system. We were able to do this only only after addaing a sufficient amount of meta-data about our fields and forms (e.g. the relationship between them, the SQL data types, etc.) I don't know how heavily you plan to make use of the dynamic attributes in reporting, so it may be a non-issue for you all, but it may be a topic for your design call. Cheers. -carl On 09/14/2011 10:13 AM, Bob Jolliffe wrote: OK. Lets try Friday morning 09h30 my time - that's 10h30 Oslo time Dial-in number: +1-424-203-8000 or Skype call to: freeconferencing.8000 Code: 553472 I know you are under pressure to finish this stuff so the intent is not to slow things up. Just keep us all pointing in same direction :-) Cheers Bob On 14 September 2011 14:40, Jo Strset stor...@gmail.com wrote: Den 14. sep. 2011 kl. 15.32 skrev Morten Olav Hansen: Can I suggest we fit in a brief design call this week to touch base? I have a freeconference number we can use. Maybe tomorrow or Friday? Lars are you around and/or available as well? Sure. I can tomorrow or Friday. Lars is in Uganda, but I'm sure he'll respond later if he has time for a conference call. I should also be able to make it.. Just give the time in advance :) Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
On 14 September 2011 15:32, Carl Leitner litlf...@ibiblio.org wrote: Hi All, I was just watching this thread and wanted to share our possibly similar experiences from iHRIS. Admittedly I have not looked at the details of how DHIS is implementing dynamic attributes in the database, but I would suspect the design pattern is similar to how iHRIS has done its form and fields. (I have also seen this design pattern in openMRS). Although this flexibility has been great, it did have a significant impact when it came to reporting time. Even with proper indices, MySQL couldn't handle multiple joins and the reports quickly became essentially unusable. As a solution we had to implement a middle layer (form caching) which sits between the data storage system and the reporting system. We were able to do this only only after addaing a sufficient amount of meta-data about our fields and forms (e.g. the relationship between them, the SQL data types, etc.) I don't know how heavily you plan to make use of the dynamic attributes in reporting, so it may be a non-issue for you all, but it may be a topic for your design call. Hi Carl Thanks for the observation. I am also not sure to what extent these will be used in reporting - though I can only imagine they will be used for something :-) Reducing to the extreme we would simply have an identifier, the class of the object and its attributes. And from an external perspective this is a neat and orthogonal way of viewing them. I even recall Murod has done something fairly similar. But as you point out, this neat reductionism can come back to bite on the performance side. So what is more likely to emerge as a pragmatic extension which builds on the history of what is already there, is a hybrid of core attributes and dynamic ones. The core attributes being hard coded columns in tables and the dynamic ones being linked through database relations. Which gives rise to the question of which should be core and which should be dynamic and why. At first pass the answer to that is probably straightforward - the core attributes already exist as column names, and are already heavily used in reporting. In time we might find a more rational basis to distinguish. From an external exchange perspective (I'm trying to make my life really simple) there is a level at which these are all simply attributes. Whether a particular system, such as dhis, strategically, historically or accidentally chooses to represent some of these as database fields and some of them as dynamic attributes is an internal detail to that system. So when I see an orgunit I might see something like: orgunit identifier(s)=.. . type=Clinic name='Bob's Clinic' alternateName='Robert's Fancy Clinic for the bewildered' geoCode=... contactPerson='Bob Jolliffe' bedcapacity=0 etc / It is up to the consumer of such an element to decide how to actually store this stuff internally and different systems might well do it differently. In DHIS I guess we would collect up all the fixed fields we have and store them, then take whatever is left and store them as dynamic or extended attributes. From the producer side we want to be able to identify the object and then dump all its attributes - implementing from DHIS perspective would imply simply dumping all the core attributes followed by whatever dynamic attributes it might have, without necessarily exposing this detail in what gets spat out.. Managing the performance/flexibility tradeoff is tricky. Particularly as we want to combine both the qualities of a performing data warehouse and a flexible metadata repository into a single system. I think the hybrid approach will take us so far. Its good to see Morten working on it. Cheers Bob Cheers. -carl On 09/14/2011 10:13 AM, Bob Jolliffe wrote: OK. Lets try Friday morning 09h30 my time - that's 10h30 Oslo time Dial-in number: +1-424-203-8000 or Skype call to: freeconferencing.8000 Code: 553472 I know you are under pressure to finish this stuff so the intent is not to slow things up. Just keep us all pointing in same direction :-) Cheers Bob On 14 September 2011 14:40, Jo Størset stor...@gmail.com wrote: Den 14. sep. 2011 kl. 15.32 skrev Morten Olav Hansen: Can I suggest we fit in a brief design call this week to touch base? I have a freeconference number we can use. Maybe tomorrow or Friday? Lars are you around and/or available as well? Sure. I can tomorrow or Friday. Lars is in Uganda, but I'm sure he'll respond later if he has time for a conference call. I should also be able to make it.. Just give the time in advance :) Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list:
Re: [Dhis2-devs] Dynamic attributes
Final thought from me for now ... On 14 September 2011 17:11, Bob Jolliffe bobjolli...@gmail.com wrote: On 14 September 2011 15:32, Carl Leitner litlf...@ibiblio.org wrote: Hi All, I was just watching this thread and wanted to share our possibly similar experiences from iHRIS. Admittedly I have not looked at the details of how DHIS is implementing dynamic attributes in the database, but I would suspect the design pattern is similar to how iHRIS has done its form and fields. (I have also seen this design pattern in openMRS). Although this flexibility has been great, it did have a significant impact when it came to reporting time. Even with proper indices, MySQL couldn't handle multiple joins and the reports quickly became essentially unusable. As a solution we had to implement a middle layer (form caching) which sits between the data storage system and the reporting system. There are some places where we do something similar. Most notably the DHIS resource tables can be thought of as something like this specifically for reporting. We were able to do this only only after addaing a sufficient amount of meta-data about our fields and forms (e.g. the relationship between them, the SQL data types, etc.) I don't know how heavily you plan to make use of the dynamic attributes in reporting, so it may be a non-issue for you all, but it may be a topic for your design call. Hi Carl Thanks for the observation. I am also not sure to what extent these will be used in reporting - though I can only imagine they will be used for something :-) Reducing to the extreme we would simply have an identifier, the class of the object and its attributes. And from an external perspective this is a neat and orthogonal way of viewing them. I even recall Murod has done something fairly similar. But as you point out, this neat reductionism can come back to bite on the performance side. So what is more likely to emerge as a pragmatic extension which builds on the history of what is already there, is a hybrid of core attributes and dynamic ones. The core attributes being hard coded columns in tables and the dynamic ones being linked through database relations. Which gives rise to the question of which should be core and which should be dynamic and why. At first pass the answer to that is probably straightforward - the core attributes already exist as column names, and are already heavily used in reporting. In time we might find a more rational basis to distinguish. From an external exchange perspective (I'm trying to make my life really simple) there is a level at which these are all simply attributes. Whether a particular system, such as dhis, strategically, historically or accidentally chooses to represent some of these as database fields and some of them as dynamic attributes is an internal detail to that system. So when I see an orgunit I might see something like: orgunit identifier(s)=.. . type=Clinic name='Bob's Clinic' alternateName='Robert's Fancy Clinic for the bewildered' geoCode=... contactPerson='Bob Jolliffe' bedcapacity=0 etc / It is up to the consumer of such an element to decide how to actually store this stuff internally and different systems might well do it differently. In DHIS I guess we would collect up all the fixed fields we have and store them, then take whatever is left and store them as dynamic or extended attributes. From the producer side we want to be able to identify the object and then dump all its attributes - implementing from DHIS perspective would imply simply dumping all the core attributes followed by whatever dynamic attributes it might have, without necessarily exposing this detail in what gets spat out.. Managing the performance/flexibility tradeoff is tricky. Particularly as we want to combine both the qualities of a performing data warehouse and a flexible metadata repository into a single system. I think the hybrid approach will take us so far. Its good to see Morten working on it. Cheers Bob Cheers. -carl On 09/14/2011 10:13 AM, Bob Jolliffe wrote: OK. Lets try Friday morning 09h30 my time - that's 10h30 Oslo time Dial-in number: +1-424-203-8000 or Skype call to: freeconferencing.8000 Code: 553472 I know you are under pressure to finish this stuff so the intent is not to slow things up. Just keep us all pointing in same direction :-) Cheers Bob On 14 September 2011 14:40, Jo Størset stor...@gmail.com wrote: Den 14. sep. 2011 kl. 15.32 skrev Morten Olav Hansen: Can I suggest we fit in a brief design call this week to touch base? I have a freeconference number we can use. Maybe tomorrow or Friday? Lars are you around and/or available as well? Sure. I can tomorrow or Friday. Lars is in Uganda, but I'm sure he'll respond later if he has time for a conference call. I should also be able to make it.. Just give the time in advance :) Jo
Re: [Dhis2-devs] Dynamic attributes
Hi - What is the planned use cases for this? This is part of my work for Rwanda, the system they are using today has quite a bit more attributes connected to each data element (or indicator as it was known in that system), and it was decided that instead of expanding the set of attributes connected to meta-data today, we would rather create a dynamic system were they would be allowed to create their own attributes. For the specific use case, what they want is: 1) Add more meta-attributes to data elements (but we also add support for indicators, organisation units) 2) Create a data repository report (data dictionary) in PDF. For this, I will expand the Export to PDF functionality that already exists in data elements. - Is this significantly different from the attributes in the tracking module (I guess these might be more dynamic, but not sure)? Unsure, I have not looked into this. But Lars gave me the go-ahead on this (and I guess they are different enough). - It doesn't seem to be linked to the general classes of nameable/identifiable object. Is it meant to be implemented just for specific meta object types? I'm a bit unsure what you mean here.. Both attribute and attributeoption implement AbstractIdentifiableObject.. - How should these be handled in the communication with external systems - is it a general component that we should think through the use of in relation to things like the mobile solutions? I guess that would be up to your team ;) I have not given that part of it any further thoughts. -- Morten Would be great to get some background info. Jo [1] https://blueprints.launchpad.net/dhis2/+spec/dynamic-attributes-for-meta-objects ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
For the specific use case, what they want is: 1) Add more meta-attributes to data elements (but we also add support for indicators, organisation units) 2) Create a data repository report (data dictionary) in PDF. For this, I will expand the Export to PDF functionality that already exists in data elements. This is also in fact a very general requirement we see many places - they want a data dictionary with dynamic attributes attached to data elements, indicators, orgunits to be able to define more comprehensive information beyond name, description etc. - Is this significantly different from the attributes in the tracking module (I guess these might be more dynamic, but not sure)? Unsure, I have not looked into this. But Lars gave me the go-ahead on this (and I guess they are different enough). Different in the sense that PatientAttribute and PatientAttributeValue have Patient-specific associations. We might investigate if this can be harmonized in the future though. ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
- It doesn't seem to be linked to the general classes of nameable/identifiable object. Is it meant to be implemented just for specific meta object types? We are planning to implement for data element, indicator, org unit only at least for this release. But it can easily be extended if we need it for more objects. The associations are unidirectional from the DataElement/IdentifiableObject - AttributeValue in order to keep the AttributeValue object clean. ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Dynamic attributes
Thanks for the responses. So I guess this might very well be a start of a general evolution in the meta model? It might be worth already now thinking a little through the possible consequences of this direction going forward. I think this is something we need :) As we work through how to represent the meta model in external formats (xml and so on), this might have implications on how generic a model we should create for these formats (and I guess the attribute definitions themselves have to represented in some way). In that context it might make sense to generalize both the objects these attributes apply to and the way we represent atrributes more generally. And also at least look at and specify likely future harmonization now? Changing the external formats once they are created is much more difficult than changing implementation details, so it might be worth it to think a little bit broader already now. A minor point - when I saw the term dynamic attributes, I was immediately thinking free-form attributes (more like tags you could put on individual org units). But I guess this is about predefined attribute types applied to (specific) meta objects(?) Maybe just calling it attributes might be good enough, if that is what it is? Jo ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] Dynamic attributes
Hi, I see there is development going on for dynamic attributes [1] on some meta objects (DataElement, Indicator, DataSet, OrganisationUnit?). Some questions: - What is the planned use cases for this? - Is this significantly different from the attributes in the tracking module (I guess these might be more dynamic, but not sure)? - It doesn't seem to be linked to the general classes of nameable/identifiable object. Is it meant to be implemented just for specific meta object types? and then I guess my main interest: - How should these be handled in the communication with external systems - is it a general component that we should think through the use of in relation to things like the mobile solutions? Would be great to get some background info. Jo [1] https://blueprints.launchpad.net/dhis2/+spec/dynamic-attributes-for-meta-objects ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
