OK cross purposes resolved ...
I think Jason you start to get the heart of the matter over permission,
auditing and publication.
One of the problems I see with the current proposal is to do with
permission. By allowing deletion via importing we cannot easily
distinguish between the authority to enter data and the authority to delete
data. That I think is possibly problematic and is what I was thinking
would be solved with a DELETE method. In principle the same could apply to
a csv payload. Except
Whereas POST (and PUT) imply an enclosed payload, I believe DELETE simply
identifies a resource thru the url. So one could delete a (slightly
mythical) datavalueset or a datavalue, but not very practical for a whole
bundle of values (not consisting of a dataset). In which case we are
forced to look instead at a more SOAP-ish RPC type mechanism in combination
with the POST (like we do with import strategy - perhaps DELETE needs to be
an optional import strategy).
Thinking further, currently someone with authority to import already has
authority to overwrite data, so having additional authority to delete isn't
actually that different. Though no less problematic.
It strikes me that we cannot easily escape the notion of publication.
Whereas our datavalue table (and related event table) maybe a slightly
wild, fluid and untamed space, once we publish data from that space (eg
donor or any other type of official reporting), something must get fixed
somewhere. Either the published data itself gets preserved (eg in a
separate table or system) or we mark it somehow as unalterable in the
source. Jim might have some thoughts on this as I recall he is addressing
similar problem with the PEPFAR reporting though I don't recall the detail
of the implementation (and I missed the presentation).
It is quite important that we get a reasonable and justifiable semantics in
place here as we are currently in the process of trying to shoehorn our dxf
and related api through IHE. So agree that we need to have a bit more
discussion on this.
Bob
On 4 October 2014 11:22, Jason Pickering jason.p.picker...@gmail.com
wrote:
Hi Bob,
Yes, I think we likely are, which I think shows we need more thinking on
this issue.
I think I also do not really know what NULL in DXF means either. Lars, is
it a ,, or ,, or ,NULL, ?
While I see the utility of using DELETE to perform this operation, it
does not solve the more urgent problem of how to delete existing values
when importing through CSV from one DHIS2 instance to the other. It also
raises some interesting questions about when a value would should be
allowed to be UPDATED/DELETED. Coming back to our friend CRIS, they had
implemented the concept of a soft delete, so that deletes could be
transmitted explicitly in XML, without having to infer it from something
like a NULL or rely on possibly unknown rules like zero significance in
upstream systems to take care of this. The data values in their system were
thus never deleted but rather marked as deleted. Maybe DHIS 1.4 had
something like this as well? But the other question here is that even if a
downstream system sends you a NULL or a zero or a request to delete a
record, do you choose to accept it? My thinking here is that I know for
instance with PSI, they say that data cannot be changed after it has been
submitted to the donor. Such restrictions may also exist with PEPFAR. So,
considering if a downstream system wants to change a data value or delete
it, do you as the upstream system choose to accept it? Right now, we have
favored speed of import over security of the data, but it might need to be
something we should look into.
Drifting here Bob, but maybe I am no longer talking at cross-purposes? :)
Regards,
Jason
On Sat, Oct 4, 2014 at 12:49 AM, Bob Jolliffe bobjolli...@gmail.com
wrote:
Jason maybe we talk at cross purposes. What you describe makes perfect
sense. If a server imports a zero or any other value it is reasonable to
expect that to replace the existing value. And if that server does not
store zeroes, then deleting what is there seems intuitively correct.
But that is not what the blueprint refers to. It suggests the importing
of nulls as a general mechanism for deletion. I think that is counter
intuitive, a little dangerous and also unclear what is null wrt dxf.
--
Jason P. Pickering
email: jason.p.picker...@gmail.com
tel:+46764147049
--
Mailing list: https://launchpad.net/~dhis2-devs-core
Post to : dhis2-devs-core@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs-core
More help : https://help.launchpad.net/ListHelp
