Re: [Sword-TAP] An example Implementation (VIDEO)
I'm just catching up here. I fully agree with what Al says about not overloading the edit-media link. #g -- Alistair Miles wrote: > Hi Richard, > > Apologies again, please bear in mind these comments are based on a partial > reading of the doc, will hopefully have more time to read in full soon. More > inline... > > On Mon, Mar 14, 2011 at 06:08:56PM +, Richard Jones wrote: >>> I notice in the current spec you overload the 'edit-media' relation to >>> define new protocol behaviour when POSTing binary content to a deposit's >>> edit-media URI. >> I'm not sure what you mean, sorry. We haven't specified any >> behaviour for POST on the edit-media URI (EM-URI in the document). >> We do a PUT on that URI which I think is consistent with the AtomPub >> approach (replace the old file with the new file). We do do a POST >> on the edit URI (Edit-URI in the document) which isn't covered by >> AtomPub at all, but is consistent with a RESTful approach. Is it >> that that bothers you? > > Sorry, I must've mis-read first time. Now I'm reading section 6.6.1 "Adding > New Packages or Files to a Container"... > > ...OK, yes I think I would object to using the edit-URI for this purpose. > Well, > actually, no, I don't care what URI you use, what I would object to is > specifying the use of the 'edit' link relation to indicate which URI can be > used for this purpose. > > I'm just trying to come at this from a REST/hypermedia point of view. > > What this spec effectively does is to change the meaning of the 'edit' > link relation, or in this case to "overload" it by giving it an additional > meaning over what IANA (via the AtomPub spec) has to say about it. > >>From a practical point of view, as a client developer, how do I know when > the 'edit' link means only what it says in the IANA registry, and when the > 'edit' link also means what it says in the SWORD spec? > > E.g., say my client receives an Atom entry document. The client finds an > 'edit' link, and because it's programmed with knowledge of atompub, it then > knows what URI to use to update or delete the entry. But how does my client > additionally know that the URI at the end of the 'edit' link also supports > adding new packages or files to a container? Should it look for other sword: > foreign markup in the entry document, then modify its understanding of the > 'edit' link relation based on the presence of this markup? Or should it just > assume that the 'edit' link means this in all cases, and just attempt the > operation (which may fail if it was actually just talking to a plain old > atompub server)? > > Instead of specifying what operations the Edit-URI should support, which is > effectively redefining/overloading the meaning of the 'edit' link relation, > I would suggest that the SWORD specification simply define a new link > relation, e.g.: > > http://purl.org/net/sword/rel/append-package-contents > > Client developers would then know to use this link relation when looking for > the URI to use to POST additional package contents too. And server developers > would be free to implement this behaviour using whatever URIs they choose - > they could do it with the edit-URI, or they could do it with a completely > different URI, it's up to them, and it doesn't matter for the client because > the client is hypermedia-driven. > > To labour this point a bit more, by specifying that the edit-URI has to > be used for this purpose, you are placing unnecessary constraints on how > servers are implemented. I.e., as a server implementer, I'm now forced to > extend/override/modify my existing server-side code for handling requests to > edit-URIs to implement this behaviour. If instead you just define a new link > relation, then server developers are free to stick this new behaviour at any > URI of their choosing, which gives the server-developer far more freedom in > how > the behaviour is implemented. And it's also simple and crystal-clear for > client > developers how a client decides which URIs to use for what protocol behaviour. > > Similarly for any other new protocol operations you want to specify, I would > suggest defining new link relations, rather than specifying what URIs should > be used or changing the meaning of any existing link relations like 'edit' > or 'edit-media'. > > So, e.g., for "Adding New Metadata to a Resource", rather than specifying that > the edit-URI should be used, the spec could define a new link relation like: > > http://purl.org/net/sword/rel/append-metadata > > Similarly for "Adding New Metadata and Packages or Files to a Resource > with Multipart". > > Am I making sense? > > Cheers, > > Alistair > > > [1] > http://swordapp.org/wp-content/uploads/2011/03/SWORDProfile.html#protocoloperations_addingcontent_binary > > >>> I wonder if an alternative pattern might give you a bit more flexibility, >>> and be a bit easier to implement, without having to define any new protocol >>> behaviour. >>> >>> Let's
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Ian, On 18/03/11 09:11, Ian Stuart wrote: > On 17/03/11 19:37, Richard Jones wrote: >>> In this second, expanded, view there are three things one need to define >>> within the discussion >>> >>> 1) What the singular Thing is: a (zip|xml|csv|xyz) file >>> 2) What "standard" the manifest file is written to (METS, EPrintsXML, >>> etc) >>> 3) What "standard" the metadata file is written to (DC, QDC, MODS, >>> EPrintsXML, BibTek, etc...) >>> >>> Now, one could define a new identifier for each combination of manifest >>> & metadata, or one can describe them separately >>> One has great flexibility& expandability, the other uses fewer header >>> fields. >> >> I think at the moment we're going for a URI which describes the >> combination. This is partly because /at the moment/ repositories tend to >> support a finite set of Packages which consist of their favourite >> metadata formats and their favourite structural manifests. So not all >> possible permutations of structural and descriptive data is currently >> likely. This may, of course, change. > > This is not a problem... we just need to be up-front about it. > > This mean, for example, that "METSDSpaceSIP" actually means "Zip file, > with binary files included (subdirectories allowed). Manifest called > 'mets.xml', which contains the meta-data in epcdx format." > > I'm happy with that. but it does mean that this will spawn a LARGE > number of distinct variations, each of which needs a unique IRI. > (Again, not a problem: the package I develop for the OA-RJ broker has an > IRI in my opendepot.org name-space) Yup, that's exactly what it means. Otherwise we're into media feature sets, which we sort of discussed earlier on the list - seems way too complicated for the moment. I'm happy that people will mint IRIs for the formats that they use regularly. Cheers, Richard > -- Enable your software for Intel(R) Active Management Technology to meet the growing manageability and security demands of your customers. Businesses are taking advantage of Intel(R) vPro (TM) technology - will your software be a part of the solution? Download the Intel(R) Manageability Checker today! http://p.sf.net/sfu/intel-dev2devmar ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Richard, On Thu, Mar 17, 2011 at 08:17:28PM +, Richard Jones wrote: > Hi Alistair, > > Thanks for this detailed analysis. I believe your point is > completely valid, and I hadn't considered it before. Some thoughts > in line ... > > >Apologies again, please bear in mind these comments are based on a partial > >reading of the doc, will hopefully have more time to read in full soon. More > >inline... > > > >On Mon, Mar 14, 2011 at 06:08:56PM +, Richard Jones wrote: > >>>I notice in the current spec you overload the 'edit-media' relation to > >>>define new protocol behaviour when POSTing binary content to a deposit's > >>>edit-media URI. > >> > >>I'm not sure what you mean, sorry. We haven't specified any > >>behaviour for POST on the edit-media URI (EM-URI in the document). > >>We do a PUT on that URI which I think is consistent with the AtomPub > >>approach (replace the old file with the new file). We do do a POST > >>on the edit URI (Edit-URI in the document) which isn't covered by > >>AtomPub at all, but is consistent with a RESTful approach. Is it > >>that that bothers you? > > > >Sorry, I must've mis-read first time. Now I'm reading section 6.6.1 "Adding > >New Packages or Files to a Container"... > > > >...OK, yes I think I would object to using the edit-URI for this purpose. > >Well, > >actually, no, I don't care what URI you use, what I would object to is > >specifying the use of the 'edit' link relation to indicate which URI can be > >used for this purpose. > > > >I'm just trying to come at this from a REST/hypermedia point of view. > > > >What this spec effectively does is to change the meaning of the 'edit' > >link relation, or in this case to "overload" it by giving it an additional > >meaning over what IANA (via the AtomPub spec) has to say about it. > > > >>From a practical point of view, as a client developer, how do I know when > >the 'edit' link means only what it says in the IANA registry, and when the > >'edit' link also means what it says in the SWORD spec? > > Yup, this is a key insight. We are in want of polymorphism in @rel > values ... > > >E.g., say my client receives an Atom entry document. The client finds an > >'edit' link, and because it's programmed with knowledge of atompub, it then > >knows what URI to use to update or delete the entry. But how does my client > >additionally know that the URI at the end of the 'edit' link also supports > >adding new packages or files to a container? Should it look for other sword: > >foreign markup in the entry document, then modify its understanding of the > >'edit' link relation based on the presence of this markup? Or should it just > >assume that the 'edit' link means this in all cases, and just attempt the > >operation (which may fail if it was actually just talking to a plain old > >atompub server)? > > For simplicity in the profile this is what I would expect. As it is > in the profile SWORDv2 allows for any URI to respond with Method Not > Allowed or Not Implemented. > > But from a client implementation point of view, is this too indistinct? I think client developers will want some way to differentiate, so that clients only offer the additional functionality to the user if they know the functionality is supported by the service. I.e., it's about user-experience, client developers will want to avoid presenting messages to the user like "sorry, the request has failed...". Clients can, of course, send an OPTIONS request to find out what methods are supported at any particular URI, then adapt the functionality they present to a user, but that may not be fine-grained enough, and coding that sort of thing on the client is more complicated than simply baking in knowledge of what various different link relations mean. > We could ensure that the server always announces to the client that > it is a sword server, by mandating the use of a sword:version > element in the atom:entry (deposit receipt). This is already > mandated in the app:service element, so it would be a minimal > extension. Yep, that would be a possible work-around for clients, if you wanted to avoid minting new link relations. > >Instead of specifying what operations the Edit-URI should support, which is > >effectively redefining/overloading the meaning of the 'edit' link relation, > >I would suggest that the SWORD specification simply define a new link > >relation, e.g.: > > > >http://purl.org/net/sword/rel/append-package-contents > > > >Client developers would then know to use this link relation when looking for > >the URI to use to POST additional package contents too. And server developers > >would be free to implement this behaviour using whatever URIs they choose - > >they could do it with the edit-URI, or they could do it with a completely > >different URI, it's up to them, and it doesn't matter for the client because > >the client is hypermedia-driven. > > So this would be an explicit way of just asserting that the URI can > be used for th
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Richard, (See comment inline below) On Thu, 2011-03-17 at 19:37 +, Richard Jones wrote: > Hi Ian, > > On 15/03/11 11:55, Ian Stuart wrote: > > On 14/03/11 21:21, Richard Jones wrote: > >> I think that a large part of what SWORD adds to AtomPub is how you talk > >> about packaging. We're definitely not going to go down the route of > >> specifying package formats or support levels outside of AtomPub's basic > >> definitions. What I'd be particularly interested to know is whether you > >> think that SWORD currently provides you with enough tools to talk > >> /about/ packaging? > >> > >> We have, for example, had a couple of mentions of the desire to ONLY use > >> mime-types, which we know is insufficient for describing packages - what > >> is the mime type of a DSpace METS package? And I have also verified that > >> the registration process for mime-types is not strictly trivial. Do the > >> extra features for providing Package information during deposit, > >> Accept-Package headers for retrieval, and sword:packaging elements in > >> the service documents and atom entries provide you with a framework > >> which will work for your requirements? > > From various conversations I've had with people, the concept of > > "packaging" is confusing for people. > > > > For many people, a "package" is a simple thing... and therefore can be > > assigned some kind of identifier (hence the idea of hooking into MIME > > Types) > > > > For others, me included, a "Package" is a concept that is composed of > > multiple parts: > > 1) There is the singular Thing that holds everything (liken this to a > > cardboard box) > > 2) Inside the Thing, there is a file of descriptive data [metadata] and > > some number of binary files (liken this to a Manilla file and some > > data-disks in the box) > > 3) There may also be a Manifest file that describes what the contents of > > the box are, and how they relate to each other. > > It is this definition of Package that we need to be able to support > within SWORD. > > Could the problem be down to the name (as you perhaps hint at here). > Would we be better changing the names of "Package" related terms to > something like "FormatProfile" or something? > > > In this second, expanded, view there are three things one need to define > > within the discussion > > > > 1) What the singular Thing is: a (zip|xml|csv|xyz) file > > 2) What "standard" the manifest file is written to (METS, EPrintsXML, > > etc) > > 3) What "standard" the metadata file is written to (DC, QDC, MODS, > > EPrintsXML, BibTek, etc...) > > > > Now, one could define a new identifier for each combination of manifest > > & metadata, or one can describe them separately > > One has great flexibility & expandability, the other uses fewer header > > fields. > > I think at the moment we're going for a URI which describes the > combination. This is partly because /at the moment/ repositories tend > to support a finite set of Packages which consist of their favourite > metadata formats and their favourite structural manifests. So not all > possible permutations of structural and descriptive data is currently > likely. This may, of course, change. Do we mean one URI for all 3? Looking at Mets packages as an example, I've just skim read the Mets Profiles docs and they don't appear to encompass the mime-type. We could get round this by wrapping a Mets profile in our own SWORD package specs which specify the Mets profile and a mime-type. Is that the plan ? Cheers, Robin. > > Thoughts? > > Cheers, > > Richard > > > > > -- > Colocation vs. Managed Hosting > A question and answer guide to determining the best fit > for your organization - today and in the future. > http://p.sf.net/sfu/internap-sfd2d > ___ > Sword-app-techadvisorypanel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
On 17/03/11 19:37, Richard Jones wrote: >> In this second, expanded, view there are three things one need to define >> within the discussion >> >> 1) What the singular Thing is: a (zip|xml|csv|xyz) file >> 2) What "standard" the manifest file is written to (METS, EPrintsXML, >> etc) >> 3) What "standard" the metadata file is written to (DC, QDC, MODS, >> EPrintsXML, BibTek, etc...) >> >> Now, one could define a new identifier for each combination of manifest >> & metadata, or one can describe them separately >> One has great flexibility & expandability, the other uses fewer header >> fields. > > I think at the moment we're going for a URI which describes the > combination. This is partly because /at the moment/ repositories tend to > support a finite set of Packages which consist of their favourite > metadata formats and their favourite structural manifests. So not all > possible permutations of structural and descriptive data is currently > likely. This may, of course, change. This is not a problem... we just need to be up-front about it. This mean, for example, that "METSDSpaceSIP" actually means "Zip file, with binary files included (subdirectories allowed). Manifest called 'mets.xml', which contains the meta-data in epcdx format." I'm happy with that. but it does mean that this will spawn a LARGE number of distinct variations, each of which needs a unique IRI. (Again, not a problem: the package I develop for the OA-RJ broker has an IRI in my opendepot.org name-space) -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Alistair, Thanks for this detailed analysis. I believe your point is completely valid, and I hadn't considered it before. Some thoughts in line ... > Apologies again, please bear in mind these comments are based on a partial > reading of the doc, will hopefully have more time to read in full soon. More > inline... > > On Mon, Mar 14, 2011 at 06:08:56PM +, Richard Jones wrote: >>> I notice in the current spec you overload the 'edit-media' relation to >>> define new protocol behaviour when POSTing binary content to a deposit's >>> edit-media URI. >> >> I'm not sure what you mean, sorry. We haven't specified any >> behaviour for POST on the edit-media URI (EM-URI in the document). >> We do a PUT on that URI which I think is consistent with the AtomPub >> approach (replace the old file with the new file). We do do a POST >> on the edit URI (Edit-URI in the document) which isn't covered by >> AtomPub at all, but is consistent with a RESTful approach. Is it >> that that bothers you? > > Sorry, I must've mis-read first time. Now I'm reading section 6.6.1 "Adding > New Packages or Files to a Container"... > > ...OK, yes I think I would object to using the edit-URI for this purpose. > Well, > actually, no, I don't care what URI you use, what I would object to is > specifying the use of the 'edit' link relation to indicate which URI can be > used for this purpose. > > I'm just trying to come at this from a REST/hypermedia point of view. > > What this spec effectively does is to change the meaning of the 'edit' > link relation, or in this case to "overload" it by giving it an additional > meaning over what IANA (via the AtomPub spec) has to say about it. > >> From a practical point of view, as a client developer, how do I know when > the 'edit' link means only what it says in the IANA registry, and when the > 'edit' link also means what it says in the SWORD spec? Yup, this is a key insight. We are in want of polymorphism in @rel values ... > E.g., say my client receives an Atom entry document. The client finds an > 'edit' link, and because it's programmed with knowledge of atompub, it then > knows what URI to use to update or delete the entry. But how does my client > additionally know that the URI at the end of the 'edit' link also supports > adding new packages or files to a container? Should it look for other sword: > foreign markup in the entry document, then modify its understanding of the > 'edit' link relation based on the presence of this markup? Or should it just > assume that the 'edit' link means this in all cases, and just attempt the > operation (which may fail if it was actually just talking to a plain old > atompub server)? For simplicity in the profile this is what I would expect. As it is in the profile SWORDv2 allows for any URI to respond with Method Not Allowed or Not Implemented. But from a client implementation point of view, is this too indistinct? We could ensure that the server always announces to the client that it is a sword server, by mandating the use of a sword:version element in the atom:entry (deposit receipt). This is already mandated in the app:service element, so it would be a minimal extension. > Instead of specifying what operations the Edit-URI should support, which is > effectively redefining/overloading the meaning of the 'edit' link relation, > I would suggest that the SWORD specification simply define a new link > relation, e.g.: > > http://purl.org/net/sword/rel/append-package-contents > > Client developers would then know to use this link relation when looking for > the URI to use to POST additional package contents too. And server developers > would be free to implement this behaviour using whatever URIs they choose - > they could do it with the edit-URI, or they could do it with a completely > different URI, it's up to them, and it doesn't matter for the client because > the client is hypermedia-driven. So this would be an explicit way of just asserting that the URI can be used for that purpose, but in the profile we could say that the Package Contents URI MAY be the same as the Edit-URI. That's sort of a question, and sort of just me reflecting back what you're saying to confirm that I've understood. If I have understood, I like the approach - we wouldn't need to include the sword:version in the atom:entry as I suggest above in this case. > To labour this point a bit more, by specifying that the edit-URI has to > be used for this purpose, you are placing unnecessary constraints on how > servers are implemented. I.e., as a server implementer, I'm now forced to > extend/override/modify my existing server-side code for handling requests to > edit-URIs to implement this behaviour. If instead you just define a new link > relation, then server developers are free to stick this new behaviour at any > URI of their choosing, which gives the server-developer far more freedom in > how > the behaviour is implemented. And it's also simple and c
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Ian, On 15/03/11 11:55, Ian Stuart wrote: > On 14/03/11 21:21, Richard Jones wrote: >> I think that a large part of what SWORD adds to AtomPub is how you talk >> about packaging. We're definitely not going to go down the route of >> specifying package formats or support levels outside of AtomPub's basic >> definitions. What I'd be particularly interested to know is whether you >> think that SWORD currently provides you with enough tools to talk >> /about/ packaging? >> >> We have, for example, had a couple of mentions of the desire to ONLY use >> mime-types, which we know is insufficient for describing packages - what >> is the mime type of a DSpace METS package? And I have also verified that >> the registration process for mime-types is not strictly trivial. Do the >> extra features for providing Package information during deposit, >> Accept-Package headers for retrieval, and sword:packaging elements in >> the service documents and atom entries provide you with a framework >> which will work for your requirements? > From various conversations I've had with people, the concept of > "packaging" is confusing for people. > > For many people, a "package" is a simple thing... and therefore can be > assigned some kind of identifier (hence the idea of hooking into MIME > Types) > > For others, me included, a "Package" is a concept that is composed of > multiple parts: > 1) There is the singular Thing that holds everything (liken this to a > cardboard box) > 2) Inside the Thing, there is a file of descriptive data [metadata] and > some number of binary files (liken this to a Manilla file and some > data-disks in the box) > 3) There may also be a Manifest file that describes what the contents of > the box are, and how they relate to each other. It is this definition of Package that we need to be able to support within SWORD. Could the problem be down to the name (as you perhaps hint at here). Would we be better changing the names of "Package" related terms to something like "FormatProfile" or something? > In this second, expanded, view there are three things one need to define > within the discussion > > 1) What the singular Thing is: a (zip|xml|csv|xyz) file > 2) What "standard" the manifest file is written to (METS, EPrintsXML, > etc) > 3) What "standard" the metadata file is written to (DC, QDC, MODS, > EPrintsXML, BibTek, etc...) > > Now, one could define a new identifier for each combination of manifest > & metadata, or one can describe them separately > One has great flexibility & expandability, the other uses fewer header > fields. I think at the moment we're going for a URI which describes the combination. This is partly because /at the moment/ repositories tend to support a finite set of Packages which consist of their favourite metadata formats and their favourite structural manifests. So not all possible permutations of structural and descriptive data is currently likely. This may, of course, change. Thoughts? Cheers, Richard -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Richard, Apologies again, please bear in mind these comments are based on a partial reading of the doc, will hopefully have more time to read in full soon. More inline... On Mon, Mar 14, 2011 at 06:08:56PM +, Richard Jones wrote: > >I notice in the current spec you overload the 'edit-media' relation to > >define new protocol behaviour when POSTing binary content to a deposit's > >edit-media URI. > > I'm not sure what you mean, sorry. We haven't specified any > behaviour for POST on the edit-media URI (EM-URI in the document). > We do a PUT on that URI which I think is consistent with the AtomPub > approach (replace the old file with the new file). We do do a POST > on the edit URI (Edit-URI in the document) which isn't covered by > AtomPub at all, but is consistent with a RESTful approach. Is it > that that bothers you? Sorry, I must've mis-read first time. Now I'm reading section 6.6.1 "Adding New Packages or Files to a Container"... ...OK, yes I think I would object to using the edit-URI for this purpose. Well, actually, no, I don't care what URI you use, what I would object to is specifying the use of the 'edit' link relation to indicate which URI can be used for this purpose. I'm just trying to come at this from a REST/hypermedia point of view. What this spec effectively does is to change the meaning of the 'edit' link relation, or in this case to "overload" it by giving it an additional meaning over what IANA (via the AtomPub spec) has to say about it. >From a practical point of view, as a client developer, how do I know when the 'edit' link means only what it says in the IANA registry, and when the 'edit' link also means what it says in the SWORD spec? E.g., say my client receives an Atom entry document. The client finds an 'edit' link, and because it's programmed with knowledge of atompub, it then knows what URI to use to update or delete the entry. But how does my client additionally know that the URI at the end of the 'edit' link also supports adding new packages or files to a container? Should it look for other sword: foreign markup in the entry document, then modify its understanding of the 'edit' link relation based on the presence of this markup? Or should it just assume that the 'edit' link means this in all cases, and just attempt the operation (which may fail if it was actually just talking to a plain old atompub server)? Instead of specifying what operations the Edit-URI should support, which is effectively redefining/overloading the meaning of the 'edit' link relation, I would suggest that the SWORD specification simply define a new link relation, e.g.: http://purl.org/net/sword/rel/append-package-contents Client developers would then know to use this link relation when looking for the URI to use to POST additional package contents too. And server developers would be free to implement this behaviour using whatever URIs they choose - they could do it with the edit-URI, or they could do it with a completely different URI, it's up to them, and it doesn't matter for the client because the client is hypermedia-driven. To labour this point a bit more, by specifying that the edit-URI has to be used for this purpose, you are placing unnecessary constraints on how servers are implemented. I.e., as a server implementer, I'm now forced to extend/override/modify my existing server-side code for handling requests to edit-URIs to implement this behaviour. If instead you just define a new link relation, then server developers are free to stick this new behaviour at any URI of their choosing, which gives the server-developer far more freedom in how the behaviour is implemented. And it's also simple and crystal-clear for client developers how a client decides which URIs to use for what protocol behaviour. Similarly for any other new protocol operations you want to specify, I would suggest defining new link relations, rather than specifying what URIs should be used or changing the meaning of any existing link relations like 'edit' or 'edit-media'. So, e.g., for "Adding New Metadata to a Resource", rather than specifying that the edit-URI should be used, the spec could define a new link relation like: http://purl.org/net/sword/rel/append-metadata Similarly for "Adding New Metadata and Packages or Files to a Resource with Multipart". Am I making sense? Cheers, Alistair [1] http://swordapp.org/wp-content/uploads/2011/03/SWORDProfile.html#protocoloperations_addingcontent_binary > > >I wonder if an alternative pattern might give you a bit more flexibility, > >and be a bit easier to implement, without having to define any new protocol > >behaviour. > > > >Let's assume you've done a GET on the service document URI and chosen a > >collection URI to work with already. > > > >As described in the spec, you then create a new deposit by POSTing some > >packaged binary content to the collection URI. In return you get a 201 with > >an Atom entry document. > > > >The returned Atom ent
Re: [Sword-TAP] An example Implementation (VIDEO)
On 14/03/11 21:21, Richard Jones wrote: > I think that a large part of what SWORD adds to AtomPub is how you talk > about packaging. We're definitely not going to go down the route of > specifying package formats or support levels outside of AtomPub's basic > definitions. What I'd be particularly interested to know is whether you > think that SWORD currently provides you with enough tools to talk > /about/ packaging? > > We have, for example, had a couple of mentions of the desire to ONLY use > mime-types, which we know is insufficient for describing packages - what > is the mime type of a DSpace METS package? And I have also verified that > the registration process for mime-types is not strictly trivial. Do the > extra features for providing Package information during deposit, > Accept-Package headers for retrieval, and sword:packaging elements in > the service documents and atom entries provide you with a framework > which will work for your requirements? From various conversations I've had with people, the concept of "packaging" is confusing for people. For many people, a "package" is a simple thing... and therefore can be assigned some kind of identifier (hence the idea of hooking into MIME Types) For others, me included, a "Package" is a concept that is composed of multiple parts: 1) There is the singular Thing that holds everything (liken this to a cardboard box) 2) Inside the Thing, there is a file of descriptive data [metadata] and some number of binary files (liken this to a Manilla file and some data-disks in the box) 3) There may also be a Manifest file that describes what the contents of the box are, and how they relate to each other. In this second, expanded, view there are three things one need to define within the discussion 1) What the singular Thing is: a (zip|xml|csv|xyz) file 2) What "standard" the manifest file is written to (METS, EPrintsXML, etc) 3) What "standard" the metadata file is written to (DC, QDC, MODS, EPrintsXML, BibTek, etc...) Now, one could define a new identifier for each combination of manifest & metadata, or one can describe them separately One has great flexibility & expandability, the other uses fewer header fields. -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Folks, Please keep the packaging discussions on-list if you prefer, or you can move the discussion to the main sword technical list where practitioners might have some input for you. https://lists.sourceforge.net/lists/listinfo/sword-app-tech SWORD has to support your packaging needs, so it will be good to keep up to date on them. Cheers, Richard On 22/02/11 16:22, David Tarrant wrote: > Could that conversation be taken off list please. I still feel that the > packaging conversation is clouding some of the more basic needs of SWORDv2. > > I'll spawn another thread on that and the Dev8D conversations when I have > time. > > Cheers > > Dave T > > > > On 22 Feb 2011, at 16:15, Ian Stuart wrote: > >> On 22/02/11 16:02, Robert D. Sanderson wrote: >>> >>> I'm happy to discuss ORE if that's of interest. >> Rob, >> >> You'll need to explain it to me first and I'm thinking I need a >> handle on Linked Data before you can get very far with that ;-) >> >> >> -- >> >> Ian Stuart. >> Developer: Open Access Repository Junction and OpenDepot.org >> Bibliographics and Multimedia Service Delivery team, >> EDINA, >> The University of Edinburgh. >> >> http://edina.ac.uk/ >> >> This email was sent via the University of Edinburgh. >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> >> -- >> Index, Search& Analyze Logs and other IT data in Real-Time with Splunk >> Collect, index and harness all the fast moving IT data generated by your >> applications, servers and devices whether physical, virtual or in the cloud. >> Deliver compliance at lower cost and gain new business insights. >> Free Software Download: http://p.sf.net/sfu/splunk-dev2dev >> ___ >> Sword-app-techadvisorypanel mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel > > -- > Index, Search& Analyze Logs and other IT data in Real-Time with Splunk > Collect, index and harness all the fast moving IT data generated by your > applications, servers and devices whether physical, virtual or in the cloud. > Deliver compliance at lower cost and gain new business insights. > Free Software Download: http://p.sf.net/sfu/splunk-dev2dev > ___ > Sword-app-techadvisorypanel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel > -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Ian, I think that a large part of what SWORD adds to AtomPub is how you talk about packaging. We're definitely not going to go down the route of specifying package formats or support levels outside of AtomPub's basic definitions. What I'd be particularly interested to know is whether you think that SWORD currently provides you with enough tools to talk /about/ packaging? We have, for example, had a couple of mentions of the desire to ONLY use mime-types, which we know is insufficient for describing packages - what is the mime type of a DSpace METS package? And I have also verified that the registration process for mime-types is not strictly trivial. Do the extra features for providing Package information during deposit, Accept-Package headers for retrieval, and sword:packaging elements in the service documents and atom entries provide you with a framework which will work for your requirements? Cheers, Richard On 18/02/11 14:28, Ian Stuart wrote: > On 18/02/11 12:52, David Tarrant wrote: >> >> On 18 Feb 2011, at 12:49, Ian Stuart wrote: >> >>> SWORD is a transport mechanism... we all understand that - but sword can >>> either be a bling truck >>> (http://farm1.static.flickr.com/120/307424194_ccb7df1246.jpg) or a >>> container >>> (http://www.shipping-container-modification.com/images/standard_large.jpg) >> >> and both are really expensive!!! > The Shipping container is an un-patented design open access, if you > like ;-) > >> I think that, if you want to upload a package, don't expect to be able >> to edit parts of the package. >> >> I feel if you want to edit using packages, then you delete the previous >> version and upload a whole new package. >> Supporting more features than that is going to be very hard in the >> packaging scenario. > The question still comes down to at some point, the client and the > server need to exchange "metadata" and "files" - how is this negotiated > and how is the data transferred? > > Even in your video, you are transferring a small set of metadata items > from client to server. In your example, you control both ends of the > link, so you know the fields to transfer. What happens when the objects > get more complex, and you don't control both ends? > > Copying binary files is (relatively) easy copying sets of metadata > from one system to another requires "packaging" > > "Packaging" is the area I'm interested in. > > -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Tim, >> If the server supported adding further media resource to the package, you >> could advertise this by including an app:collection element within the feed >> document, as per [2]. I.e., clients would do a POST to the package-contents >> collection URI, as per standard Atom protocol for creating media resources >> in a collection. >> >> If you wanted to delete any media resources within the package, then each >> media resource would have it's own URI which you could send a DELETE to. If >> you wanted to delete the whole package, you could still do a delete on the >> deposit's edit-media URI as specified in the current SWORD protocol spec. > [snip] > > Your comment is along the same lines as what I have suggested (but > probably not as clearly). I'm pushing this as an alternative to > packaging though. My feeling is it is simpler to spec (hence implement) > multiple URIs to represent a package than it is to agree to a metadata > format and file structure inside an archive. In essence I think we're all arguing the same point from different starting points. The purpose of the Statement has always been to supply to the client a list of the resources on the server, so that then you have URIs on which you can do operations. But what we haven't done is made this suitably clear in the documentation/discussion/spec. I am, therefore, going to have a go at clarifying this in the next version of the draft (which will be in a week or so, once I've had time to assimilate feedback). My worry originally was that doing this explicitly would be too large a leap from SWORD 1.0 to SWORD 2.0, but it might be that we would be better off doing it now rather than waiting. > Ian Stuart's concern is the > overhead involved in the multiple HTTP requests that would be required > to build up the complex object. (Not that this precludes the > fire-and-forget ZIP of all your files approach) We definitely have to support packages and packaging. Ian's objection stands, but further to that packaging is well entrenched in our sector and we all know that it's hard to change the way that people work, and that we'll be more successful if we go to meet them rather than forcing them to come and meet us. > This doesn't escape having to agree a data model for what we're moving > around though. You have specced a top-level entry containing one file > per sub-entry. Experience would indicate you need to have another layer > of abstraction to support HTML-formatted documents (one or more HTML > files + inline content). This is actually my main concern regarding GData. Its idiom is a filesystem with folders and files, which is a widely used idiom but not something that I want to bake into SWORD. When I write something in the specification about being able to do GET, PUT, and DELETE on URIs supplied in the Statement, I'll simply indicate that the Statement can be part of any larger RDF or Atom document (such as a GData folder feed), and leave it to implementations to interpret. Hopefully we don't need to add any more semantics to handle this. Cheers, Richard -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Alistair, Sorry about the long delay in responding. It was pretty frantic trying to get a draft of the spec ready for review! Now going back over the discussions that have been on this list and looking for things to use to enhance it. Responses inline ... > I just had a skim of the SWORDProfile spec in SVN [1], I'm guessing this is > the one you're working on? Yes. The HTML version in SVN is now the principal copy, and I will sync this to the swordapp website as and when changes come along. > I had a couple of thoughts, this is the first time I've seen the proposals > for SWORD v2, and haven't been involved in any previous discussions, so > apologies if you've been through this already. Also apologies if I missed > something in the spec, I didn't go through it yet with a fine comb. > > I notice in the current spec you overload the 'edit-media' relation to > define new protocol behaviour when POSTing binary content to a deposit's > edit-media URI. I'm not sure what you mean, sorry. We haven't specified any behaviour for POST on the edit-media URI (EM-URI in the document). We do a PUT on that URI which I think is consistent with the AtomPub approach (replace the old file with the new file). We do do a POST on the edit URI (Edit-URI in the document) which isn't covered by AtomPub at all, but is consistent with a RESTful approach. Is it that that bothers you? > I wonder if an alternative pattern might give you a bit more flexibility, > and be a bit easier to implement, without having to define any new protocol > behaviour. > > Let's assume you've done a GET on the service document URI and chosen a > collection URI to work with already. > > As described in the spec, you then create a new deposit by POSTing some > packaged binary content to the collection URI. In return you get a 201 with > an Atom entry document. > > The returned Atom entry document has an "edit" link and an "edit-media" link > as usual, and also a new link - let's call it "package-contents" for now, > what it's called doesn't really matter for this email, it's the pattern I'd > like to focus on. E.g.: > > http://www.w3.org/2005/Atom";> >al's first deposit >... > href="http://example.org/deposit/foo.atom"/> > href="http://example.org/deposit/foo.media"/> > href="http://example.org/unpacked/foo"/> >... > > > If you want to update (replace) the entire packaged content for this deposit, > you would do a PUT on the edit-media URI as per standard Atom protocol. > > If you want to update (replace) the metadata for this deposit, you would do > a PUT on the edia URI, again as per standard Atom protocol. > > Now, what's implicit here is that, as a side-effect of the initial POST > request to create the deposit, the server will have unpacked the packaged > content, and is now exposing the unpacked media resources as a standard Atom > collection, which is discoverable via the "package-contents" link. > > I.e., if you were to do a GET on the package-contents link URI, the server > would return an Atom feed listing all of the media resources within the > package. E.g.: > > http://www.w3.org/2005/Atom";> >... > href="http://example.org/unpacked/foo"/> > href="http://example.org/deposit/foo.atom"/> >... >http://example.org/unpacked/foo";> > package contents for al's first deposit > */* > >... > > ... > href="http://example.org/unpacked/foo/resource1.atom"/> > href="http://example.org/unpacked/foo/resource1.media"/> > ... > > > ... > href="http://example.org/unpacked/foo/resource2.atom"/> > href="http://example.org/unpacked/foo/resource2.media"/> > ... > >... > > > If the server supported adding further media resource to the package, you > could advertise this by including an app:collection element within the feed > document, as per [2]. I.e., clients would do a POST to the package-contents > collection URI, as per standard Atom protocol for creating media resources > in a collection. This sounds very similar to what could be achieved with the sword:statement. The statement URI appears in the atom:entry and allows you to content negotiate for either an OAI-ORE or Atom Feed document describing the contents of the resource. Further to that, in order to satisfy the desire to be consistent with GData, we have added a condition that allows the Edit-URI to be content negotiated for as an atom:feed rather than an atom:entry which would then list the collections and entries in that object. Would that meet your requirements as discussed here? We have explicitly avoided saying anything about GData semantics in the spec, but we do want to ensure that we don't impede any attempt to implement it alongside SWORD. I'd be interested in your opinion on the Atom Feed serialisation of the Statement as described at: http://swordapp.org/wp-content/uploads/2011/03/SWORDProfile.html#statement_atom and any enhance
Re: [Sword-TAP] An example Implementation (VIDEO)
Could that conversation be taken off list please. I still feel that the packaging conversation is clouding some of the more basic needs of SWORDv2. I'll spawn another thread on that and the Dev8D conversations when I have time. Cheers Dave T On 22 Feb 2011, at 16:15, Ian Stuart wrote: > On 22/02/11 16:02, Robert D. Sanderson wrote: >> >> I'm happy to discuss ORE if that's of interest. > Rob, > > You'll need to explain it to me first and I'm thinking I need a > handle on Linked Data before you can get very far with that ;-) > > > -- > > Ian Stuart. > Developer: Open Access Repository Junction and OpenDepot.org > Bibliographics and Multimedia Service Delivery team, > EDINA, > The University of Edinburgh. > > http://edina.ac.uk/ > > This email was sent via the University of Edinburgh. > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > > -- > Index, Search & Analyze Logs and other IT data in Real-Time with Splunk > Collect, index and harness all the fast moving IT data generated by your > applications, servers and devices whether physical, virtual or in the cloud. > Deliver compliance at lower cost and gain new business insights. > Free Software Download: http://p.sf.net/sfu/splunk-dev2dev > ___ > Sword-app-techadvisorypanel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel -- Index, Search & Analyze Logs and other IT data in Real-Time with Splunk Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. Free Software Download: http://p.sf.net/sfu/splunk-dev2dev ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
On 22/02/11 16:02, Robert D. Sanderson wrote: > > I'm happy to discuss ORE if that's of interest. Rob, You'll need to explain it to me first and I'm thinking I need a handle on Linked Data before you can get very far with that ;-) -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Index, Search & Analyze Logs and other IT data in Real-Time with Splunk Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. Free Software Download: http://p.sf.net/sfu/splunk-dev2dev ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
I'm happy to discuss ORE if that's of interest. Rob > On 21/02/11 22:27, Graham Klyne wrote: >> On the topic of packaging, I don't think this should be progressed in >> isolation. There's a community of interest in e-Research looking at >> "Research Objects", which share many high-level requirements with >> "packages" for deposition - roughly, composite objects + metadata, in a >> simple, easy-to-construct, easy-to-deconstruct container. > > Can you send me some contacts for this? > As part of the OA-RJ work, I will be producing a consistent method for > transfer between myself and a number of repository applications. > > -- > > Ian Stuart. > Developer: Open Access Repository Junction and OpenDepot.org > Bibliographics and Multimedia Service Delivery team, > EDINA, > The University of Edinburgh. > > http://edina.ac.uk/ > > This email was sent via the University of Edinburgh. > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > > -- > Index, Search & Analyze Logs and other IT data in Real-Time with Splunk > Collect, index and harness all the fast moving IT data generated by your > applications, servers and devices whether physical, virtual or in the > cloud. > Deliver compliance at lower cost and gain new business insights. > Free Software Download: http://p.sf.net/sfu/splunk-dev2dev > ___ > Sword-app-techadvisorypanel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel > -- Index, Search & Analyze Logs and other IT data in Real-Time with Splunk Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. Free Software Download: http://p.sf.net/sfu/splunk-dev2dev ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
On 21/02/11 22:27, Graham Klyne wrote: > On the topic of packaging, I don't think this should be progressed in > isolation. There's a community of interest in e-Research looking at > "Research Objects", which share many high-level requirements with > "packages" for deposition - roughly, composite objects + metadata, in a > simple, easy-to-construct, easy-to-deconstruct container. Can you send me some contacts for this? As part of the OA-RJ work, I will be producing a consistent method for transfer between myself and a number of repository applications. -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Index, Search & Analyze Logs and other IT data in Real-Time with Splunk Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. Free Software Download: http://p.sf.net/sfu/splunk-dev2dev ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
> On 18/02/11 12:52, David Tarrant wrote: >> I think that, if you want to upload a package, don't expect to be able >> to edit parts of the package. >> >> I feel if you want to edit using packages, then you delete the previous >> version and upload a whole new package. >> Supporting more features than that is going to be very hard in the >> packaging scenario. I think I agree with much of this. My feeling is that using multiple transactions to assemble and/or edit a package can become very complex. While some systems may wish to offer such capabilities, I don't think they're a useful basis for an spec whose primary goal is interoperability. I think this is a different issue from that which Alistair Miles was raising, which was, as far as I could tell, about the use of different link-rel types for different functions rarher than overloading an existing one. In my experience, overloading can over-constrain design choices, and often lead to tears, and is probably best avoided. On the topic of packaging, I don't think this should be progressed in isolation. There's a community of interest in e-Research looking at "Research Objects", which share many high-level requirements with "packages" for deposition - roughly, composite objects + metadata, in a simple, easy-to-construct, easy-to-deconstruct container. At this stage, I'd suggest that SWORD the spec should avoid specifying package formats, and that SWORD (the application) should keep its powder dry regarding specific packaging recommendations until some clearer community consensus emerges. (Some common themes I see in several places are: ZIP file container; ORE manifest; metadata in files within the container.) #g -- [1] Bechhofer, S., De Roure, D., Gamble, M., Goble, C. and Buchan, I. (2010) Research Objects: Towards Exchange and Reuse of Digital Knowledge. In: The Future of the Web for Collaborative Science (FWCS 2010), April 2010, Raleigh, NC, USA. [2] Bechhofer, S., Ainsworth, J., Bhagat, J., Buchan, I., Couch, P., Cruickshank, D., Delderfield, M., Dunlop, I., Gamble, M., Goble, C., Michaelides, D., Missier, P., Owen, S., Newman, D., De Roure, D. and Sufi, S. (2010) Why Linked Data is Not Enough for Scientists. In: Sixth IEEE e--Science conference (e-Science 2010), December 2010, Brisbane, Australia. Ian Stuart wrote: > On 18/02/11 12:52, David Tarrant wrote: >> On 18 Feb 2011, at 12:49, Ian Stuart wrote: >> >>> SWORD is a transport mechanism... we all understand that - but sword can >>> either be a bling truck >>> (http://farm1.static.flickr.com/120/307424194_ccb7df1246.jpg) or a >>> container >>> (http://www.shipping-container-modification.com/images/standard_large.jpg) >> and both are really expensive!!! > The Shipping container is an un-patented design open access, if you > like ;-) > >> I think that, if you want to upload a package, don't expect to be able >> to edit parts of the package. >> >> I feel if you want to edit using packages, then you delete the previous >> version and upload a whole new package. >> Supporting more features than that is going to be very hard in the >> packaging scenario. > The question still comes down to at some point, the client and the > server need to exchange "metadata" and "files" - how is this negotiated > and how is the data transferred? > > Even in your video, you are transferring a small set of metadata items > from client to server. In your example, you control both ends of the > link, so you know the fields to transfer. What happens when the objects > get more complex, and you don't control both ends? > > Copying binary files is (relatively) easy copying sets of metadata > from one system to another requires "packaging" > > "Packaging" is the area I'm interested in. > > -- Index, Search & Analyze Logs and other IT data in Real-Time with Splunk Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. Free Software Download: http://p.sf.net/sfu/splunk-dev2dev ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
On 18/02/11 12:52, David Tarrant wrote: > > On 18 Feb 2011, at 12:49, Ian Stuart wrote: > >> SWORD is a transport mechanism... we all understand that - but sword can >> either be a bling truck >> (http://farm1.static.flickr.com/120/307424194_ccb7df1246.jpg) or a >> container >> (http://www.shipping-container-modification.com/images/standard_large.jpg) > > and both are really expensive!!! The Shipping container is an un-patented design open access, if you like ;-) > I think that, if you want to upload a package, don't expect to be able > to edit parts of the package. > > I feel if you want to edit using packages, then you delete the previous > version and upload a whole new package. > Supporting more features than that is going to be very hard in the > packaging scenario. The question still comes down to at some point, the client and the server need to exchange "metadata" and "files" - how is this negotiated and how is the data transferred? Even in your video, you are transferring a small set of metadata items from client to server. In your example, you control both ends of the link, so you know the fields to transfer. What happens when the objects get more complex, and you don't control both ends? Copying binary files is (relatively) easy copying sets of metadata from one system to another requires "packaging" "Packaging" is the area I'm interested in. -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
On 18/02/11 12:05, Tim Brody wrote: > Your comment is along the same lines as what I have suggested (but > probably not as clearly). I'm pushing this as an alternative to > packaging though. My feeling is it is simpler to spec (hence implement) > multiple URIs to represent a package than it is to agree to a metadata > format and file structure inside an archive. Ian Stuart's concern is the > overhead involved in the multiple HTTP requests that would be required > to build up the complex object. (Not that this precludes the > fire-and-forget ZIP of all your files approach) I /think/ that several of us are talking across several cross purposes, but I'm not sure. I think there are three types of sword conversations to open the proceedings: 1) "Hello server, I'd like to deposit this" 2) "Hello server, tell me about item 12345" 3) "Hello server, please update item 12345 with this data" In the first situation, the server responds with "Thank you. For future reference, the item is here. You can also edit the item there" With the second scenario, the server responds with "I have it. You can see it here, and modify it there" The third scenario directs the server to modify immediately, and the server would respond with "Thank you, I have successfully done that. You can see it here, and modify it there" My concern is the number of times the server & client talk before a transfer completes... and I'm coming from the position that transfer is a common & frequent task: a multiple of items each day, and each item to a multiple of locations. For an occasional transfer, on a one-to-one basis, then a conversation is fine. Once the numbers start to climb, things need to become briefer & briefer. The thing that interests me, however, is not the client-server conversation (per sae), but what happens when the client actually gets to send a record to the server. I'm interested in how we transfer the information about the files, their relationship to each other as well as the record, and the records metadata. I'm interested in how we package that information up & send it over. I'm interested in how/where the conversation happens to work out what the package is or contains. SWORD is a transport mechanism... we all understand that - but sword can either be a bling truck (http://farm1.static.flickr.com/120/307424194_ccb7df1246.jpg) or a container (http://www.shipping-container-modification.com/images/standard_large.jpg) One is universal & useful. One is really pretty, and useless :D -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
On Thu, 2011-02-17 at 15:44 +, Alistair Miles wrote: > Hi Richard, > [snip] > If the server supported adding further media resource to the package, you > could advertise this by including an app:collection element within the feed > document, as per [2]. I.e., clients would do a POST to the package-contents > collection URI, as per standard Atom protocol for creating media resources > in a collection. > > If you wanted to delete any media resources within the package, then each > media resource would have it's own URI which you could send a DELETE to. If > you wanted to delete the whole package, you could still do a delete on the > deposit's edit-media URI as specified in the current SWORD protocol spec. [snip] Your comment is along the same lines as what I have suggested (but probably not as clearly). I'm pushing this as an alternative to packaging though. My feeling is it is simpler to spec (hence implement) multiple URIs to represent a package than it is to agree to a metadata format and file structure inside an archive. Ian Stuart's concern is the overhead involved in the multiple HTTP requests that would be required to build up the complex object. (Not that this precludes the fire-and-forget ZIP of all your files approach) This doesn't escape having to agree a data model for what we're moving around though. You have specced a top-level entry containing one file per sub-entry. Experience would indicate you need to have another layer of abstraction to support HTML-formatted documents (one or more HTML files + inline content). I feel like I'm doing a lot of hand-waving so don't want to chew over SWORD in any more depth until Richard has produced the draft. All the best, Tim. -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Richard, I just had a skim of the SWORDProfile spec in SVN [1], I'm guessing this is the one you're working on? I had a couple of thoughts, this is the first time I've seen the proposals for SWORD v2, and haven't been involved in any previous discussions, so apologies if you've been through this already. Also apologies if I missed something in the spec, I didn't go through it yet with a fine comb. I notice in the current spec you overload the 'edit-media' relation to define new protocol behaviour when POSTing binary content to a deposit's edit-media URI. I wonder if an alternative pattern might give you a bit more flexibility, and be a bit easier to implement, without having to define any new protocol behaviour. Let's assume you've done a GET on the service document URI and chosen a collection URI to work with already. As described in the spec, you then create a new deposit by POSTing some packaged binary content to the collection URI. In return you get a 201 with an Atom entry document. The returned Atom entry document has an "edit" link and an "edit-media" link as usual, and also a new link - let's call it "package-contents" for now, what it's called doesn't really matter for this email, it's the pattern I'd like to focus on. E.g.: http://www.w3.org/2005/Atom";> al's first deposit ... http://example.org/deposit/foo.atom"/> http://example.org/deposit/foo.media"/> http://example.org/unpacked/foo"/> ... If you want to update (replace) the entire packaged content for this deposit, you would do a PUT on the edit-media URI as per standard Atom protocol. If you want to update (replace) the metadata for this deposit, you would do a PUT on the edia URI, again as per standard Atom protocol. Now, what's implicit here is that, as a side-effect of the initial POST request to create the deposit, the server will have unpacked the packaged content, and is now exposing the unpacked media resources as a standard Atom collection, which is discoverable via the "package-contents" link. I.e., if you were to do a GET on the package-contents link URI, the server would return an Atom feed listing all of the media resources within the package. E.g.: http://www.w3.org/2005/Atom";> ... http://example.org/unpacked/foo"/> http://example.org/deposit/foo.atom"/> ... http://example.org/unpacked/foo";> package contents for al's first deposit */* ... ... http://example.org/unpacked/foo/resource1.atom"/> http://example.org/unpacked/foo/resource1.media"/> ... ... http://example.org/unpacked/foo/resource2.atom"/> http://example.org/unpacked/foo/resource2.media"/> ... ... If the server supported adding further media resource to the package, you could advertise this by including an app:collection element within the feed document, as per [2]. I.e., clients would do a POST to the package-contents collection URI, as per standard Atom protocol for creating media resources in a collection. If you wanted to delete any media resources within the package, then each media resource would have it's own URI which you could send a DELETE to. If you wanted to delete the whole package, you could still do a delete on the deposit's edit-media URI as specified in the current SWORD protocol spec. I just wonder if this pattern would give you more flexibility over managing the contents of a deposited package, whilst at the same time requiring less specification of non-standard Atom protocol behaviour. I.e., this pattern needs no new protocol behaviour to be specified, all it needs is that you define the "package-contents" link relation (or whatever you want to call it) as pointing to an Atom collection containing the unpacked deposit contents. It also might be easier to implement, because rather than servers having to extend their Atom protocol engine with new operations, now everything is standard Atom protocol, and all servers need to do is implement a side-effect of the initial deposit request to unpack the package and store it in a new collection. On a different point, I also notice in the current spec that you overload the 'edit' link relation to define new protocol behaviour when POSTing an Atom entry document to the edit URI. I'm guessing that what you want here is partial update? I'm just wondering how easy this would be to implement, I'm just thinking that it may not be obvious how any child elements should be handled, whether they should add or replace existing elements. Telling the difference may require the server to have knowledge of element cardinalities, which would be hard to guarantee in general where clients may use extension elements. I was just thinking if what you want to do is selectively update portions of the deposit's metadata record, then you might be better of using PATCH instead of POST, and then having a discussion around the media-type recommended for PATCH requests, and how they should be processed. Some of the issues with dealing
Re: [Sword-TAP] An example Implementation (VIDEO)
Hi Dave, This looks interesting, definitely chat about it this week. In the mean time, you could try pointing your client at the Simple SWORD Server: https://sword-app.svn.sourceforge.net/svnroot/sword-app/sss/trunk/ I'm very close now to a draft of the spec to be implemented against. I don't think anything there precludes the API being used as described in your previous email. I think the SWORD and GData approaches are compatible, but would be good to test properly. See you at dev8d! Cheers, Richard On 14/02/11 16:38, David Tarrant wrote: > All, > > Further to my actual playing with some early implementations, myself and Tim > Brody have today extended early ideas to be closer to the Google Docs > extensions and thus produced a client as a test of the implementation. > > The whole thing is best demonstrated by the video which is here: > > http://www.eprints.org/software/training/3.2/videos/swordv2_eprints.ogv > > Things we have working: > > CREATE : Initial create of the object > RETRIEVE: Obviously > UPDATE: 2 way update between both client and repository > DELETE: Again 2 way. > > Additionally, Metadata can be updated to the edit-uri (also demonstrated). > > All of this uses the Edit-URI, Atom:ID and Edit-Media URIs only + the > contents URIs for handling multiple documents. > > All the update requests are backed with HEAD requests to see if content has > changed and on which side (repo or local). > > Do take a look at the video and give feedback. I think this is close to a > full SIMPLE implementation of SWORDv2, e.g. without on behalf of and other > complex headers. > > The next thing to look into is how to move the object through the workflow. > > See people at Dev8D where I can do a live demo. > > Dave T > > > > -- > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > > > > ___ > Sword-app-techadvisorypanel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ Sword-app-techadvisorypanel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
