[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Ian Stuart ian.stu...@ed.ac.uk Date: 11 January 2011 03:04 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org We're looking at two things here, are we not? 1) we want the data returned in s specific media type (zip file, xml, json, atom+xml, etc) 2) we want the content of that data to be encoded in a particular way (METSDSpaceSIP, METSOARJ, ORE, RDFa, etc) I read your email as wanting to combine the two of them in one http header field. The alternative is to use pragma fields in which case, you can do what you like :D On 07/01/11 17:36, Richard Jones wrote: Hi Folks, I'd be really interested in people's input on the following problem that I've come across in creating the first draft of the spec. It's to do with how one can content negotiate with a server for a particular package format. Allowing the Media Resource URI to abstractly refer to the contents of the resource on the sword server (as per the business case/technical design document) means that in order to specify what you want to get back from the server when requesting that resource may require content negotiation. Content negotiation uses the HTTP Accept- headers, and the main Accept header itself allows you to list mimetypes and your preferences for receiving them, but package formats aren't represented by mimetypes (for the most part). There are two ways that we might go about content negotiating for a format (such as the SWORD example format of METSDSpaceSIP) that I can see, and I'd like to solicit feedback: 1/ Use the Accept-Encoding header in some way. This header allows you to do things like: Accept-Encoding: compress, gzip which seems to suggest that we could put in the package format like: Accept-Encoding: METSDSpaceSIP Does anyone have any experience with this header and could tell us whether this seems like a reasonable usage of it? 2/ Define an extension to the application/zip mimetype which allows us to specify the package format as a parameter. Parameters are used in mimetypes to further refine their definition, as in: application/atom+xml;type=entry This is a valid mimetype, and the Atom spec defines the parameter type with possible values entry and feed so that you can more accurately identify the content of the thing you are getting back. Content negotiation explicitly allows for the use of parameters (although some of the details are a little unclear with regard to wildcards). So we could, for example, specify a parameter swordpackage which can take the URI of a package format, and construct mimetypes like application/zip;swordpackage=uri:METSDSpaceSIP (see http://tools.ietf.org/html/draft-ietf-atompub-typeparam-00) The questions here are: is this a legitimate extension/approach, would this break anything else on the web in general, and is it naive to assume that all packages have the top level mimetype of application/zip? There has also been some discussion about the OASIS CMIS standard, and I wonder if anyone is familiar enough with it to tell us how that community handles this kind of issue (if at all?). Cheers, Richard -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: David Tarrant d...@ecs.soton.ac.uk Date: 11 January 2011 03:20 Subject: Re: content negotiating for package formats To: Ian Stuart ian.stu...@ed.ac.uk Cc: techadvisorypa...@swordapp.org I agree with Ian, why can we just use the existing x-packaging header, cos that's how point (2) works in the current sword? Dave T On 10 Jan 2011, at 14:04, Ian Stuart wrote: We're looking at two things here, are we not? 1) we want the data returned in s specific media type (zip file, xml, json, atom+xml, etc) 2) we want the content of that data to be encoded in a particular way (METSDSpaceSIP, METSOARJ, ORE, RDFa, etc) I read your email as wanting to combine the two of them in one http header field. The alternative is to use pragma fields in which case, you can do what you like :D On 07/01/11 17:36, Richard Jones wrote: Hi Folks, I'd be really interested in people's input on the following problem that I've come across in creating the first draft of the spec. It's to do with how one can content negotiate with a server for a particular package format. Allowing the Media Resource URI to abstractly refer to the contents of the resource on the sword server (as per the business case/technical design document) means that in order to specify what you want to get back from the server when requesting that resource may require content negotiation. Content negotiation uses the HTTP Accept- headers, and the main Accept header itself allows you to list mimetypes and your preferences for receiving them, but package formats aren't represented by mimetypes (for the most part). There are two ways that we might go about content negotiating for a format (such as the SWORD example format of METSDSpaceSIP) that I can see, and I'd like to solicit feedback: 1/ Use the Accept-Encoding header in some way. This header allows you to do things like: Accept-Encoding: compress, gzip which seems to suggest that we could put in the package format like: Accept-Encoding: METSDSpaceSIP Does anyone have any experience with this header and could tell us whether this seems like a reasonable usage of it? 2/ Define an extension to the application/zip mimetype which allows us to specify the package format as a parameter. Parameters are used in mimetypes to further refine their definition, as in: application/atom+xml;type=entry This is a valid mimetype, and the Atom spec defines the parameter type with possible values entry and feed so that you can more accurately identify the content of the thing you are getting back. Content negotiation explicitly allows for the use of parameters (although some of the details are a little unclear with regard to wildcards). So we could, for example, specify a parameter swordpackage which can take the URI of a package format, and construct mimetypes like application/zip;swordpackage=uri:METSDSpaceSIP (see http://tools.ietf.org/html/draft-ietf-atompub-typeparam-00) The questions here are: is this a legitimate extension/approach, would this break anything else on the web in general, and is it naive to assume that all packages have the top level mimetype of application/zip? There has also been some discussion about the OASIS CMIS standard, and I wonder if anyone is familiar enough with it to tell us how that community handles this kind of issue (if at all?). Cheers, Richard -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Scott Wilson scott.bradley.wil...@gmail.com Date: 11 January 2011 03:55 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org To answer the CMIS question - AFAIK CMIS doesn't explicitly support external packaging formats (in its scope it declares that compound and virtual objects are extended features); instead it directly uses Atom's collection handling. So a CMIS client would create the Folder object and then POST each enclosed item to it, rather than POST a zip file and rely on the repository to unpackage and store it as some sort of composite object. There is a line in the CMIS charter setting it as a secondary priority, so it may become part of CMIS in the future. Packaging is also used for supporting alternative renditions of a resource - and CMIS supports this explicitly - see renditions in the OASIS CMIS spec: http://docs.oasis-open.org/cmis/CMIS/v1.0/os/cmis-spec-v1.0.html#_Toc243905395 ... however support is currently limited to retrieving renditions, and the spec doesn't specify how to create a document with multiple renditions. This could be a good topic on which to link up with the OASIS CMIS TC. S On 10 Jan 2011, at 14:20, David Tarrant wrote: I agree with Ian, why can we just use the existing x-packaging header, cos that's how point (2) works in the current sword? Dave T On 10 Jan 2011, at 14:04, Ian Stuart wrote: We're looking at two things here, are we not? 1) we want the data returned in s specific media type (zip file, xml, json, atom+xml, etc) 2) we want the content of that data to be encoded in a particular way (METSDSpaceSIP, METSOARJ, ORE, RDFa, etc) I read your email as wanting to combine the two of them in one http header field. The alternative is to use pragma fields in which case, you can do what you like :D On 07/01/11 17:36, Richard Jones wrote: Hi Folks, I'd be really interested in people's input on the following problem that I've come across in creating the first draft of the spec. It's to do with how one can content negotiate with a server for a particular package format. Allowing the Media Resource URI to abstractly refer to the contents of the resource on the sword server (as per the business case/technical design document) means that in order to specify what you want to get back from the server when requesting that resource may require content negotiation. Content negotiation uses the HTTP Accept- headers, and the main Accept header itself allows you to list mimetypes and your preferences for receiving them, but package formats aren't represented by mimetypes (for the most part). There are two ways that we might go about content negotiating for a format (such as the SWORD example format of METSDSpaceSIP) that I can see, and I'd like to solicit feedback: 1/ Use the Accept-Encoding header in some way. This header allows you to do things like: Accept-Encoding: compress, gzip which seems to suggest that we could put in the package format like: Accept-Encoding: METSDSpaceSIP Does anyone have any experience with this header and could tell us whether this seems like a reasonable usage of it? 2/ Define an extension to the application/zip mimetype which allows us to specify the package format as a parameter. Parameters are used in mimetypes to further refine their definition, as in: application/atom+xml;type=entry This is a valid mimetype, and the Atom spec defines the parameter type with possible values entry and feed so that you can more accurately identify the content of the thing you are getting back. Content negotiation explicitly allows for the use of parameters (although some of the details are a little unclear with regard to wildcards). So we could, for example, specify a parameter swordpackage which can take the URI of a package format, and construct mimetypes like application/zip;swordpackage=uri:METSDSpaceSIP (see http://tools.ietf.org/html/draft-ietf-atompub-typeparam-00) The questions here are: is this a legitimate extension/approach, would this break anything else on the web in general, and is it naive to assume that all packages have the top level mimetype of application/zip? There has also been some discussion about the OASIS CMIS standard, and I wonder if anyone is familiar enough with it to tell us how that community handles this kind of issue (if at all?). Cheers, Richard -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Ian Stuart ian.stu...@ed.ac.uk Date: 11 January 2011 04:01 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org More specifically. the Open Access Repository Junction Discovery APIs use the Accept header to determine the content type of the returned data, and it would be sensible to remain consistent: Accept defines the mime-type (application/xml, text/plain, etc...) Pragma:X-Packaging then defines the package format for the content (METSDSpaceSIP, METSOARJ, ORE, RDFa, etc) I suggest Pragma:X-Packaging, as one currently uses an 'X-Packaging' element in the http request header object when *posting* a SWORD deposit to define the packaging type of the content. -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Richard Jones rich...@oneoverzero.com Date: 12 January 2011 09:52 Subject: Re: content negotiating for package formats To: Graham Klyne g...@ninebynine.org Cc: techadvisorypa...@swordapp.org Hi Graham, On 11/01/11 19:05, Graham Klyne wrote: Richard - a small thing: rather than using the X- header convention, just pick a suitable name and request provisional registration via IANA, per [1]. This avoids messing around if the new header goes standards track. And, anyway, X- headers don't have the same reserved for experimentation status that applies for email headers. [1] http://tools.ietf.org/html/rfc3864 That's an interesting idea. Shouldn't we do this with all SWORD headers, then, though? Probably this is a decision that we need to make as a project as to whether we go right away towards something which can be put onto a real standards track in the future, or whether we stick with X- headers. I'm in favour of being able to go standards track, but there are a few things which nag at me, including: 1/ the amount of time required to do the work to even to provisional registration of the headers 2/ backwards compatibility with SWORD 1.0. If we drop, say, X-On-Behalf-Of and go for just On-Behalf-Of we'd be breaking the back compat or placing the onus on the server to interpret both headers. What do you think? I'm also wondering about your combination of content-type and internal packaging format. The media feature description framework [2] was intended to capture this kind of combination of features in a more structured fashion. Thus, I would imagine something like: Accept-media-features: (| ( type=application/zip package=METSDSpaceSIP);q=1.0 ( type=application/atom+xml atomtype=entry package=AtomSIP);q=0.8 ) This would require IANA registration of the new header field, and a new media features called package and atomtype, per [4]. Feature type is already registered [3]. [2] http://tools.ietf.org/html/rfc2533 [3] http://tools.ietf.org/html/rfc2913 [4] http://tools.ietf.org/html/rfc2506 This seems like the proper way to do it, and just comes with the same caveats as above. What is the status of this RFC? Are we safe to go ahead and use it? (incidentally, we wouldn't need to register atomtype, as the param type=entry is part of the mimetype already). I'd be interested in the rest of the advisory panel's and project team's opinions on this, as this may well define the way that I work for the duration of the project. Cheers, Richard Richard Jones wrote: Hi Folks, Thanks, this is really great stuff. On 10/01/11 16:05, Robert D. Sanderson wrote: On 7 January 2011 17:36, Richard Jonesrich...@oneoverzero.com wrote: 2/ Define an extension to the application/zip mimetype which allows us to specify the package format as a parameter. So we could, for example, specify a parameter swordpackage which can take the URI of a package format, and construct mimetypes like application/zip;swordpackage=uri:METSDSpaceSIP (see http://tools.ietf.org/html/draft-ietf-atompub-typeparam-00) The questions here are: is this a legitimate extension/approach, would this break anything else on the web in general, and is it naive to assume that all packages have the top level mimetype of application/zip? First of all, no, it's not legitimate to invent new parameters for existing mime types. RFC 2048, Section 2.2.3 ... the names, values, and meanings of any parameters must be fully specified when a media type is registered in the IETF tree ... http://www.faqs.org/rfcs/rfc2048.html So it's not legal to create a parameter swordpackage and attach it to the existing application/zip. Ok, it sounds like this option then is simply out altogether. More generally, the HTTP specification defines the accept headers as: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html Note that the extension parameter here is for the header, not the mimetype. The BNF allows the accept-extension rule ONLY after the mandatory q value in accept-params. I have to confess to having overlooked the accept-extension rule, as there wasn't an example of usage in that document. Can you give us an example as to how that is used? Which means using Accept-Encoding in this way is potentially problematic, but Accept does have provision that would make this use legitimate. Like mime types, content encodings also have a registry. See: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html section 3.5 Basically, there are two routes to avoid breaking the rules, neither easy: 1. Register new Mime Types for every packaging format. 2. Use an x- header and eventually write an RFC to standardize it. We looked at this in both SRU (eg what it would take to have a wrapper format and an internal format: SRU vs Atom, wrapping Simple DC vs METS) and in conjunction with the digital format registry for preservation purposes. So my
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Ian Stuart ian.stu...@ed.ac.uk Date: 19 January 2011 01:11 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org On 10/01/11 18:49, Richard Jones wrote: It's looking like a separate header is the way to do this, with the following couple of options immediately standing out: Accept-Features (or X-Accept-Features if it isn't sufficiently official) X-Packaging X-Accept-Packaging (which I just made up for the purposes of this discussion) Some comments on these: Accept-Features Having looked at the document [1] (thanks Graham (K)) it looks like it would give us the leeway that we need to describe requirements while ensuring that Graham (T)'s concerns (which I share) about matching up package format requirements with mimetypes would be dealt with. On the other hand, this document is 12/13 years old and the header has not made it into the HTTP content negotiation documentation and is significantly different in format to all the other Accept- headers. It could also be a substantial effort for servers to implement the full requirements of this header. X-Packaging I'm against using this in this way as it is already used to alert the server during POST as to the package format that is being supplied. The format of the header for content negotiation would have to be totally different to this usage: a list of package formats and q values for example, rather than a single definitive URI. I see scope for confusion. X-Accept-Packaging Given my concerns about X-Packaging and the comments above about Accept-Feature, perhaps there is a middle ground that we can define which does something more minimal with just mimetypes, package formats and q values in a way similar to having a mimetype that has added parameters. For example: Accept: application/zip; q=1.0, application/atom+xml;type=entry;q=0.8 X-Accept-Packaging: application/zip;{package=METSDSpaceSIP};q=1.0, application/atom+xml;type=entry;{package=AtomSIP};q=0.8 Or some other suitably neat and unambiguous serialisation which is in line with how the other Accept- headers work and also gives us the information we want in a totally definitive mimetype-package format way. This could be supplied alongside the usual Accept header so that clients which can't generate the X-Accept-Packaging header can fall back easily to the usual content negotiation route. I'm still unclear why there is a need to combine the content type (application/zip; q=1.0) with the data encoding (METSDSpaceSIP; q=1.0) Can't you say (1) I only deal in .tgz content, and (2) you can package whatevers within that content as 'Foo', 'Bar', or even 'Acme::WhiteSpaceEncoded' -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Richard Jones rich...@oneoverzero.com Date: 19 January 2011 21:06 Subject: Re: content negotiating for package formats To: Ian Stuart ian.stu...@ed.ac.uk Cc: techadvisorypa...@swordapp.org Hi Ian, On 18/01/11 12:11, Ian Stuart wrote: On 10/01/11 18:49, Richard Jones wrote: It's looking like a separate header is the way to do this, with the following couple of options immediately standing out: Accept-Features (or X-Accept-Features if it isn't sufficiently official) X-Packaging X-Accept-Packaging (which I just made up for the purposes of this discussion) Some comments on these: Accept-Features Having looked at the document [1] (thanks Graham (K)) it looks like it would give us the leeway that we need to describe requirements while ensuring that Graham (T)'s concerns (which I share) about matching up package format requirements with mimetypes would be dealt with. On the other hand, this document is 12/13 years old and the header has not made it into the HTTP content negotiation documentation and is significantly different in format to all the other Accept- headers. It could also be a substantial effort for servers to implement the full requirements of this header. X-Packaging I'm against using this in this way as it is already used to alert the server during POST as to the package format that is being supplied. The format of the header for content negotiation would have to be totally different to this usage: a list of package formats and q values for example, rather than a single definitive URI. I see scope for confusion. X-Accept-Packaging Given my concerns about X-Packaging and the comments above about Accept-Feature, perhaps there is a middle ground that we can define which does something more minimal with just mimetypes, package formats and q values in a way similar to having a mimetype that has added parameters. For example: Accept: application/zip; q=1.0, application/atom+xml;type=entry;q=0.8 X-Accept-Packaging: application/zip;{package=METSDSpaceSIP};q=1.0, application/atom+xml;type=entry;{package=AtomSIP};q=0.8 Or some other suitably neat and unambiguous serialisation which is in line with how the other Accept- headers work and also gives us the information we want in a totally definitive mimetype-package format way. This could be supplied alongside the usual Accept header so that clients which can't generate the X-Accept-Packaging header can fall back easily to the usual content negotiation route. I'm still unclear why there is a need to combine the content type (application/zip; q=1.0) with the data encoding (METSDSpaceSIP; q=1.0) Can't you say (1) I only deal in .tgz content, and (2) you can package whatevers within that content as 'Foo', 'Bar', or even 'Acme::WhiteSpaceEncoded' I think that the problem is that you can't guarantee that the list of content types and the list of packaging types are combinable in a meaningful way; Graham T's email had an example. So suppose a server can give you content type A with packaging formats X and Y, or content type B with packaging format Z: A + (X or Y) B + Z and your content negotiation header says: Accept: A; q=1.0, B; q=0.8 Accept-Packaging: Z; q=1.0, X; q=1.0 Which combination do you return? On the other hand, this is a general problem and even within the Media Feature syntax that Graham K describes in his RFC acknowledges this effectively limits the use of q values to top-level feature sets. So, you would be limited to content negotiating for: Accept-Media-Feature: A(X), B(Z), A(Y) for example; i.e. explicitly declaring your preference of the combination of content-type and packaging format. I've spent the last 3 or 4 days looking at the Media Feature stuff in detail, and I have to confess it does feel like a sledgehammer to crack a nut. At the moment I'm playing with specifying restricted version of it to see if we can get the effect that we want without the huge overhead of a full implementation. As a consequence, I'm still open to Ian's suggested approach here, provided that we can decide a) what the new HTTP header should be called, and b) what the rules for resolving content negotiation ambiguities as shown above should be. Cheers, Richard -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Ian Stuart ian.stu...@ed.ac.uk Date: 19 January 2011 21:44 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org On 19/01/11 08:06, Richard Jones wrote: and your content negotiation header says: Accept: A; q=1.0, B; q=0.8 Accept-Packaging: Z; q=1.0, X; q=1.0 Which combination do you return? I agree its not a clear-cut case. however it also has to be said that the content negotiation header isn't helping by declaring two options to be of equal value I would say: A+Z, then A+X, then B+Z, then B+X FIFO-style (after all, if you have a preference for order, try: Accept-Packaging: X; q=1.0, Z; q=1.0 instead) Even under the concept of a combined Accept model, what do you do when you receive Accept-Media-Feature: A(Z); q=1.0, A(X); q=1.0, B(Z); q=0.8, B(X); q=0.8 -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Ian Stuart ian.stu...@ed.ac.uk Date: 19 January 2011 23:33 Subject: Re: content negotiating for package formats To: Scott Wilson scott.bradley.wil...@gmail.com Cc: techadvisorypa...@swordapp.org On 19/01/11 10:16, Scott Wilson wrote: I have an excellent content package that will either work with binary data directly included or passed-by-reference, and I am working on Importers for DSpace EPrints as we speak... as outputs of the OA-RJ Broker work. I suggest we standardise on a really crappy packaging format with almost zero features, combined with a totally inadequate metadata schema. At least that way it might actually work.* Oh, so you've seen my work then ;-) -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Robert D. Sanderson rsander...@lanl.gov Date: 20 January 2011 04:53 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org I don't think you can ever get away from a degree of content negotiation, but it doesn't necessarily need to be as complex as the scenarios outlined depending on what agreements you can have for common formats in common cases. I agree with Graham and Scott. Please can we step back and clearly and completely define the scope of the issue. My understanding is as follows: There are two dimensions of content type -- packaging (eg zip, bagit, tar, mets,...) and metadata (dc, mets, ore, ead, ...) and it is not feasible to specify all of the combinations as unique keys. These combinations are used to both deposit and retrieve content packages. Client to Server (Deposit): * There is a recommended packaging format (Zip + DC in Atom) * There is a header to inform the server which packaging format is actually being sent, as well as the metadata format * There is information available as to the types of format the server will accept (in the Svc Desc) Server to Client (Retrieve): * There SHOULD be the same recommended packaging format. If a client can construct it, then it can likely read it back again. Requiring a different format just doubles the work. * There is a header to inform the client which packaging format is being sent, as well as the metadata format * There is information available as to the types of format the server can send (in Svc Desc) which may not be the same as the set of formats it will accept for deposit. * There SHOULD be a header to request a particular format. Now, call me a heretic, but I think Packaging is the wrong way round. The outermost layer is the top level content type, and hence should be in the Accept and Content-Type headers. If you download a zip file containing 5 plain text files, Content-Type is application/zip. So what is really needed is Accept-Metadata and Content-Metadata, to request the format of the metadata in the response package. If the server can't deliver the combination, then it won't do that. The same way that if you ask for Accept-Language: fr and the server can't deliver French then it won't. I disagree with Ed that there MUST be only a single format. That's never going to fly with current workflows, and would result in the interoperability of OAI-PMH's use of simple DC ... to wit, that there is syntactic interoperability, but the semantics are completely worthless due to people stuffing random content into fields because they can't say what they really mean. The Linked Data effort has shown us that even in an open world of infinite vocabularies, people *will* self-organize into support of useful relationships, and the same should apply here. Rob -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Richard Jones rich...@oneoverzero.com Date: 20 January 2011 06:26 Subject: Re: content negotiating for package formats To: Ed Summers e...@pobox.com Cc: techadvisorypa...@swordapp.org Hi Ed, On 19/01/11 13:27, Ed Summers wrote: On Wed, Jan 19, 2011 at 6:28 AM, Richard Jonesrich...@oneoverzero.com wrote: We've had a few discussions in the past about supporting some formats, and they always end up pretty divisive. So SWORD is aiming to be totally agnostic on the point, but it does need to provide the client and server a mechanism to negotiate over what format they are interchanging. If we can achieve that, that will be relatively useful in an interoperability setting, I think, particularly as many SWORD servers (particularly repositories) are able to create dissemination packages in a large number of formats (see EPrints export plugins for example). Would it be too restrictive to require SWORD collections to only support one package format? This would mean that there MUST be one and only one sword:acceptPackaging per app:collection in the service document. I think this would simplify matters significantly for implementors since: 1) there would no longer be any need for the q attribute on sword:acceptPackaging, and the requirement to interpret them. 2) X-Packaging header registration, and the need for clients to send it would go away 2) a client could only retrieve a package in the format it was deposited in. Does anyone really have an appetite for dynamically rewriting package formats as part of an HTTP request cycle? I think that would be overly restrictive, while the above gains are relatively minimal in the scheme of things. Also, as per my earlier comment about export plugins in EPrints, you could easily imagine throwing a known package format into the repository, and then asking it to give you it back in a variety of formats that you don't know how to generate yourself. I guess a better question is: do we have many SWORD implementations that support POSTing multiple package flavors to the same collection? Also, I am -1 on SWORD requiring a standard package format. I think it would be fine to list some preferences, and light-weight, community driven mechanisms for identifying them, but that's as far as SWORD should go IMHO. It's been a long standing complaint against SWORD that despite it being an interoperability standard, you can't even deposit the same package into DSpace, EPrints and Fedora, let alone other implementations that weren't funded as part of the original project. From both a practical point of view and a community perception point of view this has to be addressed. We have tried to work around the standard package format issue by adopting an Atom Multipart [1] deposit of an Atom Entry with optional embedded metadata plus a binary payload which may either be a single file or if given the Content-Type of application/zip a plain old zip file with no prescribed internal content. This ought to be satisfactory because it is not only about the most simple format you could come up with, but it also leverages the existing semantics of AtomPub, so adding it is of minimal effort. http://tools.ietf.org/html/draft-gregorio-atompub-multipart-04 Cheers, Richard -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Ian Stuart ian.stu...@ed.ac.uk Date: 20 January 2011 22:00 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org On 19/01/11 17:26, Richard Jones wrote: Also, as per my earlier comment about export plugins in EPrints, you could easily imagine throwing a known package format into the repository, and then asking it to give you it back in a variety of formats that you don't know how to generate yourself. H.. interesting! I see a future development for the OA-RJ broker there!! It's been a long standing complaint against SWORD that despite it being an interoperability standard, you can't even deposit the same package into DSpace, EPrints and Fedora, let alone other implementations that weren't funded as part of the original project. From both a practical point of view and a community perception point of view this has to be addressed. This is an issue that the OA-RJ Project is addressing with the Broker work. Our initial work is to produce a single importable package, and the importers that go with them, so that the Broker can pass on an Item to a number of target repositories. (There is a future development idea which would be to allow each target repository to identify the package format it wanted, and the Broker would transfer in that package but that is a *much* slower mechanism when dealing with multi-institutional papers) -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel
[Sword-TAP] Fwd: content negotiating for package formats
-- Forwarded message -- From: Ian Stuart ian.stu...@ed.ac.uk Date: 21 January 2011 21:46 Subject: Re: content negotiating for package formats To: techadvisorypa...@swordapp.org On 20/01/11 18:11, Julie Allinson wrote: I might be talking nonsense here, but is this something that could support 'graceful' behaviour ... one thing I noticed in testing was that EPrints will accept a METS package with epdcx but the deposit fails if there is any other metadata instead of or in addition to the epdcx embedded in the METS doc. I'm not criticising EPrints or advocating METS but it struck me that if there is a package that could be deposited knowing that it will succeed and that you could stuff all kinds of things into it which the repository will either know what to do with and do that (unpack etc.) or simply accept and store? ... so for the EPrints case, you might need a new export plug, but in the meantime you could still be making deposits. EPrints does, indeed, expect epcdx in the xmlData section of the METS data, however I have also found that EPints is much more relaxed about the structure of METS/epcdx that DPspace is I've yet to find a Fedora volunteer to try imports with ;-) It is certainly possible to write an Importer for EPrints that will accept whatever format you care to specify... and my experience is that this is easier in EPrints than DSpace (but then again, I'm a Perl Monkey :chuckle: ) Whilst on the subject of epcdx: I am swinging away from it now - there are just so many things it doesn't do well, or misses out all together. Perhaps this is an opportunity to get people from LT, Data, Article, and various other data-store types together, and try to come up with an extensible core schema that can be both cross-platform as well as cross-type? -- Ian Stuart. Developer: Open Access Repository Junction and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh. http://edina.ac.uk/ This email was sent via the University of Edinburgh. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Sword-app-techadvisorypanel mailing list Sword-app-techadvisorypanel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sword-app-techadvisorypanel