RE: subproject URI naming convention
See inline. From: Nick Chalko [mailto:[EMAIL PROTECTED] Tim Anderson wrote: Can you provide an example of a URI which can't be parsed? -Tim [1] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax *repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier* It defines *access-specifier* and *product-specifier*, but leaves *version-specifier* and *artifact-specifier* opaque, to be defined by language, platform, or artifact-specific best practices. Since version-specifier and artifact-specifier are opaque, there is no way to tell where product specifier ends. I know we have suggested version, and a Java artifact specifier. But what about other languages, Like the new cool O/S language foo. It's artifact's are called bars http://repo.com/org/foo/cat/dog/bars/bar.zip What is the product org.foo.cat or org .foo? Is cat the version name or is dog.? Perhaps there are two kings of bars, one for dogs and one for eggs. or what ever. If product-specifier is opaque, and the artifact URI doesn't meet the criteria specified by one or more of the proposals, then a tool can't look at its URI to determine what the product or version is. Does that really matter though? How can a tool sensibly interrogate an artifact it doesn't understand? organization/project is a workable solution that lets a tool make intelligent guesses based on URI only,. I like the simplicity of Top level = Organization that distribute things 2nd level = A project. (a sub organizational unit that distributes artifacts) 3/4 level = Version, (interim builds take an extra level 4/5 = Artifacts stored any what a project likes. (with best practices for Java and other languages.) The ONLY limits we have on organization and project and version is it must be valid URI character and it can not be a / (ie pchar) I'm not really fussed if product-specifer is opaque or not - I'll go with the concensus view. IMO, the repository layout is cleaner if it is opaque, and tools can still parse any URI which meet the criteria of the proposals. If product-specifier is restricted to 2 path segments, then tools can parse any URI, but the repository structure is flatter, and can't represent project heirarchies. -Tim
RE: subproject URI naming convention
To summarise, there are three possible ways to encode subprojects in URIs: 1. Status quo repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier product-specifier = organisation / project . recommend that organisation is the reverse FQDN . for subprojects, project is the concatenation of project and subproject names . tools can't determine project and suproject names from URI E.g: http://repo.apache.org/jakarta.apache.org/commons-lang 2. Introduce mandatory subproject path i.e, change product-specifier: product-specifier = organisation / project / subproject . recommend that organisation is the reverse FQDN . no need to concatenate project and subproject names . doesn't support subprojects nesting 1 . redundant directory for projects with no subprojects . tools can determine project and suproject names from URI E.g: http://repo.apache.org/org.apache.jakarta/commons/lang http://repo.apache.org/org.apache.xml/batik/batik 3. Change product-specifier so that it is opaque repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier product-specifier = path_segments . recommend that product-specifier contains: . reverse FQDN . project name . subproject name(s) . can scale to arbitrary levels of subprojects . tools must parse URIs right to left, in order to separate version-specifier and product-specifier . tools must derive organisation, project, and subproject information from meta-data E.g: http://repo.apache.org/org/apache/jakarta/commons/lang http://repo.apache.org/org/apache/xml/batik I'm beginning to prefer option 3. -Tim
RE: subproject URI naming convention
From: Nick Chalko [mailto:[EMAIL PROTECTED] Tim Anderson wrote: The fact that commons-lang and commons-io are both part of the same Jakarta Project has no meaning to a repository. True, but users browsing the repository can find them easier if they are grouped together. The only difference between commons/lang and commons-lang is the number of items in a directory. but again if we allow arbitrary number of / before the the artifact part how can we tell what the project is we are back to http://repo.com/alpha/beta/alpha/beta/dist/beta-alpha.zip http://repo.com/dist/nightly/dist/dist/dist/dist/foo.zip Silly examples but with out a RIGID spec it will happen. Someone will want to name thier project Alpha, or nightly or the orginaztion will be named dist or intrim or snapshot. Lets just pick a number of groupings one or two or three and stick with it. Allow the / to have special meaning. R, Nick The distinction between organisation and project would no longer exist: repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier product-specifier = path_segments i.e, the organisation, project, subproject etc, are encoded in the URI using 1-n path segments. To ensure that reserved words aren't included in product-specifier, it would need to be specified as: product-specifier = path_segments ~reserved reserved = formal-build-designation | interim-build-designation | latest formal-build-designation = release interim-build-designation = interim | nightly | snapshot | ... This means: . tools cannot parse organisation, project etc details from the URI . tools can extract product-specifier, version-specifier, and artifact-specifier by parsing right to left. Some examples, using valid URIs: 1. http://repo.com/alpha/beta/alpha/1.0/binaries/beta-alpha.zip[1] artifact-specifier = binaries/beta-alpha.zip version-specifier = 1.0 product-specifier = beta/alpha 2. http://repo.com/dist/dist/dist/dist/nightly/1.0/20031202/binaries/foo.zip artifact-specifier = binaries/foo.zip version-specifier = nightly/1.0/20031202 product-specifier = dist/dist/dist/dist Your examples aren't valid: . http://repo.com/alpha/beta/alpha/beta/dist/beta-alpha.zip . dist/beta-alpha.zip isn't valid according to [2]-[5] . version-specifier *could* be beta according to [6] . product-specifier *could* be alpha/beta/alpha . http://repo.com/dist/nightly/dist/dist/dist/dist/foo.zip . dist/foo.zip isn't valid according to [2]-[5] . version-specifier *could* be dist according to [6] . product-specifier *could* be dist/nightly/dist/dist, but would be invalid given: product-specifier = path_segments ~reserved -Tim [1] thinking of dropping -bin suffix for binaries. -src suffix for sources would be retained. [2] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonDistributio nArtifactSpecifier [3] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/JavaArtifacts [4] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/SignatureArtifact Specifier [5] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/LicenseArtifactSp ecifier [6] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier
RE: subproject URI naming convention
From: Tim Anderson [mailto:[EMAIL PROTECTED] Sent: Friday, 5 December 2003 8:35 AM Damn - forgot version in artifact name... [snip] Some examples, using valid URIs: 1. http://repo.com/alpha/beta/alpha/1.0/binaries/beta-alpha-1.0.zip[1] artifact-specifier = binaries/beta-alpha-1.0.zip version-specifier = 1.0 product-specifier = beta/alpha 2. http://repo.com/dist/dist/dist/dist/nightly/1.0/20031202/binaries/foo-1.0.zi p artifact-specifier = binaries/foo-1.0.zip version-specifier = nightly/1.0/20031202 product-specifier = dist/dist/dist/dist -Tim [1] thinking of dropping -bin suffix for binaries. -src suffix for sources would be retained.
RE: licensing issues for virtual artifacts (was RE: click through license support?)
From: Noel J. Bergman [mailto:[EMAIL PROTECTED] Sent: Wednesday, 26 November 2003 1:34 PM Nicola Ken Barozzi wrote [EMAIL PROTECTED] wrote: I'm failing to see the requirement for us to do [virtual artifacts] *now*. Because Apache projects using the repository would need also non-asf jars that we don't want to distribute - virtual artifacts. I still maintain that non-ASF jars are specified in meta-data made available to client tools, and thus virtual artifacts are unnecessary. The meta-data files will need to be present repository in order for the tools to work. --- Noel The idea behind virtual artifacts is that they allow users and tools to determine if an artifact is available or not. Given the artifact: http://repo.apache.org/sun/jndi/1.2.1/jars/jndi-1.2.1.jar A user browsing the repository on seeing jndi-1.2.1.jar can assume that a tool will be able to download it, regardless of whether the repository hosts it or not. Without the virtual artifact: . users might not be aware that corresponding meta-data indicates to tools that the real artifact is hosted elsewhere . tools would need to do a 2 stage lookup to find an artifact, even if its not present: 1. determine if the artifact is hosted directly 2. on failing [1], determine if there is any meta-data indicating that the tool should look elsewhere -Tim
RE: Use of '/' in ???-specifier's
From: Stephen McConnell [mailto:[EMAIL PROTECTED] Nick Chalko wrote: Tim Anderson wrote: For advocates of URI parsing, what problems are you trying to solve? * Discovery of what is available * Repository exploring. * Auto cleanup of repositories. The URI spec is too loose. I completely agree. But I just want to add that all I want is either (a) a simple structural spec that does not imply more the 20 mins of concentration, or (b) something auto-explanitory ... a.k.a. server side meta (which acording to me is in scope relative to the objective of qualifying and differentiating organization, artifact, version and all of the other semantics that are currently being generalized. Today - we are not in the 20 min spectrum. Stephen. I would dispute (a), but I'll admit I'm biased. The proposals are public and can be edited and improved by anyone. If you feel there are changes that can be made to improve readability, go ahead - I claim no ownership - much of the content has been distilled from earlier discussions and current ASF practices. As for (b), put forward a proposal. -Tim
RE: Test/Prototypical Repository
From: Ben Walding [mailto:[EMAIL PROTECTED] Sent: Tuesday, 25 November 2003 9:03 AM I'm still not convinced that binaries is better than binary as a type directory. See my original comments that must have lost in the ether (section 2) - http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED] che.orgmsgId=1124258 [snip] binaries matches existing ASF practices, for both java and C distributions. Also, the directory name need not be reflected in the artifact extension - the directory is there simply to group similar artifacts. The proposals no longer refer to a 'type' directory. E.g, for java artifacts [1], artifact-specifier is: artifact-specifier = java-artifact-specifier java-artifact-specifier = jar-artifact | war-artifact| rar-artifact | ear-artifact | tld-artifact | javadoc-artifact jar-artifact = jars / versioned-artifact-name .jar war-artifact = wars / versioned-artifact-name .war rar-artifact = rars / versioned-artifact-name .rar ear-artifact = ears / versioned-artifact-name .ear tld-specifier = tlds / versioned-artifact-name .tld javadoc-artifact = docs / versioned-artifact-name -javadoc . arc-ext arc-ext = tar.gz | zip | bzip2 | ... -Tim [1] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/JavaArtifacts
RE: Use of '/' in ???-specifier's
From: Nick Chalko [mailto:[EMAIL PROTECTED] Tim Anderson wrote: For advocates of URI parsing, what problems are you trying to solve? * Discovery of what is available * Repository exploring. * Auto cleanup of repositories. The URI spec is too loose. As far as I can tell these are legal http://repo.apache.org/alpha/alpha/alpha/alpha/alpha/alpha.jar http://repo.apache.org/1/2/3/4/5/6/7/8/9.jar We really need to harden the URI spec a little and the / is a good start. The above a legal for the URI Syntax proposal [1], but illegal according to the common build version [2] and java artifact specifiers [3]. Tools based on [2] [3] should ignore them. Is it simply a matter of restricting organisation back to a single path segment? This would allow product-specifier to be determined by parsers. Note that this was the original approach, but some people expressed a desire to be able to break down the hierarchy using reverse-FQDNs. As for auto cleanup, this is supported in part by: . version-specifier in [1] All repository URIs must include a version in the path. This: . ensures all artifacts for a particular version are grouped together . simplifies archival of artifacts for a particular version . interim-build in [2] This assigns timestamps for interim builds (nightly, snapshot etc) The repository would have to limit version naming schemes to numeric schemes to support auto cleanup fully, which is too restrictive IMO. -Tim [1] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Proposals [2] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier [3] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/JavaArtifacts
RE: Use of '/' in ???-specifier's
Not a criticism, but I'd prefer to know the requirements, before writing the tools. As far as I can tell, maven doesn't do URI parsing. I don't know a lot about Gump, but if it wants to pull down the newest versions of jars, it can via the latest version tag [1]. Avalon adds meta-data, which is supported through the statement in [2]: Projects should be able to deploy arbitrary artifacts to the repository, whether they be for end-users, or meta-data (e.g, maven's project.xml). Tools should ignore any artifact they don't understand. -Tim [1] http://nagoya.apache.org/wiki/apachewiki.cgi?action=editid=ASFRepository/Co mmonBuildVersionSpecifier [2] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier -Original Message- From: Adam R. B. Jack [mailto:[EMAIL PROTECTED] Sent: Tuesday, 25 November 2003 5:59 AM To: [EMAIL PROTECTED] Subject: Re: Use of '/' in ???-specifier's Tim Anderson wrote: For advocates of URI parsing, what problems are you trying to solve? This is a simple matter of practicality. We've agreed to delay metadata so we can get a nice/simple repository structure w/o all the differences of opinion that metadata might introduce. We basically want to take existing repository structures (Maven's, Avalon's, Gump's, etc.) and formalize them to promote consistency/re-use. We need a repository that is practical at this level, and practical includes tooling/scripting remote/local clients. Non-parsable URIs (from a loose spec) mean an unreadable repository entries, so it is impractical without metadata. We need parsable URIs so we can have the repository by it's own metadata. I see your reply to Nick references additional specification. I wonder if they just need to merge those into the full lspecification. At this stage of development tight is far better than loose. IMHO, we can make this repository as strict as we like to start with. We need a tight prototypical repository, so we can build a repository and exercise it with tools by hand. We can't keep going back and forth in the theoretical, we need a concrete reference, we need practical experience. regards, Adam
RE: Test/Prototypical Repository
Not quite. The log4j-1.2.8.zip binary should be log4j-1.2.8-bin.zip according to http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/JavaArtifacts I would expect the log4j 1.2.8 release (with debug versions of jars and binaries) to look something like: apache/ (organisation) log4j/ (project) latest/ - symlink to 1.2.8 1.2.8/ (version) binaries/ log4j-1.2.8-bin.tar.gz log4j-1.2.8-bin.tar.gz.md5 log4j-1.2.8-bin.tar.gz.pgp log4j-1.2.8-dbg-bin.tar.gz log4j-1.2.8-dbg-bin.tar.gz.md5 log4j-1.2.8-dbg-bin.tar.gz.pgp log4j-1.2.8-bin.zip log4j-1.2.8-bin.zip.md5 log4j-1.2.8-bin.zip.pgp log4j-1.2.8-dbg-bin.zip log4j-1.2.8-dbg-bin.zip.md5 log4j-1.2.8-dbg-bin.zip.pgp source/ log4j-1.2.8-src.tar.gz log4j-1.2.8-src.tar.gz.md5 log4j-1.2.8-src.tar.gz.pgp log4j-1.2.8-src.zip log4j-1.2.8-src.zip.md5 log4j-1.2.8-src.zip.pgp jars/ log4j-1.2.8.jar log4j-1.2.8.jar.md5 log4j-1.2.8.jar.pgp log4j-1.2.8-dbg.jar log4j-1.2.8-dbg.jar.md5 log4j-1.2.8-dbg.jar.pgp pgp/ KEYS licenses/ LICENSES.txt - Tim From: Adam R. B. Jack [mailto:[EMAIL PROTECTED] Sent: Tuesday, 25 November 2003 6:25 AM To: [EMAIL PROTECTED] Subject: Test/Prototypical Repository All, As a way to force me to review the specification and attempt to implement I've started a knock up repository at: http://www.apache.org/~ajack/testrepo [If we think this is a good idea we can ask infrastructure@ for a location we can all write to.] Can folks tell me if this repository fits the specification? I had problem with the top part. regards Adam -- Experience Sybase Technology... http://www.try.sybase.com
RE: licensing issues for virtual artifacts (was RE: click through license support?)
From: Nick Chalko [mailto:[EMAIL PROTECTED] Tim Anderson wrote: Can you clarify the licensing issues further? I'm having trouble seeing what the problems are. Suppose ASF has the following link in the repository: http://repo.apache.org/sun/jndi/1.2.1/jars/jndi-1.2.1.jar This is a virtual artifact, not hosted at ASF. I think what you describe was fine. I was looking the otherway. The ability to host a jar and ensure that a user accepts a license first. So what would we need for a virutal artifact. A entry-url and the rest is up to the user/tool? There are a few approaches. The following assumptions are made: . virtual artifacts which require processing redirect to a descriptor hosted within the repository which describes how to get the real artifact. . the descriptor refers either to the real artifact, or an artifact (call it target) containing the real artifact. Possible approaches: 1. descriptor includes URL of target artifact. The limitation of this approach is that the tool has to be aware of the licensing and distribution of the target artifact. 2. descriptor includes code to get the target artifact For java, this could be a scripting mechanism based on BSF, ant or jelly. This may not be portable between tools: . tool requires a dependency on the scripting mechanism . tool may not be able to specify where the artifact is downloaded to 3. descriptor refers to code which can get the artifact For java, this would include the main class and URL classpath of the code to get the artifact. This code could be hosted in the repository. For portability between tools, an API would need to be specified to give tools control over how the target artifact is processed subsequent to its download. -Tim
[proposal] signature artifact specifier v0.1
[not too happy with the terminology used here. Open to suggestions] Overview This proposal extends the URI Syntax proposal: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax Signature artifacts are artifacts used to verify the integrity of another artifact. These include PGP/GPG signatures and keys, MD5 and SHA checksums. The key aims of this proposal are to: . formalise artifact-specifier for signature artifacts; . provide a set of best practices for such artifacts; and . enable tools to construct a URI to unambigously locate a particular signature artifact using a set of known criteria URI Components == An absolute repository URI is written as follows: repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier For signature artifacts, artifact-specifier is: artifact-specifier = signature-artifact-specifier signature-artifact-specifier = key-artifact | integrity-artifact Key artifacts - For artifacts digitally signed using PGP/GPG, there is an associated KEYS artifact. key-artifact = pgp-keys pgp-keys = pgp/KEYS E.g: http://repo.apache.org/apache/ant/1.5.4/pgp/KEYS Integrity artifacts --- Each artifact may have an associated integrity artifact: integrity-artifact = artifact-specifier . sig-type sig-type = pgp | md5 | sha Where: . pgp indicates the artifact was digitally signed using PGP/GPG . md5 indicates an md5 checksum . sha indicates a SHA checksum E.g: The artifact: http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar may have integrity artifacts: http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.md5 http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.pgp http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.sha Rationale = Integrity artifacts located alongside artifacts --- This approach enables integrity artifacts to be located easily. Tool support Key artifacts - Tools can unambigously locate a key artifact given the project-version URI and signature type. E.g, given: uri = http://repo.apache.org/apache/ant/1.5.4/ sig-type = pgp The key artifact URI would be: uri = http://repo.apache.org/apache/ant/1.5.4/pgp/KEYS Integrity artifacts --- Tools can unambigously locate an integrity artifact given the repository URI of the associated artifact, and the signature type. E.g, given: uri = http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar sig-type = md5 The integrity artifact URI would be: uri = http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.md5
licensing issues for virtual artifacts (was RE: click through license support?)
Can you clarify the licensing issues further? I'm having trouble seeing what the problems are. Suppose ASF has the following link in the repository: http://repo.apache.org/sun/jndi/1.2.1/jars/jndi-1.2.1.jar This is a virtual artifact, not hosted at ASF. Via http redirection and magic, a download tool: A. pops up a browser, requiring the user to accept Sun's license B. downloads the corresponding jndi-1_2_1.zip distribution if and only if the user *manually* accepts the license C. caches the distribution locally D. extracts jndi.jar from the distribution for local use Taking the Sun license points one at a time: . (i): you distribute the Software complete and unmodified and only bundled as part of, and for the sole purpose of running, your Java applets or applications (Programs) I don't see a violation here. The repository is not distributing JNDI - its facilitating its download. The download tool is not distributing JNDI - its facilitating its use by an application. As far as I can tell, the only requirement is that the onus is on the end user to satisfy this part of the license. . (ii) the Programs add significant and primary functionality to the Software, Again, the onus is on the end user to satisfy this part of the license. . (iii) you do not distribute additional software intended to replace any component(s) of the Software Not violated by the repository nor the download tool. . (iv) you do not remove or alter any proprietary legends or notices contained in the Software, When unpacking the distribution, the tool needs to ensure that license information is retained. . (v) you only distribute the Software subject to a license agreement that protects Sun's interests consistent with the terms contained in this Agreement, and Again, the onus is on the end user to satisfy this part of the license. . (vi) you agree to defend and indemnify Sun and its licensors from and against any damages, costs, liabilities, settlement amounts and/or expenses (including attorneys' fees) incurred in connection with any claim, lawsuit or action by any third party that arises or results from the use or distribution of any and all Programs and/or Software. The ASF has not distributed the software, so it can't be liable. If this has been discussed elsewhere, could you post a link? Thanks, Tim From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Wednesday, 19 November 2003 2:46 AM To: [EMAIL PROTECTED] Subject: Re: click through license support? Nicola Ken Barozzi [EMAIL PROTECTED] wrote on 19/11/2003 01:31:13 AM: [EMAIL PROTECTED] wrote: Nicola Ken Barozzi [EMAIL PROTECTED] wrote on 15/11/2003 10:00:07 PM: Tim Anderson wrote: ... A tool can 'screen scrape' the redirected page, prompt the user to accept the license and only download if the license is accepted, If the tool is made to work like a web browser, ie show the pages and then download when the user clicks on the button, IMHO it would be perfectly acceptable. But still illegal. I still don't understand why. I mean, if: 1- the program opens the browser on the product download page 2 - the user does the download steps as usual 3 - the program gets the downloaded artifact from the local download location Why would we be breaking the license? The only difference between this approach and the usual one is that the download location is linked. We've been down this road and are working with Sun on a solution. We have (had?) a tool that would do the above in Maven ages ago. Yes, I'm aware of that. See http://maven.apache.org/sun-licensing-journey.html Very good that you have this page, thanks for the pointer. For example, the JavaMail v1.3 BCL has Supplemental License Terms which state in Point 2. : ...Sun grants you a non-exclusive, non-transferable, limited license to reproduce and distribute the Software in binary code form only, provided that (i) you distribute the Software complete and unmodified and only bundled as part of, and for the sole purpose of running, your Java applets or applications (Programs), (ii) the Programs add significant and primary functionality to the Software, (iii) you do not distribute additional software intended to replace any component(s) of the Software, (iv) you do not remove or alter any proprietary legends or notices contained in the Software, (v) you only distribute the Software subject to a license agreement that protects Sun's interests consistent with the terms contained in this Agreement, and (vi) you agree to defend and indemnify Sun and its licensors from and against any damages, costs, liabilities, settlement amounts and/or expenses (including attorneys' fees) incurred in connection with any claim, lawsuit or action by any third party that arises or results from the use or distribution of any and all Programs
RE: Use of '/' in ???-specifier's
From: Nick Chalko [mailto:[EMAIL PROTECTED] Noel J. Bergman wrote: Seems to me that there is limited utility to being able to parse the URI, and that the real key is having meta-data with which to assemble it. But others don't seem to agree with that view. They want to parse semantic information from the URI. The semantic information is there in the URL, org. project. version, artifact type, name, release type etc. People WILL try to parse it. I think it would be a Good Idea to make it easy to parse at least the major pieces into discrete chunks. Assuming most people will simply replace / with - or _ the issue is not one off URL length or URL readability, it seems to be mostly about the browseablity of of directories. In other words have all the apache projects under the apache dir, or under subdirs of apache. I think the convience of knowing exactly where org, project, and version start and stop is worth the cost to browseablity. The proposals aim to structure the repository such that: 1. artifacts can easily be located by users 2. artifacts can easily be located by tools For [2], the intention is that a maven-like approach will be used for artifact resolution i.e, the user specifies enough criteria to enable a tool to unambigously locate an artifact. This approach doesn't require artifact URIs to be parsed. For advocates of URI parsing, what problems are you trying to solve? -Tim
RE: click through license support?
From: Nick Chalko [mailto:[EMAIL PROTECTED] ASL does not require click's for it's license. I think this is out of scope. It is an important discusion for tools, but we are not doing tools here. R, Nick Yes and no. It would be useful if the repository could host virtual artifacts. This enables: . a unified view of artifacts to be presented to both users and tools . virtual artifacts to be augmented with meta-data E.g, sun jars could have associated maven project.xml files. . federation support At the very least, tools would need to be aware that they need to follow http redirects to get to the real artifact. This group could make recommendations as to how virtual artifacts could be supported.
[proposal] common distribution artifact specifier v0.1
Overview This proposal extends the URI Syntax proposal: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax It is recommended, but not required, that it be used in conjunction with the Common Build Version Specifier proposal: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier The key aims of this proposal are to: . formalise artifact-specifier for source and binary distributions; . support platform-independent and platform-specific distributions; . provide a set of best practices for such artifacts; and . enable tools to construct a URI to unambigously locate a particular distribution artifact using a set of known criteria URI Components == An absolute repository URI is written as follows: repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier For distribution artifacts, artifact-specifier is: artifact-specifier = distribution-artifact distribution-artifact = binary-artifact | source-artifact Binary artifacts binary-artifact = platform-independent-binary | platform-specific-binary platform-independent-binary = bin-prefix / versioned-artifact-name bin-suffix platform-specific-binary = bin-prefix / os-name / versioned-platform-artifact-name bin-suffix bin-prefix = binaries os-name = pchar+ bin-suffix = -bin . arc-ext arc-ext = tar.gz | zip | exe | pchar+ (pchar is per http://www.ietf.org/rfc/rfc2396.txt) E.g: binaries/linux/httpd-2.0.40-i686-pc-linux-gnu-rh73-bin.tar.gz binaries/win32/apache-2.0.48-win32-x86-bin.exe binaries/commons-cli-1.1-bin.tar.gz Source artifacts source-artifact = platform-independent-source | platform-specific-source platform-independent-source = src-prefix / versioned-artifact-name src-suffix platform-specific-source = src-prefix / os-name / versioned-platform-artifact-name src-suffix src-prefix = source src-suffix = -src . arc-ext E.g: source/commons-cli-1.1-src.zip source/solaris/httpd-2.0.43-sparc-sun-solaris-2.8-src.tar.gz Platform independent artifacts -- Platform independent artifacts include the project version: versioned-artifact-name = artifact-name - short-version [debug] artifact-name = pchar+ short-version = version-name [- timestamp] (version-name and timestamp are per http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier) E.g: ant-1.5.4-src.tar.gz ant-1.5.4-20031113.1043-bin.zip ant-1.5.4-dbg-bin.tar.gz Platform specific artifacts --- Platform-specific distribution artifacts include the project version and platform: versioned-platform-artifact-name = artifact-name - short-version platform [debug] artifact-name = pchar+ short-version = version-name [- timestamp] plaform = pchar+ os-name = pchar+ Debug information - Artifacts can indicate that they include debugging information via the -dbg suffix: debug = -dbg E.g: commons-cli-1.1-dbg-bin.zip httpd-2.0.43-sparc-sun-solaris2.8-dbg-bin.tar.gz Rationale = Artifacts in subdirectories --- Each category of artifact in this proposal is required to be located in its own directory, e.g: binaries/commons-cli-1.0.tar.gz rather than in the root project version directory. The alternative approach of placing each artifact in the root makes repository navigation harder, particularly for projects that: . deploy large numbers of artifacts . deploy artifacts for multiple platforms . deploy artifacts other than those defined by this proposal. Inclusion of version and platform in artifact names Distribution artifacts require that the version be included in their names. Platform specific artifacts are required to also specify the platform: versioned-artifact-name = artifact-name - short-version [debug] versioned-platform-artifact-name = artifact-name - short-version platform [debug] short-version = version-name [- timestamp] This ensures that it is immediately obvious to users what version and platform of an artifact they are using, subsequent to its download. The optional timestamp indicates interim builds, as per: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier Tool support Tools can unambigously locate an artifact within a project version given the following criteria: . artifact type Mandatory. . artifact name Mandatory. . OS name. Mandatory for
Summary of repository URI proposals
There are now four proposals available at: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Proposals . Repository URI Syntax http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax This specifies the core URI layout, which the other proposals extend. . Common Build Version Specifier http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier This specifies versioning for projects which perform formal and interim builds. . Common Distribution Artifact Specifier http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonDistributio nArtifactSpecifier This specifies the URI format for source and binary distribution artifacts. . Java Artifact Specifier http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/JavaArtifacts This specifies the URI formats for java artifacts. To do - . change Java Artifact Specifier to refer to Common Distribution Artifact Specifier, for source and binary distributions . factor out signature-artifact-specifier and signature-keys from Java Artifact Specifier, into a new proposal, (Signature Artifact Specifier?), as it is relevant to Common Distribution Artifact Specifier as well. . factor out license-specifier from Java Artifact Specifier, into a new proposal, (License Artifact Specifier?), as it is relevant to Common Distribution Artifact Specifier as well. . determine what additional support is required for C/C++ projects. Common Distribution Artifact Specifier aims to handle binary and source distributions for these projects - what else is required? . move common BNF somewhere else. E.g: versioned-artifact-name artifact-name short-version debug arc-ext are contained in both Java Artifact and Common Distribution Artifact Specifiers. -Tim
RE: Use of '/' in ???-specifier's
From: Stephen McConnell [mailto:[EMAIL PROTECTED] Nick Chalko wrote: Given this spec repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier What is the version of this URL http://repo.apache.org/org/apache/commons/nightly/alpha/1.0/foo.jar * Projet commons, version Nightly 1.0 alpha * Project commons-nightly, version 1.0 alpha * Project commons-nightly-alpah version 1.0 (release) I think we should tighten the spec enough so we can at least tell the access, product,version, and artifact specifiers appart. +1 Steve. A repository URI cannot be parsed simply based on repository-uri. It needs to be used in conjunction with the other proposals before any information can be derived. As stated previously, the URI doesn't match the criteria supported by the other proposals. This doesn't mean that the URI is invalid, it simply means that it isn't supported by them. Tools that are based on the proposals should therefore ignore it. Alternative approaches would be to: . limit each *-specifier to a single path segment (i.e, disallow /) Hopefully, the proposals show that this is too restrictive . include delimeters in the URI E.g, http://repo.apache.org/organisation/apache/project/foo/version/... Duplicate information, and just plain *ugly*. -Tim
RE: [proposal] URI Syntax - v0.2
I changed organisation to name-segments to support structures using reverse-FQDNs e.g: http://repo.apache.org/org/apache http://repo.apache.org/org/tigris http://repo.apache.org/com/sun while maintaining support for single segment organisation names e.g: http://repo.apache.org/oracle See the comments regarding groupId in the original proposal for background: http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]ms gNo=308 From a tool perspective, it can unambiguously locate a project when given inputs of: org.apache - must replace . with / before performing lookup org/apache oracle The implication of this is that generic tools can't parse the URI and determine what is part of the product-specifier and what is part of the version-specifier. However, I don't think this is unreasonable. There is no requirement that tools be able to parse URIs to extract meta-data. -Tim From: Anou Manavalan [mailto:[EMAIL PROTECTED] Tim, This is very nicely laid out. I have one little suggestion, In the Product Specifier, can the organization be made as just name-segment ? This avoids the confusion of / separator that separates the main things like the orgainization / project with / separating the organisation itself. I mean, replace . By - instead of / - since / is used as the main separation. Instead of this, where it is hard to say where the org ends and where the project starts, you sure can differentiate it, but http://repo.apache.org/org/apache/commons-logging this makes more sense as org / project http://repo.apache.org/org-apache/commons-logging regards, -Anou From: Tim Anderson [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: [proposal] URI Syntax - v0.2 Date: Fri, 14 Nov 2003 16:39:06 +1100 This version replaces v1.0: http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED] ache.orgms gNo=308 Overview The key aims of this proposal are: . language and artifact neutrality. It should be possible to support multiple languages and their artifacts, not just java. . it should be possible for users to easily navigate the repository and locate artifacts, including jars and release distributions. Compare this with the existing approach of separating release distributions (http://www.apache.org/dist/) and jars (http://www.ibiblio.org/maven). . it should be possible for tools to construct a URI to locate an artifact using a set of known criteria Artifacts - All files in the repository are artifacts. There is no distinction between artifacts and meta-data. Any relationships between artifacts is determined by supporting tools. Repository URI Components = An absolute repository URI is written as follows: repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier Access specifier The access specifier determines the scheme, authority, and optional repository directory prefix. There is currently no requirement for ftp, scp or file based access - only http is supported: access-specifier = http-access-specifier http-access-specifier = http://; authority / [directory /] directory = path_segments (authority and path_segments are per http://www.ietf.org/rfc/rfc2396.txt) directory is used when the repository cannot be located at the root of an absolute URI. URI examples: http://repo.apache.org/ http://repo.apache.org/pub/repository Product specifier - The product specifier specifies the organisation and project: product-specifier = organisation / project organisation = name-segments project = name-segment name-segments = name-segment *( / name_segment) name-segment = nchar+ nchar = alphanum | escaped | _ | - | ! | ~ | @ | (alphanum and escaped are per http://www.ietf.org/rfc/rfc2396.txt) organisation is the organisation name. It is arbitrary, but should be globally unique. It could be the domain name, or reverse domain name, with . replaced by /, e.g: sun/com, org/apache or simply the name of the organisation, e.g oracle. project is the project name. It is unique within an organisation. E.g, ldap, jndi, maven, commons-logging. URI examples: http://repo.apache.org/org/apache/commons-logging http://repo.apache.org/sun/jndi Version specifier - The version specifier specifies the version of the project: version-specifier = name-segments For the purposes of this proposal, version-specifier is opaque - its format is determined by language and deployment best practices. Some possible examples include: 1.0, v0.9-beta, nightly/20031113, latest, release/1.5.4 URI examples: http://repo.apache.org/apache/commons-logging/1.0 http://repo.apache.org/apache/commons-logging/1.1 http
RE: [proposal] URI Syntax - v0.2
From: Nick Chalko [mailto:[EMAIL PROTECTED] Tim Anderson wrote: From a tool perspective, it can unambiguously locate a project when given inputs of: org.apache - must replace . with / before performing lookup org/apache oracle The implication of this is that generic tools can't parse the URI and determine what is part of the product-specifier and what is part of the version-specifier. However, I don't think this is unreasonable. There is no requirement that tools be able to parse URIs to extract meta-data. -Tim I think easing the job for tools is a good goal. We must support both Humans and Tools. I would favor Humans. But both humans and tools will have problems when some orginzation decides its project name is Beta or nightly, etc I think we should consider not allowing / in many of the parts. R, Nick For tools, I think the main objective should be coming up with a set of rules which enable them to unambigously locate an artifact given a set of inputs. I believe this is possible with the two proposals so far, at least for java artifacts. -Tim
[proposal] common build version specifier - v0.1
Overview This proposal extends the URI Syntax proposal, v0.2: http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]ms gNo=367 The key aims of this proposal are to: . formalise version-specifier for projects which provide formal and interim builds; . provide a set of best practices for such projects; and . enable tools to construct a URI to unambigously locate a particular project version using a set of known criteria URI Components == An absolute repository URI is written as follows: repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier This proposal defines version-specifier as follows: version-specifier = build build = formal-build | interim-build formal-build = [formal-build-designation /] version interim-build = interim-build-designation / version [ / interim-version ] interim-version = latest | MMDD [- HHMM [SS]] formal-build-designation = release | ... interim-build-designation = interim | nightly | snapshot | ... version = latest | *pchar (pchar is per http://www.ietf.org/rfc/rfc2396.txt) Build - Builds are separated into formal and interim builds. A formal build may be a final or milestone release. e.g: 1.0, release/1.0, 1.0-beta1, release/1.0-rc1 An interim build is an informal release, produced by a nightly build or an ad-hoc snapshot. e.g: nightly/1.0/20031113, snapshot/1.2beta1. Version --- Version is either latest or arbitrary, determined by the project or deployment tool. latest always refers to the latest version of a particular build, and may be supported using symbolic links, or via http redirection. E.g: http://repo.apache.org/apache/commons-cli/release/l.0/... http://repo.apache.org/apache/commons-cli/release/l.1/... http://repo.apache.org/apache/commons-cli/release/latest/... - symlink to ../1.1 Interim version --- The interim version is either a timestamp, or latest. latest always refers to the latest interim version and may be supported using symbolic links, or via http redirection. http://repo.apache.org/apache/commons-cli/nightly/1.0/20031112/... http://repo.apache.org/apache/commons-cli/nightly/1.0/20031113/... http://repo.apache.org/apache/commons-cli/nightly/1.0/latest/... - symlink to ../20031113 Rationale = Optional build designation for formal builds formal-build is defined as: formal-build = [formal-build-designation /] version formal-build-designation = release | ... The formal-build-designation is optional for those projects which don't produce interim builds, or don't wish to add another directory for formal releases. E.g: http://repo.apache.org/apache/commons-cli/l.0/... http://repo.apache.org/apache/commons-cli/l.1/... Mandatory version in interim builds --- interim-build is defined as: interim-build = interim-build-designation / version [ / interim-version ] interim-version = latest | MMDD [- HHMM [SS]] This enables support for multiple versions of builds, if there are two or more concurrent development streams. E.g, to support nightly builds of versions 1.0 and 2.0 of commons-cli: http://repo.apache.org/apache/commons-cli/nightly/1.0/20031112/... http://repo.apache.org/apache/commons-cli/nightly/1.0/20031113/... http://repo.apache.org/apache/commons-cli/nightly/1.0/latest/... - symlink to ../20031113 http://repo.apache.org/apache/commons-cli/nightly/2.0/20031112/... http://repo.apache.org/apache/commons-cli/nightly/2.0/20031113/... http://repo.apache.org/apache/commons-cli/nightly/2.0/latest/... - symlink to ../20031113 Build designation naming conventions formal-build-designation and interim-build-designation are defined as: formal-build-designation = release | ... interim-build-designation = interim | nightly | snapshot | ... In other words, tools may use release, interim etc, but may also extend them to define their own. Tool support Tools can unambigously locate a project version given the following criteria: . formal or interim build-designation Optional. If not specified, assume formal build. . version Mandatory. . interim-version Optional. If specified, build designation must also be specified. Example 1. -- Given: organisation = apache project = commons-cli version = 1.0 build-designation = interim-version = The URI would be: http://repo.apache.org/apache/commons-cli/1.0 Example 2. -- Given: organisation = apache project = commons-cli version = latest build-designation = interim-version = The URI would be: http://repo.apache.org/apache/commons-cli/latest If two versions of commons-cli are available, 1.0 and 1.1, the URI would resolve to: http://repo.apache.org/apache/commons-cli/1.1
RE: [proposal] URI Syntax - v0.2
From: Adam R. B. Jack [mailto:[EMAIL PROTECTED] snip/ You could have the client tool be told the resource/URI by the user, and do the download/verification, yes. That said, I don't think it buys the user enough, they have to browse/locate stash the URI in some local config. I'd like to say go get me xerces from any repository it is in, but get me the latest, but I only want release not nightly/snapshot (e.g. http://www.krysalis.org/ruper/ant/reposync.htm). That to me, is useful. I don't mind being alone in my views, but I ask again -- if we don't set the bar higher than a one-way URI for download, why write a spec at all? I feel we have the potential to win big, and I'd like the ASF Repository to be a step forward towards these goals, not a step backward. I believe this is possible using the current proposals. If tools follow these when deploying artifacts, a user can can say go get me the latest formal xerces build. That said, some configuration will always be required, whether it be like maven's project.xml dependency resolution, or some other scheme. The proposals aim to avoid users explicitly using URIs. Users should be able to supply a set of criteria and the tools be responsible for constructing the URI. -Tim
RE: [proposal] common build version specifier - v0.1
From: Adam R. B. Jack [mailto:[EMAIL PROTECTED] Sent: Saturday, 15 November 2003 2:54 PM To: [EMAIL PROTECTED] Subject: Re: [proposal] common build version specifier - v0.1 Tim, I *love* your specifications, I really appreciate the clear/concise/explicit nature of them. I only wish you'd use Wiki not EyeBrowse as your persistent documentation tool. Wiki has versioning (so we can see older copies should we need to refer back) and such, and allows other to make (respectful) changes, and allows a view not cluttered throughout mail threads. I'm game to be your cut-n-paste-wallah if you really need one, but please (at least) refer to your proposals in the Wiki. regards OK. I'm not particularly wiki literate (the formatting is a bit off), but I've added/updated the proposals at: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Proposals Regards, Tim
RE: [proposal] java artifact specifier v0.1
From: Peter Donald [mailto:[EMAIL PROTECTED] On Fri, 14 Nov 2003 08:16 pm, Michal Maczka wrote: The only purpose of type in maven is to indicate how it is processed by the runtime. (ie plugins get installed, jars get added to classpath etc). It does not even specify that extension as there is a M-to-M between type and extension. ie. I don't agree that this is the only purpose of type. I think it's reasonable to have type directory as this can separate artifact produced by various tools. ... I would like to have an exclusivity for adding artifacts to directories like (jars/sources/distribution). But I could accept if somebody (some tool) is keeping its files in sibling directories. Also some exotic artifacts are making the repository harder to navigate. Sounds reasonable. But maybe the term type is not the best term for this as it does not designate a type but a arbitary subdivision of space based on usage patterns. Not sure what a better term would be? I thought of using category but opted for type simply because it is shorter. -Tim
click through license support?
One of the current problems with repositories such as http://www.ibiblio.org/maven is their inability to host products which have restrictive licensing schemes. (See http://maven.apache.org/sun-licensing-journey.html for background) E.g, ibiblio cannot host jars from Sun, because of the requirement that users must manually accept Sun's license before downloading the jars. This reduces the usefulness of using the repository for dependency resolution. I see several possible workarounds for this: Virtual hosting --- With this approach, none of the artifacts are hosted within the public repository. http redirection is used to direct 'virtual' artifact accesses to the real artifact. The limitation of this approach is that automatic artifact resolution can only work if the redirect is to the real artifact. This rules out all of the Sun jars which require acceptance of Sun's license first. A tool can 'screen scrape' the redirected page, prompt the user to accept the license and only download if the license is accepted, but this doesn't work in the general case. Direct hosting -- With this approach, artifacts are hosted within the public repository, but download is only enabled if the user agrees to the license. This implies that http redirection must be used and that tools have to be intelligent enough to handle the redirection and prompt the user. The limitation of this approach is that direct hosting can only be supported if an agreement can be made with the license holder. Thoughts?
RE: [proposal] URI Syntax - v0.2
From: Noel J. Bergman [mailto:[EMAIL PROTECTED] My input here is primarily based on writting Ruper You don't have to like the tool, I'm not trying to push the implementation I've never even seen the thing, and you are a priori assuming that I don't like it? It allows you to query what is there, query and capture oldest resources [and do a delete/clean], and download newest, etc. How does it know what OLDEST means? I see that Tim is trying to add some more structure, so maybe he's thinking that we can restrict the URI space so that a restricted notion of version assures an automatable concept of succession. The common build version specifier proposal does add structure to the version, but doesn't enable tools to determine if one version is older or newer than another. A tool could reasonably assume that version 1.0 2.0 but this is only valid for projects which follow numeric versions. For those projects which love codename versions (e.g, chicago, delta), no assumptions can be made. -Tim
RE: Use of '/' in ???-specifier's
From: Nick Chalko [mailto:[EMAIL PROTECTED] Given this spec repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier What is the version of this URL http://repo.apache.org/org/apache/commons/nightly/alpha/1.0/foo.jar * Projet commons, version Nightly 1.0 alpha * Project commons-nightly, version 1.0 alpha * Project commons-nightly-alpah version 1.0 (release) I think we should tighten the spec enough so we can at least tell the access, product,version, and artifact specifiers appart. R, Nick The URI isn't valid, according to http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier as a nightly build is an interim build: interim-build = interim-build-designation / version [ / interim-version ] interim-version = latest | MMDD [- HHMM [SS]] interim-build-designation = interim | nightly | snapshot | ... version = latest | *pchar IOW, one of alpha or 1.0 is invalid. However, given the URI: http://repo.apache.org/org/apache/commons/nightly/1.0/20031113/jars/foo-1.0. jar it is possible to determine the version using the common build version specifier and java artifact specifier proposals, by parsing the URI from right to left. It isn't possible to separate the organisation from the directory component of access-specifier however. e.g. given: http://repo.apache.org/org/apache/ it could be interpreted as: . directory = org, organisation = apache . directory = , organisation = org/apache -Tim
[proposal] java artifact specifier - v0.2
Overview This proposal extends the URI Syntax proposal: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax It is recommended, but not required, that it be used in conjunction with the Common Build Version Specifier proposal: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier The key aims of this proposal are to: . formalise artifact-specifier for java based projects . provide a set of best practices for such projects; and . enable tools to construct a URI to unambigously locate a particular java project artifact using a set of known criteria Java project artifacts include, but are not limited to: jars, wars, rars, tlds, source, binary and document distributions, and licenses. URI Components == An absolute repository URI is written as follows: repository-uri = access-specifier / product-specifier / version-specifier / artifact-specifier For java project artifacts, artifact-specifier is: artifact-specifier = java-project-artifact java-project-artifact = java-artifact | distribution-artifact | signature-artifact | signature-keys | license-artifact Java artifacts -- Java artifacts include, but are not limited to, jars, wars, rars, and tlds. java-artifact = jar-artifact | war-artifact| rar-artifact | tld-artifact | path_segments jar-artifact = jars / versioned-artifact-name .jar war-artifact = wars / versioned-artifact-name .war rar-artifact = rars / versioned-artifact-name .rar tld-specifier = tlds / versioned-artifact-name .tld (path_segments is per http://www.ietf.org/rfc/rfc2396.txt) Versioned artifact names Java and distribution artifacts include the project version: versioned-artifact-name = artifact-name - short-version [debug] artifact-name = pchar+ short-version = version-name [- timestamp] debug = -dbg (version-name and timestamp are per http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/CommonBuildVersio nSpecifier) (pchar is per http://www.ietf.org/rfc/rfc2396.txt) Artifacts can indicate that they include debugging information via the -dbg suffix. E.g: ant-1.5.4.jar ant-1.5.4-20031113.1043.jar ant-1.5.4-dbg.jar Distribution artifacts -- Distribution artifacts include binary, source, and documentation distributions: distribution-artifact = binary-artifact | source-artifact | doc-artifact binary-artifact = binaries / versioned-artifact-name -bin . arc-ext source-artifact = source / versioned-artifact-name -src . arc-ext doc-artifact = document-artifact | javadoc-artifact document-artifact = docs / versioned-artifact-name -doc . arc-ext javadoc-artifact = docs / versioned-artifact-name -javadoc . arc-ext arc-ext = tar.gz | zip | bzip2 | ... E.g: binaries/ant-1.5.4-bin.zip binaries/ant-1.5.4-dbg-bin.tar.gz source/ant-1.5.4-src.zip source/ant-1.5.4-src.tar.gz docs/ant-1.5.4-doc.zip docs/ant-1.5.4-javadoc.zip Signatures -- An artifact may have an associated PGP, MD5, or SHA signature artifact, located alongside it: signature-artifact-specifier = java-project-artifact . sig-type sig-type = pgp | md5 | sha | ... E.g: http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.md5 http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.pgp http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.sha Where a signature has associated keys, these are specified by: signature-keys = sig-type / KEYS E.g: http://repo.apache.org/apache/ant/1.5.4/pgp/KEYS Licenses license-specifier = licenses / license-name license-name = LICENSE.txt | path_segment (path_segment is per http://www.ietf.org/rfc/rfc2396.txt) Rationale = Artifacts in subdirectories --- Each category of artifact in this proposal is required to be located in its own directory, e.g: jars/commons-cli-1.0.jar rather than in the root project version directory. The alternative approach of placing each artifact in the root makes repository navigation harder, particularly for projects: . which deploy with large numbers of artifacts . which deploy artifacts other than those defined by this proposal. Inclusion of version in artifact names -- Java and distribution artifacts require that the version be included in their names: versioned-artifact-name = artifact-name - short-version [debug] short-version = version-name [- timestamp] This ensures that it is immediately obvious to users what version of an artifact they are using, subsequent to its download. The optional timestamp indicates interim builds, as per:
RE: Parsable URI (Re: [proposal] URI Syntax - v0.2)
The URI proposals so far specify URIs which are just as parseable as those currently in use by maven's repository [1]. The only caveat is that they need to be parsed from right to left, as the organisation [2] part of product-specifier cannot be separated from the directory part of access-specifier, without prior knowledge of the repository structure. E.g: if a repository has its root at: http://www.apache.org/repository And the organisation of a project is: org/apache And the project name is: commons-cli The URI: http://www.apache.org/repository/org/apache/commons-cli needs to be parsed from right to left to determine that the project is commons-cli. Without knowing that the repository has its root at: http://www.apache.org/repository; the organisation cannot be determined. Like maven's repository, which doesn't impose any version naming convention, tools trying to parse the URI need to make guesses as to which version is older or newer. -Tim [1] http://www.ibiblio.org/maven [2] http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax From: Adam R. B. Jack [mailto:[EMAIL PROTECTED] Noel wrote: You don't have to like the tool, I'm not trying to push the implementation I've never even seen the thing, and you are a priori assuming that I don't like it? No Neol, I'm not that emoition, I meant it dispassionately and without inference, maybe it just read differently. That was more 'one' doesn't have to like it. [I know this list has (in the past) slipped into implementation codebase factions, and I was hoping not to encourage that.] So perhaps I should've writen ... One doesn't have to like this tool/implementation, but the results are valuable at layer 1. It allows you to query what is there, query and capture oldest resources [and do a delete/clean], and download newest, etc. How does it know what OLDEST means? I see that Tim is trying to add some more structure, so maybe he's thinking that we can restrict the URI space so that a restricted notion of version assures an automatable concept of succession. Ruper parses all the attributes of the resources, including the version, and either do (pchar) string comparisons or (in versions case) structured comparisons. Much as there are a few different flavours of a versions they pretty much fall into a parsable pattern. Ruper (through Version) strictly parses the string in a number of different ways (known formats) until one matches. Again, the most important aspect of parsing the URI is knowing what is separated from what, that this pchar is a version, this pchar is a type (or whatever). If values can by groked within that, great, if not, it is still Some find such a tool useful, I'd like to believe that apache users (admins and external users) would find it useful. I don't disagree. I simply said that if you view the repository solution as a layer of specifications, the lowest layer can be a syntax that does not require semantics such as an automatable concept of succession. If we need that, we can add it either by a convention within the URI space, or by other means. We all agree to layers, but I am testing what are the minimum things we'll accept for layter one. I beleive that the repository needs to be 'tooling readable', hence the URI needs to be parse, the other aspects (can an attributed be fully groked) can come later. Again, I need to get to the wiki to put a proposal and pros/cons, I'll try next week. Absolutely. But that may require something more than the URI schema. :-) But if it doesn't have to, should it? I'm trying to determine what we ought will accept at the lowest level. I think clean up is important, I like the other aspects. I agree that much should be done via metadata (e.g. dependencies) however writting potentially shared/conflicting files to a repository is a scary step, and I'd like to see how much we can do with atomic artefacts. I feel we have the potential to win big, and I'd like the ASF Repository to be a step forward towards these goals, not a step backward. I agree. But one layer at a time. :-) Yes, and we are doing layer one -- without metadata, we still need to determine our minimum expectations. If URI is this contentious/involved, I could see metadata as being a long drawn out process one we don't agree on as a whole. Maybe this first layer is the hardest, but I'd like it to be the one giving the most rewards so we aren't all sitting waiting for metadata. regards, Adam
RE: [VOTE] Where is version in URI Syntax
I've restructured the wiki page at http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/WhereIsVersionInU RISytnax, and removed the part about symbolic links. -Tim -Original Message- From: Nick Chalko [mailto:[EMAIL PROTECTED] Sent: Friday, 14 November 2003 11:00 AM To: [EMAIL PROTECTED] Subject: Re: [VOTE] Where is version in UIR Syntax Tim Anderson wrote: I have a few comments on the content of that page: 1. Not sure why the discussion and the proposals are separate, given the partial duplication of pros and cons for each. Would prefer to see these merged. Good point, feel free to merge them. and add your pro cons. We will need this for later, when people ask us Why, we can point them to the wiki summary. 2. Version be a mandatory component of artifact filename Pros: . Artifacts become identifiable when *downloaded* from the repository. . This is not compatible with the current ASF scheme. Neither maven, nor dist require version in the artifact filename. Cons: . Presumes to know requirements of other repository users, for which we have no requirements. 3. Version in directory Cons: . I don't see how the need for a 'latest' symbolic link is a con. There is no uniform way at ASF at the moment to indicate the latest version. . Scheme not currently used by ASF. 4. There has been no discussion on how to cope with nightly or snapshot builds, which could change the version syntax. E.g: 1. Subdir per build: http://repo.apache.org/apache/commons-cli/nightly/20031112/... http://repo.apache.org/apache/commons-cli/nightly/20031113/... 2. Embedded in version: http://repo.apache.org/apache/commons-cli/nightly-20031112/... http://repo.apache.org/apache/commons-cli/nightly-20031113/... I'm leaning towards the former, as browsing is simpler. OTOH, this then leads to the possibility of nightly, snapshot, release etc being mandatory in product-specifier: product-specifier = organisation / project / rtype / version rtype = nightly | snapshot | release | ... -Tim -Original Message- From: Nick Chalko [mailto:[EMAIL PROTECTED] Sent: Friday, 14 November 2003 9:51 AM To: [EMAIL PROTECTED] Subject: Re: [VOTE] Where is version in UIR Syntax Current count. 2 For version dir with optional version on artifact name. 3 for version dir and versioned artifact name. Make sure you voice your opinion. Nick Chalko wrote: Lets see where we stand on the version. Please go to http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/WhereIs VersionInURISytnax and vote for the Proposal you prefer. Add pro's and con's as you see fit. Lets see how close we are to a consensus so wee can move on to other parts of the URISyntax. R, Nick
URI Syntax: nightly and release builds
The URISyntax proposal is silent on how to handle nightly, release, snapshot, and latest builds. This should be formalised. The current proposal has: product-specifier = organisation / project / version where: version = *pchar To support nightlies etc, this leads to the possibility of artifacts named: http://repo.apache.org/apache/commons-cli/1.0/... http://repo.apache.org/apache/commons-cli/1.1/... http://repo.apache.org/apache/commons-cli/latest/... - link to ../1.1 http://repo.apache.org/apache/commons-cli/nightly-20031112/... http://repo.apache.org/apache/commons-cli/nightly-20031113/... http://repo.apache.org/apache/commons-cli/nightly-latest/... - link to ../nightly-latest Where *latest is a symlink to the latest version, to aid navigation. Option 1. Specify version format To formalise the above, product-specifier could be changed to: product-specifier = organisation / project / [rtype -] version rtype = nightly | snapshot version = latest | MMDD [- HHMM [SS]] | *pchar Cons: . clutters the repository . doesn't follow existing conventions, e.g: http://cvs.apache.org/builds/jakarta-commons/nightly/commons-cli/ . no facility to indicate snapshots or nightlies of a particular version, if two or more versions are being developed concurrently. Option 2. Add build directory - To reduce clutter, a new directory could be introduced to separate releases from nightly and snapshot builds i.e: product-specifier = organisation / project / rtype / version rtype = release | nightly | snapshot version = latest | MMDD [- HHMM [SS]] | *pchar E.g: http://repo.apache.org/apache/commons-cli/release/l.0/... http://repo.apache.org/apache/commons-cli/release/l.1/... http://repo.apache.org/apache/commons-cli/release/latest/... - symlink to ../1.1 http://repo.apache.org/apache/commons-cli/nightly/20031112/... http://repo.apache.org/apache/commons-cli/nightly/20031113/... - symlink to ../20031113 http://repo.apache.org/apache/commons-cli/snapshot/20030901-1032/... http://repo.apache.org/apache/commons-cli/snapshot/latest/... - symlink to ../20030901-1032 Cons: . no facility to indicate snapshots or nightlies of a particular version, if two or more versions are being developed concurrently. Option 3. Concurrent version nightly/snapshot builds To allow nightlies and snapshots of multiple versions, product-specifier could be changed to: product-specifier = organisation / project / build build = release-build | interim-build release-build = release / version interim-build = itype / version / MMDD [- HHMM [SS]] itype = nightly | snapshot version = latest | *pchar E.g: http://repo.apache.org/apache/commons-cli/release/l.0/... http://repo.apache.org/apache/commons-cli/release/l.1/... http://repo.apache.org/apache/commons-cli/release/latest/... - symlink to ../1.1 http://repo.apache.org/apache/commons-cli/nightly/1.0/20031112/... http://repo.apache.org/apache/commons-cli/nightly/1.0/20031113/... http://repo.apache.org/apache/commons-cli/nightly/1.0/latest/... - symlink to ../20031113 http://repo.apache.org/apache/commons-cli/nightly/2.0/20031112/... http://repo.apache.org/apache/commons-cli/nightly/2.0/20031113/... http://repo.apache.org/apache/commons-cli/nightly/2.0/latest/... - symlink to ../20031113 http://repo.apache.org/apache/commons-cli/snapshot/1.0/20030901-1032/... http://repo.apache.org/apache/commons-cli/snapshot/1.0/latest/... - symlink to ../20030901-1032 http://repo.apache.org/apache/commons-cli/snapshot/2.0/20031101-1452/... http://repo.apache.org/apache/commons-cli/snapshot/2.0/latest/... - symlink to ../20031101-1452 Option 4. Concurrent version interim builds An alternative to Option 3 would be to remove any distinction from nightly and snapshot builds, as the difference IMO is only cosmetic: product-specifier = organisation / project / build build = release-build | interim-build release-build = release / version interim-build = interim / version / MMDD [- HHMM [SS]] version = latest | *pchar E.g: http://repo.apache.org/apache/commons-cli/release/l.0/... http://repo.apache.org/apache/commons-cli/release/l.1/... http://repo.apache.org/apache/commons-cli/release/latest/... - symlink to ../1.1 http://repo.apache.org/apache/commons-cli/interim/1.0/20031112/... http://repo.apache.org/apache/commons-cli/interim/1.0/20031113/... http://repo.apache.org/apache/commons-cli/interim/1.0/latest/... - symlink to ../20031113 http://repo.apache.org/apache/commons-cli/interim/2.0/20031112/... http://repo.apache.org/apache/commons-cli/interim/2.0/20031113/... http://repo.apache.org/apache/commons-cli/interim/2.0/latest/... - symlink to ../20031113
RE: [proposal] java artifact specifier v0.1
From: Peter Donald [mailto:[EMAIL PROTECTED] On Mon, 10 Nov 2003 06:31 pm, Tim Anderson wrote: URI Components -- An absolute repository URI is written as follows: repository-uri = access-specifier / product-specifier / artifact-specifier For java artifacts, artifact-specifier is: artifact-specifier = [type /] artifact type = jars | wars | rars | tlds | binaries | source | licenses | ... artifact = artifact-name [- version] [- stype][.ext] artifact-name = *pchar stype = bin | src ext = jar | war | rar | tld | tar.gz | zip | ... type seems to be a rather artificial component. There should be no artefacts with the same name but different types thus no conflicts if they are in the same directory. ie Why not have http://repo.apache.org/apache/ant/1.5.4/ant-1.5.4.jar http://repo.apache.org/apache/ant/1.5.4/ant-1.5.4.jar.md5 http://repo.apache.org/apache/ant/1.5.4/ant-optional-1.5.4.jar http://repo.apache.org/apache/ant/1.5.4/ant-optional-1.5.4.jar.md5 http://repo.apache.org/apache/ant/1.5.4/ant-bin-1.5.4.zip http://repo.apache.org/apache/ant/1.5.4/ant-bin-1.5.4.zip.md5 http://repo.apache.org/apache/ant/1.5.4/ant-bin-1.5.4.tar.gz http://repo.apache.org/apache/ant/1.5.4/ant-bin-1.5.4.tar.gz.asc http://repo.apache.org/apache/ant/1.5.4/ant-src-1.5.4.zip http://repo.apache.org/apache/ant/1.5.4/ant-src-1.5.4.zip.md5 http://repo.apache.org/apache/ant/1.5.4/ant-src-1.5.4.tar.gz http://repo.apache.org/apache/ant/1.5.4/ant-src-1.5.4.tar.gz.asc http://repo.apache.org/apache/ant/1.5.4/LICENSE-1.5.4.txt http://repo.apache.org/apache/ant/1.5.4/KEYS-1.5.4 Also stype seems like another artificial distinction. Is not two artefacts with different stypes just two different artefacts. ie ant-src-1.5.4.tar.gz == artifact-name == ant-src ant-bin-1.5.4.tar.gz == artifact-name == ant-bin It seems that is just adding extra distinctions that are not really needed. And I can't see any benefit of these extra distinctions. 'type' is there to logically group artifacts, to aid users browsing the repository. For products with many artifacts, this makes navigating the repository easier. The values of both 'type' and 'stype' are arbitrary - the suggested values for the java artifact specifier try to follow conventions used at apache. -Tim
RE: [proposal] java artifact specifier v0.1
Take a look at the proposals: http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/JavaArtifacts Hopefully it will be clear. -Tim -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Thursday, 13 November 2003 1:39 PM To: [EMAIL PROTECTED] Subject: Re: [proposal] java artifact specifier v0.1 Hi, From: Tim Anderson [EMAIL PROTECTED] 'type' is there to logically group artifacts, to aid users browsing the repository. For products with many artifacts, this makes navigating the repository easier. So the sole purpose of this is to make navigation easier? Isn't that what the groupID is meant to be used for? How does type/groupID differ in their purposes?
RE: [VOTE] Where is version in UIR Syntax
I have a few comments on the content of that page: 1. Not sure why the discussion and the proposals are separate, given the partial duplication of pros and cons for each. Would prefer to see these merged. 2. Version be a mandatory component of artifact filename Pros: . Artifacts become identifiable when *downloaded* from the repository. . This is not compatible with the current ASF scheme. Neither maven, nor dist require version in the artifact filename. Cons: . Presumes to know requirements of other repository users, for which we have no requirements. 3. Version in directory Cons: . I don't see how the need for a 'latest' symbolic link is a con. There is no uniform way at ASF at the moment to indicate the latest version. . Scheme not currently used by ASF. 4. There has been no discussion on how to cope with nightly or snapshot builds, which could change the version syntax. E.g: 1. Subdir per build: http://repo.apache.org/apache/commons-cli/nightly/20031112/... http://repo.apache.org/apache/commons-cli/nightly/20031113/... 2. Embedded in version: http://repo.apache.org/apache/commons-cli/nightly-20031112/... http://repo.apache.org/apache/commons-cli/nightly-20031113/... I'm leaning towards the former, as browsing is simpler. OTOH, this then leads to the possibility of nightly, snapshot, release etc being mandatory in product-specifier: product-specifier = organisation / project / rtype / version rtype = nightly | snapshot | release | ... -Tim -Original Message- From: Nick Chalko [mailto:[EMAIL PROTECTED] Sent: Friday, 14 November 2003 9:51 AM To: [EMAIL PROTECTED] Subject: Re: [VOTE] Where is version in UIR Syntax Current count. 2 For version dir with optional version on artifact name. 3 for version dir and versioned artifact name. Make sure you voice your opinion. Nick Chalko wrote: Lets see where we stand on the version. Please go to http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/WhereIs VersionInURISytnax and vote for the Proposal you prefer. Add pro's and con's as you see fit. Lets see how close we are to a consensus so wee can move on to other parts of the URISyntax. R, Nick
RE: Comments on URI Syntax
From: Stephen McConnell [mailto:[EMAIL PROTECTED] Tim Anderson wrote: From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] From the requirements at http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Requirements: ASF Repository shall ... allow browsing and downloading of artifacts by humans via normal web browser. Requiring a version to be part of the artifact file name when the artifact is only useful to end users (e.g README), reduces clarity. But it does increase usability sometimes. README for which version? An example: http://repo.apache.org/apache/commons-dbcp/1.1/README The README is for version 1.1 of commons-dbcp. By implication - the README is not an artifact but a feature of a version. Is that a reasonable conclusion? Stephen. Why make the distinction? I view everything a project deploys as an artifact. Some artifacts will only be useful to end users (e.g, README, LICENSE.txt etc), others will be useful to tools. -Tim
[proposal] java artifact specifier v0.1
Overview The aim of this proposal is to specify the URI syntax for artifacts for java-based projects. It extends the URI syntax proposal: http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]ms gNo=308 Java project artifacts include, but are not limited to: jars, wars, rars, tlds, source and binary distributions, and licenses. URI Components -- An absolute repository URI is written as follows: repository-uri = access-specifier / product-specifier / artifact-specifier For java artifacts, artifact-specifier is: artifact-specifier = [type /] artifact type = jars | wars | rars | tlds | binaries | source | licenses | ... artifact = artifact-name [- version] [- stype][.ext] artifact-name = *pchar stype = bin | src ext = jar | war | rar | tld | tar.gz | zip | ... Checksums PGP signatures -- All artifacts may have an associated md5 checksum, of the form: artifact-specifier .md5 or a PGP signatures, of the form: artifact-specifier .asc Examples jars: http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar http://repo.apache.org/apache/ant/1.5.4/jars/ant-1.5.4.jar.md5 http://repo.apache.org/apache/ant/1.5.4/jars/ant-optional-1.5.4.jar http://repo.apache.org/apache/ant/1.5.4/jars/ant-optional-1.5.4.jar.md5 binaries: http://repo.apache.org/apache/ant/1.5.4/binaries/ant-1.5.4-bin.zip http://repo.apache.org/apache/ant/1.5.4/binaries/ant-1.5.4-bin.zip.md5 http://repo.apache.org/apache/ant/1.5.4/binaries/ant-1.5.4-bin.tar.gz http://repo.apache.org/apache/ant/1.5.4/binaries/ant-1.5.4-bin.tar.gz.asc source: http://repo.apache.org/apache/ant/1.5.4/source/ant-1.5.4-src.zip http://repo.apache.org/apache/ant/1.5.4/source/ant-1.5.4-src.zip.md5 http://repo.apache.org/apache/ant/1.5.4/source/ant-1.5.4-src.tar.gz http://repo.apache.org/apache/ant/1.5.4/source/ant-1.5.4-src.tar.gz.asc licenses: http://repo.apache.org/apache/ant/1.5.4/licenses/LICENSE.txt PGP keys: http://repo.apache.org/apache/ant/1.5.4/KEYS
RE: Comments on URI Syntax
http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]ms gNo=266 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Sunday, 9 November 2003 7:28 PM To: [EMAIL PROTECTED] Subject: RE: Comments on URI Syntax Where is Tim's Layout? -- dIon Gillard, Multitask Consulting Blog: http://blogs.codehaus.org/people/dion/ Pub Key:http://blogs.codehaus.org/people/dion/public-key.asc Noel J. Bergman [EMAIL PROTECTED] wrote on 09/11/2003 06:22:51 PM: Jason, I think that Tim's ideas were pretty well-thought out and reflect a workable consensus. The changes you are making to his ideas, if I read the correctly, are to mandate a couple of things that he did not rule out, but permitted to remain optional. Having them as optional does not strike me as a problem. Best practices can always suggest that optional elements be used, and we'll discover in practice how broadly the rule(s) should apply. We should make sure that folks like William Rowe and others who have commented on the repository structure lately take a look at, and provide feedback on, Tim's layout. --- Noel
RE: Comments on URI Syntax
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] From the requirements at http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Requirements: ASF Repository shall ... allow browsing and downloading of artifacts by humans via normal web browser. Requiring a version to be part of the artifact file name when the artifact is only useful to end users (e.g README), reduces clarity. But it does increase usability sometimes. README for which version? An example: http://repo.apache.org/apache/commons-dbcp/1.1/README The README is for version 1.1 of commons-dbcp. -Tim
RE: URI/URL Syntax -- little nits to be aware of
From: Stephen McConnell [mailto:[EMAIL PROTECTED] Tim Anderson wrote: I take the view that everything in the repository is an artifact. Tools can exclude the artifacts they don't need - there can't be any language agnostic support for this, without adding metadata. Tim: How do you address something like the following: http://www.ibiblio.org/maven/ant/jars/ant-1.5.jar http://www.ibiblio.org/maven/ant/jars/ant-1.5.jar.md5 I don't see the md5 file as an artifact - instead I consider it to be meta data about the jar artifact. The md5 file is an artifact. Its meta data for the jar, for those tools that understand it. -Tim PS - is anyone else having problems with this list? I never received my original response to this.
