Re: RE: Scope/Phasing
[snip] I don't intend to inflame anyone so please don't take it in the wrong way but the Maven PMC sounds like a closed society to me. The maven-new details are hidden pending some PMC conclusions. The Wagon details are hidden as well again pending PMC conclusions. There just seems to me to be an extreme amount of caution/secrecy with regard to involving the community which counteracts the benefits of having one in the first place. [snip] Can we please keep conjecture and flame-bait to private email? Dion, this was certainly not my intention and I appologize for comming across that way - that was the reason for the disclaimer at the begining. Sincerely, Alex
Re: [Possible Incubation] Apache Repo
Roy T. Fielding wrote: Small note - some of the participants on the [EMAIL PROTECTED] are discussing the actual requirements - which from my (and other) point(s) of view go beyond a file-system http protocol cut-and-dried implementation solution. Some consider this area to be much more than an HTTP download handler. In fact - if the scope of a repository model were limited to that then would would be missing a really big opportunity to do this in a way is of real value to multiple projects. Yes - you can assume some simplistic models down low - but hopefully this is not just about plumbing but also about addressing the requirements across different abstractions that will ultimately ensure that semantic assumptions are consistent across multiple repository-enabled applications. The requirement is that ASF-owned software be distributed in an efficient (for our costs), reliable (for our maintainers), and user-friendly way. I would add one more requirement to above statement - namely machine-friendly. There is an emerging requirement for application driven downloading that has the potential to significantly exceed the classic browser driven requirements that the ASF is addressing today. This has a direct impact on ASF costs, reliability, and utility. Feel free to consider any number of designs that may accomplish that requirement, but don't mistake a design opinion for a requirement. In particular, do not under any circumstances hold up implementation of the repository in pursuit of perfection -- a current implementation can be replaced at any time we see fit. Just trying to clarify, is this correct? I hope not - it would not meet Avalon project requirements relative to repository-aware applications. I much prefer Roy's terminology a single storage facility to look like a repository with multiple interfaces. Roy's statement *does* encompass the scope of requirements that I see as relevant. Hello? Avalon project requirements do not encompass repository needs, and certainly do not define them. I don't think that I have suggested this. Avalon requirements deal with at least three layers of abstraction with respect to server side facilities. At the lowest level these requirements are rather close to the requirements you have outlined above. As far as second and third level requirements are concerned - these place functional requirements on the respective underlying facilities. My objective are rather simple - the basic facility should be a platform on which higher level facilities can be established without resorting to inefficiencies or workarounds. I should also point out that Avalon is not alone is this respect. Other projects are evolving towards repository-awareness. Identifying and collecting those requirements (many of which are project/application specific) are central to the delivery of a basic repository that is extendible to meet the needs of a significant usage model (in terms of ASF near-term impact). Avalon users might prefer a given interface to the repository, which I assume the Avalon community is more than capable of defining and developing on their own. Clearly yes. The work within Avalon has *done* exactly this and as a result - issues with current approaches have discovered and near term requirements have been identified. These aspects are the things that Avalon has to contribute to general model. I'll restate my earlier comment - the simplistic http + file layout assumptions do not meet Avalon project requirements relative to repository-aware applications. In fact this is probably the key point - is this initiative about dealing with ASF download requirements, or, does this initiative address the emerging and potentially much larger repository aware application scenario? If this is simply about safe downloading then my assertions are clearly inappropriate. The commonality that is required by repository is determining the easiest way for all of our projects to provide artifacts and their authenticity-proving signatures such that the general public can get what they want, when they want, at a minimum cost to us. The tools that retrieve from the repository are not within its scope. Just to clarify - I completely agree that the development of tools *using* a repository are out of scope. However - these very same tools (at least those that exist today) are in my opinion *totally within scope* in terms of establishing near term requirements on the ASF and the repository solutions the ASF provides. Anything (and I do mean anything) beyond that is a software project that the ASF has not authorized, and certainly won't be developed here unless it is within a PMC. People are certainly welcome to propose such a project on incubator, but that is not the repository project. Repository is a task to be accomplished! Relative to present or short-term needs? Please note that this is not an argumentative question. It is a question concerning
Re: [Possible Incubation] Apache Repo
On Sat, 2003-11-08 at 21:47, Stephen McConnell wrote: Roy T. Fielding wrote: Small note - some of the participants on the [EMAIL PROTECTED] are discussing the actual requirements - which from my (and other) point(s) of view go beyond a file-system http protocol cut-and-dried implementation solution. Some consider this area to be much more than an HTTP download handler. In fact - if the scope of a repository model were limited to that then would would be missing a really big opportunity to do this in a way is of real value to multiple projects. Yes - you can assume some simplistic models down low - but hopefully this is not just about plumbing but also about addressing the requirements across different abstractions that will ultimately ensure that semantic assumptions are consistent across multiple repository-enabled applications. The requirement is that ASF-owned software be distributed in an efficient (for our costs), reliable (for our maintainers), and user-friendly way. I would add one more requirement to above statement - namely machine-friendly. There is an emerging requirement for application driven downloading that has the potential to significantly exceed the classic browser driven requirements that the ASF is addressing today. This has a direct impact on ASF costs, reliability, and utility. I have challenged you to give me a scenerio that I can't satisfy with something like the current Maven repository. Instead you drone on ad nauseum about the theoretical. Let's have a concrete example. I don't want to read your email essays and I don't want to listen to you foisting your solutions on us under the guise of trying to satisfy requirements i.e. we don't need Merlin+LDAP for the base-line repository so give it a rest because that's exactly what the initial messages sounded like to me. I clearly expressed and stated that any required information can be stored in the repository as ancillary artifacts which can be processed in application specific ways. The goal is to remain simple while being able satisfy all future requirements. I am willing to show you over course of next week with a simple example using Plexus that I can use what exists to create a Repository Aware Application. If you can actually provide a concrete use case that isn't wildly far fetched and shows that something akin to the Maven repository doesn't satisfy your requirements then I will listen to you. I am not willing to accept your long winded tomes of techno babble on the theoretical abstractions of the all singing, all dancing repository. As Roy state the repository can evolve and starting simple is definitely the right way to start. One of my favourite quotes is from Patterns for Evolving Frameworks by Don Roberts and Ralph Johnson in PLoP volume 3: quote People develop abstractions by generalizing from concrete examples. Every attempt to determine the correct abstractions on paper without actually developing a running system is doomed to failure. No one is that smart. A framework is a resuable design, so you develop it by looking at the things it is supposed to be designed of. The more examples you look at, the more general your framework will be. /quote So please lets not stray into the abstract gobblygook and stay focused on concrete examples. -- jvz. Jason van Zyl [EMAIL PROTECTED] http://tambora.zenplex.org In short, man creates for himself a new religion of a rational and technical order to justify his work and to be justified in it. -- Jacques Ellul, The Technological Society
Re: [Possible Incubation] Apache Repo
On Sun, 2003-11-09 at 00:35, Stephen McConnell wrote: Jason: I must confess that I am intrigued by your approach to collaboration! That's because you're at least as deficient as I am in the realm of collaboration. Neither you or I are any great shining examples of an ideal collaborator. You'll have to bear with me while I try to make ammends. I will try to bear with you. Following your assertion that I am droning on ad nauseum about the theoretical - combined with your follow-up with a request for specifics, I am obliged to conlude that your post concerning nauseum was simply an incorrect assertion on you part, or, your post is intended to establish an entry in a drone catalogue for the benefit of other like minded persons. While I appreciate that you are very busy working on initiative outside the scope and control of the ASF - I would *really* appreciate your guidance on this subject. Did you simply make a mistake, or are you intentions to propergate that mistake? I can't glean a single coherent sentiment in this message. But it is shorter, so we're getting somewhere. Stephen. -- jvz. Jason van Zyl [EMAIL PROTECTED] http://tambora.zenplex.org In short, man creates for himself a new religion of a rational and technical order to justify his work and to be justified in it. -- Jacques Ellul, The Technological Society
Re: [Possible Incubation] Apache Repo
It is usually unwise to mix insults with requests. However, the point of collaboration is not to obtain the civility of a collegial discussion over tea; the point is to accomplish the task. Continual discussion of issues that are not relevant to the task being collaborated upon is not collaboration -- it is being a pain in the ass. Folks, you don't have to respond to every comment, and especially not this one. If it isn't relevant, you'll get more done by just ignoring it and moving on. Roy
RE: [Possible Incubation] Apache Repo
Neither you or I are any great shining examples of an ideal collaborator. You'll have to bear with me while I try to make ammends. I will try to bear with you. Let take this at face value: someone admitting to faults, pledging to improve, asking for allowances while making even more mistakes in the interim, and offering the same in return. Few people cannot stand to improve their inter-personal communication. --- Noel
Re: Comments on URI Syntax
On Sun, 2003-11-09 at 01:41, Tim Anderson wrote: I have a few comments on the proposed URI Syntax, from http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/URISyntax. quote Compromise URI http://host/project/version/artifact-[version;].ext For example http://repo.apache.org/org-apache-ant/1.5.1/ant-1.5.1.jar http://repo.apache.org/org-apache-ant/1.5.1/ant-testutil-1.5.1.jar http://repo.apache.org/org-apache-ant/1.5.1/LICENSE.txt /quote Having the version in the path certainly doesn't hurt readability and it definitely will make the structure more navigable as it keeps a massive number of artifacts from piling up in one place. And of course you have the by product of faster indexing and quicker hits by the file system and if transfered to another storage mechanism the reduction of the bit per bucket can only be a good thing. Simple ideas are good ones. Good idea! +1 1. This should be written as: http://host/project/version/artifact[-version].ext as the '-' is only required if the version is present. I think the version should always be present. People will use the repository directly and I think that's ok so you if you copy an artifact somewhere by mistake it is always nice to have as much information as possible so making the version optional I don't think is a great idea. 2. Does '.ext' need to be mandatory? I'm assuming that a project is free to deploy whatever it likes into the repository, not all of which should be forced to have extensions (e.g, Unix shell scripts, README files). I don't think they need to be, but I haven't thought about that one much. We have presumed so in Maven because artifacts have generally been archives but there's no reason there has to be an extension. 3. project is too limiting as it is required to be globally unique, resulting in unwieldy names like: jakarta-commons-logging or org.apache.jakarta.commons.logging I would prefer to see this split into: organisation/product where: . organisation is arbitrary, but globally unique. It could be the domain name, e.g sun.com, the reverse domain name e.g org.apache, or simply the name of the organisation, e.g oracle. . project is the project name, unique within the organisation, e.g: jndi, ldap, commons-logging etc. What we've discussed in Maven-land is using something like a groupId which might look like: org.apache.maven and actually split on the dot to make a directory. So basically organization by FQDN. Something which would also make indexing easier in filesystems and I think makes it easier to navigate by a person. 4. artifact is too limiting as it groups all artifacts for one project in a single directory. For projects producing large no.s of artifacts, it becomes difficult for users to browse. The httpd project for example produces multiple binaries, for different platforms (see http://www.apache.org/dist/httpd/) The requirement that -version is prepended to the artifact name also doesn't support language specific requirements. I would prefer to see this split into: [type/][platform/]artifact where: . type is optional and arbitrary, determined by the deployment tool. E.g: jars, binaries, docs etc. . platform is optional and arbitrary, determined by the deployment tool. Having the type I think is good and has worked for Maven. +1 . artifact is determined by the deployment tool, and includes: . the artifact name . the version (optional) . the platform (optional) . the extension (optional) . the type (optional) E.g, -src, -bin etc. This allows the repository to cater for language-specific deployment tools. For java, artifact could be: artifact-name[-version][-type][.ext] E.g: . LICENSE.txt . ant-1.5.1.jar . ant-1.5.1-src.zip For C binaries, artifact could be: artifact-name-version-platform.ext E.g: . httpd-2.0.43-sparc-sun-solaris2.8.tar.gz In summary, I think the URI should be of the form: http://host/organisation/project/version/[type/][platform/]arti fact, For organization I would suggest a groupId where most projects would use their FQDN and split on the dot for directory structure. Also the manditory use of a version in the artifact name as even in your example below the LICENSE.txt could potentially change from one release to another and you wouldn't want to copy one version over another by mistake and distribute it. An Unlikely example yes, but possible if the version is not in the artifact itself. I also think the type should be required. So my attempt for further refinement would be this: http://host/groupId/project/version/type/[platform/]/artifact-version[.ext] with the format of artifact determined by the deployment tool. For example: http://repo.apache.org/apache/ant/1.5.4/LICENSE.txt
Re: [Possible Incubation] Apache Repo
Roy T. Fielding wrote: It is about safe downloading of dependencies from a virtual repository that extends across mirrored systems on a heterogeneous, multi-organizational network. The underlying infrastructure is going to be file based because it will be replicated with rsync. I sure as hell don't want anything more complex than that at the base level. Building interfaces on top of that is trivial and not in the least impacted in terms of efficiency -- there are no methods of storing large objects more efficient than a modern filesystem. Start simple and layer on top of that. Roy: I am completely aware of all of the above. What I asked was the question concerning scope relative to the ASF. The assumtions within the filesystem have a direct impact on the economics aspects and utility of deployment. Are you (representing the board) focussing on immediate or short term requirements? [ ] immediate [ ] short term Which one it is ? Stephen. -- Stephen J. McConnell mailto:[EMAIL PROTECTED]
Re: [Possible Incubation] Apache Repo
Noel J. Bergman wrote: My idea of collaboration is something *totally* different. It sure can be once you get rid of anyone who doesn't agree with you. Neither of you has the most perfect record on collaboration. Fine. Please drop it and focus on the actual task to be addressed. Sniping at each other is not going to resolve the technical issues. Noel, it seems that despite this is not the first email of this kind, the thing keeps dragging on :-/ Hence I strongly ask everyone on this list to completely ignore and snip comments and emails that are not in the scope of this discussion. *sigh* -- Nicola Ken Barozzi [EMAIL PROTECTED] - verba volant, scripta manent - (discussions get forgotten, just code remains) -
RE: Comments on URI Syntax
Where is Tim's Layout? -- dIon Gillard, Multitask Consulting Blog: http://blogs.codehaus.org/people/dion/ Pub Key:http://blogs.codehaus.org/people/dion/public-key.asc Noel J. Bergman [EMAIL PROTECTED] wrote on 09/11/2003 06:22:51 PM: Jason, I think that Tim's ideas were pretty well-thought out and reflect a workable consensus. The changes you are making to his ideas, if I read the correctly, are to mandate a couple of things that he did not rule out, but permitted to remain optional. Having them as optional does not strike me as a problem. Best practices can always suggest that optional elements be used, and we'll discover in practice how broadly the rule(s) should apply. We should make sure that folks like William Rowe and others who have commented on the repository structure lately take a look at, and provide feedback on, Tim's layout. --- Noel
RE: Comments on URI Syntax
http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]ms gNo=266 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Sunday, 9 November 2003 7:28 PM To: [EMAIL PROTECTED] Subject: RE: Comments on URI Syntax Where is Tim's Layout? -- dIon Gillard, Multitask Consulting Blog: http://blogs.codehaus.org/people/dion/ Pub Key:http://blogs.codehaus.org/people/dion/public-key.asc Noel J. Bergman [EMAIL PROTECTED] wrote on 09/11/2003 06:22:51 PM: Jason, I think that Tim's ideas were pretty well-thought out and reflect a workable consensus. The changes you are making to his ideas, if I read the correctly, are to mandate a couple of things that he did not rule out, but permitted to remain optional. Having them as optional does not strike me as a problem. Best practices can always suggest that optional elements be used, and we'll discover in practice how broadly the rule(s) should apply. We should make sure that folks like William Rowe and others who have commented on the repository structure lately take a look at, and provide feedback on, Tim's layout. --- Noel
RE: Comments on URI Syntax
On Sun, 2003-11-09 at 02:22, Noel J. Bergman wrote: Jason, I think that Tim's ideas were pretty well-thought out and reflect a workable consensus. The changes you are making to his ideas, if I read the correctly, are to mandate a couple of things that he did not rule out, but permitted to remain optional. Having them as optional does not strike me as a problem. Best practices can always suggest that optional elements be used, and we'll discover in practice how broadly the rule(s) should apply. We should make sure that folks like William Rowe and others who have commented on the repository structure lately take a look at, and provide feedback on, Tim's layout. If someone else wants to act as secretary that's cool but I wanted to try and collect the ideas expressed so far in a small document. I'm not a huge fan of the wiki. If someone else has started some coherent documentation I won't step on anyone's toes but I'll help codify any existing docs there are. --- Noel -- jvz. Jason van Zyl [EMAIL PROTECTED] http://tambora.zenplex.org In short, man creates for himself a new religion of a rational and technical order to justify his work and to be justified in it. -- Jacques Ellul, The Technological Society
RE: Proposals
peter royal wrote: On Nov 7, 2003, at 5:37 PM, Stephen McConnell wrote: Searching is certainly on the my agenda. We need it for IDE related development, repository management, and intelligent query based artifact aquisition (and HTTP in this context is not an ideal solution). But that's a layer that would sit on top of the repository, right? That is my thought as well. First we need a layout, of which I like Tim's the best so far. On top of that we can add meta-data, and then tools that use the meta-data. For dumb servers, we can have tools that create static meta-data, which would also be more efficient. Smart servers might support dynamically creating, or simply reformatting, meta-data for different clients, e.g., Maven, Avalon or JNLP, from a single source, but otherwise we just store meta-data statically. The real smarts would be on the client side, where the metadata would be most often used. But since the meta-data would be present at a URL within the layout, I don't think that we need to deal with it right now. Just the layout, itself, which can be presented to infrastructure and the projects. Once we've a layout, existing tools can make use of it, and people can start discussing the next layer, which would be meta-data. At least that's my view. :-) --- Noel
RE: URI/URL Syntax -- little nits to be aware of
1) I have angst over the version in the URI (as a 'directory') only because of the likely need for symbolic links for 'latest'. I think this is a burden on publishing tools, and leads to errors (what if two tool were publishing at same time, can symbolic links be created remotely, etc.) That said, read on... I am also -1 for separate directories per version 2) Version in the filename has it's issues also -- e.g. jakarta-servlet-api-4-1.1 -- is that version 4 or 1? (It is 1.1 of jakarta-servlet-api-4.) 3) Some folks like to use _ not - for such separators. Some also like to use periods in resource names. Both make resource parsing hard. Why do you want to parse strings which describe versions? If you want to impose on anyone how they should version their artifacts? There is zero % of chance for that. Version can be anything like: build-994 or 1.2.alpha-5 or 4-1.1 and we should just decide where string which describe a version is included in the URL and stop there. Michal
Re: Comments on URI Syntax
[EMAIL PROTECTED] wrote: From the requirements at http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Requirements: ASF Repository shall ... allow browsing and downloading of artifacts by humans via normal web browser. Requiring a version to be part of the artifact file name when the artifact is only useful to end users (e.g README), reduces clarity. But it does increase usability sometimes. README for which version? Good point! Is not a README a feature of an artifact? Stephen. -- Stephen J. McConnell mailto:[EMAIL PROTECTED]
Re: URI/URL Syntax -- little nits to be aware of
Tim Anderson wrote: I take the view that everything in the repository is an artifact. Tools can exclude the artifacts they don't need - there can't be any language agnostic support for this, without adding metadata. Tim: How do you address something like the following: http://www.ibiblio.org/maven/ant/jars/ant-1.5.jar http://www.ibiblio.org/maven/ant/jars/ant-1.5.jar.md5 I don't see the md5 file as an artifact - instead I consider it to be meta data about the jar artifact. Stephen. -- Stephen J. McConnell mailto:[EMAIL PROTECTED]
RE: Comments on URI Syntax
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] From the requirements at http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Requirements: ASF Repository shall ... allow browsing and downloading of artifacts by humans via normal web browser. Requiring a version to be part of the artifact file name when the artifact is only useful to end users (e.g README), reduces clarity. But it does increase usability sometimes. README for which version? An example: http://repo.apache.org/apache/commons-dbcp/1.1/README The README is for version 1.1 of commons-dbcp. -Tim
Re: Comments on URI Syntax
Tim Anderson wrote: From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] From the requirements at http://nagoya.apache.org/wiki/apachewiki.cgi?ASFRepository/Requirements: ASF Repository shall ... allow browsing and downloading of artifacts by humans via normal web browser. Requiring a version to be part of the artifact file name when the artifact is only useful to end users (e.g README), reduces clarity. But it does increase usability sometimes. README for which version? An example: http://repo.apache.org/apache/commons-dbcp/1.1/README The README is for version 1.1 of commons-dbcp. By implication - the README is not an artifact but a feature of a version. Is that a reasonable conclusion? Stephen. -- Stephen J. McConnell mailto:[EMAIL PROTECTED]
RE: URI/URL Syntax -- little nits to be aware of
From: Stephen McConnell [mailto:[EMAIL PROTECTED] Tim Anderson wrote: I take the view that everything in the repository is an artifact. Tools can exclude the artifacts they don't need - there can't be any language agnostic support for this, without adding metadata. Tim: How do you address something like the following: http://www.ibiblio.org/maven/ant/jars/ant-1.5.jar http://www.ibiblio.org/maven/ant/jars/ant-1.5.jar.md5 I don't see the md5 file as an artifact - instead I consider it to be meta data about the jar artifact. The md5 file is an artifact. Its meta data for the jar, for those tools that understand it. -Tim PS - is anyone else having problems with this list? I never received my original response to this.
