catalog for optional feature and layout discovery
Hello, After reading a few of the conversations on the list I have come up with some ideas. First Jason mentioned the use of decorations to embellish the functionality of the repository. After some thought I will have to agree even though I did not at first. Keeping the repository simple is a good thing. Now there were discussions with Tim regarding the layout of the repository and how the URL was to be designed. Then Steve talked about using meta data (the notion of associating attributes/properties with artifacts). I think this can be used as meta data but that's one application of it. I think these attributes can be used for just about any application. So how do we enable a flexible repository that allows clients to discover the optional services/features implemented as decoration on the repository? Also how do we enable any kind of regular repository structure so we do not have to restrict the layout of the repository? If the repository published a simple manifest or catalog of sorts in a fixed location so client APIs (in any language) can access it, then we can use this to store the following: 1). List of optional decorations offered by the repository 2). List of directory structure and naming rules 3). Miscellaneous seed information required to negotiate with the repo 3 is just filler cuz I did not want to have just two points ;-). Ok the list of decorative services, features or what have you describe the optional functionality offered by the repository. One such feature may be the use of attributes for associating relational information with a repo artifact. The potential features are limited only by the imagination. Just the other day I was using xdelta (the binary diff tool) and it occured to me that the repository could offer xdelta generation on the fly for applications that want to upgrade but use diffs instead of pulling down massive binaries. This is just an idea and its here to show how pleasantly carried away we can get with decorations so don't interpret me as trying to add it to the repository ;-). The point is with this catalog we can publish these features - the means to do that can be debates but we should not spend time on debating what features to add. At the end of the day this is left to the discretion of the user. We just need to allow for all the possibilities. Now in terms of #2 we can make the structure of the repository and the naming conventions used user definable by including naming and structure rules for the directory layout of the repository. These rules tell clients how to navigate the repository to get at the artifacts they may be interested in. And the published decorations tell the client how it can take advantage of optional services offered by the repository. I'm also trying to concieve of how client API's can selectively leverage these feature using plugable repository feature extensions. Also trying to get a feel for what this might look like and the use cases. These are just some ideas that have sparked in my head as a consequence of the recent activity on the list. Thought I'd put them out to be discussed. Thoughts? Comments? Alex
Re: RE: Scope/Phasing
> [snip] > > I don't intend to inflame anyone so please don't take it in the wrong > way > > but the Maven PMC sounds like a closed society to me. The maven-new > details > > > > are hidden pending some PMC conclusions. The Wagon details are hidden > > as well again pending PMC conclusions. There just seems to me to be an > > extreme amount of caution/secrecy with regard to involving the community > > > which counteracts the benefits of having one in the first place. > [snip] > > Can we please keep conjecture and flame-bait to private email? Dion, this was certainly not my intention and I appologize for comming across that way - that was the reason for the disclaimer at the begining. Sincerely, Alex
Re: Re: Merging repository discussions
> I have been out of commission for the better part of this year but > Plexus was slated to integrate Maven's repositories directly. Of late > I've been discussing this with the Loom folk and ~10 people who have > access to the Plexus-based version of Maven. Yes, I remember checking out maven-new from the maven cvs repository. If you remember, you told me of its eventual move to Codehaus. So it's new home is there I take it? Is there a way I can take a look at it? Also I'm not thinking of the repository as only part of a component container or just a Maven thing. I'm concerned with creating a single free repository. All the folks out there across projects can use it including Loom and Plexus at Codehaus. It should not matter what project uses it. The key is to make it a standard and not have a dozen implementations out there causing incompatibilities. > One of the the first tools to be released is something called Wagon > which Michal has been refactoring since the first versions of the > artifact handling mechanisms were checked into the maven-new repository. > It's Michal's baby and I will definitely encourage him to join this > list. Wagon was at first dependent on Plexus but that has now changed to > be more PicoComponent-esque whereby it can be embedded or used in any > Java tool or application. Michal has actually endeavoured to make Wagon > work with Loom as Peter Donald will most likely get to using Wagon in > Loom before I get to using it in Plexus. So Wagon is a repository API implemented at Codehaus? > Right, the Component Software nirvanna :-) If you haven't read Component > Software by Syperski I highly recommend it. There are also a plethora of > papers at citeseer on the topic of confederated repositories of > components so really just artifacts in essence. It's always nice to dream ;-). However outcome is not my primary goal. I just want a standard repository API for all repo aware apps to use. I also want to explore the use of directories coupled with this idea. > > > The ideas being discussed revolve around the notion of an attribute > > database to be used to centrally manage project related artifact > > attributes. This way the repository becomes a queriable artifact directory > > as well as a artifact store. > > I think as a decoration many things can be built on the notion of the Yes the decorator pattern is what I was inferring to - good catch. What about putting the POM and potentially other related app specific info extending the POM and keeping it allin a queriable centralized directory? > repository. There is certainly no reason that hinting types of artifacts > can't be introduced in order to enable the construction of large > metadata repositories for querying. Perhaps we want to get away from having properties files or XML files floating around just to mark up the repo with metadata. Even thought this can server as a make shift solution for now. We need to manage the relational info in a form that can be indexed and queried not one that requres it to be snarfed down and parsed et. cetera. Without a database solution the repo may become very difficult to manage with several metadata files as it grows. > > > Are these ideas appealing to the Maven community? > > Most certainly and is definitely in sync with the component form that > Maven will start to take in the next couple of months. That's great to hear. Is this maven-new you are referring to? Is maven-new going to come back to Apache? I took your advice and chilled out rather than trying to componentize maven. I'm glad to see you've gotten so far with it. Thanks, Alex
Fwd: Re: [part due] What is an ArtifactDatabase?
Niclas, Very sound advice and keen observations ... you're right! Time will tell and we need to start making some baby steps in the right direction for now. I think involvement with the Maven people in defining what a repository is for one single Apache repository is most important first. Extention of this repository with artifact attributes to enable the application based characterization of artifacts is the next step. The POM can be stored there and so can other application specific deployment attributes if the Maven people want to go there. The nice thing is if done right we can query for these artifacts based on their attributes to build a very powerful base to all platforms. The ideas to come out of Maven and Merlin regarding a repository and its use are very powerful. Maven invented it and Merlin showed us how to use it to manage an environment. Both are very powerful innovations. >From what I can see even Loom at jContainer is following that same route and >this proves the fact that a connection exists. The biggest danger today is >the potential for community fragmentation. Apache does not need two copies of >the same thing. The technical details will come together but let's do it right as one community. I think this is where the [EMAIL PROTECTED] list is going. In the words of that great American hero, G.I. Joe, "that's half the battle." Alex > >>From: Niclas Hedhman <[EMAIL PROTECTED]> > >>Date: 2003/11/06 Thu PM 09:29:42 EST > >>To: "Avalon Developers List" > >>Subject: Re: Repository, Bootstrapping and Embedding > >> > >>On Friday 07 November 2003 05:12, Stephen McConnell wrote: > >> > >> > I believe apps are artifacts that can be kept in the repo with its > dependencies. There are no limitations. Why treat application artifacts > any differently? > > > >>I basically agree. And, yes, more "try and see" is probably required. > >> > >> > >> > >>>I'm mainly thinking here about the stuff that Niclas Hedhman has been > >>>talking about. Personally I think its a different viewpoint on the > >>>component model and I'm still figuring out what that viewpoint > >>>encompasses and what its relationship is to existing content and service. > >>> > >>> > >>Basically, > >>almost ever single application needs something beyond "code" and > >>"resources" ( > >>resource as in ClassLoader.getResourceAsStream() ). > >> > >>At _application_ level, we have bunches of stuff that is pretty much > >>common, > >>for instance; > >> > >>1. Help system > >>2. Installation instructions. > >>3. Temporary storage. > >>4. Configuration files. > >> > >>but no matter what list I produce, someone will say "and ...", so for the > >>moment, we just accept "stuff". > >> > >>Looking at the capability of installers, it is evident that there is a > >>great > >>deal of requirement surrounding "stuff", may it be due to platform, > >>language, > >>installation choices, or a dozen other conditions and combinations between > >>conditions. I think you get the picture. > >> > >>NOW, bringing this to our scenario; > >>Often components are aware of many "conditions" in the deployment scenario, > >>and should (this is something new from head) expose not only the "stuff" > >>(artifacts) but the conditions for each such artifact. > >>How this can be done? No clue at the moment, but FSM and/or scripting comes > >>to > >>mind. > >> > >>At the far end, one could imagine that a set of conditions are given, and > >>the > >>application self-assemble the "stuff" into a running application. Why not? > >>What would that do to server (or client for that matter) management? > >> > >>We have a long way ahead, but even the longest journeys start with small > >>steps. > >> > >> > >>Niclas
Re: RE: Proposals
> > Well, let's make sure that we get input from the httpd release folks before > we re-design the layout of the library. I just want to make sure that we > have a consensus across projects that everyone can live with, not just Java > projects, and which works for the full spectrum of projects. > > --- Noel I totally second that. The repository concept should support the entire gambit of artifacts that can be generated. It must be language and application neutral hence very extensible. There should be some caution when deciding upon a rigid repository structure that must have the artifact type as a path component. Something tells me this could lead to trouble. On a side note: Taging artifacts using attributes can help acheive this as well. It's another potential tool that could be very liberating to those designing the repository and its conventions. I would try to keep the file structure very generic while using artifact attributes and some queriable engine to ask for the right kinds of artifacts. Both webdav/deltaV and directories can play a role here. As you know you can associate properties/attributes with artifacts using webdav. You can also acheive this by using a directory as the relational engine with a webserver as the artifact/content store. Nice thing is, you can wrap the JNDI around it all too and switch URL schemes to do different things: use LDAP for relational queries on attributes and use http/ftp for content retrieval. The neat thing is we can use the protocol that best suites the activity. My $0.02 Alex
Merging repository discussions
Hi Jason, > Just a note to any Maven developers who are interested in discussing the > format of the future repository layout for Apache are welcome to join in > at [EMAIL PROTECTED] I think we have lots to offer in this area so > if you have some pop into the discussions. Very cool you have good timing. Have you had a chance to look at some of the discussions on the avalon-dev list regarding an API for building repository aware Avalon applications. It would be nice to merge our discussions and these topics together as one community rather than fragmenting it. For starters here are links to some of those related threads: http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]&msgNo=22035 http://nagoya.apache.org/eyebrowse/[EMAIL PROTECTED]&msgNo=22038 I separated it into two parts as two trails to track each peice separately. I may have gotten too creative here ;-). Basically we're shooting for having a means to associate attributes with artifacts within the repository and have a generic API for repository aware and/or repository bootstrapping applications. The ideas being discussed revolve around the notion of an attribute database to be used to centrally manage project related artifact attributes. This way the repository becomes a queriable artifact directory as well as a artifact store. Are these ideas appealing to the Maven community? What can we do to consolidate our efforts? Sincerely, Alex Karasulu
