Gregory Kick wrote:
On 8/22/07, Jason van Zyl <[EMAIL PROTECTED]> wrote:
On 22 Aug 07, at 8:24 AM 22 Aug 07, Gregory Kick wrote:
Ok, so I'm coming to the conversation slightly late in the game, but
all of this discussion reminds me of something that's been in the back
of my mind for sometime. With all of the effort put forth in the
various repository managers, front-ends, proxies, mirrors, etc. why
doesn't the whole repository mechanism get a bit more intelligence in
order to solve some of these basic problems?
Exactly. In short they are. Trying to do anything reasonably
intelligent with an unintelligent mechanism like a simple web server
is very hard.
AFAIK (and correct me if i'm wrong), http is almost exclusively for
fetching artifacts, so a basic RESTful service could go a long way
towards standardizing some of the things that seem to be pretty
commonly needed. For example:
- authentication/authorization, which is what would solve the problem
discussed in this thread
- basic listing (e.g. what artifacts exist for a given group)
- search (depending on how much logic you really want to give to the
repository)
- standard, machine-readable output. no more scraping apache
directory listings...
- consistent, accurate, real-time metadata
- server-side deployment validation/atomic deploys
The list could probably go on, but those are just a few quick ideas.
It just seems like there's functionality that I would like to be able
to expect from a repository, but don't get because it's not really
anything more than a big directory. Then, all of the more
sophisticated repository management tools can spend time focusing on
more advanced features like proxying, etc.
Now, let me take a moment to address some the concerns that I'm sure
are going to come out of this.
I'm sure that someone will object to having to run a repository
process.
I don't believe this will be the case if it is very simple, and not
very onerous. If starting from scratch setting up a little Java
application should be easier then setting up Apache. This is the goal
I would think, at least for things that I'm working on for clients.
Well, aside from the file wagon (which doesn't make for a
very good remote repository :-)
It does if you use a shared file system, many people actually use
this. And simple Window file perms work here to protect it.
How do you deal with concurrent writes?
That's a nightmare problem with the local repository anyway.
One team of ours had to scrap switching to Maven as it filled the disk
on their shared homes drive... and if they changed the system default to
use a common local repository they kept on getting broken builds from
two people installing the same (usually snapshot) artifact at the same time!
It's more of a problem for continuous integration servers that run
builds in parallel.
However the concurrent write problem is solved for remote repositories,
please solve it for local repositories as well!
, all of them require some sort of
server somewhere. Granted, a lot of people have sshd or httpd or ftpd
running so it's not an extra process, but there's still _something_
running on the other end no matter what. Also, I'd be pretty
surprised to find that a significant portion of the people running a
remote repository weren't running a servlet container already as well.
Plus, there's always jetty or winstone if you want a quick install.
No argument that in the very near future I don't think anyone will
use a Maven repository without a front-end application. Web servers
are just too stupid, security is annoying, you have no transactional
behaviour, and you have to expose the internal structure.
I'm also pretty sure that there's going to be some question of
backwards compatibility. Well, seeing that I'm suggesting a RESTful
service, it could have the exact same structure as what's being used
now. I'm not entirely sure I'd recommend that, but it _could_. If it
didn't you could always just add a new layout.
The repository managers will have an API, and can be exposed via REST
which will end up being one of the favored modes of operation. But I
see applications interacting directly with client side APIs to get
things done. Like Maven deployments, releases, searching, what have you.
This seems like it'd end up being problematic for a couple of reasons.
The first is that when you say that it _can_ be exposed via REST this
implies too much flexibility in what the resource URIs look like. To
provide a decent user experience from both the application and the
browser, there has to be some standardization for the rest api. Not
that the current layout doesn't have a standard interface, but the
format of the results for directory listings and the URIs for
additional functionality would end up being all over the place.
The other issue is that, from a developer perspective, I'd rather
there not be a standard java api. If the job of a repository is to
serve artifacts, that should be the only contract to which I'm bound
(especially if the maven code is still compiling with 1.4). Unless
there's a really compelling reason to support other transports, any
additional api just adds another layer. Of course, the maven-built RI
could add whatever api it wants, but I can't think of a scenario in
which you'd need to support its invocation from anything other than
the transport.
Now, what seems like it'd be the highest hurdle: the central
repository. The content would have to be migrated and it'd have to be
hosted. Of course, I don't really have a good answer for either.
Migration seemed to be a pain the first time, but at least the
community would have some experience... :-)
Nothing would need to be migrated, there's not reason that the call
that is currently made could not be adapted. With a simple web server
there are issues of migration, with an application then you just map
the URL to what the application knows and respond as the client
expects. Not actually having a real application on repositories is
limiting but we evolved in a normal way I think and this is the next
step for Maven.
But the case that Atlassian wants solved would be very simple with
something like Proximity even today. One simple access controller
that blocked the source jars for particular users and you're set.
So, that's it. Any thoughts?
On 8/22/07, James William Dumay <[EMAIL PROTECTED]> wrote:
Jason,
So why can't it all be in one repository? You have people who buy
your products with a non-source license and you give them access to
binaries from a Maven repository instead of an installer? Or you
have
updaters that use a Maven repository so you only need the binaries
for this? This last case I could understand.
We cannot have just one repository and give access to paying
customers
because we also have a community of plugin developers which rely on
various artifacts that are part of our products.
This would mean for every developer that would like to write a
plugin we
would have to give them access to the repository - this would include
our private sources.
For your own artifacts that you are producing why do they need to be
public at all? I ask because I do something similar and I don't
understand what it is you're actually trying to accomplish. This
aside it still doesn't change the fact Maven wasn't designed to have
the primary artifact separated from the secondaries and simply
causes
a discontinuity in tooling. This is currently how it works. For
anyone trying to debug their copy of an Atlassian product the Maven
IDE integration wouldn't work.
The jars that are released to public without their sources and
javadocs
are not something the developers building and modifying the products
would be wanting source and javadocs for anyway. They are there
simply
deployed so that they can build the source code that they are
licensed
to modify. Internally, our developers would be able to access the
primary artifacts, sources, javadocs and assemblies.
In your case would it not be easier to give people read-only access
to the SCM, they build it as you build it and have access to a
password protected repository that contains everything required for
building. I don't understand your need to separate it the binaries
from javadoc and sources.
Customers currently download the sources from the website, not
checking
them out from the repository. Having a password protected repository
would mean we would have to add more admin overhead to the Developer
Network team.
I believe a user should be able to configure their maven builds to
allow
the deployment of attached artifacts to a specified repository. Users
who would use this feature would have to be aware of the
consequences of
splitting these artifacts away from their primary artifacts.
James
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Gregory Kick
http://kickstyle.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder and PMC Chair, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
