Re: long project names & repositories
Brett Porter wrote: > the smartfrog solution is brute force unforgiving: you must declare > the SHA1 or MD5 value in a download Right... I'm sure users wanting security will put up with a certain level of pain. I'm still not sure how you securely publish the value initially (though this certainly prevents later tampering). I'd still like to think this through a little more. > The reason for sha1 emphasis is primarily that md5 is doomed: its too > easy to break, 2-5 years left in it before it is as trusted as DES. > Now, when it is finally broken, .rpm is the first juicy target. But > md5 is old; it would be good if maven2 had .sha1 everywhere right from > the outset. Sadly, SHA-1 is also "doomed"...See, e.g. http://www.schneier.com/blog/archives/2005/02/cryptanalysis_o.html Assuming goal is just checksum, however, this is np. Also, SHA-1 still definitely has more runway than MD5. Quoting from Shneier, "Jon Callas, PGP's CTO, put it best: 'It's time to walk, but not run, to the fire exits. You don't see smoke, but the fire alarms have gone off.'" Phil Yes, I've always stated our md5s will be nothing more than checksum verification that your download wasn't corrupted, rather than a security feature. As for m2... no problem: http://jira.codehaus.org/browse/MNG-287 I'm pretty sure it's trivial to add. > yes. a legal string of valid names would be good, better than an > exclusion list (as there are so many characters in unicode land. For > x-platform support, that means lowercase ascii alphanumeric plus a few > separators for all of the different elements: > > [a-z0-9.-+_] > > The weakness here is that it doesnt address the wants/needs of the > non-ascii world., but since we are constructing HTTP URLs from the > strings, we need to use that subset, not just those chars that are > valid in filenames. Yep, I definitely agree. We have readable names in our project descriptors which should support non-ASCII. - Brett
Re: long project names & repositories
On Apr 11, 2005 5:27 PM, Brett Porter <[EMAIL PROTECTED]> wrote: > Sorry, I meant: http.proxyHost, http.proxyPort and counterparts as > standard names. > > - Brett well, you could do, but you need to work out 1. how to set up the options for every app you start on the command line 2. how to set up the options for every JAR file you run by double clicking. It can be done via our friend the registry in windows, but it isnt easy. Even then, it doesnt address the variability of laptop networking. In my past research, I had a lot of data on how laptops got used, and primarily it was networking, not power that mattered: http://iseran.com/Steve/notebook_slides.pdf http://www.hpl.hp.com/techreports/2000/HPL-2000-21.html So I went on to do a util that looked at system context and changed things, network, power, screen saver, etc. depending upon what it thought was happening http://www.hpl.hp.com/techreports/2001/HPL-2001-158.html dynamic adaption of IE proxy settings work, though only through IE restarts. I think if I'd do it again, I'd route everything through a localhost proxy which relayed stuff as approprite. -steve
Re: long project names & repositories
Sorry, I meant: http.proxyHost, http.proxyPort and counterparts as standard names. - Brett On Apr 12, 2005 2:09 AM, Steve Loughran <[EMAIL PROTECTED]> wrote: > On Apr 11, 2005 4:39 PM, Brett Porter <[EMAIL PROTECTED]> wrote: > > > If you needed that, couldn't the default system properties for proxies be > > used? > > > > 1. On windows, this is hidden in a bit of the registry the API keeps > from from you, plus the java.util.prefs stuff escapes stuff *wierdly*, > as if they didnt expect the registry APIs to be unicode or something. > > 2. Linux. >
Re: long project names & repositories
On Apr 11, 2005 4:39 PM, Brett Porter <[EMAIL PROTECTED]> wrote: > If you needed that, couldn't the default system properties for proxies be > used? > 1. On windows, this is hidden in a bit of the registry the API keeps from from you, plus the java.util.prefs stuff escapes stuff *wierdly*, as if they didnt expect the registry APIs to be unicode or something. 2. Linux.
Re: long project names & repositories
> you know, I wonder if we can't create a defacto place in the Java1.4 > preferences world for storing proxy information across apps. There is > something for applets, but nothing for apps in general. > > Imagine a System node "org.apache.jakarta.proxy" that would be read by > everything that cares about proxies -they could even implement a > listener to respond to dynamic changes, so that were I to write a > laptop utility that switched the proxy on and off, apps could get the > change. > If you needed that, couldn't the default system properties for proxies be used? - Brett
Re: long project names & repositories
On Apr 11, 2005 2:02 AM, Brett Porter <[EMAIL PROTECTED]> wrote: > > one problem I have for both systems is proxies; at work I cant go to > > remote http servers without proxy setup. What does maven do? > > wagon has a proxy configuration that it passes on to the JDK stuff as > system properties, or to httpclient as configuration depending on the > implementation you use. The proxy is configured via your main Maven > config file for your user/installation. you know, I wonder if we can't create a defacto place in the Java1.4 preferences world for storing proxy information across apps. There is something for applets, but nothing for apps in general. Imagine a System node "org.apache.jakarta.proxy" that would be read by everything that cares about proxies -they could even implement a listener to respond to dynamic changes, so that were I to write a laptop utility that switched the proxy on and off, apps could get the change.
Re: long project names & repositories
> There is no good solution here. Really. sure - I understand that. All I am trying to think of is something as secure as your proposal, but less intrusive to the user. Or at least a decent way to aid them with tools. Really, I've been in favour of using bouncycastle and PGP for a long time, so I guess the time has come to finally bite down and implement it. > Or team's security person doesnt believe in anything > other than clean build code, tagged CVS releases, with personal > signing. That's actually not too bad as they publish all that built stuff to the company repository and don't have to do any of this :) > The only reason I can get away with coding the maven lib > support is that he is away right now... :) > The best source of keys (both SHA1 and MD5) will be the PGP signed > announcements of releases. That puts PGP at the base of the trust > chain. but we cant automated PGP checks without bouncycastle on the > path. I realise this is not something that can be OOTB with Ant, but I'm sure security conscious folks would be happy to add it. For Maven, I definitely want to go down this path so we can make signing the release part of the deployment process too. Anyway, I'll let the list know if I or someone else in the team finally get to the point of moving on it. - Brett
Re: long project names & repositories
On Apr 11, 2005 2:02 PM, Brett Porter <[EMAIL PROTECTED]> wrote: > > the smartfrog solution is brute force unforgiving: you must declare > > the SHA1 or MD5 value in a download > > Right... I'm sure users wanting security will put up with a certain > level of pain. I'm still not sure how you securely publish the value > initially (though this certainly prevents later tampering). > > I'd still like to think this through a little more. There is no good solution here. Really. For example, say maven and apache include the public keys of the apache CA: how do you verify that the versions that ship with your IDE, or with the tomcat version that came with your app server havent been subverted. Similarly: how do you verify that the SHA1/MD5 key that you are putting in your template is the correct one, and not one that is from somewhere malicious. You need to trust someone, somewhere, and that is the weak point for attack. Or team's security person doesnt believe in anything other than clean build code, tagged CVS releases, with personal signing. The only reason I can get away with coding the maven lib support is that he is away right now... The best source of keys (both SHA1 and MD5) will be the PGP signed announcements of releases. That puts PGP at the base of the trust chain. but we cant automated PGP checks without bouncycastle on the path.
Re: long project names & repositories
> the smartfrog solution is brute force unforgiving: you must declare > the SHA1 or MD5 value in a download Right... I'm sure users wanting security will put up with a certain level of pain. I'm still not sure how you securely publish the value initially (though this certainly prevents later tampering). I'd still like to think this through a little more. > The reason for sha1 emphasis is primarily that md5 is doomed: its too > easy to break, 2-5 years left in it before it is as trusted as DES. > Now, when it is finally broken, .rpm is the first juicy target. But > md5 is old; it would be good if maven2 had .sha1 everywhere right from > the outset. Yes, I've always stated our md5s will be nothing more than checksum verification that your download wasn't corrupted, rather than a security feature. As for m2... no problem: http://jira.codehaus.org/browse/MNG-287 I'm pretty sure it's trivial to add. > yes. a legal string of valid names would be good, better than an > exclusion list (as there are so many characters in unicode land. For > x-platform support, that means lowercase ascii alphanumeric plus a few > separators for all of the different elements: > > [a-z0-9.-+_] > > The weakness here is that it doesnt address the wants/needs of the > non-ascii world., but since we are constructing HTTP URLs from the > strings, we need to use that subset, not just those chars that are > valid in filenames. Yep, I definitely agree. We have readable names in our project descriptors which should support non-ASCII. - Brett
Re: long project names & repositories
On Apr 11, 2005 12:51 PM, Brett Porter <[EMAIL PROTECTED]> wrote:
> (I'm assuming you meant to reply to all by the content - it happens
> frequently with other gmail users - sorry if I'm out of place
> repeating your message)
no, that's gmail for you.
>
> At the very least we should continue to swap notes, especially when it
> comes to implementing the security stuff so that we have one,
> consistent solution.
yes.
>
> I'm still thinking over the inclusion of the sha1 in the dependency
> definition. The reasoning seems sound, but something about it just
> gives that gut feeling that it isn't the best way to go - maybe it is
> still just the niggling feeling of too much work for the user.
the smartfrog solution is brute force unforgiving: you must declare
the SHA1 or MD5 value in a download
commons-logging-1.02 extends MavenArtifact {
project "jakarta.commons";
artifact "commons-logging";
version "1.02;
sha1 "4f..2b";
}
It means to switch you'd need to update both the version and the sha1
together, but that's ok. You cold declare a sub version
commons-logging-1.4 extends commons-logging-1.02 {
version "1.4";
sha1 "45facb...";
}
Then you'd change your references when you run your app
App extends Java {
packages [commons-logging1.04, ];
}
the runtime would work it all out. For ant, with its property driven
override, its a lot fiddlier.
The reason for sha1 emphasis is primarily that md5 is doomed: its too
easy to break, 2-5 years left in it before it is as trusted as DES.
Now, when it is finally broken, .rpm is the first juicy target. But
md5 is old; it would be good if maven2 had .sha1 everywhere right from
the outset.
>
> > I see.
> >
> > One more question regarding m2 names
> >
> > You map a project name of "org.apache.axis" to org/apache/ant. What
> > about artifact names. I've been defaulting to assuming
> > if (artifact==null) { artifact=project}
> >
> > but if project is now dotted, should I default to the last element of
> > the project name? Or require an artifact name if the project is
> > dotted?
>
> I don't quite understand this - sorry. We have groupId and artifactId
> which combine to be globally unique (where artifactId is unique within
> a single groupId). Only the groupId is modified - if the artifactId
> had a '.' then the name would remain the same, ie:
>
> org/apache/ant/ant.optional/1.6.3/ant.optional-1.6.3.jar
OK. And the artifact ID is compulsory.
>
> To ease confusion, perhaps we will probably make . and / illegal in
> the artifactId (along with anything else note valid or desired in a
> path name - spaces, for example).
yes. a legal string of valid names would be good, better than an
exclusion list (as there are so many characters in unicode land. For
x-platform support, that means lowercase ascii alphanumeric plus a few
separators for all of the different elements:
[a-z0-9.-+_]
The weakness here is that it doesnt address the wants/needs of the
non-ascii world., but since we are constructing HTTP URLs from the
strings, we need to use that subset, not just those chars that are
valid in filenames.
Re: long project names & repositories
(I'm assuming you meant to reply to all by the content - it happens
frequently with other gmail users - sorry if I'm out of place
repeating your message)
> I'd probably retain my own in the Ant codebase, but SmartFrog could
> take wagon -as long as our signing it doesnt break it (smartfrog runs
> a secure classloader for in-JVM code); whoever deploys to an
> infrastructure has to sign every jar with a key they have told every
> running daemon to trust; no central CA. At the very least, I could use
> httpclient rather than the stuff in java.net, stuff that is maintained
> by someone who has clearly never read the HTTP spec.
we're over at wagon-dev@maven.apache.org if you need anything. If Ant
isn't going to use it, I probably won't put as much time into making a
public face for it just yet unless anyone else is interested in
helping out. It works fine, but needs doco, mainly.
> The reason for keeping ant self-contained is
>
> -all the Http download code is already there in the Get task
>
> -If we do it without any dependent libraries, then we can include in
> the core tasks and so be 100% sure all ant installations have it built
> in. We can then use it for bootstrapping
>
> -Although the repository stuff is designed with extra plug in
> repositories, if maven2 was built in, I could make it the default
> layout.
You're welcome to checkout the source code (or just import the
classes) and add it to ant.jar if that'll help. We don't have any
additional dependencies.
However, it's fine if you are happy to continue as is too - we'll most
likely release ant tasks to wrap around maven-artifact to give Ant
users transitive dependencies, snapshot handling, etc. outside of
Maven.
At the very least we should continue to swap notes, especially when it
comes to implementing the security stuff so that we have one,
consistent solution.
I'm still thinking over the inclusion of the sha1 in the dependency
definition. The reasoning seems sound, but something about it just
gives that gut feeling that it isn't the best way to go - maybe it is
still just the niggling feeling of too much work for the user.
> I see.
>
> One more question regarding m2 names
>
> You map a project name of "org.apache.axis" to org/apache/ant. What
> about artifact names. I've been defaulting to assuming
> if (artifact==null) { artifact=project}
>
> but if project is now dotted, should I default to the last element of
> the project name? Or require an artifact name if the project is
> dotted?
I don't quite understand this - sorry. We have groupId and artifactId
which combine to be globally unique (where artifactId is unique within
a single groupId). Only the groupId is modified - if the artifactId
had a '.' then the name would remain the same, ie:
org/apache/ant/ant.optional/1.6.3/ant.optional-1.6.3.jar
To ease confusion, perhaps we will probably make . and / illegal in
the artifactId (along with anything else note valid or desired in a
path name - spaces, for example).
Cheers,
Brett
Re: long project names & repositories
> Ant is still debating whether or not to EOL 1.2 support, following > Sun's lead. I hadnt been over enthused about -and certainly against > dropping 1.3 support given how some platforms (e.g. FreeBSD) are still > 1.3 only. right... so if we were to roll back the API and the lightweight HTTP provider to 1.2 (ie, make all the exceptions deal with a Throwable cause rather than giving it to Exception in the constructor), would you prefer to be using Wagon in the repository task, or are you going to maintain your own implementation? > Your mention of httpclient reminds me how atrociously inconsistent > java.net is on the older platforms. As long as servers dont require > auth or multi cookie support we'll be ok. probably. as long as we dont > rely on reliable content-length headers or parsing of any error > responses. Right. By default, we use the JDK provided code for HTTP (the "lightweight" wagon). We also have one based on httpclient (which is what is used in m1), which some people needed behind certain NTLM proxies that the JDK didn't seem to support (at least until JDK 1.4, but even then I don't think it worked with all the ones we saw people trying to use). Switching implementations should just be a configuration setting. > I'll look at coding ant and smartfrog support, write the unit tests, > etc, etc, let you know how it worked. > > one problem I have for both systems is proxies; at work I cant go to > remote http servers without proxy setup. What does maven do? wagon has a proxy configuration that it passes on to the JDK stuff as system properties, or to httpclient as configuration depending on the implementation you use. The proxy is configured via your main Maven config file for your user/installation. Cheers, Brett
Re: long project names & repositories
On Apr 10, 2005 2:06 AM, Brett Porter <[EMAIL PROTECTED]> wrote: > > planetapache.org knows everything :) > > Yes, I just caught your post :) > > > This is cool. > > -what is the local cache name/layout? > > Configurable, defaults to ~/.m2/repository and uses the "default" > layout, which is the new one. > OK. > > > > I'll add maven2 lib support to ant CVS_HEAD and make it the default > > there, same for Smartfrog; in both cases I'll provide support for the > > old layout somehow too. That'll be a good test of how well architected > > the different classes are -if adding a new repository layout only > > changes one class. > > Earlier, there was some talk about whether it would be appropriate to > get Wagon up to release quality and use that. What are your thoughts > on this? > > The main limitation is that it currently requires JDK 1.4, which > conflicts with your message: > http://mail-archives.apache.org/mod_mbox/www-repository/200501.mbox/[EMAIL > PROTECTED] > It doesn't have any additional dependencies (I may have added some > utils, but we can take them back out). I'm pretty sure the API could > be cut back to 1.2 if that is required, though it would be nicer to > keep it. It is possible that it could be an optional feature for 1.4 > users... how do you feel about this? Ant is still debating whether or not to EOL 1.2 support, following Sun's lead. I hadnt been over enthused about -and certainly against dropping 1.3 support given how some platforms (e.g. FreeBSD) are still 1.3 only. Your mention of httpclient reminds me how atrociously inconsistent java.net is on the older platforms. As long as servers dont require auth or multi cookie support we'll be ok. probably. as long as we dont rely on reliable content-length headers or parsing of any error responses. > > We can move the maven-artifact (or portions thereof) code to Wagon > too. It currently only depends on the container, but I am about to > take a pass at cleaning it up, so I can send the design past here to > see if we are on the same page if you are interested. > > Some of the wagon providers require JDK 1.4 and external dependencies > (eg, the httpclient version of the http provider - not the default, > and the SCP implementation). I don't imagine it is a problem to > document this. > > As for your other mail: > - classifier is for stuff derived from a single groupId:artifactId > (shares the POM). eg -client for an ejb client, -bin and -src on a > dist. ok > - yes, extension should be able to be null or empty - will fix good. that'll be my first contrib to the maven source tree :) > - we have really been trying to discourage the absence of a version in > the repository, but we can have that restriction in Maven, not in > maven-artifact if that is what you want. The current layour really does rely on some kind of version artifact; and you cant do proper versioned repositories without it. your policy makes sense to me. > > So, how do we move forward on this? I'll look at coding ant and smartfrog support, write the unit tests, etc, etc, let you know how it worked. one problem I have for both systems is proxies; at work I cant go to remote http servers without proxy setup. What does maven do?
Re: long project names & repositories
> planetapache.org knows everything :) Yes, I just caught your post :) > This is cool. > -what is the local cache name/layout? Configurable, defaults to ~/.m2/repository and uses the "default" layout, which is the new one. > > I'll add maven2 lib support to ant CVS_HEAD and make it the default > there, same for Smartfrog; in both cases I'll provide support for the > old layout somehow too. That'll be a good test of how well architected > the different classes are -if adding a new repository layout only > changes one class. Earlier, there was some talk about whether it would be appropriate to get Wagon up to release quality and use that. What are your thoughts on this? The main limitation is that it currently requires JDK 1.4, which conflicts with your message: http://mail-archives.apache.org/mod_mbox/www-repository/200501.mbox/[EMAIL PROTECTED] It doesn't have any additional dependencies (I may have added some utils, but we can take them back out). I'm pretty sure the API could be cut back to 1.2 if that is required, though it would be nicer to keep it. It is possible that it could be an optional feature for 1.4 users... how do you feel about this? We can move the maven-artifact (or portions thereof) code to Wagon too. It currently only depends on the container, but I am about to take a pass at cleaning it up, so I can send the design past here to see if we are on the same page if you are interested. Some of the wagon providers require JDK 1.4 and external dependencies (eg, the httpclient version of the http provider - not the default, and the SCP implementation). I don't imagine it is a problem to document this. As for your other mail: - classifier is for stuff derived from a single groupId:artifactId (shares the POM). eg -client for an ejb client, -bin and -src on a dist. - yes, extension should be able to be null or empty - will fix - we have really been trying to discourage the absence of a version in the repository, but we can have that restriction in Maven, not in maven-artifact if that is what you want. So, how do we move forward on this? Cheers, Brett
Re: long project names & repositories
On Apr 9, 2005 2:25 AM, Brett Porter <[EMAIL PROTECTED]> wrote:
http://cvs.apache.org/viewcvs.cgi/maven-components/maven-artifact/src/main/java/org/apache/maven/artifact/repository/layout/DefaultRepositoryLayout.java?rev=1.8&view=markup
>
path.append( artifact.getBaseVersion() ).append( '/' );
path.append( artifact.getArtifactId() ).append( '-' ).append(
artifact.getVersion() );
if ( artifact.hasClassifier() )
{
path.append( '-' ).append( artifact.getClassifier() );
}
path.append( '.' ).append( artifactHandler.extension() );
-this assumes that version!=null. Is this the case.
-what's a classifer
-what if you want an artifact that has no "." at the end? A null
extension should be a null extension; an empty extension to .
Re: long project names & repositories
On Apr 9, 2005 2:25 AM, Brett Porter <[EMAIL PROTECTED]> wrote: > > 1. I see that a Maven2 alpha is out: is it still using the Maven1 > > repository structure? > > News travels fast... I was going to post about that here this morning :) planetapache.org knows everything :) > > New repository: http://www.ibiblio.org/maven2/ > > m2 can use the m1 structure, but by default it uses a new one. I > believe it is almost identical to the one from the wiki (which I can't > seem to find since the wiki was upgraded?), with the omission of > /$type/ in the directory structure. > > http://cvs.apache.org/viewcvs.cgi/maven-components/maven-artifact/src/main/java/org/apache/maven/artifact/repository/layout/DefaultRepositoryLayout.java?rev=1.8&view=markup > > > 3. One question I have is about our policy for longer project names, > > "org.apache.something". I have stub code that replaces that with > > org/apache/something and then looks for the usual > > org/apache/something/jars/artifact[-version][.extension] afterwards. > > Is this wise or not? > > Yes, we've taken that approach, starting with Maven: > http://www.ibiblio.org/maven2/org/apache/maven/ > > More about the m2 repository: > - all the metadata have been converted to Maven 2's POM format, to the > best of our ability > - new Maven 1.x artifacts still come in through the Maven 1.x > repository, and are automatically converted and synced to /maven2/, so > for example ASF releases will be published automatically > > I think that's all. Let me know if you have any questions or issues. > This is cool. -what is the local cache name/layout? I'll add maven2 lib support to ant CVS_HEAD and make it the default there, same for Smartfrog; in both cases I'll provide support for the old layout somehow too. That'll be a good test of how well architected the different classes are -if adding a new repository layout only changes one class. -steve
Re: long project names & repositories
> 1. I see that a Maven2 alpha is out: is it still using the Maven1 > repository structure? News travels fast... I was going to post about that here this morning :) New repository: http://www.ibiblio.org/maven2/ m2 can use the m1 structure, but by default it uses a new one. I believe it is almost identical to the one from the wiki (which I can't seem to find since the wiki was upgraded?), with the omission of /$type/ in the directory structure. http://cvs.apache.org/viewcvs.cgi/maven-components/maven-artifact/src/main/java/org/apache/maven/artifact/repository/layout/DefaultRepositoryLayout.java?rev=1.8&view=markup > 3. One question I have is about our policy for longer project names, > "org.apache.something". I have stub code that replaces that with > org/apache/something and then looks for the usual > org/apache/something/jars/artifact[-version][.extension] afterwards. > Is this wise or not? Yes, we've taken that approach, starting with Maven: http://www.ibiblio.org/maven2/org/apache/maven/ More about the m2 repository: - all the metadata have been converted to Maven 2's POM format, to the best of our ability - new Maven 1.x artifacts still come in through the Maven 1.x repository, and are automatically converted and synced to /maven2/, so for example ASF releases will be published automatically I think that's all. Let me know if you have any questions or issues. Cheers, Brett
