Additional fixes in -r676516 and -r676526..
Summary of the fixes:
1. generate checksums for the generated merged metadata
2. put the merged metadata & its checksums in the first repo where the
metadata was found. the filename for the merged metadata is
maven-metadata-${repoGroupId}.xml to handle instances when a repository
belongs to multiple groups.
3. merge the metadata if and only if all the following conditions are
met:
- the request was made to a repository group url
- the requested resource is a metadata file or a checksum file of a
metadata
- the requested metadata file is at the project level, not version
level
4. added test case when metadata checksums are requested after the
metadata file (simulate Maven requests)
Thanks,
Deng
On Sun, Jul 13, 2008 at 10:08 PM, Maria Odea Ching <[EMAIL PROTECTED]>
wrote:
> Hi Nico,
>
> Could you verify the fix i committed in trunk -r676322 if it works for you?
> I tested it out using ranges and I was able to get the artifacts from the
> different repos.
>
> Thanks,
> Deng
>
>
> On Fri, Jul 11, 2008 at 6:59 PM, Maria Odea Ching <[EMAIL PROTECTED]>
> wrote:
>
>> On Fri, Jul 11, 2008 at 6:35 PM, nicolas de loof <[EMAIL PROTECTED]>
>> wrote:
>>
>>> I've looked at MetaDataTools class.It merges the metadatas from multiple
>>> repositories in a target file. This could be refactored into another
>>> method
>>> to return the ArchivaRepositoryMetadata objet, so that it can be used for
>>> repo grouping.
>>
>>
>> I'm using the RepositoryMetadataMerge.merge(..) to merge the metadata from
>> each repository into
>> a main metadata file, so I don't need to refactor anything. I have still
>> yet to test it though. I'll commit it in svn asap
>> so you could also take a look :)
>>
>> The "*write to file*" process should also be skipped when there is no
>>> concrete metadata (availableVersions or Plugins) to avoid creating
>>> "empty"
>>> metadatas in managed repo for requested artifacts that aren't available.
>>>
>>> Could this be a security issue with archiva ? requesting a fake artifact
>>> metadata creates a new File. Requesting thousand of them with a random
>>> name
>>> will create thousand files. This could be the basis of a "brute force"
>>> deny
>>> of service attack, with file system becoming full. Or maybe I'm a
>>> paranoid ?
>>
>>
>> This is indeed alarming! I'll try replicating this and send my results
>> back..
>>
>>
>>>
>>> Nicolas.
>>
>>
>> Thanks,
>> Deng
>>
>>
>>>
>>> 2008/7/10 Maria Odea Ching <[EMAIL PROTECTED]>:
>>>
>>> > Okay, thanks Nico.. I think I can do the merging stuff. Will be back
>>> online
>>> > later when I get home..
>>> >
>>> > -Deng
>>> >
>>> > On Thu, Jul 10, 2008 at 6:14 PM, nicolas de loof <[EMAIL PROTECTED]>
>>> > wrote:
>>> >
>>> > > 2nd option may be simplier but not really nice, let's consider this
>>> use
>>> > > case
>>> > > :
>>> > >
>>> > > A user has an issue with a maven plugin. As a nice guy, he creates an
>>> > > issue,
>>> > > attach a patch, but to get quick fix he deploys a snapshot to its
>>> > corporate
>>> > > repo.
>>> > >
>>> > > To get it using a group repository ("public" proxying public repos +
>>> > > "corporate"), we need to merge the metadatas from both repositories,
>>> so
>>> > > that
>>> > > the custom snapshot will get listed in metadatas + all available
>>> version
>>> > on
>>> > > public snapshot repositories.
>>> > >
>>> > > From my understanding, this will require some "*list available
>>> files*"
>>> > > logic
>>> > > + "*merge metadatas*" in ArchivaDavResource.createResource(), and not
>>> > just
>>> > > "return resource;"
>>> > >
>>> > > I've commited the "*list available files*" part that was trivial.
>>> > >
>>> > > I'm not used with Dav (jackRabit) API and metadata file format, so
>>> any
>>> > help
>>> > > for the "*merge metadatas*" would be welcome (TODO at line 263).
>>> This
>>> > > would
>>> > > require to create an "in memory" virtual resource for the merged
>>> > metadata.
>>> > >
>>> > > Nico.
>>> > >
>>> > >
>>> > >
>>> > > 2008/7/10 Maria Odea Ching <[EMAIL PROTECTED]>:
>>> > >
>>> > > > Oh ok, sorry I thought it was a blank file returned. I haven't
>>> tried
>>> > > > replicating it yet, was just looking through the codes.
>>> > > > I guess either of the 2 options would work.. I'm still finishing up
>>> > some
>>> > > > things in the search, I'll see what I can do for 872 later tonight.
>>> > > > Unless of course you want to work on it? :)
>>> > > >
>>> > > > Thanks,
>>> > > > Deng
>>> > > >
>>> > > > On Thu, Jul 10, 2008 at 5:13 PM, nicolas de loof <
>>> [EMAIL PROTECTED]>
>>> > > > wrote:
>>> > > >
>>> > > > > This mean the First metadata file found in on repo of the group
>>> is
>>> > > > returned
>>> > > > > !
>>> > > > >
>>> > > > > This is what happen : I get a metadat file with no <version>
>>> included
>>> > > > (not
>>> > > > > an blank file, but an XML construct with just root element) :
>>> this is
>>> > > the
>>> > > > > content of my first repo in the group, that did not retrieve the
>>> > > expected
>>> > > > > artifact.
>>> > > > >
>>> > > > > I made a simple test : place "release" repo before "corporate" in
>>> the
>>> > > > > group,
>>> > > > > and I the get the expected metadata.
>>> > > > >
>>> > > > > My conclusion is we have two options :
>>> > > > >
>>> > > > > - support metadata merge from repositories in the group (not just
>>> > > "first
>>> > > > > win")
>>> > > > > - don't create metadata in managed repo if no artifact was found
>>> from
>>> > > > > proxies.
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > 2008/7/10 Maria Odea Ching <[EMAIL PROTECTED]>:
>>> > > > >
>>> > > > > > The ArchivaVirtualDavResource is for the dav browse
>>> (directories),
>>> > > not
>>> > > > > for
>>> > > > > > artifact requests..
>>> > > > > >
>>> > > > > > Hmm, the virtual repos work this way:
>>> > > > > > - client requests for an artifact
>>> > > > > > - RepoServlet handles the request and goes to
>>> > > ArchivaDavSessionProvider
>>> > > > > > - in ArchivaDavSessionProvider, the request is checked if it is
>>> a
>>> > > repo
>>> > > > > > group
>>> > > > > > url or a regular repo url.
>>> > > > > > If it is, then loop through the repos under the group. The
>>> first
>>> > > > > > requested artifact found among the repos
>>> > > > > > is returned (proxied or already existing). The metadata
>>> > > update/merge
>>> > > > > > happens when the artifact is
>>> > > > > > fetched from the proxies (but this is on a per repository
>>> level,
>>> > > > > there's
>>> > > > > > no metadata of metadata files
>>> > > > > > from each repo being merged at the repo group level)
>>> > > > > >
>>> > > > > > The same process goes if the request made is for a metadata
>>> file. I
>>> > > > don't
>>> > > > > > know why a blank metadata file was returned.. :(
>>> > > > > >
>>> > > > > > Thanks,
>>> > > > > > Deng
>>> > > > > >
>>> > > > > > On Thu, Jul 10, 2008 at 3:39 PM, nicolas de loof <
>>> > [EMAIL PROTECTED]
>>> > > >
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > > > I just started investigating on this. I can't find where the
>>> > > > metadatas
>>> > > > > > are
>>> > > > > > > merged when a repository group is set. From my understanding,
>>> > > > > > > ArchivaVirtualDavResource is used with a List<File> of
>>> metadatas
>>> > > from
>>> > > > > > each
>>> > > > > > > repo in the group. But I don't find WHERE the metadata files
>>> are
>>> > > > merged
>>> > > > > > > into
>>> > > > > > > a single XML content...
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > 2008/7/10 Maria Odea Ching <[EMAIL PROTECTED]>:
>>> > > > > > >
>>> > > > > > > > Ok, thanks Nico, Dan :)
>>> > > > > > > > I'll look into this further.. I guess we could include this
>>> in
>>> > > > 1.1.1
>>> > > > > > > which
>>> > > > > > > > is scheduled to be released right after 1.1 to fix another
>>> > > blocker
>>> > > > in
>>> > > > > > > 1.1.
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > > Deng
>>> > > > > > > >
>>> > > > > > > > On Thu, Jul 10, 2008 at 2:37 PM, nicolas de loof <
>>> > > > [EMAIL PROTECTED]
>>> > > > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > > MRM-872 created for this, with the concrete case I fall
>>> into.
>>> > > > > > > > >
>>> > > > > > > > > If confirmed, this is a blocker for repository group use
>>> > > > > > > > >
>>> > > > > > > > > 2008/7/10 Dan Tran <[EMAIL PROTECTED]>:
>>> > > > > > > > >
>>> > > > > > > > > > sounds like a blocking bug? would love to see this
>>> fixed
>>> > in
>>> > > > 1.1
>>> > > > > > :-)
>>> > > > > > > > > >
>>> > > > > > > > > > On Wed, Jul 9, 2008 at 7:11 PM, Maria Odea Ching <
>>> > > > > > [EMAIL PROTECTED]>
>>> > > > > > > > > > wrote:
>>> > > > > > > > > > > Hi Nico,
>>> > > > > > > > > > >
>>> > > > > > > > > > > That looks like a new bug :( The metadata should be
>>> just
>>> > > the
>>> > > > > same
>>> > > > > > > as
>>> > > > > > > > > the
>>> > > > > > > > > > > regular metadata of a managed repo (not belonging to
>>> a
>>> > > group)
>>> > > > > as
>>> > > > > > it
>>> > > > > > > > > uses
>>> > > > > > > > > > the
>>> > > > > > > > > > > same code for webdav & proxying. Could you file a
>>> jira
>>> > for
>>> > > > > this?
>>> > > > > > > > > > >
>>> > > > > > > > > > > Thanks,
>>> > > > > > > > > > > Deng
>>> > > > > > > > > > >
>>> > > > > > > > > > > On Wed, Jul 9, 2008 at 8:45 PM, nicolas de loof <
>>> > > > > > > [EMAIL PROTECTED]>
>>> > > > > > > > > > wrote:
>>> > > > > > > > > > >
>>> > > > > > > > > > >> Hy,
>>> > > > > > > > > > >>
>>> > > > > > > > > > >> I get strange behaviour wen using repository group :
>>> > > > > > > > > > >>
>>> > > > > > > > > > >> From a fresh maven installation (empty local repo),
>>> with
>>> > > > > > settings
>>> > > > > > > > set
>>> > > > > > > > > to
>>> > > > > > > > > > >> mirror central to my archiva repository group URL,
>>> the
>>> > > > eclipse
>>> > > > > > 2.5
>>> > > > > > > > > > plugin
>>> > > > > > > > > > >> can't find the required
>>> > > > > org.eclipse.core:resources:[3.1.0,4.0.0)
>>> > > > > > > > > > >>
>>> > > > > > > > > > >> The maven-metadata.xml returned by the group URL is
>>> > emty.
>>> > > > > > > requesting
>>> > > > > > > > > > >> metadatas from individual managed repos in the group
>>> are
>>> > > > fine.
>>> > > > > > Is
>>> > > > > > > > this
>>> > > > > > > > > a
>>> > > > > > > > > > >> known limitation ?
>>> > > > > > > > > > >>
>>> > > > > > > > > > >> Nico.
>>> > > > > > > > > > >>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>
