Re: Maven central and indexing

[Hervé BOUTEMY]

FYI, Nexus Indexer was given to Apache Maven by Sonatype in 2010:
http://incubator.apache.org/ip-clearance/maven-indexer.html

Then became Maven Indexer
http://maven.apache.org/maven-indexer/

or for better documentation:
http://maven.apache.org/maven-indexer-archives/maven-indexer-LATEST/

Regards,

Hervé

Le vendredi 10 mars 2017, 20:23:07 CET Bernd Eckenfels a écrit :
> Hello,
> Related: http://blog.sonatype.com/2008/12/central-repository-downloading-the
> -nexus-index/ And especially the news of a public available dataset (by
> Google):http://takari.io/2015/10/28/google-maven-central.html
> 
> Gruss
> Bernd



-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Maven central and indexing

[Laird Nelson]

On Fri, Mar 10, 2017 at 2:56 PM Laird Nelson  wrote:

> Oh!  I did just notice that you can search on a field called "tags" in
> search.maven.org.  It is not immediately clear to me where the contents
> for this field come from.  Does anyone know?
>

Hmm; it looks to be a Nexus-specific feature, but one that you can control:
https://github.com/sonatype/nexus-maven-plugins/tree/master/staging/maven-plugin#tagging-staging-repositories

Now, if you don't use the Nexus staging plugin it looks like this field is
still populated automatically somehow—presumably from various meaningful
words in the description, or name, or something—but I can't find any
documentation concerning what it's auto-populated with.

Here is a completely arbitrary example Maven central query where you can
see the tag array:
http://search.maven.org/solrsearch/select?q=g:com.datathings+AND+a:greycat-memory+AND+v:3=xml
It looks like you have to have a query that results in a single artifact to
see the actual tags, or else they don't show up in the results.  (I of
course have no idea whether the author of this artifact explicitly tagged
the thing or whether these tags are auto-populated.)

Best,
Laird

Re: Maven central and indexing

[Laird Nelson]

On Fri, Mar 10, 2017 at 2:18 PM Laird Nelson  wrote:

> On Fri, Mar 10, 2017 at 1:34 PM Manfred Moser 
> wrote:
>
> Laird Nelson wrote on 2017-03-10 13:03:
> > I am also not sure I'm smart enough to grok all the implications of the
> > Google-related post.  That itself still does not expose an index, say, of
> > the contents of all MANIFEST.MF files, right?  Do I read properly that
> such
> > tools are in development, and to the best of everyone's knowledge an
> index
> > of the contents of the contents :-) of Maven central does not exist?
>
>
Oh!  I did just notice that you can search on a field called "tags" in
search.maven.org.  It is not immediately clear to me where the contents for
this field come from.  Does anyone know?

For example, from just bashing around with it, I can search for everything
with "compile" as the value for "tags":

http://search.maven.org/#search%7Cga%7C1%7Ctags%3Acompile

What's even more interesting is that if I search for everything with
"sbtplugin" as the value for "tags", I get some results whose artifacts do
not appear to contain anything with the string "sbtplugin" in them (I
opened a few jar files and checked).  That suggests to me that perhaps
something is deploying these and specifying the tags at deploy time?  But
the maven-deploy-plugin does not seem to advertise such a facility.

I've found no documentation for this index field.  Can someone tell me how
it is populated?  It seems staggeringly useful.

Best,
Laird

Re: Maven central and indexing

[Laird Nelson]

On Fri, Mar 10, 2017 at 1:34 PM Manfred Moser 
wrote:

> Laird Nelson wrote on 2017-03-10 13:03:
> > I am also not sure I'm smart enough to grok all the implications of the
> > Google-related post.  That itself still does not expose an index, say, of
> > the contents of all MANIFEST.MF files, right?  Do I read properly that
> such
> > tools are in development, and to the best of everyone's knowledge an
> index
> > of the contents of the contents :-) of Maven central does not exist?
>
> No publically available one at this stage that I am aware of..
>

Thank you again.  I also noticed that the repository located at
https://maven-central.storage.googleapis.com/ described in Jason's post
 has not been
updated in at least a year so I'm assuming this project died or went
elsewhere.  Noted here for posterity.

Have a good weekend,
Best,
Laird

Re: Maven central and indexing

[Manfred Moser]

Laird Nelson wrote on 2017-03-10 13:03:

> On Fri, Mar 10, 2017 at 12:23 PM Bernd Eckenfels 
> wrote:
> 
>> Hello,
>> Related:
>> http://blog.sonatype.com/2008/12/central-repository-downloading-the-nexus-index/
>> And especially the news of a public available dataset (by Google):
>> http://takari.io/2015/10/28/google-maven-central.html
> 
> 
> Thank you!  I'm aware of the Nexus index, though I haven't probed it
> deeply; perhaps it has more that the simple GAV-oriented index exposed by
> search.maven.org (if it does not, then it doesn't solve my problem).

It does not.. 

> I am also not sure I'm smart enough to grok all the implications of the
> Google-related post.  That itself still does not expose an index, say, of
> the contents of all MANIFEST.MF files, right?  Do I read properly that such
> tools are in development, and to the best of everyone's knowledge an index
> of the contents of the contents :-) of Maven central does not exist?

No publically available one at this stage that I am aware of.. 

manfred

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Maven central and indexing

[Laird Nelson]

On Fri, Mar 10, 2017 at 12:23 PM Bernd Eckenfels 
wrote:

> Hello,
> Related:
> http://blog.sonatype.com/2008/12/central-repository-downloading-the-nexus-index/
> And especially the news of a public available dataset (by Google):
> http://takari.io/2015/10/28/google-maven-central.html


Thank you!  I'm aware of the Nexus index, though I haven't probed it
deeply; perhaps it has more that the simple GAV-oriented index exposed by
search.maven.org (if it does not, then it doesn't solve my problem).

I am also not sure I'm smart enough to grok all the implications of the
Google-related post.  That itself still does not expose an index, say, of
the contents of all MANIFEST.MF files, right?  Do I read properly that such
tools are in development, and to the best of everyone's knowledge an index
of the contents of the contents :-) of Maven central does not exist?

Best,
Laird

Re: Maven central and indexing

[Bernd Eckenfels]

Hello,
Related: 
http://blog.sonatype.com/2008/12/central-repository-downloading-the-nexus-index/
And especially the news of a public available dataset (by 
Google):http://takari.io/2015/10/28/google-maven-central.html

Gruss
Bernd
-- 
http://bernd.eckenfels.net




On Fri, Mar 10, 2017 at 8:14 PM +0100, "Laird Nelson"  
wrote:










(This may be a Sonatype question, but it does concern Maven central and the
Maven community so I thought I'd ask here.)

Currently, you can search Maven central efficiently using search.maven.org.
You can search on coordinates and also by class (i.e. "what artifact
contains a class whose name contains the string 'Frobnicator'?").

How would one go about lobbying for additional Solr/Lucene indices on other
things, like, say, META-INF/MANIFEST.MF header contents, or the presence of
a certain resource (com/foo/bar/Frobnicator.properties etc.), or full-text
search on non-binary resources? Stuff like that?  Or is there some service
I'm unaware of out there that provides this information that I could then
use to resolve the resulting artifact pointers?

Best,
Laird

Maven central and indexing

[Laird Nelson]

(This may be a Sonatype question, but it does concern Maven central and the
Maven community so I thought I'd ask here.)

Currently, you can search Maven central efficiently using search.maven.org.
You can search on coordinates and also by class (i.e. "what artifact
contains a class whose name contains the string 'Frobnicator'?").

How would one go about lobbying for additional Solr/Lucene indices on other
things, like, say, META-INF/MANIFEST.MF header contents, or the presence of
a certain resource (com/foo/bar/Frobnicator.properties etc.), or full-text
search on non-binary resources? Stuff like that?  Or is there some service
I'm unaware of out there that provides this information that I could then
use to resolve the resulting artifact pointers?

Best,
Laird