Yes, you are right. The lucene dependency causes a lot of trouble and will cause headaches with each version change of one of the dependencies. What are the requirements for a replacement? - We want to store hierarchical data? - We want to store metadata for nodes ? - Fulltext search (only metadata or for artifacts too?) - Blob / Artifact storage (I don't think so, but not so familiar with the archiva artifact model)?
Maybe some graph database may be an alternative. Don't know if the license of neo4j is compatible to the apache license, and I think it brings lucene as dependency too. I will have a look. Problem is, if there is fulltext search needed, I think, for most of the frameworks we get a lucene dependency, if it's embedded. Other alternatives: - Implement fulltext search by our own (index of the metadata stored via the archiva api) and use the lucene dependency that comes from the maven-indexer - Jcr Oak with Solr. Solr is not embedded, must run as its own application (war). Greetings Martin Am Samstag, 24. Juni 2017, 14:05:26 CEST schrieb Olivier Lamy: > well this gonna be a pain. > IMHO we need to find a new alternative to jcr oak. > And something not using Lucene as it's a real pain to have different > librairies using lucene as they do not update in the same time (and Lucene > break backward compat so quickly...) > Any ideas? I'd like to have something embedded (but with a possible > external server configuration). > There is currently a Cassandra implementation. I was not satisfied about > performance but I guess I did that 4yo ago so can be improved for sure :-) > Maybe orientdb? > What else? > > On 24 June 2017 at 09:50, Olivier Lamy <[email protected]> wrote: > > well the issue is non compatible version of Lucene for Maven Indexer and > > Oak (well I can try push a patch to Oak for upgrading...) > > > > On 24 June 2017 at 08:41, Olivier Lamy <[email protected]> wrote: > >> Hi > >> Maven Indexer 6.0-SNAPSHOT doesn't need anymore plexus bridge. > >> I'm working on it in the branch ( feature/jcr_oak ) > >> Not sure why but I have intermittent failure with store-jcr module. > >> I definitely agree on the upgrade. > >> Well we can simply detect it's not oak compatible and schedule a full > >> reindex (maybe with a message in logs and ui?) > >> But we need to be sure we can still read central index and not sure about > >> possible lucene conflict with oak and maven indexer. > >> We can work on this branch? (I created a Jenkins job for it > >> https://builds.apache.org/view/A-D/view/Archiva/job/archi > >> va-jcr-oak-branch/) > >> If you prefer master I would say no worries neither. > >> Something else to look at is upgrading maven-core etc... > >> Anyway > >> Cheers > >> Olivier > >> > >> On 22 June 2017 at 19:16, Martin <[email protected]> wrote: > >>> Hi, > >>> > >>> upgrading the maven indexer leads to some major changes. > >>> Lucene is used by maven-indexer and also by jackrabbit. Jackrabbit > >>> sticks to > >>> the old 3.x version and, as I see it, they will not move to a newer > >>> version. > >>> There is Jackrabbit Oak as alternative. > >>> I tried a proof of concept and could replace the jackrabbit > >>> implementation of > >>> metadata-store-jcr with a oak implementation. At least I got the unit > >>> tests of > >>> this module all to pass. > >>> But switching to Oak has some drawbacks: > >>> - The repository format changed and we must provide a way to migrate > >>> (either > >>> migrate the existing repository or create a new one by reindexing) > >>> - The lucene version used is newer but does not match to the version > >>> from the > >>> maven-indexer dependencies. There may come up some incompatibilities > >>> that are > >>> not solvable without using a modified version of one of the both. Or > >>> there may > >>> be the possibility to switch to solr (as separate component) and get rid > >>> of > >>> the lucene dependencies for jcr inside the archiva project. > >>> > >>> Switching to maven-indexer 6.0-SNAPSHOT means some changes too: > >>> - The Plexus-Sisu-Bridge does not work as before. > >>> - We must migrate from the NexusIndexer to the indexer API. > >>> > >>> So switching to the new indexer and oak means more work as expected and > >>> some > >>> risks regarding new incompatibility problems. And I think this cannot be > >>> done > >>> without broken master builds for some time period. > >>> > >>> So, what should we do? I think maven indexer is one of the core > >>> components of > >>> archiva, and we should utilize the 3.x-version to migrate to the new > >>> indexer > >>> version, even if this means switching to jcr oak. Otherwise it would > >>> mean to > >>> stick to the old version for the next years. > >>> @Olivier, regarding the maven-indexer / sisu-Bridge API changes, I hope > >>> you > >>> can provide useful help. > >>> > >>> I committed the PoC to the branch feature/jcr_oak. There are some > >>> modules > >>> where the tests do not pass (mainly because of the indexer API changes). > >>> > >>> Any comments? > >>> > >>> Cheers > >>> > >>> Martin > >>> > >>> Am Dienstag, 13. Juni 2017, 09:07:35 CEST schrieb Olivier Lamy: > >>> > forget it but we need to ensure we can read maven index files.... > >>> > > >>> > On 13 June 2017 at 17:06, Olivier Lamy <[email protected]> wrote: > >>> > > Hi, > >>> > > Remember jackrabbit depends on Lucene as well so upgrading Lucene > >>> > >>> can be a > >>> > >>> > > problem here. > >>> > > Regarding maven-indexer yes we can depend on a snapshot until the > >>> > >>> release. > >>> > >>> > > I can release it ;-) > >>> > > > >>> > > On 13 June 2017 at 06:06, Martin <[email protected]> wrote: > >>> > >> Hi, > >>> > >> > >>> > >> the lucene version depends on the maven indexer. But I'm not sure > >>> > >>> about > >>> > >>> > >> the > >>> > >> current state of maven-indexer. The version has not changed since > >>> > >>> some > >>> > >>> > >> 2013. > >>> > >> > >>> > >> There are commits on the master branch since then, and the lucene > >>> > >>> version > >>> > >>> > >> has > >>> > >> been changed too, but no releases were tagged. > >>> > >> Does it make sense to switch to the maven-indexer 6.0-SNAPSHOT? > >>> > >> > >>> > >> As I know there are new compact index formats with new lucene > >>> > >>> versions > >>> > >>> > >> but I'm > >>> > >> not sure if this is relevant for the maven indexes. > >>> > >> > >>> > >> Cheers > >>> > >> > >>> > >> Martin > >>> > > > >>> > > -- > >>> > > Olivier Lamy > >>> > > http://twitter.com/olamy | http://linkedin.com/in/olamy > >> > >> -- > >> Olivier Lamy > >> http://twitter.com/olamy | http://linkedin.com/in/olamy > > > > -- > > Olivier Lamy > > http://twitter.com/olamy | http://linkedin.com/in/olamy
