Re: The Lucene Solr Gradle Build Game plan

[Dawid Weiss]

I looked at the solr ref guide build and started converting it to
Gradle but have a question to Mark (because he coordinates the
effort).

What immediately jumps into face is the decision problem -- do we want
to emulate what ant does at the moment or do we want to clean it up
(breaking file/ folder structure and causing incompatibility with ant
build).

I went the "compatible" way and started porting ant tasks but it's
quite awkward. For example -- there are template properties that refer
to ivy version properties... we could emulate/ compute these but it's
a pain. The way the module is currently structured is also awkward -
it'd be more natural to have a separate java project with the "tools"
required to compile extra stuff and just reference it from the manual
build (and this would be a plain module, not a java module). This
would limit the need for customizing source sets, classpaths, etc.

My few initial tasks syncing sources, setting up infrastructure to
filter templates and compiling the required tools are here:
https://github.com/apache/lucene-solr/compare/jira/SOLR-13452_gradle_7_refguide...dweiss:jira/SOLR-13452_gradle_7_refguide?expand=1

I'll stop and wait for feedback (especially on the ivy versions issue)
before I resume.

Dawid

On Wed, Sep 25, 2019 at 6:20 PM Dawid Weiss  wrote:
>
> Never mind, I've got it.
>
> D.
>
> On Wed, Sep 25, 2019 at 7:59 AM Dawid Weiss  wrote:
> >
> > Hi Cassandra,
> >
> > > I’m more than happy to share more details our current build so we can 
> > > replicate some of the above steps, but I’m stuck without a lot more basic 
> > > Gradle skills that I don’t have time to acquire with day-job/personal 
> > > life commitments. I put it into a separate branch so we could iterate a 
> > > little easier, can anyone help?
> >
> > Where is this branch you made changes on? If you can point me at the
> > corresponding ant code I'll try to help you out.
> >
> > Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: rat problems?

[Diego Ceccarelli (BLOOMBERG/ LONDON)]

I re-triggered and it went away, thanks! If it happens again I'll send the 
output..


From: a...@apache.org At: 09/25/19 19:39:06To:  Diego Ceccarelli (BLOOMBERG/ 
LONDON ) ,  dev@lucene.apache.org
Subject: Re: rat problems?

RAT is an Apache tool used for checking release conditions.

Can you paste the output? It can be a transient issue — maybe try re triggering?

On Wed, 25 Sep 2019 at 23:24, Diego Ceccarelli (BLOOMBERG/ LONDON) 
 wrote:

Hi all, 

Sometimes precommit fails due to 'rat problems', what are rat problems? should 
I get a cat? :)

Thanks
-- 
Regards,

Atri
Apache Concerted

Re: rat problems?

[Atri Sharma]

RAT is an Apache tool used for checking release conditions.

Can you paste the output? It can be a transient issue — maybe try re
triggering?

On Wed, 25 Sep 2019 at 23:24, Diego Ceccarelli (BLOOMBERG/ LONDON) <
dceccarel...@bloomberg.net> wrote:

> Hi all,
>
> Sometimes precommit fails due to 'rat problems', what are rat problems?
> should I get a cat? :)
>
> Thanks
>
-- 
Regards,

Atri
Apache Concerted

rat problems?

[Diego Ceccarelli (BLOOMBERG/ LONDON)]

Hi all, 

Sometimes precommit fails due to 'rat problems', what are rat problems? should 
I get a cat? :)

Thanks

Re: The Lucene Solr Gradle Build Game plan

[Dawid Weiss]

Never mind, I've got it.

D.

On Wed, Sep 25, 2019 at 7:59 AM Dawid Weiss  wrote:
>
> Hi Cassandra,
>
> > I’m more than happy to share more details our current build so we can 
> > replicate some of the above steps, but I’m stuck without a lot more basic 
> > Gradle skills that I don’t have time to acquire with day-job/personal life 
> > commitments. I put it into a separate branch so we could iterate a little 
> > easier, can anyone help?
>
> Where is this branch you made changes on? If you can point me at the
> corresponding ant code I'll try to help you out.
>
> Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: The Lucene Solr Gradle Build Game plan

[Gézapeti]

Hi Cassandra and Dawid,

I don't really understand how the ref guide build works, but I managed to
make the solr-ref-guide project compile on calling gradle build
I've pushed my small changes to
https://github.com/gezapeti/lucene-solr/tree/jira/SOLR-13452_gradle_7_refguide
Dawid, you can find Cassandra's work at
https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_7_refguide

Cassandra, is there a way to generate a single-page html version of the
ref-guide?

Hope it helps
gp


On Wed, Sep 25, 2019 at 8:00 AM Dawid Weiss  wrote:

> Hi Cassandra,
>
> > I’m more than happy to share more details our current build so we can
> replicate some of the above steps, but I’m stuck without a lot more basic
> Gradle skills that I don’t have time to acquire with day-job/personal life
> commitments. I put it into a separate branch so we could iterate a little
> easier, can anyone help?
>
> Where is this branch you made changes on? If you can point me at the
> corresponding ant code I'll try to help you out.
>
> Dawid
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Welcome Atri Sharma as Lucene/Solr committer

[Tommaso Teofili]

Congrats and welcome Atri!

Regards,
Tommaso

On Wed, 25 Sep 2019 at 08:55, Atri Sharma  wrote:

> Thank you so much!
>
> On Tue, 24 Sep 2019 at 23:05, Furkan KAMACI 
> wrote:
>
>> Hi,
>>
>> Congrats Atri!
>>
>> Kind Regards,
>> Furkan KAMACI
>>
>> On Thu, Sep 19, 2019 at 5:47 PM Atri Sharma  wrote:
>>
>>> Thank you!
>>>
>>> On Thu, Sep 19, 2019 at 2:04 PM Uwe Schindler  wrote:
>>> >
>>> > Welcome!
>>> >
>>> > Congratulations and I hope to work together with you in the future
>>> (although I am a bit busy at the moment, so I have not much time).
>>> >
>>> > Uwe
>>> >
>>> > -
>>> > Uwe Schindler
>>> > Achterdiek 19, D-28357 Bremen
>>> 
>>> > https://www.thetaphi.de
>>> > eMail: u...@thetaphi.de
>>> >
>>> > > -Original Message-
>>> > > From: Adrien Grand 
>>> > > Sent: Wednesday, September 18, 2019 9:12 AM
>>> > > To: Lucene Dev 
>>> > > Subject: Welcome Atri Sharma as Lucene/Solr committer
>>> > >
>>> > > Hi all,
>>> > >
>>> > > Please join me in welcoming Atri Sharma as Lucene/ Solr committer!
>>> > >
>>> > > If you are following activity on Lucene, this name will likely sound
>>> > > familiar to you: Atri has been very busy trying to improve Lucene
>>> over
>>> > > the past months. In particular, Atri recently started improving our
>>> > > top-hits optimizations like early termination on sorted indexes and
>>> > > WAND, when indexes are searched using multiple threads.
>>> > >
>>> > > Congratulations and welcome! It is a tradition to introduce yourself
>>> > > with a brief bio.
>>> > >
>>> > > --
>>> > > Adrien
>>> > >
>>> > > -
>>> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> > > For additional commands, e-mail: dev-h...@lucene.apache.org
>>> >
>>> >
>>> > -
>>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>>> >
>>>
>>>
>>> --
>>> Regards,
>>>
>>> Atri
>>> Apache Concerted
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>> --
> Regards,
>
> Atri
> Apache Concerted
>

Solr Test Improvement PR From the Gradle Branch.

[Mark Miller]

So my Gradle landing might be a little delayed, I've burned myself out a
little trying to get Solr tests to perform at top speed due to Gradle's
"dumb" parallel test distribution.

I extended the Gradle test task and made my own testFast target that is
"smart" and has been fun to play with, but that only drew me back into the
test quagmire I've been in and out of before. As soon as I saw tests start
to move faster, I was hooked again. The worst kind of addiction, every
additional diminishing savings costing more blood.

In the end I fixed a bunch more bs, pulled in a bunch of improvements and
fixes around tests from the Starburst branch that had not come over yet,
and have generally been wrapping my magic lasso around this test suite. A
bunch of that includes stuff in the code that was causing test problems or
slowness. I've put a lot of emphasis on moving actual slow stuff
to @Nightly runs and fixing not actual slow stuff.

On my 18 core dev machine, I've gotten down to about 7 minutes for all the
Solr modules currently in the Gradle build (there might be a new one not
there).

There is a bunch of clean up I have to do, and review of some things, and
again it's a rather large ''patch".

I'd like to get it out of my gradle branch and into master before moving in
Gradle, so I plan to put up a PR when I can in the near future and am
hoping for some help with review.

Thanks,

-- 
- Mark

http://about.me/markrmiller

Re: Welcome Atri Sharma as Lucene/Solr committer

[Atri Sharma]

Thank you so much!

On Tue, 24 Sep 2019 at 23:05, Furkan KAMACI  wrote:

> Hi,
>
> Congrats Atri!
>
> Kind Regards,
> Furkan KAMACI
>
> On Thu, Sep 19, 2019 at 5:47 PM Atri Sharma  wrote:
>
>> Thank you!
>>
>> On Thu, Sep 19, 2019 at 2:04 PM Uwe Schindler  wrote:
>> >
>> > Welcome!
>> >
>> > Congratulations and I hope to work together with you in the future
>> (although I am a bit busy at the moment, so I have not much time).
>> >
>> > Uwe
>> >
>> > -
>> > Uwe Schindler
>> > Achterdiek 19, D-28357 Bremen
>> 
>> > https://www.thetaphi.de
>> > eMail: u...@thetaphi.de
>> >
>> > > -Original Message-
>> > > From: Adrien Grand 
>> > > Sent: Wednesday, September 18, 2019 9:12 AM
>> > > To: Lucene Dev 
>> > > Subject: Welcome Atri Sharma as Lucene/Solr committer
>> > >
>> > > Hi all,
>> > >
>> > > Please join me in welcoming Atri Sharma as Lucene/ Solr committer!
>> > >
>> > > If you are following activity on Lucene, this name will likely sound
>> > > familiar to you: Atri has been very busy trying to improve Lucene over
>> > > the past months. In particular, Atri recently started improving our
>> > > top-hits optimizations like early termination on sorted indexes and
>> > > WAND, when indexes are searched using multiple threads.
>> > >
>> > > Congratulations and welcome! It is a tradition to introduce yourself
>> > > with a brief bio.
>> > >
>> > > --
>> > > Adrien
>> > >
>> > > -
>> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>>
>> --
>> Regards,
>>
>> Atri
>> Apache Concerted
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>> --
Regards,

Atri
Apache Concerted

Re: The Lucene Solr Gradle Build Game plan

[Dawid Weiss]

Hi Cassandra,

> I’m more than happy to share more details our current build so we can 
> replicate some of the above steps, but I’m stuck without a lot more basic 
> Gradle skills that I don’t have time to acquire with day-job/personal life 
> commitments. I put it into a separate branch so we could iterate a little 
> easier, can anyone help?

Where is this branch you made changes on? If you can point me at the
corresponding ant code I'll try to help you out.

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org