Re: [jira] Jan Høydahl mentioned you on SOLR-13648 (Jira) (JIRA)

[Dawid Weiss]

Hi Jan,

Funny, I don't have permission to see or comment on:
https://issues.apache.org/jira/browse/SOLR-13648

Anybody knows why this is the case?

Yes, simple-xml is a dependency of Carrot2 (clustering contrib's
default implementation). I'm working on having it removed on next
iteration of Carrot2 so it won't be long before it's gone.

Dawid

On Mon, Sep 16, 2019 at 11:09 PM Jan Høydahl (Jira)  wrote:
>
>
>  [ 
> https://issues.apache.org/jira/browse/SOLR-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>  ]
>
> Jan Høydahl mentioned you on SOLR-13648
> ---
>
> [~dawid.weiss] do you know if simple-xml is a dependency of Carrot2 or other 
> part of the clustering contrib, or if it can be removed/replaced?
>
> > Key: SOLR-13648
>
> > View Online: https://issues.apache.org/jira/browse/SOLR-13648
> > Add Comment: 
> > https://issues.apache.org/jira/browse/SOLR-13648#add-comment
>
> Hint: You can mention someone in an issue description or comment by typing  
> "@" in front of their username.
>
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Notes from the committers' meeting at Activate 2019

[David Smiley]

+1 to all that Gus said.  On a fresh project then indeed perhaps we would
use GitHub issues but it's not nearly so compelling at this point with so
much rich history in JIRA, not to mention those issues being referenced in
commit messages.

Is the point to lower barriers for contributors that are not in our
community yet (thus don't have an ASF JIRA account)?  Well... they don't
*have* to create the JIRA issue if a committer is sufficiently interested
in the issue at hand to do it.  And as mentioned this could be automated.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Sep 16, 2019 at 6:27 PM Gus Heck  wrote:

> FWIW, One thing that needs to be figured out is how github would
> accommodate security issues (or how the process for those issues would
> change).  Does github have the ability to assign roles and visibility
> (could be I haven't really worked with organizations on GitHub, all my
> clients have been on jira ... except the one that has trello, and another
> with gitlab... neither of which have impressed me very much )?
>
> Additionally, I'm slightly leery of the move since I don't (yet) see the
> real benefit that pays for the splitting of the records into two systems.
> Can issues be migrated to github? I've not really been on a large project
> that used only GitHub, can someone explain what we *gain* by moving to
> GitHub issues. At least two things are lost: continuity and familiarity. My
> impression is that there are a lot fewer features, but maybe I've just not
> been exposed to them.
>
> Part of me worries that this is the usual cycle of "this is simpler" (mass
> migration ensues) "but it doesn't x, y and z" (x, y and z are added) "wow
> this is complex, but THAT is simpler" (mass migration ensues...) "hmm
> p, q an z are missing" (p q and z are added  and cycle repeats). So I'm
> skeptical of any "gain" hanging it's hat solely on "simplicity". Jira used
> to be the simpler, snazzier, sexier alternative to bugzilla...
>
> Sell me, I'm all ears, but not seeing it yet :)
>
> -Gus
>
> On Mon, Sep 16, 2019 at 3:57 PM Jan Høydahl  wrote:
>
>> Is there any reason at all that we need to hold on to JIRA? ASF allows us
>> to move all issue handling over to GH, I'd like us to consider such a move.
>>
>> Until then, I made a script that "diffs" GH and JIRA and outputs a
>> report, see
>> https://github.com/apache/lucene-solr/tree/master/dev-tools/scripts#githubprspy
>>
>> Running the tool shows that we're not very successful in keeping the two
>> in sync, we also forget to close PRs after JIRAs are resolved:
>>
>> $ ./githubPRs.py
>> Lucene/Solr Github PR report
>> 
>> Number of open Pull Requests: 208
>>
>> PRs lacking JIRA reference in title
>>   #882: Add versions.lock entry for
>> "org.apache.yetus:audience-annotations"
>>   #880: Tweak header format.
>>   [… many more ]
>>
>> Open PRs with a resolved JIRA
>>   #723: SOLR-13545 status=Closed, resolution=Fixed,
>> resolutiondate=2019-06-22T13:04:47.000+ (SOLR-13545: AutoClose stream
>> in ContentStreamUpdateRequest)
>>   #643: SOLR-13391 status=Resolved, resolution=Resolved,
>> resolutiondate=2019-04-12T14:09:27.000+ (SOLR-13391: Add variance and
>> standard deviation stream evaluators)
>>   [… many more]
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> 16. sep. 2019 kl. 20:57 skrev Andrzej Białecki :
>>
>>
>> On 16 Sep 2019, at 19:38, Yonik Seeley  wrote:
>>
>> >  - PR is opened - should automatically create a jira if it doesn’t
>> exist yet
>>
>> What were the reasons behind this? Shouldn't a JIRA just be optional if
>> we started with a PR?
>>
>>
>> That’s why we need to discuss this :)
>>
>> I remember that at some point ASF treated JIRA as the system of record
>> for the legal validation of contributions.
>>
>> I also remember that at some point we used to say that if a discussion
>> didn’t happen in JIRA then it didn’t exist, and that any decisions
>> regarding code should be recorded in JIRA because we can’t expect people to
>> monitor and contribute / object to decisions happening elsewhere.
>>
>> —
>>
>> Andrzej Białecki
>>
>>
>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>

Re: Notes from the committers' meeting at Activate 2019

[Gus Heck]

FWIW, One thing that needs to be figured out is how github would
accommodate security issues (or how the process for those issues would
change).  Does github have the ability to assign roles and visibility
(could be I haven't really worked with organizations on GitHub, all my
clients have been on jira ... except the one that has trello, and another
with gitlab... neither of which have impressed me very much )?

Additionally, I'm slightly leery of the move since I don't (yet) see the
real benefit that pays for the splitting of the records into two systems.
Can issues be migrated to github? I've not really been on a large project
that used only GitHub, can someone explain what we *gain* by moving to
GitHub issues. At least two things are lost: continuity and familiarity. My
impression is that there are a lot fewer features, but maybe I've just not
been exposed to them.

Part of me worries that this is the usual cycle of "this is simpler" (mass
migration ensues) "but it doesn't x, y and z" (x, y and z are added) "wow
this is complex, but THAT is simpler" (mass migration ensues...) "hmm
p, q an z are missing" (p q and z are added  and cycle repeats). So I'm
skeptical of any "gain" hanging it's hat solely on "simplicity". Jira used
to be the simpler, snazzier, sexier alternative to bugzilla...

Sell me, I'm all ears, but not seeing it yet :)

-Gus

On Mon, Sep 16, 2019 at 3:57 PM Jan Høydahl  wrote:

> Is there any reason at all that we need to hold on to JIRA? ASF allows us
> to move all issue handling over to GH, I'd like us to consider such a move.
>
> Until then, I made a script that "diffs" GH and JIRA and outputs a report,
> see
> https://github.com/apache/lucene-solr/tree/master/dev-tools/scripts#githubprspy
>
> Running the tool shows that we're not very successful in keeping the two
> in sync, we also forget to close PRs after JIRAs are resolved:
>
> $ ./githubPRs.py
> Lucene/Solr Github PR report
> 
> Number of open Pull Requests: 208
>
> PRs lacking JIRA reference in title
>   #882: Add versions.lock entry for "org.apache.yetus:audience-annotations"
>   #880: Tweak header format.
>   [… many more ]
>
> Open PRs with a resolved JIRA
>   #723: SOLR-13545 status=Closed, resolution=Fixed,
> resolutiondate=2019-06-22T13:04:47.000+ (SOLR-13545: AutoClose stream
> in ContentStreamUpdateRequest)
>   #643: SOLR-13391 status=Resolved, resolution=Resolved,
> resolutiondate=2019-04-12T14:09:27.000+ (SOLR-13391: Add variance and
> standard deviation stream evaluators)
>   [… many more]
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> 16. sep. 2019 kl. 20:57 skrev Andrzej Białecki :
>
>
> On 16 Sep 2019, at 19:38, Yonik Seeley  wrote:
>
> >  - PR is opened - should automatically create a jira if it doesn’t exist
> yet
>
> What were the reasons behind this? Shouldn't a JIRA just be optional if we
> started with a PR?
>
>
> That’s why we need to discuss this :)
>
> I remember that at some point ASF treated JIRA as the system of record for
> the legal validation of contributions.
>
> I also remember that at some point we used to say that if a discussion
> didn’t happen in JIRA then it didn’t exist, and that any decisions
> regarding code should be recorded in JIRA because we can’t expect people to
> monitor and contribute / object to decisions happening elsewhere.
>
> —
>
> Andrzej Białecki
>
>
>

-- 
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)

Re: The Lucene Solr Gradle Build Game plan

[Mark Miller]

This is no reproduce output yet, in my mind that is waiting with the tests
seed work that I started on but needs review and finishing.

tests_jvms is jvms and no user param naming is yet thought out or final
when it comes to modifying the build. I've instead stuck to exposing few
things and expecting them to get exposed as needed.

tests_jvms is basically a default translation from ant, I'm hoping someone
else want's to own what happens here.

bq. how do we get *all* of the logging from failed tests to be written to
stdout/stderr when a test fail?

Most things are pretty easy in Gradle. If you want to do it or want me to
do it, please file a JIRA. If it's not a quick fix for someone, it would be
great to start tracking what each individual thinks is critical in JIRA.

All the test stuff needs review and some tweaks (has not changed sine last
email thread which you may not have caught), last I remember Dawid signed
up for it starting around today (15th) ;) See previous email chain.

bq. What's the plan as far as things like "ant beast"

beast stuff will come later

tests.jvms is tests_jvms :) The other settings along these lines are either
part of the test Runner (not most) or the Ant test launcher from
RandomizedTesting. We will have some form of beasting, but that's almost
last in line in terms of what I'm doing now.

- Mark

On Mon, Sep 16, 2019 at 1:59 PM Chris Hostetter 
wrote:

>
> Some misc questions after playing around with gradle on this branch for a
> bit today in no particular order...
>
> : tests_jvms=5
> ...
> : test_jvms is controlled by us and defaults to the number of cores
> detected
> : / 2.
>
> If tests_jvms is a lucene/solr specific property, that needs to
> be in a users "multi-project" "~/.gradle" file ... shouldn't we name it
> namespace it with something like "lucene_solr_test_jvms" to make that
> clear reduce future confusion/collsion?  (as i anticipate this wo't be the
> last prop we may need)
>
> How do we get "reproduce with" type output (by default) when a test fails?
>
> For that matter, how do we get *all* of the logging from failed tests to
> be written to stdout/stderr when a test fail?  ... this is pretty
> important for making jenkin's console log a good "one stop shop" for
> understanding everything that went wrong in a build.
>
> ("--debug" and "--info" seem to do this, but they write a *TON* more
> gradle specific shit then "ant test" type logging use to produce by
> default, and don't care if the test passes or not ... which would make
> them way too excessively verbose for a jenkins build)
>
> What's the plan as far as things like "ant beast", "-Dtests.dups",
> "-Dtests.iters" & "-Dtests.jvms" ? ... those types of options are all
> pretty critical for diagnosing and troubleshooting failures.
> ("-Dtests.jvms=1" is important for figuring out if/how the execution of
> one test might polute static variables in the JVM that cause a failure in
> a subsequent class in the sane JVM)
>
> is there a simple option to prevent gradle from using curses even though
> it detects it's being run in a tty?
>
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Mark

http://about.me/markrmiller

Re: Notes from the committers' meeting at Activate 2019

[Jan Høydahl]

Is there any reason at all that we need to hold on to JIRA? ASF allows us to 
move all issue handling over to GH, I'd like us to consider such a move.

Until then, I made a script that "diffs" GH and JIRA and outputs a report, see 
https://github.com/apache/lucene-solr/tree/master/dev-tools/scripts#githubprspy

Running the tool shows that we're not very successful in keeping the two in 
sync, we also forget to close PRs after JIRAs are resolved:

$ ./githubPRs.py 
Lucene/Solr Github PR report

Number of open Pull Requests: 208

PRs lacking JIRA reference in title
  #882: Add versions.lock entry for "org.apache.yetus:audience-annotations"
  #880: Tweak header format.
  [… many more ]

Open PRs with a resolved JIRA
  #723: SOLR-13545 status=Closed, resolution=Fixed, 
resolutiondate=2019-06-22T13:04:47.000+ (SOLR-13545: AutoClose stream in 
ContentStreamUpdateRequest)
  #643: SOLR-13391 status=Resolved, resolution=Resolved, 
resolutiondate=2019-04-12T14:09:27.000+ (SOLR-13391: Add variance and 
standard deviation stream evaluators)
  [… many more]

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 16. sep. 2019 kl. 20:57 skrev Andrzej Białecki :
> 
> 
>> On 16 Sep 2019, at 19:38, Yonik Seeley > > wrote:
>> 
>> >  - PR is opened - should automatically create a jira if it doesn’t exist 
>> > yet
>> 
>> What were the reasons behind this? Shouldn't a JIRA just be optional if we 
>> started with a PR?
> 
> That’s why we need to discuss this :)
> 
> I remember that at some point ASF treated JIRA as the system of record for 
> the legal validation of contributions.
> 
> I also remember that at some point we used to say that if a discussion didn’t 
> happen in JIRA then it didn’t exist, and that any decisions regarding code 
> should be recorded in JIRA because we can’t expect people to monitor and 
> contribute / object to decisions happening elsewhere.
> 
> —
> 
> Andrzej Białecki
> 

Re: Intervals in Solr json request

[Mikhail Khludnev]

Jason,
Here we go https://issues.apache.org/jira/browse/SOLR-13764


On Mon, Sep 16, 2019 at 3:05 PM Jason Gerlowski 
wrote:

> Hi Mikhail,
>
> I'm having trouble understanding the exact syntax you're proposing.
> Is there a jira where the syntax is described in a little more detail?
>  If not, would you care to put together a writeup on a jira somewhere?
>  It's hard (for me at least) to weigh in as things are currently.
>
> Best,
>
> Jason
>
> On Sun, Sep 8, 2019 at 3:25 PM Mikhail Khludnev  wrote:
> >
> > Ok. It might be a parser referring to a json object under some new
> property
> >
> > {
> >"query": {
> >"jinterval":"just a name"  // introducing new QPPlugin
> >   },
> >"jparams": {   // introducing new top-level entry
> > "just a name": {
> >   "or":["foo",
> >   "bar",
> >{ "unordered":
> > ["bag",
> >   "baz",
> >   "ban",
> >{ "phrase": ["moo","foo"]}
> >  ]
> > }
> >   ],
> >"field":"text_content"
> >  }
> >  }
> > }
> >
> > Can we consider it as a spec for the new feature?
> >
> > On Sun, Sep 8, 2019 at 12:16 AM Mikhail Khludnev 
> wrote:
> >>
> >> Thanks for your warm responses. I encounter Intervals, and considering
> introducing them in Solr JSON Request API.
> >> Following Query DSL approach gives me something like
> >> "interval":{  "or":["foo",
> >>   "bar",
> >>{"interval": { "unordered":
> >> ["bag",
> >>   "baz",
> >>   "ban",
> >>{ "interval":{ "phrase":
> ["moo","foo"]} }
> >>  ]}
> >>   }
> >> ],
> >>"field":"text_content"}
> >> So, it implies creating {!inteval} query parser, which handles local
> param in a certain way, eg  it shouldn't support "or" and "phrase" an the
> same node.
> >> Not sure how to propagate "filed" to term nodes.
> >>
> >> I'd rather want to have more control over syntax and JsonQueryConverter.
> >>  "interval":{  "or":["foo",
> >>   "bar",
> >>{ "unordered":
> >> ["bag",
> >>   "baz",
> >>   "ban",
> >>{ "phrase": ["moo","foo"]}
> >>  ]}
> >>   }
> >>   ],
> >>"field":"text_content"}
> >>
> >> Any ideas, preferences?
> >>
> >> On Sat, Sep 7, 2019 at 12:03 AM Mikhail Khludnev 
> wrote:
> >>>
> >>> Hello,
> >>>
> >>> Finally we let users to send span queries via XML (yeah) query parser.
> But I feel awkward to invoke XML under Json. Straightforward approach lead
> us to bunch of span[Or|And|Not|Etc] QParser plugins. Are there any more
> elegant ideas?
> >>>
> >>> --
> >>> Sincerely yours
> >>> Mikhail Khludnev
> >>
> >>
> >>
> >> --
> >> Sincerely yours
> >> Mikhail Khludnev
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
Sincerely yours
Mikhail Khludnev

Re: The Lucene Solr Gradle Build Game plan

[Chris Hostetter]

Some misc questions after playing around with gradle on this branch for a 
bit today in no particular order...

: tests_jvms=5
...
: test_jvms is controlled by us and defaults to the number of cores detected
: / 2.

If tests_jvms is a lucene/solr specific property, that needs to 
be in a users "multi-project" "~/.gradle" file ... shouldn't we name it 
namespace it with something like "lucene_solr_test_jvms" to make that 
clear reduce future confusion/collsion?  (as i anticipate this wo't be the 
last prop we may need)

How do we get "reproduce with" type output (by default) when a test fails? 

For that matter, how do we get *all* of the logging from failed tests to 
be written to stdout/stderr when a test fail?  ... this is pretty 
important for making jenkin's console log a good "one stop shop" for 
understanding everything that went wrong in a build.  

("--debug" and "--info" seem to do this, but they write a *TON* more 
gradle specific shit then "ant test" type logging use to produce by 
default, and don't care if the test passes or not ... which would make 
them way too excessively verbose for a jenkins build)

What's the plan as far as things like "ant beast", "-Dtests.dups", 
"-Dtests.iters" & "-Dtests.jvms" ? ... those types of options are all 
pretty critical for diagnosing and troubleshooting failures.  
("-Dtests.jvms=1" is important for figuring out if/how the execution of 
one test might polute static variables in the JVM that cause a failure in 
a subsequent class in the sane JVM)

is there a simple option to prevent gradle from using curses even though 
it detects it's being run in a tty?


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Notes from the committers' meeting at Activate 2019

[Andrzej Białecki]

> On 16 Sep 2019, at 19:38, Yonik Seeley  wrote:
> 
> >  - PR is opened - should automatically create a jira if it doesn’t exist yet
> 
> What were the reasons behind this? Shouldn't a JIRA just be optional if we 
> started with a PR?

That’s why we need to discuss this :)

I remember that at some point ASF treated JIRA as the system of record for the 
legal validation of contributions.

I also remember that at some point we used to say that if a discussion didn’t 
happen in JIRA then it didn’t exist, and that any decisions regarding code 
should be recorded in JIRA because we can’t expect people to monitor and 
contribute / object to decisions happening elsewhere.

—

Andrzej Białecki

Re: The Lucene Solr Gradle Build Game plan

[Mark Miller]

Maybe making a backup first would improve things?

My first feeling was just let the user do this, but I work in a bunch of
envs and it's very nice to have.

At worst the task description and any doc can say WARNING!: we are going to
edit your file if you do this, there will be a .bak or something.

Mark

On Mon, Sep 16, 2019 at 1:36 PM Mark Miller  wrote:

> That's against Gradle best practices, if you look it up, this is the way
> Gradle is intended to work. Settings like
>
> That task description can warn about editing your file, but I feel it's
> pretty safe given that you have to decide to use it.
>
> We don't remove anything and we don't change controversial settings
> (unless they break us, like configure on demand).
>
> Settings like org.gradle.workers.max are used by the Daemon and meant to
> be pretty universal as Daemons are intended to be reused.
>
> Another property is ours - if you have overridden it, asking for these
> settings changes will change it. Good?
>
> Configure on demand will break us and is experimental and is noted around
> a variety of bugs, so we disable it.
>
> Sure, just smashing up someones ~/.gradle/gradle.properties is not nice,
> but making required and performance adjustments at request seems great to
> me.
>
> - Mark
>
> On Sun, Sep 15, 2019 at 10:56 PM Gus Heck  wrote:
>
>> Haven't looked at your code, but it sounds a bit scary for you (or
>> whomever comes after you) to touch my ~/.gradle/gradle.properties.
>> Sometimes I have some pretty important stuff there Can we use the
>> project specific gradle.properties in the project root dir and make it
>> .gitignore?
>>
>> On Sun, Sep 15, 2019 at 8:47 PM Mark Miller 
>> wrote:
>>
>>> Okay, there is now a task called defaultUserConfig
>>>
>>> You can do gw defaultUserConfig to set some recommended settings in
>>> ~/.gradle/gradle.properties. By default they should be more relaxed, but we
>>> can tune as needed.
>>>
>>> For better build performance but also more resource usage you can do:
>>>
>>> gw defaultUserConfig --style=aggressive
>>>
>>> Mark
>>>
>>> On Sun, Sep 15, 2019 at 5:50 PM Mark Miller 
>>> wrote:
>>>
 By the way, I did hear about the hack day and some Gradle testing,
 which is great, and very useful. Totally needed, but we also need a bit of
 a more deep core developer view of things vs the old build as well. The
 type of stuff that’s much harder to tease out than verifying all the new
 build targets and such. Of course a lot of that can come after we switch,
 but I have a sneaky feeling some core devs will have deep opinions about
 certain things.

 Ill add a new task for default config setup.

 Mark

 On Sun, Sep 15, 2019 at 5:12 PM Mark Miller 
 wrote:

> I'll just detail it out here as well:
>
> You can configure parallelism in ~/.gradle/gradle.properties
>
> org.gradle.workers.max=2
> tests_jvms=5
>
> org.gradle.workers.max is controlled by gradle and defaults to the
> number of cores detected - I wish I could change to divided by 2.
> test_jvms is controlled by us and defaults to the number of cores
> detected / 2.
>
> org.gradle.workers.max  controls the total number of jvms that will be
> run in parallel - for tasks or tests or whatever gradle is doing.
> test_jvms controls how many parallel jvms a module will use for tests,
> but that is also limited by org.gradle.workers.max
>
> You should try setting both to cores / 2 and work down from there if
> needed.
>
> When running tests across multiple modules, org.gradle.workers.max is
> the actual limit for test jvms spun up because each module is only limited
> to test_jvms, but ALL tasks are limited to org.gradle.workers.max and
> module tasks are run in parallel by default.
>
> By setting them the same, we get similar behavior whether we run tests
> from the root directory of the project (all tests) or from a single module
> (say solr-core).
>
> On Sun, Sep 15, 2019 at 5:03 PM Mark Miller 
> wrote:
>
>> I've added more about that here:
>> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build
>>
>> It's configurable, but difficult for us to choose a default based on
>> cores as far as I can tell and gradles default, which is based on cores
>> detected, is too high, especially because of hyperthreading.
>>
>> Probably the best we can do is default it to a hard 2 or 4 and let
>> people raise it depending on what wait you want to error. In both cases 
>> the
>> majority of people will want to change it.
>>
>> On Sun, Sep 15, 2019 at 3:39 PM Gus Heck  wrote:
>>
>>> Not sure if you heard, but about a half a dozen folks tried it out
>>> on macs and one on windows at the hack day on Tuesday before Activate. 
>>> It
>>> caused some scrambling for sharing of power bricks (a single 

Re: The Lucene Solr Gradle Build Game plan

[Mark Miller]

That's against Gradle best practices, if you look it up, this is the way
Gradle is intended to work. Settings like

That task description can warn about editing your file, but I feel it's
pretty safe given that you have to decide to use it.

We don't remove anything and we don't change controversial settings (unless
they break us, like configure on demand).

Settings like org.gradle.workers.max are used by the Daemon and meant to be
pretty universal as Daemons are intended to be reused.

Another property is ours - if you have overridden it, asking for these
settings changes will change it. Good?

Configure on demand will break us and is experimental and is noted around a
variety of bugs, so we disable it.

Sure, just smashing up someones ~/.gradle/gradle.properties is not nice,
but making required and performance adjustments at request seems great to
me.

- Mark

On Sun, Sep 15, 2019 at 10:56 PM Gus Heck  wrote:

> Haven't looked at your code, but it sounds a bit scary for you (or
> whomever comes after you) to touch my ~/.gradle/gradle.properties.
> Sometimes I have some pretty important stuff there Can we use the
> project specific gradle.properties in the project root dir and make it
> .gitignore?
>
> On Sun, Sep 15, 2019 at 8:47 PM Mark Miller  wrote:
>
>> Okay, there is now a task called defaultUserConfig
>>
>> You can do gw defaultUserConfig to set some recommended settings in
>> ~/.gradle/gradle.properties. By default they should be more relaxed, but we
>> can tune as needed.
>>
>> For better build performance but also more resource usage you can do:
>>
>> gw defaultUserConfig --style=aggressive
>>
>> Mark
>>
>> On Sun, Sep 15, 2019 at 5:50 PM Mark Miller 
>> wrote:
>>
>>> By the way, I did hear about the hack day and some Gradle testing, which
>>> is great, and very useful. Totally needed, but we also need a bit of a more
>>> deep core developer view of things vs the old build as well. The type of
>>> stuff that’s much harder to tease out than verifying all the new build
>>> targets and such. Of course a lot of that can come after we switch, but I
>>> have a sneaky feeling some core devs will have deep opinions about certain
>>> things.
>>>
>>> Ill add a new task for default config setup.
>>>
>>> Mark
>>>
>>> On Sun, Sep 15, 2019 at 5:12 PM Mark Miller 
>>> wrote:
>>>
 I'll just detail it out here as well:

 You can configure parallelism in ~/.gradle/gradle.properties

 org.gradle.workers.max=2
 tests_jvms=5

 org.gradle.workers.max is controlled by gradle and defaults to the
 number of cores detected - I wish I could change to divided by 2.
 test_jvms is controlled by us and defaults to the number of cores
 detected / 2.

 org.gradle.workers.max  controls the total number of jvms that will be
 run in parallel - for tasks or tests or whatever gradle is doing.
 test_jvms controls how many parallel jvms a module will use for tests,
 but that is also limited by org.gradle.workers.max

 You should try setting both to cores / 2 and work down from there if
 needed.

 When running tests across multiple modules, org.gradle.workers.max is
 the actual limit for test jvms spun up because each module is only limited
 to test_jvms, but ALL tasks are limited to org.gradle.workers.max and
 module tasks are run in parallel by default.

 By setting them the same, we get similar behavior whether we run tests
 from the root directory of the project (all tests) or from a single module
 (say solr-core).

 On Sun, Sep 15, 2019 at 5:03 PM Mark Miller 
 wrote:

> I've added more about that here:
> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build
>
> It's configurable, but difficult for us to choose a default based on
> cores as far as I can tell and gradles default, which is based on cores
> detected, is too high, especially because of hyperthreading.
>
> Probably the best we can do is default it to a hard 2 or 4 and let
> people raise it depending on what wait you want to error. In both cases 
> the
> majority of people will want to change it.
>
> On Sun, Sep 15, 2019 at 3:39 PM Gus Heck  wrote:
>
>> Not sure if you heard, but about a half a dozen folks tried it out on
>> macs and one on windows at the hack day on Tuesday before Activate. It
>> caused some scrambling for sharing of power bricks (a single run of the
>> tests eats 70% of a fully charged 2018 macbook pro battery in 45 min), 
>> but
>> the good news is it only failed on well known flaky tests and on the one
>> windows machine and that in the PDF building for the ref guide (and there
>> was that small bit with the error message and the AtomicBoolean that I
>> fixed). I've heard some opinions that maybe we don't need the PDF version
>> in the future, The one bit of feedback that came out of it was it would 

Re: Notes from the committers' meeting at Activate 2019

[Yonik Seeley]

>  - PR is opened - should automatically create a jira if it doesn’t exist
yet

What were the reasons behind this? Shouldn't a JIRA just be optional if we
started with a PR?

-Yonik

Re: Git notifications and threading

[Jan Høydahl]

Agree, would be much better if the issue title came first. When reading mails 
on my phone I have to open each mail to see which issue it is about since the 
email client will only display N first chars of the mail title :(

Jan Høydahl

> 16. sep. 2019 kl. 17:42 skrev Erick Erickson :
> 
> Hmmm, I’ve noticed one thing that would be nice. Currently I’m using OS X 
> mail client. It tries to thread conversations
> 
> Ideally, anything with the title containing:
> 
> SOLR-13734 JWTAuthPlugin to support multiple issuers”
> 
> would be grouped together. But since the title is prepended with:
> "[GitHub] [lucene-solr] gerlowskija commented on issue #860: “
> 
> Anyone know of a work-around? Or am I misremembering that I just put all the 
> Git notifications in a folder and used to ignore them?
> 
> Erick
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Notes from the committers' meeting at Activate 2019

[Andrzej Białecki]

> On 16 Sep 2019, at 17:55, Ishan Chattopadhyaya  
> wrote:
> 
>> Committers attending (at least some part of) the meeting, in no particular 
>> order: Erik Hatcher, Anshum Gupta, David Smiley, Gus Heck, Noble Paul, Varun 
>> Thacker, Ishan Chattopadhyaya, Tomás Löbbe, and yours truly.
> 
> Also, Tim Allison, Joel Bernstein, Jason Gerlowski. (Did we miss out
> someone else too?)

Gah, of course, sorry guys - I was even sitting next to them…

> 
> On Mon, Sep 16, 2019 at 7:24 AM Andrzej Białecki  wrote:
>> 
>> Hey folks,
>> 
>> Some of the committers attended the Activate 2019 conference, which took 
>> place in Washington, DC on Sep 10-13.
>> 
>> The schedule was packed, so we managed to only have a ~1hr meeting during a 
>> lunch break - nonetheless, I think it was still very productive!
>> 
>> Committers attending (at least some part of) the meeting, in no particular 
>> order: Erik Hatcher, Anshum Gupta, David Smiley, Gus Heck, Noble Paul, Varun 
>> Thacker, Ishan Chattopadhyaya, Tomás Löbbe, and yours truly.
>> 
>> Here are the notes I took - those attending, feel free to clear up any 
>> errors, omissions or misunderstandings.
>> 
>> - Clean up tests that needlessly use AbstractDistribZk…
>>- because this class creates a control collection, which in many cases is 
>> not needed.
>> 
>> - Consider reusing a single MiniSolrCloudCluster instance for multiple test 
>> suites
>>- always use unique collection names per suite / test
>>- some suites won’t be able to use this due to a particular setup or 
>> side-effects (sysprops, expected metrics, etc)
>>- those that can should execute much faster
>> 
>> - Deprecations in 8x - we still need to actually remove the stuff from 
>> master:
>>- old blob store
>>- old spatial
>>- other things?
>> 
>> - Replace NamedList with MapWriter?
>>- avoid creating objects during serialization
>>- big undertaking, but transition piece by piece
>>- example: ExportHandler / ExportWriter
>>- new API should use MapWriter instead of NamedList / Map
>>- public API changes have to go through deprecation in 8x and removal 
>> only in 9
>> 
>> - We have three different and partially incomplete faceting impls
>>- do we want to do something about it to reduce confusion and code 
>> footprint?
>> 
>> - V2 APIs are incomplete, there’s no workflow to maintain them in sync with 
>> v1. Proposed strategy to improve this:
>>- move SolrJ to v2 - this could be done soon
>>- move Solr internally to use v2
>>- move tests to use v2 by default.
>>- RefGuide in 9.0 should show v2 examples by default
>>- deprecate v1
>>- come up with a better way of creating v2 api metadata (annotations?)
>> 
>> - Promote GitHub-centric approach to dev & collaboration
>>- PRs as the main method for submitting contributions
>>- How to Contribute should be the first section of the github page
>>- PR is opened - should automatically create a jira if it doesn’t exist 
>> yet
>>- discourage using patches when code review is expected.
>>- PR is more inviting for collaboration than a patch
>>- downside: PR is only for a single branch (no backport integration)
>>- travis integration?
>>- or use Github Actions for automated precommits, tests
>> 
>> - Javadocs, typos, small ref guide changes should not require a Jira issue 
>> with its overheads
>> 
>> —--
>> 
>> Andrzej Białecki
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 8.3 release

[Adrien Grand]

+1 to start working on 8.3

Did you mean "start a vote" when you wrote "release the artifacts"? It
got me wondering because I don't think we frequently managed to go
from cutting a branch to releasing artifacts in so little time in the
past.

On Mon, Sep 16, 2019 at 5:52 PM Ishan Chattopadhyaya
 wrote:
>
> Hi all,
> We have a lot of unreleased features and fixes. I propose that we cut
> a 8.3 branch in two weeks (in order to have sufficient time to bake in
> all in-progress features). If there are no objections to doing so, I
> can volunteer for the release as an RM and plan for cutting a release
> branch on 30 September (and release the artifacts about 3-4 days after
> that).
>
> WDYT?
> Regards,
> Ishan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>


-- 
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

BadApple report

[Erick Erickson]

I’m going to suspend these until we build up a better backlog of tests since a 
number of machines weren’t being collected by Hoss’ rollups. I’ll continue to 
gather the rollups every week, but for a while I don’t think it’s worth 
cluttering your inbox.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 8.3 release

[Gus Heck]

Sounds good to me except I think we need to make ref guide part of
releases. We are now able to commit docs along with code and an issue
should not be resolved and features not merged unless docs are in and ready
for release.

On Mon, Sep 16, 2019, 11:52 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Hi all,
> We have a lot of unreleased features and fixes. I propose that we cut
> a 8.3 branch in two weeks (in order to have sufficient time to bake in
> all in-progress features). If there are no objections to doing so, I
> can volunteer for the release as an RM and plan for cutting a release
> branch on 30 September (and release the artifacts about 3-4 days after
> that).
>
> WDYT?
> Regards,
> Ishan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Notes from the committers' meeting at Activate 2019

[Ishan Chattopadhyaya]

> Committers attending (at least some part of) the meeting, in no particular 
> order: Erik Hatcher, Anshum Gupta, David Smiley, Gus Heck, Noble Paul, Varun 
> Thacker, Ishan Chattopadhyaya, Tomás Löbbe, and yours truly.

Also, Tim Allison, Joel Bernstein, Jason Gerlowski. (Did we miss out
someone else too?)

On Mon, Sep 16, 2019 at 7:24 AM Andrzej Białecki  wrote:
>
> Hey folks,
>
> Some of the committers attended the Activate 2019 conference, which took 
> place in Washington, DC on Sep 10-13.
>
> The schedule was packed, so we managed to only have a ~1hr meeting during a 
> lunch break - nonetheless, I think it was still very productive!
>
> Committers attending (at least some part of) the meeting, in no particular 
> order: Erik Hatcher, Anshum Gupta, David Smiley, Gus Heck, Noble Paul, Varun 
> Thacker, Ishan Chattopadhyaya, Tomás Löbbe, and yours truly.
>
> Here are the notes I took - those attending, feel free to clear up any 
> errors, omissions or misunderstandings.
>
> - Clean up tests that needlessly use AbstractDistribZk…
> - because this class creates a control collection, which in many cases is 
> not needed.
>
> - Consider reusing a single MiniSolrCloudCluster instance for multiple test 
> suites
> - always use unique collection names per suite / test
> - some suites won’t be able to use this due to a particular setup or 
> side-effects (sysprops, expected metrics, etc)
> - those that can should execute much faster
>
> - Deprecations in 8x - we still need to actually remove the stuff from master:
> - old blob store
> - old spatial
> - other things?
>
> - Replace NamedList with MapWriter?
> - avoid creating objects during serialization
> - big undertaking, but transition piece by piece
> - example: ExportHandler / ExportWriter
> - new API should use MapWriter instead of NamedList / Map
> - public API changes have to go through deprecation in 8x and removal 
> only in 9
>
> - We have three different and partially incomplete faceting impls
> - do we want to do something about it to reduce confusion and code 
> footprint?
>
> - V2 APIs are incomplete, there’s no workflow to maintain them in sync with 
> v1. Proposed strategy to improve this:
> - move SolrJ to v2 - this could be done soon
> - move Solr internally to use v2
> - move tests to use v2 by default.
> - RefGuide in 9.0 should show v2 examples by default
> - deprecate v1
> - come up with a better way of creating v2 api metadata (annotations?)
>
> - Promote GitHub-centric approach to dev & collaboration
> - PRs as the main method for submitting contributions
> - How to Contribute should be the first section of the github page
> - PR is opened - should automatically create a jira if it doesn’t exist 
> yet
> - discourage using patches when code review is expected.
> - PR is more inviting for collaboration than a patch
> - downside: PR is only for a single branch (no backport integration)
> - travis integration?
> - or use Github Actions for automated precommits, tests
>
> - Javadocs, typos, small ref guide changes should not require a Jira issue 
> with its overheads
>
> —--
>
> Andrzej Białecki
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Direct I/O

[Michael Sokolov]

https://bugs.openjdk.java.net/browse/JDK-8189192 makes it appear that
Direct I/O is (or may be?) available now in JDK's since JDK10. Should
we try using that API in NativeUnixDirectory in order to avoid JNI
calls?

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

8.3 release

[Ishan Chattopadhyaya]

Hi all,
We have a lot of unreleased features and fixes. I propose that we cut
a 8.3 branch in two weeks (in order to have sufficient time to bake in
all in-progress features). If there are no objections to doing so, I
can volunteer for the release as an RM and plan for cutting a release
branch on 30 September (and release the artifacts about 3-4 days after
that).

WDYT?
Regards,
Ishan

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Git notifications and threading

[Erick Erickson]

Hmmm, I’ve noticed one thing that would be nice. Currently I’m using OS X mail 
client. It tries to thread conversations

Ideally, anything with the title containing:

SOLR-13734 JWTAuthPlugin to support multiple issuers”

would be grouped together. But since the title is prepended with:
"[GitHub] [lucene-solr] gerlowskija commented on issue #860: “

Anyone know of a work-around? Or am I misremembering that I just put all the 
Git notifications in a folder and used to ignore them?

Erick
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Intervals in Solr json request

[Jason Gerlowski]

Hi Mikhail,

I'm having trouble understanding the exact syntax you're proposing.
Is there a jira where the syntax is described in a little more detail?
 If not, would you care to put together a writeup on a jira somewhere?
 It's hard (for me at least) to weigh in as things are currently.

Best,

Jason

On Sun, Sep 8, 2019 at 3:25 PM Mikhail Khludnev  wrote:
>
> Ok. It might be a parser referring to a json object under some new property
>
> {
>"query": {
>"jinterval":"just a name"  // introducing new QPPlugin
>   },
>"jparams": {   // introducing new top-level entry
> "just a name": {
>   "or":["foo",
>   "bar",
>{ "unordered":
> ["bag",
>   "baz",
>   "ban",
>{ "phrase": ["moo","foo"]}
>  ]
> }
>   ],
>"field":"text_content"
>  }
>  }
> }
>
> Can we consider it as a spec for the new feature?
>
> On Sun, Sep 8, 2019 at 12:16 AM Mikhail Khludnev  wrote:
>>
>> Thanks for your warm responses. I encounter Intervals, and considering 
>> introducing them in Solr JSON Request API.
>> Following Query DSL approach gives me something like
>> "interval":{  "or":["foo",
>>   "bar",
>>{"interval": { "unordered":
>> ["bag",
>>   "baz",
>>   "ban",
>>{ "interval":{ "phrase": 
>> ["moo","foo"]} }
>>  ]}
>>   }
>> ],
>>"field":"text_content"}
>> So, it implies creating {!inteval} query parser, which handles local param 
>> in a certain way, eg  it shouldn't support "or" and "phrase" an the same 
>> node.
>> Not sure how to propagate "filed" to term nodes.
>>
>> I'd rather want to have more control over syntax and JsonQueryConverter.
>>  "interval":{  "or":["foo",
>>   "bar",
>>{ "unordered":
>> ["bag",
>>   "baz",
>>   "ban",
>>{ "phrase": ["moo","foo"]}
>>  ]}
>>   }
>>   ],
>>"field":"text_content"}
>>
>> Any ideas, preferences?
>>
>> On Sat, Sep 7, 2019 at 12:03 AM Mikhail Khludnev  wrote:
>>>
>>> Hello,
>>>
>>> Finally we let users to send span queries via XML (yeah) query parser. But 
>>> I feel awkward to invoke XML under Json. Straightforward approach lead us 
>>> to bunch of span[Or|And|Not|Etc] QParser plugins. Are there any more 
>>> elegant ideas?
>>>
>>> --
>>> Sincerely yours
>>> Mikhail Khludnev
>>
>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Notes from the committers' meeting at Activate 2019

[Andrzej Białecki]

Hey folks,

Some of the committers attended the Activate 2019 conference, which took place 
in Washington, DC on Sep 10-13.

The schedule was packed, so we managed to only have a ~1hr meeting during a 
lunch break - nonetheless, I think it was still very productive!

Committers attending (at least some part of) the meeting, in no particular 
order: Erik Hatcher, Anshum Gupta, David Smiley, Gus Heck, Noble Paul, Varun 
Thacker, Ishan Chattopadhyaya, Tomás Löbbe, and yours truly.

Here are the notes I took - those attending, feel free to clear up any errors, 
omissions or misunderstandings.

- Clean up tests that needlessly use AbstractDistribZk…
- because this class creates a control collection, which in many cases is 
not needed.

- Consider reusing a single MiniSolrCloudCluster instance for multiple test 
suites
- always use unique collection names per suite / test
- some suites won’t be able to use this due to a particular setup or 
side-effects (sysprops, expected metrics, etc)
- those that can should execute much faster

- Deprecations in 8x - we still need to actually remove the stuff from master:
- old blob store
- old spatial
- other things?

- Replace NamedList with MapWriter?
- avoid creating objects during serialization
- big undertaking, but transition piece by piece
- example: ExportHandler / ExportWriter
- new API should use MapWriter instead of NamedList / Map
- public API changes have to go through deprecation in 8x and removal only 
in 9

- We have three different and partially incomplete faceting impls
- do we want to do something about it to reduce confusion and code 
footprint?

- V2 APIs are incomplete, there’s no workflow to maintain them in sync with v1. 
Proposed strategy to improve this:
- move SolrJ to v2 - this could be done soon
- move Solr internally to use v2
- move tests to use v2 by default.
- RefGuide in 9.0 should show v2 examples by default
- deprecate v1
- come up with a better way of creating v2 api metadata (annotations?)

- Promote GitHub-centric approach to dev & collaboration
- PRs as the main method for submitting contributions 
- How to Contribute should be the first section of the github page
- PR is opened - should automatically create a jira if it doesn’t exist yet
- discourage using patches when code review is expected.
- PR is more inviting for collaboration than a patch
- downside: PR is only for a single branch (no backport integration)
- travis integration?
- or use Github Actions for automated precommits, tests

- Javadocs, typos, small ref guide changes should not require a Jira issue with 
its overheads

—--

Andrzej Białecki


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org