Re: Notes from the committers' meeting at Activate 2019

[Gus Heck]

I wrote quickly and didn't expound much, let me clarify that my comments
are in reference to having bug tracking in GitHub. Using the mirror doesn't
bother me since the system of record is apache gitbox (the GitHub mirror is
WAY better UI than gitbox). Having the record of what bugs were resolved in
what versions and the thought that went into it, only exist outside apache
is all I'm worried about. I'm not anti git, nor am I anti code review in
GitHub, as long as major direction changes and decisions s are summarized
in jira, or in code comments. I also have generally assumed that pull
request reviews were meant to mostly be code review, i.e. focused comments
on specific lines or classes . Discussion of how to solve bugs or possible
directions for features would be in jira.

In more concrete terms one thing I will definitely miss if we go to GitHub
is the tabular view of issues, especially the ability to sort by issue ID
which I use regularly to get a view of (roughly) chronologically history of
changes on a topic. I really find their way of listing issues very hard to
read. It would be much easier to scan down a column of milestones for
example.

By the way, how would we plan to handle fix versions in GitHub issues?
Milestones? What about affects version etc.

And I don't agree that "everyone is on GitHub" I still have yet to
encounter a single client site that was using GitHub (~20 clients in 6
years ranging from 1 man startups to Fortune 50 companies). Plenty using
git, often bitbucket, but nobody using github itself. Plenty of open source
projects (including minor ones I started) use GitHub... But the idea that
folks out there won't know how to adapt to anything other than GitHub seems
preposterous to me. The only folks who are not going to be able to absorb a
new bug tracking system are the very junior devs, few of whom will be
looking to contribute to Solr I think. Honestly the popularity of python as
a teaching language is a much bigger threat to our ability to attract fresh
folks than our bug tracker choice...

So I'm not at all against GitHub having some role, but the degree of
dependency on outside services seems important. I guess it's possible to
see it as a question of what you consider peripheral vs core. I think
records of the issues are core, but code review comments not so much.

Also it seems ironic that I'm writing this mail to clarify I'm not entirely
against Github as I wait for a *forced* reboot to finish on my Windows
machine... One that hit while I was in the middle of something else...

-Gus


On Tue, Sep 17, 2019, 9:01 PM Mark Miller  wrote:

> Also, just FYI, I say that as someone that kind of prefers patches and
> JIRA for 90% of what I do. I’ve been doing this same shit for like half my
> life, I’m not looking for fancy new tools for the hell of it either. I like
> patches. It’s just going to happen. And I see a huge pool of free resources
> in the meantime, wow those workflow limits are not too bad at all. I could
> stop another new test that takes 2 minutes from coming in non nightly. Now
> that’s practically interesting.
>
> Mark
>
> On Tue, Sep 17, 2019 at 7:39 PM Mark Miller  wrote:
>
>> I think that is a little over the top.
>>
>> As it is the majority of dev and pr's and action is moving to GitHub,
>> whether anyone is from Syria or not.
>>
>> If we decided, like most other communities on Gods green earth, to tell
>> our community we are going GitHub first it and expect committers to not
>> avoid all of our checks by just sticking to patches, the practical
>> differences don't have to be much beyond that. Apache GitBox is not going
>> away, it's easy to clearly spell out that those without access to GitHub
>> can provide a patch as we would allow any committer without access or moral
>> quandaries the same obviously. Making how to contribute a patch and use
>> JIRA alternate doc for those with GitHub issues is pretty low effort.
>>
>> JIRA is a little different, I'm not as sold on leaving it, but really
>> it's the same thing if almost all of the dev community starts using it for
>> the bulk of what would be in JIRA, already lots of JIRA related comments
>> and review have gone there - most stuff is just split instead of "free and
>> available" - GitHub is lacking, JIRA is lacking.  Given that every damn
>> company and project is on GitHub, this is just the way it will continue to
>> go. So leaving JIRA up for history and those without access to GitHub would
>> be the same too.
>>
>> And if M$ does anything with GitHub, the universe will collectively move
>> on, with 90% of the world in the same spot. Great opportunity will emerge
>> if that happens. Join me in a startup :)
>>
>> It sounds great to be like, freedom, TOS and "Sad!" but practically it's
>> all meaningless.
>>
>> This is happening and will happen. Like I once said Git was inevitable
>> and just shut up until it came, this is the same.
>>
>> "Us" as a community deciding to embrace it just means 3-4 old 

Re: 8.3 release

[Anshum Gupta]

I think it makes sense to just bundle the code and ref guide release into
one. Right now, it's been Cassandra and Hoss who have taken care of the ref
guide, but that shouldn't be the case.

It might mean that the ref guide is a little behind when the code releases,
but then we should just be better at committing the documentation when we
commit the code. Hopefully, we'll get better at that and the ref guide
releases would contain all new features but overall, having a different
release/voting process for the ref guide is a lot of overhead that isn't
really needed.


On Tue, Sep 17, 2019 at 4:31 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Hi Adrien,
> Indeed, meant to write about starting a vote.
>
> @Gus, I'll have to let Cassandra weigh in on this one as I'm not very
> familiar with the ref guide release process.
>
> Regards,
> Ishan
>
> On Mon, 16 Sep, 2019, 7:28 PM Adrien Grand,  wrote:
>
>> +1 to start working on 8.3
>>
>> Did you mean "start a vote" when you wrote "release the artifacts"? It
>> got me wondering because I don't think we frequently managed to go
>> from cutting a branch to releasing artifacts in so little time in the
>> past.
>>
>> On Mon, Sep 16, 2019 at 5:52 PM Ishan Chattopadhyaya
>>  wrote:
>> >
>> > Hi all,
>> > We have a lot of unreleased features and fixes. I propose that we cut
>> > a 8.3 branch in two weeks (in order to have sufficient time to bake in
>> > all in-progress features). If there are no objections to doing so, I
>> > can volunteer for the release as an RM and plan for cutting a release
>> > branch on 30 September (and release the artifacts about 3-4 days after
>> > that).
>> >
>> > WDYT?
>> > Regards,
>> > Ishan
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>>
>> --
>> Adrien
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

-- 
Anshum Gupta

Re: Notes from the committers' meeting at Activate 2019

[Mark Miller]

Also, just FYI, I say that as someone that kind of prefers patches and JIRA
for 90% of what I do. I’ve been doing this same shit for like half my life,
I’m not looking for fancy new tools for the hell of it either. I like
patches. It’s just going to happen. And I see a huge pool of free resources
in the meantime, wow those workflow limits are not too bad at all. I could
stop another new test that takes 2 minutes from coming in non nightly. Now
that’s practically interesting.

Mark

On Tue, Sep 17, 2019 at 7:39 PM Mark Miller  wrote:

> I think that is a little over the top.
>
> As it is the majority of dev and pr's and action is moving to GitHub,
> whether anyone is from Syria or not.
>
> If we decided, like most other communities on Gods green earth, to tell
> our community we are going GitHub first it and expect committers to not
> avoid all of our checks by just sticking to patches, the practical
> differences don't have to be much beyond that. Apache GitBox is not going
> away, it's easy to clearly spell out that those without access to GitHub
> can provide a patch as we would allow any committer without access or moral
> quandaries the same obviously. Making how to contribute a patch and use
> JIRA alternate doc for those with GitHub issues is pretty low effort.
>
> JIRA is a little different, I'm not as sold on leaving it, but really it's
> the same thing if almost all of the dev community starts using it for the
> bulk of what would be in JIRA, already lots of JIRA related comments and
> review have gone there - most stuff is just split instead of "free and
> available" - GitHub is lacking, JIRA is lacking.  Given that every damn
> company and project is on GitHub, this is just the way it will continue to
> go. So leaving JIRA up for history and those without access to GitHub would
> be the same too.
>
> And if M$ does anything with GitHub, the universe will collectively move
> on, with 90% of the world in the same spot. Great opportunity will emerge
> if that happens. Join me in a startup :)
>
> It sounds great to be like, freedom, TOS and "Sad!" but practically it's
> all meaningless.
>
> This is happening and will happen. Like I once said Git was inevitable and
> just shut up until it came, this is the same.
>
> "Us" as a community deciding to embrace it just means 3-4 old curmudgeons
> in a year won't as likely still be holding onto old ways for the sake of a
> imagined victim. Anyone that doesn't want to accept the GitHub TOS would
> get the same deal as someone from Siria. They will get the same 2nd citizen
> experience they are currently enjoying and that will continue to grow.
>
> And whatever you say or whatever the day, the practical difference of what
> happens will be zilch except for one thing: some people will feel better
> about bucking the community even if they are not from Syria or morally
> against the GitHub TOS.
>
> I'm a big fan of the kicking of screaming way, but generally only in my
> personal life. Professionally, I like to embrace the practical.
>
>
>
> On Tue, Sep 17, 2019 at 4:59 PM Anshum Gupta 
> wrote:
>
>> Ok, I buy that reason for leaving the ASF controlled mechanism.
>>
>> On Tue, Sep 17, 2019 at 2:16 PM Chris Hostetter 
>> wrote:
>>
>>>
>>> : Is there any reason at all that we need to hold on to JIRA? ASF allows
>>> : us to move all issue handling over to GH, I'd like us to consider such
>>> a
>>> : move.
>>>
>>> In my opinion, migrating from JIRA to Github "issues" would be a
>>> terrible
>>> idea.
>>>
>>> I have no objections to the goal of "encouraging" and "facilitating"
>>> contributions via github from people already using github -- but making
>>> github the defacto (or *sole*) way to participate and contribute code
>>> means pressuring people into accepting the github TOS (not just
>>> now, but whatever those might be in the future) as well as making it
>>> virtually impossible for people to participate if they are in locations
>>> github has decided to block (ie: Iran, Syria, and Crimea ATM + whomever
>>> else the US decides to sanction down the road)
>>>
>>> Opening up, or expanding, pathways for participation is one thing --
>>> I'm all in favor of that (even if I personally can't stand those
>>> avenues).
>>>
>>> But *closing* existing path ways that are currently entirely "open" and
>>> "free" to anyone that wants to participate w/o any limitations or TOS
>>> other then "Provide an ASF controled and owned website with an email
>>> address" ... that's just sad.
>>>
>>>
>>> -Hoss
>>> http://www.lucidworks.com/
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>
>>
>> --
>> Anshum Gupta
>>
>
>
> --
> - Mark
>
> http://about.me/markrmiller
>
-- 
- Mark

http://about.me/markrmiller

Re: Notes from the committers' meeting at Activate 2019

[Mark Miller]

I think that is a little over the top.

As it is the majority of dev and pr's and action is moving to GitHub,
whether anyone is from Syria or not.

If we decided, like most other communities on Gods green earth, to tell our
community we are going GitHub first it and expect committers to not avoid
all of our checks by just sticking to patches, the practical differences
don't have to be much beyond that. Apache GitBox is not going away, it's
easy to clearly spell out that those without access to GitHub can provide a
patch as we would allow any committer without access or moral quandaries
the same obviously. Making how to contribute a patch and use JIRA alternate
doc for those with GitHub issues is pretty low effort.

JIRA is a little different, I'm not as sold on leaving it, but really it's
the same thing if almost all of the dev community starts using it for the
bulk of what would be in JIRA, already lots of JIRA related comments and
review have gone there - most stuff is just split instead of "free and
available" - GitHub is lacking, JIRA is lacking.  Given that every damn
company and project is on GitHub, this is just the way it will continue to
go. So leaving JIRA up for history and those without access to GitHub would
be the same too.

And if M$ does anything with GitHub, the universe will collectively move
on, with 90% of the world in the same spot. Great opportunity will emerge
if that happens. Join me in a startup :)

It sounds great to be like, freedom, TOS and "Sad!" but practically it's
all meaningless.

This is happening and will happen. Like I once said Git was inevitable and
just shut up until it came, this is the same.

"Us" as a community deciding to embrace it just means 3-4 old curmudgeons
in a year won't as likely still be holding onto old ways for the sake of a
imagined victim. Anyone that doesn't want to accept the GitHub TOS would
get the same deal as someone from Siria. They will get the same 2nd citizen
experience they are currently enjoying and that will continue to grow.

And whatever you say or whatever the day, the practical difference of what
happens will be zilch except for one thing: some people will feel better
about bucking the community even if they are not from Syria or morally
against the GitHub TOS.

I'm a big fan of the kicking of screaming way, but generally only in my
personal life. Professionally, I like to embrace the practical.



On Tue, Sep 17, 2019 at 4:59 PM Anshum Gupta  wrote:

> Ok, I buy that reason for leaving the ASF controlled mechanism.
>
> On Tue, Sep 17, 2019 at 2:16 PM Chris Hostetter 
> wrote:
>
>>
>> : Is there any reason at all that we need to hold on to JIRA? ASF allows
>> : us to move all issue handling over to GH, I'd like us to consider such
>> a
>> : move.
>>
>> In my opinion, migrating from JIRA to Github "issues" would be a terrible
>> idea.
>>
>> I have no objections to the goal of "encouraging" and "facilitating"
>> contributions via github from people already using github -- but making
>> github the defacto (or *sole*) way to participate and contribute code
>> means pressuring people into accepting the github TOS (not just
>> now, but whatever those might be in the future) as well as making it
>> virtually impossible for people to participate if they are in locations
>> github has decided to block (ie: Iran, Syria, and Crimea ATM + whomever
>> else the US decides to sanction down the road)
>>
>> Opening up, or expanding, pathways for participation is one thing --
>> I'm all in favor of that (even if I personally can't stand those avenues).
>>
>> But *closing* existing path ways that are currently entirely "open" and
>> "free" to anyone that wants to participate w/o any limitations or TOS
>> other then "Provide an ASF controled and owned website with an email
>> address" ... that's just sad.
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>
> --
> Anshum Gupta
>


-- 
- Mark

http://about.me/markrmiller

Re: 8.3 release

[Ishan Chattopadhyaya]

Hi Adrien,
Indeed, meant to write about starting a vote.

@Gus, I'll have to let Cassandra weigh in on this one as I'm not very
familiar with the ref guide release process.

Regards,
Ishan

On Mon, 16 Sep, 2019, 7:28 PM Adrien Grand,  wrote:

> +1 to start working on 8.3
>
> Did you mean "start a vote" when you wrote "release the artifacts"? It
> got me wondering because I don't think we frequently managed to go
> from cutting a branch to releasing artifacts in so little time in the
> past.
>
> On Mon, Sep 16, 2019 at 5:52 PM Ishan Chattopadhyaya
>  wrote:
> >
> > Hi all,
> > We have a lot of unreleased features and fixes. I propose that we cut
> > a 8.3 branch in two weeks (in order to have sufficient time to bake in
> > all in-progress features). If there are no objections to doing so, I
> > can volunteer for the release as an RM and plan for cutting a release
> > branch on 30 September (and release the artifacts about 3-4 days after
> > that).
> >
> > WDYT?
> > Regards,
> > Ishan
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
>
> --
> Adrien
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Notes from the committers' meeting at Activate 2019

[Anshum Gupta]

Ok, I buy that reason for leaving the ASF controlled mechanism.

On Tue, Sep 17, 2019 at 2:16 PM Chris Hostetter 
wrote:

>
> : Is there any reason at all that we need to hold on to JIRA? ASF allows
> : us to move all issue handling over to GH, I'd like us to consider such a
> : move.
>
> In my opinion, migrating from JIRA to Github "issues" would be a terrible
> idea.
>
> I have no objections to the goal of "encouraging" and "facilitating"
> contributions via github from people already using github -- but making
> github the defacto (or *sole*) way to participate and contribute code
> means pressuring people into accepting the github TOS (not just
> now, but whatever those might be in the future) as well as making it
> virtually impossible for people to participate if they are in locations
> github has decided to block (ie: Iran, Syria, and Crimea ATM + whomever
> else the US decides to sanction down the road)
>
> Opening up, or expanding, pathways for participation is one thing --
> I'm all in favor of that (even if I personally can't stand those avenues).
>
> But *closing* existing path ways that are currently entirely "open" and
> "free" to anyone that wants to participate w/o any limitations or TOS
> other then "Provide an ASF controled and owned website with an email
> address" ... that's just sad.
>
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
Anshum Gupta

Re: Notes from the committers' meeting at Activate 2019

[Gus Heck]

"means pressuring people into accepting the github TOS"

That's a really good point. Hadn't thought of that and it's definitely not
ok to put github in control (or make them able to force a sudden burden of
work when we don't like what they did). Apache should determine it's own
destiny, and for that to be true, Apache must have it's own infrastructure.
Keep in mind who owns Github, and ponder what, prior to their embracing
open source, they might have done to give IIS an edge over httpd... for
example. They are playing nice now, but there's no guarantee that will
remain true for all time.

On Tue, Sep 17, 2019 at 5:16 PM Chris Hostetter 
wrote:

>
> : Is there any reason at all that we need to hold on to JIRA? ASF allows
> : us to move all issue handling over to GH, I'd like us to consider such a
> : move.
>
> In my opinion, migrating from JIRA to Github "issues" would be a terrible
> idea.
>
> I have no objections to the goal of "encouraging" and "facilitating"
> contributions via github from people already using github -- but making
> github the defacto (or *sole*) way to participate and contribute code
> means pressuring people into accepting the github TOS (not just
> now, but whatever those might be in the future) as well as making it
> virtually impossible for people to participate if they are in locations
> github has decided to block (ie: Iran, Syria, and Crimea ATM + whomever
> else the US decides to sanction down the road)
>
> Opening up, or expanding, pathways for participation is one thing --
> I'm all in favor of that (even if I personally can't stand those avenues).
>
> But *closing* existing path ways that are currently entirely "open" and
> "free" to anyone that wants to participate w/o any limitations or TOS
> other then "Provide an ASF controled and owned website with an email
> address" ... that's just sad.
>
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)

Re: Notes from the committers' meeting at Activate 2019

[Chris Hostetter]

: Is there any reason at all that we need to hold on to JIRA? ASF allows 
: us to move all issue handling over to GH, I'd like us to consider such a 
: move.

In my opinion, migrating from JIRA to Github "issues" would be a terrible 
idea.

I have no objections to the goal of "encouraging" and "facilitating" 
contributions via github from people already using github -- but making 
github the defacto (or *sole*) way to participate and contribute code 
means pressuring people into accepting the github TOS (not just 
now, but whatever those might be in the future) as well as making it 
virtually impossible for people to participate if they are in locations 
github has decided to block (ie: Iran, Syria, and Crimea ATM + whomever 
else the US decides to sanction down the road)

Opening up, or expanding, pathways for participation is one thing -- 
I'm all in favor of that (even if I personally can't stand those avenues).

But *closing* existing path ways that are currently entirely "open" and 
"free" to anyone that wants to participate w/o any limitations or TOS 
other then "Provide an ASF controled and owned website with an email 
address" ... that's just sad.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Notes from the committers' meeting at Activate 2019

[Anshum Gupta]

I think standardizing the approach to contributing would be a great thing.
It might mean not accommodating everyone's preference, but then it would
mean that the people who want to review are all on the same page. Else
people who prefer git will rarely look at patches, and the other way around.

The major concern that was brought up during the committer meeting was
around the history that is in JIRA archives. However, that shouldn't stop
us from creating "all" new issues in GitHub if everyone agrees.

Anshum

On Tue, Sep 17, 2019 at 1:59 PM Mark Miller  wrote:

> I think I may prefer JIRA to GitHub Issues. I'm not sure.
>
> Anyway, it's worth wondering if we can just move JIRA to GitHub. GitHub is
> now a first class mirror for Apache that we can commit to, but they still
> keep a primary copy of our code at Apache. Perhaps that is only a code
> concern now though.
>
> It appears others have moved:
> https://accumulo.apache.org/blog/2018/03/16/moving-to-github-issues.html
>
> As far as the GitHub vs patch issue, I don't want tell everyone they can't
> use patches, I usually prefer them, but, if we pushed everything through
> GitHub we can take super advantage of their free and way nicer than what we
> will ever have at Apache, Actions CI stuff. I'm not saying that will drive
> unit tests now or something, but for things like precommit and other checks
> (beasting on new tests or whatever), everyone going through that would be
> great. Would be awesome to project ourselves well in front of prs, or end
> up using the 2 branches and promoting changes when they are checked more
> thoroughly.
>
> If we started using Git primarily that way, not using issues is probably
> more friction than we need ...
>
> Mark
>
> On Tue, Sep 17, 2019 at 2:46 PM David Smiley 
> wrote:
>
>> Well put Jason.  I think we didn't discuss it in any further detail than
>> what you said right here -- basically cater to GitHub PRs and either
>> discourage or undocument "patch file" based contributions.  That's it in a
>> sentence.  We all nodded our head to that, albeit some of us like me
>> confess to enjoying the ease of patch based contributions.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Tue, Sep 17, 2019 at 9:46 AM Jason Gerlowski 
>> wrote:
>>
>>> I missed the part of the meeting/lunch when our use of Github came up.
>>> Can anyone that was present summarize the discussion in a little more
>>> detail?
>>>
>>> re: issues on github.  There are challenges like avoiding
>>> fragmentation and keeping multiple issue sources up to date, but those
>>> are problems that probably shrink or go away with appropriate
>>> automation.  IMO at least, issue reporting is largely the same whether
>>> you're on Github, JIRA, trello, etc.  You fill out a form, set the
>>> appropriate tags and fields, etc.  I'm fine w/ changes there, but it's
>>> hard for me to imagine how that would have much/any impact on barrier
>>> to entry.
>>>
>>> I see our code-contribution process as a much bigger driver of the
>>> barrier-to-entry. First time contributors must learn how to generate
>>> patches.  They receive code-review on JIRA, so they get one chunk of
>>> text in a comment.  And contributions have very weak version control
>>> (identically named SOLR-.patch files piling up on the same issue).
>>> Github has concrete benefits here.  If the goal is to make it easier
>>> for contributors to get involved, making Github first-class for code
>>> contributions might be where the real gains are.  (We allow Github
>>> PR's, but could steer people towards them more clearly: rewrite
>>> HowToContribute docs to assume Github, hide the patch process for new
>>> contributors, set up workflows in Github to run precommit/tests on
>>> PRs, etc.)
>>>
>>> On Tue, Sep 17, 2019 at 8:56 AM Eric Pugh
>>>  wrote:
>>> >
>>> > Ah..   The mythical committer just sitting around waiting to be
>>> “interested in the issue” that you have created!   Having said that, I
>>> think thats a separate challenge!
>>> >
>>> >
>>> > On Sep 17, 2019, at 12:38 AM, David Smiley 
>>> wrote:
>>> >
>>> > +1 to all that Gus said.  On a fresh project then indeed perhaps we
>>> would use GitHub issues but it's not nearly so compelling at this point
>>> with so much rich history in JIRA, not to mention those issues being
>>> referenced in commit messages.
>>> >
>>> > Is the point to lower barriers for contributors that are not in our
>>> community yet (thus don't have an ASF JIRA account)?  Well... they don't
>>> *have* to create the JIRA issue if a committer is sufficiently interested
>>> in the issue at hand to do it.  And as mentioned this could be automated.
>>> >
>>> > ~ David Smiley
>>> > Apache Lucene/Solr Search Developer
>>> > http://www.linkedin.com/in/davidwsmiley
>>> >
>>> >
>>> > On Mon, Sep 16, 2019 at 6:27 PM Gus Heck  wrote:
>>> >>
>>> >> FWIW, One thing that needs to be figured out is how github would
>>> accommodate 

Re: Notes from the committers' meeting at Activate 2019

[Mark Miller]

I think I may prefer JIRA to GitHub Issues. I'm not sure.

Anyway, it's worth wondering if we can just move JIRA to GitHub. GitHub is
now a first class mirror for Apache that we can commit to, but they still
keep a primary copy of our code at Apache. Perhaps that is only a code
concern now though.

It appears others have moved:
https://accumulo.apache.org/blog/2018/03/16/moving-to-github-issues.html

As far as the GitHub vs patch issue, I don't want tell everyone they can't
use patches, I usually prefer them, but, if we pushed everything through
GitHub we can take super advantage of their free and way nicer than what we
will ever have at Apache, Actions CI stuff. I'm not saying that will drive
unit tests now or something, but for things like precommit and other checks
(beasting on new tests or whatever), everyone going through that would be
great. Would be awesome to project ourselves well in front of prs, or end
up using the 2 branches and promoting changes when they are checked more
thoroughly.

If we started using Git primarily that way, not using issues is probably
more friction than we need ...

Mark

On Tue, Sep 17, 2019 at 2:46 PM David Smiley 
wrote:

> Well put Jason.  I think we didn't discuss it in any further detail than
> what you said right here -- basically cater to GitHub PRs and either
> discourage or undocument "patch file" based contributions.  That's it in a
> sentence.  We all nodded our head to that, albeit some of us like me
> confess to enjoying the ease of patch based contributions.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, Sep 17, 2019 at 9:46 AM Jason Gerlowski 
> wrote:
>
>> I missed the part of the meeting/lunch when our use of Github came up.
>> Can anyone that was present summarize the discussion in a little more
>> detail?
>>
>> re: issues on github.  There are challenges like avoiding
>> fragmentation and keeping multiple issue sources up to date, but those
>> are problems that probably shrink or go away with appropriate
>> automation.  IMO at least, issue reporting is largely the same whether
>> you're on Github, JIRA, trello, etc.  You fill out a form, set the
>> appropriate tags and fields, etc.  I'm fine w/ changes there, but it's
>> hard for me to imagine how that would have much/any impact on barrier
>> to entry.
>>
>> I see our code-contribution process as a much bigger driver of the
>> barrier-to-entry. First time contributors must learn how to generate
>> patches.  They receive code-review on JIRA, so they get one chunk of
>> text in a comment.  And contributions have very weak version control
>> (identically named SOLR-.patch files piling up on the same issue).
>> Github has concrete benefits here.  If the goal is to make it easier
>> for contributors to get involved, making Github first-class for code
>> contributions might be where the real gains are.  (We allow Github
>> PR's, but could steer people towards them more clearly: rewrite
>> HowToContribute docs to assume Github, hide the patch process for new
>> contributors, set up workflows in Github to run precommit/tests on
>> PRs, etc.)
>>
>> On Tue, Sep 17, 2019 at 8:56 AM Eric Pugh
>>  wrote:
>> >
>> > Ah..   The mythical committer just sitting around waiting to be
>> “interested in the issue” that you have created!   Having said that, I
>> think thats a separate challenge!
>> >
>> >
>> > On Sep 17, 2019, at 12:38 AM, David Smiley 
>> wrote:
>> >
>> > +1 to all that Gus said.  On a fresh project then indeed perhaps we
>> would use GitHub issues but it's not nearly so compelling at this point
>> with so much rich history in JIRA, not to mention those issues being
>> referenced in commit messages.
>> >
>> > Is the point to lower barriers for contributors that are not in our
>> community yet (thus don't have an ASF JIRA account)?  Well... they don't
>> *have* to create the JIRA issue if a committer is sufficiently interested
>> in the issue at hand to do it.  And as mentioned this could be automated.
>> >
>> > ~ David Smiley
>> > Apache Lucene/Solr Search Developer
>> > http://www.linkedin.com/in/davidwsmiley
>> >
>> >
>> > On Mon, Sep 16, 2019 at 6:27 PM Gus Heck  wrote:
>> >>
>> >> FWIW, One thing that needs to be figured out is how github would
>> accommodate security issues (or how the process for those issues would
>> change).  Does github have the ability to assign roles and visibility
>> (could be I haven't really worked with organizations on GitHub, all my
>> clients have been on jira ... except the one that has trello, and another
>> with gitlab... neither of which have impressed me very much )?
>> >>
>> >> Additionally, I'm slightly leery of the move since I don't (yet) see
>> the real benefit that pays for the splitting of the records into two
>> systems. Can issues be migrated to github? I've not really been on a large
>> project that used only GitHub, can someone explain what we *gain* by moving
>> to GitHub issues. At 

Release Announcement: General Availability of Java 13 / JDK 13

[Rory O'Donnell]

 Hi Uwe & Dawid,

*Release Announcement: General Availability of Java 13 / JDK 13 [1] *

 * JDK 13, the reference implementation of Java 13, is now Generally
   Available.
 * GPL-licensed OpenJDK builds from Oracle are available here:
   https://jdk.java.net/13
 * Release notes - https://jdk.java.net/13/release-notes

This release includes the following five features:

 * 350: Dynamic CDS Archives
 * 351: ZGC: Uncommit Unused Memory
 * 353: Reimplement the Legacy Socket API
 * 354: Switch Expressions (Preview)
 * 355: Text Blocks (Preview)

Along with many smaller enhancements and bug fixes.

Thanks to everyone who contributed JDK 13, whether by creating features 
or enhancements, logging  bugs, or downloading and testing the 
early-access builds.


*JDK 14 EA build 14, under both the GPL and Oracle EA licenses, is now 
available at **http://jdk.java.net/14**.*


 * JEPs targeted to JDK 14, so far
 o 352 - Non-Volatile Mapped Byte Buffers
   
 * Release Notes
 o https://jdk.java.net/14/release-notes
 * Recent Bug fixes of Interest
 o Build 14:
 + 8229785: MethodType::fromMethodDescriptorString requires
   "getClassLoader" permission
 + 8212117: Classes are now loaded and linked by Class.forName()
 + 8228854: Default ErrorListener No Longer Reports Warnings
   and Errors to the Console
 * Changes in this build
   

   [14]

*Quality Report for September 2019 is available*

 * 
https://wiki.openjdk.java.net/display/quality/Quality+Outreach+report+September+2019

Rgds,Rory

[1] 
https://mail.openjdk.java.net/pipermail/jdk-dev/2019-September/003335.html


--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA, Dublin,Ireland

Re: The Lucene Solr Gradle Build Game plan

[Gus Heck]

I mentioned when we chatted at activate that I had a build in the past that
was doing some pulling of gems... Looked back and it was actually using
https://github.com/robfletcher/gradle-compass which was doing the gem
management. Seems to use https://github.com/jruby-gradle/jruby-gradle-plugin to
pull gems (if I'm reading their code right). That could be helpful in
automating the requirements for the html ref guide.

On Tue, Sep 17, 2019 at 1:32 PM Cassandra Targett 
wrote:

> I got started on the Ref Guide build last night. The biggest change there
> is to use the asciidoctor-gradle-plugin instead of using the
> asciidoctor-ant plugin.
>
> So far I’ve got it working enough to build a single page of the Ref Guide
> into a PDF file. Baby steps ;)
>
> By itself that only gets us the PDF, while our HTML is built with several
> Ruby gems which we had to require people to install locally. However, the
> asciidoctor-gradle plugin includes JRuby, so fingers crossed we’ll finally
> be able to get that working and end up in a better place than we are today,
> and with a more unified build configuration than we currently have.
>
> I’m not ready to push anything to the branch yet, but hopefully will be in
> the next day or two.
>
> Cassandra
> On Sep 17, 2019, 5:02 AM -0500, Dawid Weiss ,
> wrote:
>
> [...] last I remember Dawid signed up for it starting around today (15th)
> ;)
>
>
> Ah... so you remembered?... Kind of hoped you forgot. :)
>
> I'll take a look and try to tackle some of Chris's questions. Are we
> free to commit to that gradle branch or do you prefer PRs?
>
> D.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)

Re: Notes from the committers' meeting at Activate 2019

[David Smiley]

Well put Jason.  I think we didn't discuss it in any further detail than
what you said right here -- basically cater to GitHub PRs and either
discourage or undocument "patch file" based contributions.  That's it in a
sentence.  We all nodded our head to that, albeit some of us like me
confess to enjoying the ease of patch based contributions.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Sep 17, 2019 at 9:46 AM Jason Gerlowski 
wrote:

> I missed the part of the meeting/lunch when our use of Github came up.
> Can anyone that was present summarize the discussion in a little more
> detail?
>
> re: issues on github.  There are challenges like avoiding
> fragmentation and keeping multiple issue sources up to date, but those
> are problems that probably shrink or go away with appropriate
> automation.  IMO at least, issue reporting is largely the same whether
> you're on Github, JIRA, trello, etc.  You fill out a form, set the
> appropriate tags and fields, etc.  I'm fine w/ changes there, but it's
> hard for me to imagine how that would have much/any impact on barrier
> to entry.
>
> I see our code-contribution process as a much bigger driver of the
> barrier-to-entry. First time contributors must learn how to generate
> patches.  They receive code-review on JIRA, so they get one chunk of
> text in a comment.  And contributions have very weak version control
> (identically named SOLR-.patch files piling up on the same issue).
> Github has concrete benefits here.  If the goal is to make it easier
> for contributors to get involved, making Github first-class for code
> contributions might be where the real gains are.  (We allow Github
> PR's, but could steer people towards them more clearly: rewrite
> HowToContribute docs to assume Github, hide the patch process for new
> contributors, set up workflows in Github to run precommit/tests on
> PRs, etc.)
>
> On Tue, Sep 17, 2019 at 8:56 AM Eric Pugh
>  wrote:
> >
> > Ah..   The mythical committer just sitting around waiting to be
> “interested in the issue” that you have created!   Having said that, I
> think thats a separate challenge!
> >
> >
> > On Sep 17, 2019, at 12:38 AM, David Smiley 
> wrote:
> >
> > +1 to all that Gus said.  On a fresh project then indeed perhaps we
> would use GitHub issues but it's not nearly so compelling at this point
> with so much rich history in JIRA, not to mention those issues being
> referenced in commit messages.
> >
> > Is the point to lower barriers for contributors that are not in our
> community yet (thus don't have an ASF JIRA account)?  Well... they don't
> *have* to create the JIRA issue if a committer is sufficiently interested
> in the issue at hand to do it.  And as mentioned this could be automated.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Mon, Sep 16, 2019 at 6:27 PM Gus Heck  wrote:
> >>
> >> FWIW, One thing that needs to be figured out is how github would
> accommodate security issues (or how the process for those issues would
> change).  Does github have the ability to assign roles and visibility
> (could be I haven't really worked with organizations on GitHub, all my
> clients have been on jira ... except the one that has trello, and another
> with gitlab... neither of which have impressed me very much )?
> >>
> >> Additionally, I'm slightly leery of the move since I don't (yet) see
> the real benefit that pays for the splitting of the records into two
> systems. Can issues be migrated to github? I've not really been on a large
> project that used only GitHub, can someone explain what we *gain* by moving
> to GitHub issues. At least two things are lost: continuity and familiarity.
> My impression is that there are a lot fewer features, but maybe I've just
> not been exposed to them.
> >>
> >> Part of me worries that this is the usual cycle of "this is simpler"
> (mass migration ensues) "but it doesn't x, y and z" (x, y and z are added)
> "wow this is complex, but THAT is simpler" (mass migration ensues...)
> "hmm p, q an z are missing" (p q and z are added  and cycle repeats). So
> I'm skeptical of any "gain" hanging it's hat solely on "simplicity". Jira
> used to be the simpler, snazzier, sexier alternative to bugzilla...
> >>
> >> Sell me, I'm all ears, but not seeing it yet :)
> >>
> >> -Gus
> >>
> >> On Mon, Sep 16, 2019 at 3:57 PM Jan Høydahl 
> wrote:
> >>>
> >>> Is there any reason at all that we need to hold on to JIRA? ASF allows
> us to move all issue handling over to GH, I'd like us to consider such a
> move.
> >>>
> >>> Until then, I made a script that "diffs" GH and JIRA and outputs a
> report, see
> https://github.com/apache/lucene-solr/tree/master/dev-tools/scripts#githubprspy
> >>>
> >>> Running the tool shows that we're not very successful in keeping the
> two in sync, we also forget to close PRs after JIRAs are resolved:
> >>>
> >>> $ ./githubPRs.py
> 

Re: Github PR Collaboration Question

[David Smiley]

RE the idea of shash-merge from a PR:  that would be cool were it not for
CHANGES.txt; ah well.

+1 to "put it in the PR template checklist"

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Sep 17, 2019 at 2:37 PM Jan Høydahl  wrote:

> I like the idea to put it in the PR template checklist. Or you can ask the
> contributor to check that box. I once made a PR against another PR branch
> and it worked but was too many steps.
>
> I like the idea of squash-merging from PR branch since (I believe) the
> commit will then have the credit of the contributor which then shows up in
> his/her stats on GH which is nice.
>
> Jan Høydahl
>
> > 17. sep. 2019 kl. 16:17 skrev Jason Gerlowski :
> >
> > Github does have an option that fork-owners can click when they create
> > a PR that will give those with karma on the upstream repo the same
> > karma on their PR branch.  [1] That would solve this problem somewhat.
> > But it's still up to users to choose that themselves.  Maybe it makes
> > sense to mention this as an optional checklist item in the PR template
> > that was recently added.
> >
> > Still curious about other approaches though if anyone has suggestions.
> >
> > [1]
> https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork
> >
> >> On Tue, Sep 17, 2019 at 10:12 AM Jason Gerlowski 
> wrote:
> >>
> >> Hey all,
> >>
> >> I've hit a small snag reviewing a few PRs on github recently.  Wanted
> >> to see if anyone has any suggestions for my workflow:
> >>
> >> I’ve found myself in the position a few times where I want to add a
> >> few small changes to a contributor’s PR…either to help them with a
> >> piece they haven’t gotten to yet, or to show what I’m suggesting with
> >> a particular review comment, or to clean up little things like
> >> whitespace formatting, etc.  In the patch world, this is easy to do
> >> without requiring review/acking by other contributors…you apply the
> >> latest patch, make your additional changes, and re-upload to jira.
> >> But in Github, you have to juggle branch/PR ownership.  PR’s are often
> >> from personal forks, where others don't have write access there.  So I
> >> can't add to the "main" PR without either (a) asking the contributor
> >> to give me the right karma, or (b) opening my own "secondary" PR
> >> against their "main" PR branch, and asking them to review/merge my
> >> "secondary" PR.
> >>
> >> Is there some simpler approach I'm missing?  I love the in-line
> >> comments that Github supports for code-review, and it seems like more
> >> committers are starting to use it.  I'd love to figure out how to make
> >> it work for me.  But two developers collaborating on the same PR seems
> >> like a pretty fundamental use case to be so heavyweight.  This must be
> >> a problem that's solved, right?
> >>
> >> Best,
> >>
> >> Jason
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: The Lucene Solr Gradle Build Game plan

[Mark Miller]

bq. Are we free to commit to that gradle branch or do you prefer PRs?

Commit away unless you are looking for feedback and a PR makes it easier.

I held onto it for long enough :) Time to get some more blood in this thing.

Also, if there are any things or parts that are important and you don't
have time for, file a JIRA and assign me and I will do it. Largely, I am
looking more for input and collaboration and what's more critically missing
then manual labor. At the same time, we need some labor too, because if
this goes in with me doing 99% of it, there is something that smells wrong
about moving.

bq. I’m not ready to push anything to the branch yet, but hopefully will be
in the next day or two.

Thanks so much Cassandra! Let me know if I can help on any trouble.

- Mark

On Tue, Sep 17, 2019 at 12:31 PM Cassandra Targett 
wrote:

> I got started on the Ref Guide build last night. The biggest change there
> is to use the asciidoctor-gradle-plugin instead of using the
> asciidoctor-ant plugin.
>
> So far I’ve got it working enough to build a single page of the Ref Guide
> into a PDF file. Baby steps ;)
>
> By itself that only gets us the PDF, while our HTML is built with several
> Ruby gems which we had to require people to install locally. However, the
> asciidoctor-gradle plugin includes JRuby, so fingers crossed we’ll finally
> be able to get that working and end up in a better place than we are today,
> and with a more unified build configuration than we currently have.
>
> I’m not ready to push anything to the branch yet, but hopefully will be in
> the next day or two.
>
> Cassandra
> On Sep 17, 2019, 5:02 AM -0500, Dawid Weiss ,
> wrote:
>
> [...] last I remember Dawid signed up for it starting around today (15th)
> ;)
>
>
> Ah... so you remembered?... Kind of hoped you forgot. :)
>
> I'll take a look and try to tackle some of Chris's questions. Are we
> free to commit to that gradle branch or do you prefer PRs?
>
> D.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Mark

http://about.me/markrmiller

Re: Github PR Collaboration Question

[Jan Høydahl]

I like the idea to put it in the PR template checklist. Or you can ask the 
contributor to check that box. I once made a PR against another PR branch and 
it worked but was too many steps.

I like the idea of squash-merging from PR branch since (I believe) the commit 
will then have the credit of the contributor which then shows up in his/her 
stats on GH which is nice.

Jan Høydahl

> 17. sep. 2019 kl. 16:17 skrev Jason Gerlowski :
> 
> Github does have an option that fork-owners can click when they create
> a PR that will give those with karma on the upstream repo the same
> karma on their PR branch.  [1] That would solve this problem somewhat.
> But it's still up to users to choose that themselves.  Maybe it makes
> sense to mention this as an optional checklist item in the PR template
> that was recently added.
> 
> Still curious about other approaches though if anyone has suggestions.
> 
> [1] 
> https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork
> 
>> On Tue, Sep 17, 2019 at 10:12 AM Jason Gerlowski  
>> wrote:
>> 
>> Hey all,
>> 
>> I've hit a small snag reviewing a few PRs on github recently.  Wanted
>> to see if anyone has any suggestions for my workflow:
>> 
>> I’ve found myself in the position a few times where I want to add a
>> few small changes to a contributor’s PR…either to help them with a
>> piece they haven’t gotten to yet, or to show what I’m suggesting with
>> a particular review comment, or to clean up little things like
>> whitespace formatting, etc.  In the patch world, this is easy to do
>> without requiring review/acking by other contributors…you apply the
>> latest patch, make your additional changes, and re-upload to jira.
>> But in Github, you have to juggle branch/PR ownership.  PR’s are often
>> from personal forks, where others don't have write access there.  So I
>> can't add to the "main" PR without either (a) asking the contributor
>> to give me the right karma, or (b) opening my own "secondary" PR
>> against their "main" PR branch, and asking them to review/merge my
>> "secondary" PR.
>> 
>> Is there some simpler approach I'm missing?  I love the in-line
>> comments that Github supports for code-review, and it seems like more
>> committers are starting to use it.  I'd love to figure out how to make
>> it work for me.  But two developers collaborating on the same PR seems
>> like a pretty fundamental use case to be so heavyweight.  This must be
>> a problem that's solved, right?
>> 
>> Best,
>> 
>> Jason
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: The Lucene Solr Gradle Build Game plan

[Cassandra Targett]

I got started on the Ref Guide build last night. The biggest change there is to 
use the asciidoctor-gradle-plugin instead of using the asciidoctor-ant plugin.

So far I’ve got it working enough to build a single page of the Ref Guide into 
a PDF file. Baby steps ;)

By itself that only gets us the PDF, while our HTML is built with several Ruby 
gems which we had to require people to install locally. However, the 
asciidoctor-gradle plugin includes JRuby, so fingers crossed we’ll finally be 
able to get that working and end up in a better place than we are today, and 
with a more unified build configuration than we currently have.

I’m not ready to push anything to the branch yet, but hopefully will be in the 
next day or two.

Cassandra
On Sep 17, 2019, 5:02 AM -0500, Dawid Weiss , wrote:
> > [...] last I remember Dawid signed up for it starting around today (15th) ;)
>
> Ah... so you remembered?... Kind of hoped you forgot. :)
>
> I'll take a look and try to tackle some of Chris's questions. Are we
> free to commit to that gradle branch or do you prefer PRs?
>
> D.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

Re: Direct I/O

[Uwe Schindler]

We discussed this already on Berlinbuzzwords (Mike and Michael). Yes it's 
possible and may work for merges where block io is possible. But most of us 
said: it's fine to not use io cache for merging, but it won't make pages hot. 
So merges are invisible to OS, so you have to warm merged segments if you write 
directly. If you read directly on merging, you won't pollute cache with one 
time reads, but it also won't use cache if already cached.
We should better make a proposal for f/madvise. The jdk people are open for 
that, and I am jdk committer now, so I can make a prototype.

Uwe

Am September 17, 2019 4:48:26 PM UTC schrieb Dawid Weiss 
:
>Isn't that restricted to aligned block-only access though? I can
>imagine this would complicate the implementation if somebody wanted to
>use it directly.
>
>Dawid
>
>On Tue, Sep 17, 2019 at 5:37 PM Michael McCandless
> wrote:
>>
>> Whoa!  That would be awesome -- no more JNI to use Direct I/O?
>> Looks like you use it like this:
>>
>> FileChannel fc = FileChannel.open(p, StandardOpenOption.WRITE,
>>   ExtendedOpenOption.DIRECT
>>
>> But it looks like you need to enable the jdk.unsupported module,
>added with http://openjdk.java.net/jeps/260
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Mon, Sep 16, 2019 at 11:55 AM Michael Sokolov 
>wrote:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8189192 makes it appear
>that
>>> Direct I/O is (or may be?) available now in JDK's since JDK10.
>Should
>>> we try using that API in NativeUnixDirectory in order to avoid JNI
>>> calls?
>>>
>>>
>-
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>
>-
>To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>For additional commands, e-mail: dev-h...@lucene.apache.org

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Re: Direct I/O

[Dawid Weiss]

Isn't that restricted to aligned block-only access though? I can
imagine this would complicate the implementation if somebody wanted to
use it directly.

Dawid

On Tue, Sep 17, 2019 at 5:37 PM Michael McCandless
 wrote:
>
> Whoa!  That would be awesome -- no more JNI to use Direct I/O?
> Looks like you use it like this:
>
> FileChannel fc = FileChannel.open(p, StandardOpenOption.WRITE,
>   ExtendedOpenOption.DIRECT
>
> But it looks like you need to enable the jdk.unsupported module, added with 
> http://openjdk.java.net/jeps/260
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Mon, Sep 16, 2019 at 11:55 AM Michael Sokolov  wrote:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8189192 makes it appear that
>> Direct I/O is (or may be?) available now in JDK's since JDK10. Should
>> we try using that API in NativeUnixDirectory in order to avoid JNI
>> calls?
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Adding env-var options for the Solr Prometheus Exporter

[Houston Putman]

Yep, I'll work on that this afternoon.

On Tue, Sep 17, 2019 at 12:24 PM Anshum Gupta 
wrote:

> Sounds like a good idea, Houston.
> Do you plan on creating a JIRA and PR ?
>
> On Tue, Sep 17, 2019 at 9:17 AM Houston Putman 
> wrote:
>
>> The startup options for the Prometheus Exporter are pretty sparse
>> compared to what the Solr start script offers.
>>
>> I'm looking at adding some options that mirror what Solr offers, such as
>>
>>- SOLR_HEAP
>>- SOLR_JAVA_MEM
>>
>>  Maybe some GC options as well.
>>
>> Having just the memory settings available would let us start the
>> prometheus exporter with more than 500 Mb of heap, which right now isn't
>> possible as the max heap is hard coded here
>> 
>> .
>>
>> This should be pretty easy to add, and we can make it sync with the
>> naming convention used by the Solr start script. I'll probably create a
>> JIRA and patch soon unless anyone has concerns or objections.
>>
>> - Houston Putman
>>
>
>
> --
> Anshum Gupta
>

Re: Adding env-var options for the Solr Prometheus Exporter

[Anshum Gupta]

Sounds like a good idea, Houston.
Do you plan on creating a JIRA and PR ?

On Tue, Sep 17, 2019 at 9:17 AM Houston Putman 
wrote:

> The startup options for the Prometheus Exporter are pretty sparse compared
> to what the Solr start script offers.
>
> I'm looking at adding some options that mirror what Solr offers, such as
>
>- SOLR_HEAP
>- SOLR_JAVA_MEM
>
>  Maybe some GC options as well.
>
> Having just the memory settings available would let us start the
> prometheus exporter with more than 500 Mb of heap, which right now isn't
> possible as the max heap is hard coded here
> 
> .
>
> This should be pretty easy to add, and we can make it sync with the naming
> convention used by the Solr start script. I'll probably create a JIRA and
> patch soon unless anyone has concerns or objections.
>
> - Houston Putman
>


-- 
Anshum Gupta

Adding env-var options for the Solr Prometheus Exporter

[Houston Putman]

The startup options for the Prometheus Exporter are pretty sparse compared
to what the Solr start script offers.

I'm looking at adding some options that mirror what Solr offers, such as

   - SOLR_HEAP
   - SOLR_JAVA_MEM

 Maybe some GC options as well.

Having just the memory settings available would let us start the prometheus
exporter with more than 500 Mb of heap, which right now isn't possible as
the max heap is hard coded here

.

This should be pretty easy to add, and we can make it sync with the naming
convention used by the Solr start script. I'll probably create a JIRA and
patch soon unless anyone has concerns or objections.

- Houston Putman

Re: Direct I/O

[Michael McCandless]

Whoa!  That would be awesome -- no more JNI to use Direct I/O?
Looks like you use it like this:

FileChannel fc = FileChannel.open(p, StandardOpenOption.WRITE,
  ExtendedOpenOption.DIRECT

But it looks like you need to enable the jdk.unsupported module, added with
http://openjdk.java.net/jeps/260

Mike McCandless

http://blog.mikemccandless.com


On Mon, Sep 16, 2019 at 11:55 AM Michael Sokolov  wrote:

> https://bugs.openjdk.java.net/browse/JDK-8189192 makes it appear that
> Direct I/O is (or may be?) available now in JDK's since JDK10. Should
> we try using that API in NativeUnixDirectory in order to avoid JNI
> calls?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Github PR Collaboration Question

[Jason Gerlowski]

Github does have an option that fork-owners can click when they create
a PR that will give those with karma on the upstream repo the same
karma on their PR branch.  [1] That would solve this problem somewhat.
But it's still up to users to choose that themselves.  Maybe it makes
sense to mention this as an optional checklist item in the PR template
that was recently added.

Still curious about other approaches though if anyone has suggestions.

[1] 
https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork

On Tue, Sep 17, 2019 at 10:12 AM Jason Gerlowski  wrote:
>
> Hey all,
>
> I've hit a small snag reviewing a few PRs on github recently.  Wanted
> to see if anyone has any suggestions for my workflow:
>
> I’ve found myself in the position a few times where I want to add a
> few small changes to a contributor’s PR…either to help them with a
> piece they haven’t gotten to yet, or to show what I’m suggesting with
> a particular review comment, or to clean up little things like
> whitespace formatting, etc.  In the patch world, this is easy to do
> without requiring review/acking by other contributors…you apply the
> latest patch, make your additional changes, and re-upload to jira.
> But in Github, you have to juggle branch/PR ownership.  PR’s are often
> from personal forks, where others don't have write access there.  So I
> can't add to the "main" PR without either (a) asking the contributor
> to give me the right karma, or (b) opening my own "secondary" PR
> against their "main" PR branch, and asking them to review/merge my
> "secondary" PR.
>
> Is there some simpler approach I'm missing?  I love the in-line
> comments that Github supports for code-review, and it seems like more
> committers are starting to use it.  I'd love to figure out how to make
> it work for me.  But two developers collaborating on the same PR seems
> like a pretty fundamental use case to be so heavyweight.  This must be
> a problem that's solved, right?
>
> Best,
>
> Jason

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Github PR Collaboration Question

[Jason Gerlowski]

Hey all,

I've hit a small snag reviewing a few PRs on github recently.  Wanted
to see if anyone has any suggestions for my workflow:

I’ve found myself in the position a few times where I want to add a
few small changes to a contributor’s PR…either to help them with a
piece they haven’t gotten to yet, or to show what I’m suggesting with
a particular review comment, or to clean up little things like
whitespace formatting, etc.  In the patch world, this is easy to do
without requiring review/acking by other contributors…you apply the
latest patch, make your additional changes, and re-upload to jira.
But in Github, you have to juggle branch/PR ownership.  PR’s are often
from personal forks, where others don't have write access there.  So I
can't add to the "main" PR without either (a) asking the contributor
to give me the right karma, or (b) opening my own "secondary" PR
against their "main" PR branch, and asking them to review/merge my
"secondary" PR.

Is there some simpler approach I'm missing?  I love the in-line
comments that Github supports for code-review, and it seems like more
committers are starting to use it.  I'd love to figure out how to make
it work for me.  But two developers collaborating on the same PR seems
like a pretty fundamental use case to be so heavyweight.  This must be
a problem that's solved, right?

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Notes from the committers' meeting at Activate 2019

[Jason Gerlowski]

I missed the part of the meeting/lunch when our use of Github came up.
Can anyone that was present summarize the discussion in a little more
detail?

re: issues on github.  There are challenges like avoiding
fragmentation and keeping multiple issue sources up to date, but those
are problems that probably shrink or go away with appropriate
automation.  IMO at least, issue reporting is largely the same whether
you're on Github, JIRA, trello, etc.  You fill out a form, set the
appropriate tags and fields, etc.  I'm fine w/ changes there, but it's
hard for me to imagine how that would have much/any impact on barrier
to entry.

I see our code-contribution process as a much bigger driver of the
barrier-to-entry. First time contributors must learn how to generate
patches.  They receive code-review on JIRA, so they get one chunk of
text in a comment.  And contributions have very weak version control
(identically named SOLR-.patch files piling up on the same issue).
Github has concrete benefits here.  If the goal is to make it easier
for contributors to get involved, making Github first-class for code
contributions might be where the real gains are.  (We allow Github
PR's, but could steer people towards them more clearly: rewrite
HowToContribute docs to assume Github, hide the patch process for new
contributors, set up workflows in Github to run precommit/tests on
PRs, etc.)

On Tue, Sep 17, 2019 at 8:56 AM Eric Pugh
 wrote:
>
> Ah..   The mythical committer just sitting around waiting to be “interested 
> in the issue” that you have created!   Having said that, I think thats a 
> separate challenge!
>
>
> On Sep 17, 2019, at 12:38 AM, David Smiley  wrote:
>
> +1 to all that Gus said.  On a fresh project then indeed perhaps we would use 
> GitHub issues but it's not nearly so compelling at this point with so much 
> rich history in JIRA, not to mention those issues being referenced in commit 
> messages.
>
> Is the point to lower barriers for contributors that are not in our community 
> yet (thus don't have an ASF JIRA account)?  Well... they don't *have* to 
> create the JIRA issue if a committer is sufficiently interested in the issue 
> at hand to do it.  And as mentioned this could be automated.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Mon, Sep 16, 2019 at 6:27 PM Gus Heck  wrote:
>>
>> FWIW, One thing that needs to be figured out is how github would accommodate 
>> security issues (or how the process for those issues would change).  Does 
>> github have the ability to assign roles and visibility (could be I haven't 
>> really worked with organizations on GitHub, all my clients have been on jira 
>> ... except the one that has trello, and another with gitlab... neither of 
>> which have impressed me very much )?
>>
>> Additionally, I'm slightly leery of the move since I don't (yet) see the 
>> real benefit that pays for the splitting of the records into two systems. 
>> Can issues be migrated to github? I've not really been on a large project 
>> that used only GitHub, can someone explain what we *gain* by moving to 
>> GitHub issues. At least two things are lost: continuity and familiarity. My 
>> impression is that there are a lot fewer features, but maybe I've just not 
>> been exposed to them.
>>
>> Part of me worries that this is the usual cycle of "this is simpler" (mass 
>> migration ensues) "but it doesn't x, y and z" (x, y and z are added) "wow 
>> this is complex, but THAT is simpler" (mass migration ensues...) "hmm p, 
>> q an z are missing" (p q and z are added  and cycle repeats). So I'm 
>> skeptical of any "gain" hanging it's hat solely on "simplicity". Jira used 
>> to be the simpler, snazzier, sexier alternative to bugzilla...
>>
>> Sell me, I'm all ears, but not seeing it yet :)
>>
>> -Gus
>>
>> On Mon, Sep 16, 2019 at 3:57 PM Jan Høydahl  wrote:
>>>
>>> Is there any reason at all that we need to hold on to JIRA? ASF allows us 
>>> to move all issue handling over to GH, I'd like us to consider such a move.
>>>
>>> Until then, I made a script that "diffs" GH and JIRA and outputs a report, 
>>> see 
>>> https://github.com/apache/lucene-solr/tree/master/dev-tools/scripts#githubprspy
>>>
>>> Running the tool shows that we're not very successful in keeping the two in 
>>> sync, we also forget to close PRs after JIRAs are resolved:
>>>
>>> $ ./githubPRs.py
>>> Lucene/Solr Github PR report
>>> 
>>> Number of open Pull Requests: 208
>>>
>>> PRs lacking JIRA reference in title
>>>   #882: Add versions.lock entry for "org.apache.yetus:audience-annotations"
>>>   #880: Tweak header format.
>>>   [… many more ]
>>>
>>> Open PRs with a resolved JIRA
>>>   #723: SOLR-13545 status=Closed, resolution=Fixed, 
>>> resolutiondate=2019-06-22T13:04:47.000+ (SOLR-13545: AutoClose stream 
>>> in ContentStreamUpdateRequest)
>>>   #643: SOLR-13391 status=Resolved, resolution=Resolved, 
>>> 

Re: Notes from the committers' meeting at Activate 2019

[Eric Pugh]

Ah..   The mythical committer just sitting around waiting to be “interested in 
the issue” that you have created!   Having said that, I think thats a separate 
challenge!


> On Sep 17, 2019, at 12:38 AM, David Smiley  wrote:
> 
> +1 to all that Gus said.  On a fresh project then indeed perhaps we would use 
> GitHub issues but it's not nearly so compelling at this point with so much 
> rich history in JIRA, not to mention those issues being referenced in commit 
> messages.
> 
> Is the point to lower barriers for contributors that are not in our community 
> yet (thus don't have an ASF JIRA account)?  Well... they don't *have* to 
> create the JIRA issue if a committer is sufficiently interested in the issue 
> at hand to do it.  And as mentioned this could be automated.
> 
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley 
> 
> 
> On Mon, Sep 16, 2019 at 6:27 PM Gus Heck  > wrote:
> FWIW, One thing that needs to be figured out is how github would accommodate 
> security issues (or how the process for those issues would change).  Does 
> github have the ability to assign roles and visibility (could be I haven't 
> really worked with organizations on GitHub, all my clients have been on jira 
> ... except the one that has trello, and another with gitlab... neither of 
> which have impressed me very much )?
> 
> Additionally, I'm slightly leery of the move since I don't (yet) see the real 
> benefit that pays for the splitting of the records into two systems. Can 
> issues be migrated to github? I've not really been on a large project that 
> used only GitHub, can someone explain what we *gain* by moving to GitHub 
> issues. At least two things are lost: continuity and familiarity. My 
> impression is that there are a lot fewer features, but maybe I've just not 
> been exposed to them. 
> 
> Part of me worries that this is the usual cycle of "this is simpler" (mass 
> migration ensues) "but it doesn't x, y and z" (x, y and z are added) "wow 
> this is complex, but THAT is simpler" (mass migration ensues...) "hmm p, 
> q an z are missing" (p q and z are added  and cycle repeats). So I'm 
> skeptical of any "gain" hanging it's hat solely on "simplicity". Jira used to 
> be the simpler, snazzier, sexier alternative to bugzilla...
> 
> Sell me, I'm all ears, but not seeing it yet :)
> 
> -Gus
> 
> On Mon, Sep 16, 2019 at 3:57 PM Jan Høydahl  > wrote:
> Is there any reason at all that we need to hold on to JIRA? ASF allows us to 
> move all issue handling over to GH, I'd like us to consider such a move.
> 
> Until then, I made a script that "diffs" GH and JIRA and outputs a report, 
> see 
> https://github.com/apache/lucene-solr/tree/master/dev-tools/scripts#githubprspy
>  
> 
> 
> Running the tool shows that we're not very successful in keeping the two in 
> sync, we also forget to close PRs after JIRAs are resolved:
> 
> $ ./githubPRs.py 
> Lucene/Solr Github PR report
> 
> Number of open Pull Requests: 208
> 
> PRs lacking JIRA reference in title
>   #882: Add versions.lock entry for "org.apache.yetus:audience-annotations"
>   #880: Tweak header format.
>   [… many more ]
> 
> Open PRs with a resolved JIRA
>   #723: SOLR-13545 status=Closed, resolution=Fixed, 
> resolutiondate=2019-06-22T13:04:47.000+ (SOLR-13545: AutoClose stream in 
> ContentStreamUpdateRequest)
>   #643: SOLR-13391 status=Resolved, resolution=Resolved, 
> resolutiondate=2019-04-12T14:09:27.000+ (SOLR-13391: Add variance and 
> standard deviation stream evaluators)
>   [… many more]
> 
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com 
> 
>> 16. sep. 2019 kl. 20:57 skrev Andrzej Białecki > >:
>> 
>> 
>>> On 16 Sep 2019, at 19:38, Yonik Seeley >> > wrote:
>>> 
>>> >  - PR is opened - should automatically create a jira if it doesn’t exist 
>>> > yet
>>> 
>>> What were the reasons behind this? Shouldn't a JIRA just be optional if we 
>>> started with a PR?
>> 
>> That’s why we need to discuss this :)
>> 
>> I remember that at some point ASF treated JIRA as the system of record for 
>> the legal validation of contributions.
>> 
>> I also remember that at some point we used to say that if a discussion 
>> didn’t happen in JIRA then it didn’t exist, and that any decisions regarding 
>> code should be recorded in JIRA because we can’t expect people to monitor 
>> and contribute / object to decisions happening elsewhere.
>> 
>> —
>> 
>> Andrzej Białecki
>> 
> 
> 
> 
> -- 
> http://www.needhamsoftware.com  (work)
> http://www.the111shift.com  (play)

___
Eric Pugh | Founder & CEO | OpenSource 

Re: The Lucene Solr Gradle Build Game plan

[Dawid Weiss]

> [...] last I remember Dawid signed up for it starting around today (15th) ;)

Ah... so you remembered?... Kind of hoped you forgot. :)

I'll take a look and try to tackle some of Chris's questions. Are we
free to commit to that gradle branch or do you prefer PRs?

D.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] Jan Høydahl mentioned you on SOLR-13648 (Jira) (JIRA)

[Jan Høydahl]

Thanks Dawid. That JIRA was a private/protected one, that's why. I'll email you 
the contents in private :)

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 17. sep. 2019 kl. 07:52 skrev Dawid Weiss :
> 
> Hi Jan,
> 
> Funny, I don't have permission to see or comment on:
> https://issues.apache.org/jira/browse/SOLR-13648
> 
> Anybody knows why this is the case?
> 
> Yes, simple-xml is a dependency of Carrot2 (clustering contrib's
> default implementation). I'm working on having it removed on next
> iteration of Carrot2 so it won't be long before it's gone.
> 
> Dawid
>