Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

I am going to resolve those JIRAs now.

2019년 9월 9일 (월) 오전 9:46, Hyukjin Kwon 님이 작성:

> Yup, no worries. I roughly set the one week delay considering the official
> release date :D
>
> On Mon, 9 Sep 2019, 09:45 Dongjoon Hyun,  wrote:
>
>> Thank you, Hyukjin.
>>
>> +1 for closing according to 2.3.x EOL.
>>
>> For the timing, please do that after the official 2.3.4 release
>> announcement.
>>
>> Bests,
>> Dongjoon.
>>
>> On Sun, Sep 8, 2019 at 16:27 Sean Owen  wrote:
>>
>>> I think simply closing old issues with no activity in a long time is
>>> OK. The "Affected Version" is somewhat noisy, so not even particularly
>>> important to also query, but yeah I see some value in trying to limit
>>> the scope this way.
>>>
>>> On Sat, Sep 7, 2019 at 10:15 PM Hyukjin Kwon 
>>> wrote:
>>> >
>>> > HI all,
>>> >
>>> > We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x)
>>> in order to make it
>>> > the manageable size before.
>>> > Since Spark 2.3.4 will be EOL release, I plan to do this again roughly
>>> in a week.
>>> >
>>> > The JIRAs that has not been updated for the last year, and having
>>> affect version of EOL releases will be:
>>> >   - Resolved as 'Incomplete' status
>>> >   - Has a 'bulk-closed' label.
>>> >
>>> > I plan to use this JQL
>>> >
>>> > project = SPARK
>>> >   AND status in (Open, "In Progress", Reopened)
>>> >   AND (
>>> > affectedVersion = EMPTY OR
>>> > NOT (affectedVersion in versionMatch("^3.*")
>>> >   OR affectedVersion in versionMatch("^2.4.*")
>>> > )
>>> >   )
>>> >   AND updated <= -52w
>>> >
>>> >
>>> > You could click this link and check.
>>> >
>>> >
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w
>>> >
>>> > Please let me know if you guys have any concern or opinion on this.
>>> >
>>> > Thanks.
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

Yup, no worries. I roughly set the one week delay considering the official
release date :D

On Mon, 9 Sep 2019, 09:45 Dongjoon Hyun,  wrote:

> Thank you, Hyukjin.
>
> +1 for closing according to 2.3.x EOL.
>
> For the timing, please do that after the official 2.3.4 release
> announcement.
>
> Bests,
> Dongjoon.
>
> On Sun, Sep 8, 2019 at 16:27 Sean Owen  wrote:
>
>> I think simply closing old issues with no activity in a long time is
>> OK. The "Affected Version" is somewhat noisy, so not even particularly
>> important to also query, but yeah I see some value in trying to limit
>> the scope this way.
>>
>> On Sat, Sep 7, 2019 at 10:15 PM Hyukjin Kwon  wrote:
>> >
>> > HI all,
>> >
>> > We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x) in
>> order to make it
>> > the manageable size before.
>> > Since Spark 2.3.4 will be EOL release, I plan to do this again roughly
>> in a week.
>> >
>> > The JIRAs that has not been updated for the last year, and having
>> affect version of EOL releases will be:
>> >   - Resolved as 'Incomplete' status
>> >   - Has a 'bulk-closed' label.
>> >
>> > I plan to use this JQL
>> >
>> > project = SPARK
>> >   AND status in (Open, "In Progress", Reopened)
>> >   AND (
>> > affectedVersion = EMPTY OR
>> > NOT (affectedVersion in versionMatch("^3.*")
>> >   OR affectedVersion in versionMatch("^2.4.*")
>> > )
>> >   )
>> >   AND updated <= -52w
>> >
>> >
>> > You could click this link and check.
>> >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w
>> >
>> > Please let me know if you guys have any concern or opinion on this.
>> >
>> > Thanks.
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: Resolving all JIRAs affecting EOL releases

[Dongjoon Hyun]

Thank you, Hyukjin.

+1 for closing according to 2.3.x EOL.

For the timing, please do that after the official 2.3.4 release
announcement.

Bests,
Dongjoon.

On Sun, Sep 8, 2019 at 16:27 Sean Owen  wrote:

> I think simply closing old issues with no activity in a long time is
> OK. The "Affected Version" is somewhat noisy, so not even particularly
> important to also query, but yeah I see some value in trying to limit
> the scope this way.
>
> On Sat, Sep 7, 2019 at 10:15 PM Hyukjin Kwon  wrote:
> >
> > HI all,
> >
> > We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x) in
> order to make it
> > the manageable size before.
> > Since Spark 2.3.4 will be EOL release, I plan to do this again roughly
> in a week.
> >
> > The JIRAs that has not been updated for the last year, and having affect
> version of EOL releases will be:
> >   - Resolved as 'Incomplete' status
> >   - Has a 'bulk-closed' label.
> >
> > I plan to use this JQL
> >
> > project = SPARK
> >   AND status in (Open, "In Progress", Reopened)
> >   AND (
> > affectedVersion = EMPTY OR
> > NOT (affectedVersion in versionMatch("^3.*")
> >   OR affectedVersion in versionMatch("^2.4.*")
> > )
> >   )
> >   AND updated <= -52w
> >
> >
> > You could click this link and check.
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w
> >
> > Please let me know if you guys have any concern or opinion on this.
> >
> > Thanks.
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: Resolving all JIRAs affecting EOL releases

[Sean Owen]

I think simply closing old issues with no activity in a long time is
OK. The "Affected Version" is somewhat noisy, so not even particularly
important to also query, but yeah I see some value in trying to limit
the scope this way.

On Sat, Sep 7, 2019 at 10:15 PM Hyukjin Kwon  wrote:
>
> HI all,
>
> We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x) in order 
> to make it
> the manageable size before.
> Since Spark 2.3.4 will be EOL release, I plan to do this again roughly in a 
> week.
>
> The JIRAs that has not been updated for the last year, and having affect 
> version of EOL releases will be:
>   - Resolved as 'Incomplete' status
>   - Has a 'bulk-closed' label.
>
> I plan to use this JQL
>
> project = SPARK
>   AND status in (Open, "In Progress", Reopened)
>   AND (
> affectedVersion = EMPTY OR
> NOT (affectedVersion in versionMatch("^3.*")
>   OR affectedVersion in versionMatch("^2.4.*")
> )
>   )
>   AND updated <= -52w
>
>
> You could click this link and check.
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w
>
> Please let me know if you guys have any concern or opinion on this.
>
> Thanks.

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Resolving all JIRAs affecting EOL releases

[Takeshi Yamamuro]

Thanks for the kind explanation.
I see and it looks ok to me.

On Sun, Sep 8, 2019 at 1:54 PM Hyukjin Kwon  wrote:

> Thanks for checking it.
>
> I think it's fine by two reasons below:
>
> 1. It has another condition for such cases - one year time range.
>   Basically, such PRs have not been merged for one year, which I believe
> are not likely merged soon.
>   The JIRA status will be updated when such PRs are merged anyway.
>
> 2. The JIRAs and PRs ideally should be updated. If the PR authors forgot
> to update affected versions,
> it could be a good ping to update the affected versions in its JIRA,
> which is a good practice I believe.
>
> FWIW, currently setting 'in-Progress' doesn't properly work. It has been
> few months.
> I raised this issue several times at
> http://apache-spark-developers-list.1001551.n3.nabble.com/In-Apache-Spark-JIRA-spark-dev-github-jira-sync-py-not-running-properly-td27077.html
>  because
> it blocked me to search JIRAs. I had to change my JQL to check JIRAs. It's
> still not being fixed. I don't know who to ask about this.
>
> If this is not being fixed, we might not have to care about 'In-Progress'
> anymore.
>
>
> 2019년 9월 8일 (일) 오후 1:31, Takeshi Yamamuro 님이 작성:
>
>> Hi, Hyukjin,
>>
>> I checked entries in the list and I found that some entries have
>> 'In-Progress' in their status and have oepn prs (e.g., SPARK-25211
>> ).
>> We can also close these PRs according to the bulk close?
>> (But, we might need to check the corresponding PRs manually?)
>>
>> Bests,
>> Takeshi
>>
>>
>> On Sun, Sep 8, 2019 at 12:15 PM Hyukjin Kwon  wrote:
>>
>>> HI all,
>>>
>>> We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x) in
>>> order to make it
>>> the manageable size before.
>>> Since Spark 2.3.4 will be EOL release, I plan to do this again roughly
>>> in a week.
>>>
>>> The JIRAs that has not been updated for the last year, and having affect
>>> version of EOL releases will be:
>>>   - Resolved as 'Incomplete' status
>>>   - Has a 'bulk-closed' label.
>>>
>>> I plan to use this JQL
>>>
>>> project = SPARK
>>>   AND status in (Open, "In Progress", Reopened)
>>>   AND (
>>> affectedVersion = EMPTY OR
>>> NOT (affectedVersion in versionMatch("^3.*")
>>>   OR affectedVersion in versionMatch("^2.4.*")
>>> )
>>>   )
>>>   AND updated <= -52w
>>>
>>>
>>> You could click this link and check.
>>>
>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w
>>>
>>> Please let me know if you guys have any concern or opinion on this.
>>>
>>> Thanks.
>>>
>>
>>
>> --
>> ---
>> Takeshi Yamamuro
>>
>

-- 
---
Takeshi Yamamuro

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

Thanks for checking it.

I think it's fine by two reasons below:

1. It has another condition for such cases - one year time range.
  Basically, such PRs have not been merged for one year, which I believe
are not likely merged soon.
  The JIRA status will be updated when such PRs are merged anyway.

2. The JIRAs and PRs ideally should be updated. If the PR authors forgot to
update affected versions,
it could be a good ping to update the affected versions in its JIRA,
which is a good practice I believe.

FWIW, currently setting 'in-Progress' doesn't properly work. It has been
few months.
I raised this issue several times at
http://apache-spark-developers-list.1001551.n3.nabble.com/In-Apache-Spark-JIRA-spark-dev-github-jira-sync-py-not-running-properly-td27077.html
 because
it blocked me to search JIRAs. I had to change my JQL to check JIRAs. It's
still not being fixed. I don't know who to ask about this.

If this is not being fixed, we might not have to care about 'In-Progress'
anymore.


2019년 9월 8일 (일) 오후 1:31, Takeshi Yamamuro 님이 작성:

> Hi, Hyukjin,
>
> I checked entries in the list and I found that some entries have
> 'In-Progress' in their status and have oepn prs (e.g., SPARK-25211
> ).
> We can also close these PRs according to the bulk close?
> (But, we might need to check the corresponding PRs manually?)
>
> Bests,
> Takeshi
>
>
> On Sun, Sep 8, 2019 at 12:15 PM Hyukjin Kwon  wrote:
>
>> HI all,
>>
>> We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x) in
>> order to make it
>> the manageable size before.
>> Since Spark 2.3.4 will be EOL release, I plan to do this again roughly in
>> a week.
>>
>> The JIRAs that has not been updated for the last year, and having affect
>> version of EOL releases will be:
>>   - Resolved as 'Incomplete' status
>>   - Has a 'bulk-closed' label.
>>
>> I plan to use this JQL
>>
>> project = SPARK
>>   AND status in (Open, "In Progress", Reopened)
>>   AND (
>> affectedVersion = EMPTY OR
>> NOT (affectedVersion in versionMatch("^3.*")
>>   OR affectedVersion in versionMatch("^2.4.*")
>> )
>>   )
>>   AND updated <= -52w
>>
>>
>> You could click this link and check.
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w
>>
>> Please let me know if you guys have any concern or opinion on this.
>>
>> Thanks.
>>
>
>
> --
> ---
> Takeshi Yamamuro
>

Re: Resolving all JIRAs affecting EOL releases

[Takeshi Yamamuro]

Hi, Hyukjin,

I checked entries in the list and I found that some entries have
'In-Progress' in their status and have oepn prs (e.g., SPARK-25211
).
We can also close these PRs according to the bulk close?
(But, we might need to check the corresponding PRs manually?)

Bests,
Takeshi


On Sun, Sep 8, 2019 at 12:15 PM Hyukjin Kwon  wrote:

> HI all,
>
> We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x) in
> order to make it
> the manageable size before.
> Since Spark 2.3.4 will be EOL release, I plan to do this again roughly in
> a week.
>
> The JIRAs that has not been updated for the last year, and having affect
> version of EOL releases will be:
>   - Resolved as 'Incomplete' status
>   - Has a 'bulk-closed' label.
>
> I plan to use this JQL
>
> project = SPARK
>   AND status in (Open, "In Progress", Reopened)
>   AND (
> affectedVersion = EMPTY OR
> NOT (affectedVersion in versionMatch("^3.*")
>   OR affectedVersion in versionMatch("^2.4.*")
> )
>   )
>   AND updated <= -52w
>
>
> You could click this link and check.
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w
>
> Please let me know if you guys have any concern or opinion on this.
>
> Thanks.
>


-- 
---
Takeshi Yamamuro

Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

HI all,

We have resolved JIRAs that targets EOL releases (up to Spark 2.2.x) in
order to make it
the manageable size before.
Since Spark 2.3.4 will be EOL release, I plan to do this again roughly in a
week.

The JIRAs that has not been updated for the last year, and having affect
version of EOL releases will be:
  - Resolved as 'Incomplete' status
  - Has a 'bulk-closed' label.

I plan to use this JQL

project = SPARK
  AND status in (Open, "In Progress", Reopened)
  AND (
affectedVersion = EMPTY OR
NOT (affectedVersion in versionMatch("^3.*")
  OR affectedVersion in versionMatch("^2.4.*")
)
  )
  AND updated <= -52w


You could click this link and check.

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20(affectedVersion%20%3D%20EMPTY%20OR%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)))%20AND%20updated%20%3C%3D%20-52w

Please let me know if you guys have any concern or opinion on this.

Thanks.

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

I took an action for those JIRAs.

The JIRAs that has not been updated for the last year, and having affect
version of EOL releases were now:
  - Resolved as 'Incomplete' status
  - Has a 'bulk-closed' label.

Thanks guys.

2019년 5월 21일 (화) 오전 8:35, shane knapp 님이 작성:

> alright, i found 3 jiras that i was able to close:
>
>1. SPARK-19612 
>2.
>   1. SPARK-22996 
>  2.
> 1. SPARK-22766
> 
> 2.
> 3.
>
>
> On Sun, May 19, 2019 at 6:43 PM Hyukjin Kwon  wrote:
>
>> Thanks Shane .. the URL I linked somehow didn't work in other people
>> browser. Hope this link works:
>>
>>
>> https://issues.apache.org/jira/browse/SPARK-23492?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w
>>
>> I will take an action around this time tomorrow considering there were
>> some more changes to make at the last minute.
>>
>>
>> 2019년 5월 19일 (일) 오후 6:39, Hyukjin Kwon 님이 작성:
>>
>>> I will add one more condition for "updated". So, it will additionally
>>> avoid things updated within one year but left open against EOL releases.
>>>
>>> project = SPARK
>>>   AND status in (Open, "In Progress", Reopened)
>>>   AND (
>>> affectedVersion = EMPTY OR
>>> NOT (affectedVersion in versionMatch("^3.*")
>>>   OR affectedVersion in versionMatch("^2.4.*")
>>>   OR affectedVersion in versionMatch("^2.3.*")
>>> )
>>>   )
>>>   AND updated <= -52w
>>>
>>>
>>> https://issues.apache.org/jira/issues/?filter=12344168=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w
>>>
>>> This still reduces JIRAs under 1000 which I originally targeted.
>>>
>>>
>>>
>>> 2019년 5월 19일 (일) 오후 6:08, Sean Owen 님이 작성:
>>>
 I'd only tweak this to perhaps not close JIRAs that have been updated
 recently -- even just avoiding things updated in the last month. For
 example this would close
 https://issues.apache.org/jira/browse/SPARK-27758 which was opened
 Friday (though, for other reasons it should probably be closed). Still I
 don't mind it under the logic that it has been reported against 2.1.0.

 On the other hand, I'd go further and close _anything_ not updated in a
 long time, like a year (or 2 if feeling conservative). That is there's
 probably a lot of old cruft out there that wasn't marked with an Affected
 Version, before that was required.

 On Sat, May 18, 2019 at 10:48 PM Hyukjin Kwon 
 wrote:

> Thanks guys.
>
> This thread got more than 3 PMC votes without any objection. I
> slightly edited JQL from Abdeali's suggestion (thanks, Abdeali).
>
>
> JQL:
>
> project = SPARK
>   AND status in (Open, "In Progress", Reopened)
>   AND (
> affectedVersion = EMPTY OR
> NOT (affectedVersion in versionMatch("^3.*")
>   OR affectedVersion in versionMatch("^2.4.*")
>   OR affectedVersion in versionMatch("^2.3.*")
> )
>   )
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)
>
>
> It means we will resolve all JIRAs that have EOL releases as affected
> versions, including no version specified in affected versions - this will
> reduce open JIRAs under 900.
>
> Looks I can use a bulk action feature in JIRA. Tomorrow at the similar
> time, I will
> - Label those JIRAs as 'bulk-closed'
> - Resolve them via `Incomplete` status.
>
> Please double check the list and let me know if you guys have any
> concern.
>
>
>
>
>
> 2019년 5월 18일 (토) 오후 12:22, Dongjoon Hyun 님이
> 작성:
>
>> +1, too.
>>
>> Thank you, 

Re: Resolving all JIRAs affecting EOL releases

[shane knapp]

alright, i found 3 jiras that i was able to close:

   1. SPARK-19612 
   2.
  1. SPARK-22996 
 2.
1. SPARK-22766

2.
3.


On Sun, May 19, 2019 at 6:43 PM Hyukjin Kwon  wrote:

> Thanks Shane .. the URL I linked somehow didn't work in other people
> browser. Hope this link works:
>
>
> https://issues.apache.org/jira/browse/SPARK-23492?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w
>
> I will take an action around this time tomorrow considering there were
> some more changes to make at the last minute.
>
>
> 2019년 5월 19일 (일) 오후 6:39, Hyukjin Kwon 님이 작성:
>
>> I will add one more condition for "updated". So, it will additionally
>> avoid things updated within one year but left open against EOL releases.
>>
>> project = SPARK
>>   AND status in (Open, "In Progress", Reopened)
>>   AND (
>> affectedVersion = EMPTY OR
>> NOT (affectedVersion in versionMatch("^3.*")
>>   OR affectedVersion in versionMatch("^2.4.*")
>>   OR affectedVersion in versionMatch("^2.3.*")
>> )
>>   )
>>   AND updated <= -52w
>>
>>
>> https://issues.apache.org/jira/issues/?filter=12344168=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w
>>
>> This still reduces JIRAs under 1000 which I originally targeted.
>>
>>
>>
>> 2019년 5월 19일 (일) 오후 6:08, Sean Owen 님이 작성:
>>
>>> I'd only tweak this to perhaps not close JIRAs that have been updated
>>> recently -- even just avoiding things updated in the last month. For
>>> example this would close
>>> https://issues.apache.org/jira/browse/SPARK-27758 which was opened
>>> Friday (though, for other reasons it should probably be closed). Still I
>>> don't mind it under the logic that it has been reported against 2.1.0.
>>>
>>> On the other hand, I'd go further and close _anything_ not updated in a
>>> long time, like a year (or 2 if feeling conservative). That is there's
>>> probably a lot of old cruft out there that wasn't marked with an Affected
>>> Version, before that was required.
>>>
>>> On Sat, May 18, 2019 at 10:48 PM Hyukjin Kwon 
>>> wrote:
>>>
 Thanks guys.

 This thread got more than 3 PMC votes without any objection. I slightly
 edited JQL from Abdeali's suggestion (thanks, Abdeali).


 JQL:

 project = SPARK
   AND status in (Open, "In Progress", Reopened)
   AND (
 affectedVersion = EMPTY OR
 NOT (affectedVersion in versionMatch("^3.*")
   OR affectedVersion in versionMatch("^2.4.*")
   OR affectedVersion in versionMatch("^2.3.*")
 )
   )


 https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)


 It means we will resolve all JIRAs that have EOL releases as affected
 versions, including no version specified in affected versions - this will
 reduce open JIRAs under 900.

 Looks I can use a bulk action feature in JIRA. Tomorrow at the similar
 time, I will
 - Label those JIRAs as 'bulk-closed'
 - Resolve them via `Incomplete` status.

 Please double check the list and let me know if you guys have any
 concern.





 2019년 5월 18일 (토) 오후 12:22, Dongjoon Hyun 님이
 작성:

> +1, too.
>
> Thank you, Hyukjin!
>
> Bests,
> Dongjoon.
>
>
> On Fri, May 17, 2019 at 9:07 AM Imran Rashid
>  wrote:
>
>> +1, thanks for taking this on
>>
>> On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon 
>> wrote:
>>
>>> oh, wait. 'Incomplete' can still make sense in this way then.
>>> Yes, I am good with 'Incomplete' too.
>>>
>>> 2019년 5월 16일 

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

Thanks Shane .. the URL I linked somehow didn't work in other people
browser. Hope this link works:

https://issues.apache.org/jira/browse/SPARK-23492?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w

I will take an action around this time tomorrow considering there were some
more changes to make at the last minute.


2019년 5월 19일 (일) 오후 6:39, Hyukjin Kwon 님이 작성:

> I will add one more condition for "updated". So, it will additionally
> avoid things updated within one year but left open against EOL releases.
>
> project = SPARK
>   AND status in (Open, "In Progress", Reopened)
>   AND (
> affectedVersion = EMPTY OR
> NOT (affectedVersion in versionMatch("^3.*")
>   OR affectedVersion in versionMatch("^2.4.*")
>   OR affectedVersion in versionMatch("^2.3.*")
> )
>   )
>   AND updated <= -52w
>
>
> https://issues.apache.org/jira/issues/?filter=12344168=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w
>
> This still reduces JIRAs under 1000 which I originally targeted.
>
>
>
> 2019년 5월 19일 (일) 오후 6:08, Sean Owen 님이 작성:
>
>> I'd only tweak this to perhaps not close JIRAs that have been updated
>> recently -- even just avoiding things updated in the last month. For
>> example this would close
>> https://issues.apache.org/jira/browse/SPARK-27758 which was opened
>> Friday (though, for other reasons it should probably be closed). Still I
>> don't mind it under the logic that it has been reported against 2.1.0.
>>
>> On the other hand, I'd go further and close _anything_ not updated in a
>> long time, like a year (or 2 if feeling conservative). That is there's
>> probably a lot of old cruft out there that wasn't marked with an Affected
>> Version, before that was required.
>>
>> On Sat, May 18, 2019 at 10:48 PM Hyukjin Kwon 
>> wrote:
>>
>>> Thanks guys.
>>>
>>> This thread got more than 3 PMC votes without any objection. I slightly
>>> edited JQL from Abdeali's suggestion (thanks, Abdeali).
>>>
>>>
>>> JQL:
>>>
>>> project = SPARK
>>>   AND status in (Open, "In Progress", Reopened)
>>>   AND (
>>> affectedVersion = EMPTY OR
>>> NOT (affectedVersion in versionMatch("^3.*")
>>>   OR affectedVersion in versionMatch("^2.4.*")
>>>   OR affectedVersion in versionMatch("^2.3.*")
>>> )
>>>   )
>>>
>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)
>>>
>>>
>>> It means we will resolve all JIRAs that have EOL releases as affected
>>> versions, including no version specified in affected versions - this will
>>> reduce open JIRAs under 900.
>>>
>>> Looks I can use a bulk action feature in JIRA. Tomorrow at the similar
>>> time, I will
>>> - Label those JIRAs as 'bulk-closed'
>>> - Resolve them via `Incomplete` status.
>>>
>>> Please double check the list and let me know if you guys have any
>>> concern.
>>>
>>>
>>>
>>>
>>>
>>> 2019년 5월 18일 (토) 오후 12:22, Dongjoon Hyun 님이 작성:
>>>
 +1, too.

 Thank you, Hyukjin!

 Bests,
 Dongjoon.


 On Fri, May 17, 2019 at 9:07 AM Imran Rashid
  wrote:

> +1, thanks for taking this on
>
> On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon 
> wrote:
>
>> oh, wait. 'Incomplete' can still make sense in this way then.
>> Yes, I am good with 'Incomplete' too.
>>
>> 2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon 님이 작성:
>>
>>> I actually recently used 'Incomplete'  a bit when the JIRA is
>>> basically too poorly formed (like just copying and pasting an error) ...
>>>
>>> I was thinking about 'Unresolved' status or `Auto Closed' too. I
>>> double checked they can be reopen as well after resolution.
>>>
>>> [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
>>> [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]
>>>
>>> 2019년 5월 16일 (목) 오전 11:04, 

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

I will add one more condition for "updated". So, it will additionally avoid
things updated within one year but left open against EOL releases.

project = SPARK
  AND status in (Open, "In Progress", Reopened)
  AND (
affectedVersion = EMPTY OR
NOT (affectedVersion in versionMatch("^3.*")
  OR affectedVersion in versionMatch("^2.4.*")
  OR affectedVersion in versionMatch("^2.3.*")
)
  )
  AND updated <= -52w

https://issues.apache.org/jira/issues/?filter=12344168=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w

This still reduces JIRAs under 1000 which I originally targeted.



2019년 5월 19일 (일) 오후 6:08, Sean Owen 님이 작성:

> I'd only tweak this to perhaps not close JIRAs that have been updated
> recently -- even just avoiding things updated in the last month. For
> example this would close https://issues.apache.org/jira/browse/SPARK-27758 
> which
> was opened Friday (though, for other reasons it should probably be closed).
> Still I don't mind it under the logic that it has been reported against
> 2.1.0.
>
> On the other hand, I'd go further and close _anything_ not updated in a
> long time, like a year (or 2 if feeling conservative). That is there's
> probably a lot of old cruft out there that wasn't marked with an Affected
> Version, before that was required.
>
> On Sat, May 18, 2019 at 10:48 PM Hyukjin Kwon  wrote:
>
>> Thanks guys.
>>
>> This thread got more than 3 PMC votes without any objection. I slightly
>> edited JQL from Abdeali's suggestion (thanks, Abdeali).
>>
>>
>> JQL:
>>
>> project = SPARK
>>   AND status in (Open, "In Progress", Reopened)
>>   AND (
>> affectedVersion = EMPTY OR
>> NOT (affectedVersion in versionMatch("^3.*")
>>   OR affectedVersion in versionMatch("^2.4.*")
>>   OR affectedVersion in versionMatch("^2.3.*")
>> )
>>   )
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)
>>
>>
>> It means we will resolve all JIRAs that have EOL releases as affected
>> versions, including no version specified in affected versions - this will
>> reduce open JIRAs under 900.
>>
>> Looks I can use a bulk action feature in JIRA. Tomorrow at the similar
>> time, I will
>> - Label those JIRAs as 'bulk-closed'
>> - Resolve them via `Incomplete` status.
>>
>> Please double check the list and let me know if you guys have any concern.
>>
>>
>>
>>
>>
>> 2019년 5월 18일 (토) 오후 12:22, Dongjoon Hyun 님이 작성:
>>
>>> +1, too.
>>>
>>> Thank you, Hyukjin!
>>>
>>> Bests,
>>> Dongjoon.
>>>
>>>
>>> On Fri, May 17, 2019 at 9:07 AM Imran Rashid
>>>  wrote:
>>>
 +1, thanks for taking this on

 On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon 
 wrote:

> oh, wait. 'Incomplete' can still make sense in this way then.
> Yes, I am good with 'Incomplete' too.
>
> 2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon 님이 작성:
>
>> I actually recently used 'Incomplete'  a bit when the JIRA is
>> basically too poorly formed (like just copying and pasting an error) ...
>>
>> I was thinking about 'Unresolved' status or `Auto Closed' too. I
>> double checked they can be reopen as well after resolution.
>>
>> [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
>> [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]
>>
>> 2019년 5월 16일 (목) 오전 11:04, Sean Owen 님이 작성:
>>
>>> Agree, anything without an Affected Version should be old enough to
>>> time out.
>>> I might use "Incomplete" or something as the status, as we haven't
>>> otherwise used that. Maybe that's simpler than a label. But, anything 
>>> like
>>> that sounds good.
>>>
>>> On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon 
>>> wrote:
>>>
 BTW, affected version became a required field (I don't remember
 when exactly was .. I believe it's around when we work on Spark 2.3):

 [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]

 So, including all EOL versions and affected versions not specified
 will roughly work.
 Using "Cannot Reproduce" as its status and 'bulk-closed' label
 makes the best sense to me.

 Okie. I want to open this roughly for a week 

Re: Resolving all JIRAs affecting EOL releases

[Sean Owen]

I'd only tweak this to perhaps not close JIRAs that have been updated
recently -- even just avoiding things updated in the last month. For
example this would close
https://issues.apache.org/jira/browse/SPARK-27758 which
was opened Friday (though, for other reasons it should probably be closed).
Still I don't mind it under the logic that it has been reported against
2.1.0.

On the other hand, I'd go further and close _anything_ not updated in a
long time, like a year (or 2 if feeling conservative). That is there's
probably a lot of old cruft out there that wasn't marked with an Affected
Version, before that was required.

On Sat, May 18, 2019 at 10:48 PM Hyukjin Kwon  wrote:

> Thanks guys.
>
> This thread got more than 3 PMC votes without any objection. I slightly
> edited JQL from Abdeali's suggestion (thanks, Abdeali).
>
>
> JQL:
>
> project = SPARK
>   AND status in (Open, "In Progress", Reopened)
>   AND (
> affectedVersion = EMPTY OR
> NOT (affectedVersion in versionMatch("^3.*")
>   OR affectedVersion in versionMatch("^2.4.*")
>   OR affectedVersion in versionMatch("^2.3.*")
> )
>   )
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)
>
>
> It means we will resolve all JIRAs that have EOL releases as affected
> versions, including no version specified in affected versions - this will
> reduce open JIRAs under 900.
>
> Looks I can use a bulk action feature in JIRA. Tomorrow at the similar
> time, I will
> - Label those JIRAs as 'bulk-closed'
> - Resolve them via `Incomplete` status.
>
> Please double check the list and let me know if you guys have any concern.
>
>
>
>
>
> 2019년 5월 18일 (토) 오후 12:22, Dongjoon Hyun 님이 작성:
>
>> +1, too.
>>
>> Thank you, Hyukjin!
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Fri, May 17, 2019 at 9:07 AM Imran Rashid 
>> wrote:
>>
>>> +1, thanks for taking this on
>>>
>>> On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon 
>>> wrote:
>>>
 oh, wait. 'Incomplete' can still make sense in this way then.
 Yes, I am good with 'Incomplete' too.

 2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon 님이 작성:

> I actually recently used 'Incomplete'  a bit when the JIRA is
> basically too poorly formed (like just copying and pasting an error) ...
>
> I was thinking about 'Unresolved' status or `Auto Closed' too. I
> double checked they can be reopen as well after resolution.
>
> [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
> [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]
>
> 2019년 5월 16일 (목) 오전 11:04, Sean Owen 님이 작성:
>
>> Agree, anything without an Affected Version should be old enough to
>> time out.
>> I might use "Incomplete" or something as the status, as we haven't
>> otherwise used that. Maybe that's simpler than a label. But, anything 
>> like
>> that sounds good.
>>
>> On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon 
>> wrote:
>>
>>> BTW, affected version became a required field (I don't remember when
>>> exactly was .. I believe it's around when we work on Spark 2.3):
>>>
>>> [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]
>>>
>>> So, including all EOL versions and affected versions not specified
>>> will roughly work.
>>> Using "Cannot Reproduce" as its status and 'bulk-closed' label makes
>>> the best sense to me.
>>>
>>> Okie. I want to open this roughly for a week before taking an actual
>>> action for this. If there's no more feedback, I will do as I said ^ next
>>> week.
>>>
>>>
>>> 2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:
>>>
 +1 in favor of some sort of JIRA cleanup.

 My only request is that we attach some sort of 'bulk-closed' label
 to issues that we close via JIRA filter batch operations (and resolve 
 the
 issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
 makes it easier to audit what was closed, simplifying the process of
 identifying and re-opening valid issues caught in our dragnet.


 On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:

> I gave up looking through JIRAs a long time ago, so, big respect
> for
> continuing to try to triage them. I am afraid we're missing a few
> important bug reports in the torrent, but most JIRAs are not
> well-formed, just questions, stale, or simply things that won't be
> added. I do think it's important to reflect that reality, and so
> I'm
> always in favor of 

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

Thanks guys.

This thread got more than 3 PMC votes without any objection. I slightly
edited JQL from Abdeali's suggestion (thanks, Abdeali).


JQL:

project = SPARK
  AND status in (Open, "In Progress", Reopened)
  AND (
affectedVersion = EMPTY OR
NOT (affectedVersion in versionMatch("^3.*")
  OR affectedVersion in versionMatch("^2.4.*")
  OR affectedVersion in versionMatch("^2.3.*")
)
  )

https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)


It means we will resolve all JIRAs that have EOL releases as affected
versions, including no version specified in affected versions - this will
reduce open JIRAs under 900.

Looks I can use a bulk action feature in JIRA. Tomorrow at the similar
time, I will
- Label those JIRAs as 'bulk-closed'
- Resolve them via `Incomplete` status.

Please double check the list and let me know if you guys have any concern.





2019년 5월 18일 (토) 오후 12:22, Dongjoon Hyun 님이 작성:

> +1, too.
>
> Thank you, Hyukjin!
>
> Bests,
> Dongjoon.
>
>
> On Fri, May 17, 2019 at 9:07 AM Imran Rashid 
> wrote:
>
>> +1, thanks for taking this on
>>
>> On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon  wrote:
>>
>>> oh, wait. 'Incomplete' can still make sense in this way then.
>>> Yes, I am good with 'Incomplete' too.
>>>
>>> 2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon 님이 작성:
>>>
 I actually recently used 'Incomplete'  a bit when the JIRA is basically
 too poorly formed (like just copying and pasting an error) ...

 I was thinking about 'Unresolved' status or `Auto Closed' too. I double
 checked they can be reopen as well after resolution.

 [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
 [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]

 2019년 5월 16일 (목) 오전 11:04, Sean Owen 님이 작성:

> Agree, anything without an Affected Version should be old enough to
> time out.
> I might use "Incomplete" or something as the status, as we haven't
> otherwise used that. Maybe that's simpler than a label. But, anything like
> that sounds good.
>
> On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon 
> wrote:
>
>> BTW, affected version became a required field (I don't remember when
>> exactly was .. I believe it's around when we work on Spark 2.3):
>>
>> [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]
>>
>> So, including all EOL versions and affected versions not specified
>> will roughly work.
>> Using "Cannot Reproduce" as its status and 'bulk-closed' label makes
>> the best sense to me.
>>
>> Okie. I want to open this roughly for a week before taking an actual
>> action for this. If there's no more feedback, I will do as I said ^ next
>> week.
>>
>>
>> 2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:
>>
>>> +1 in favor of some sort of JIRA cleanup.
>>>
>>> My only request is that we attach some sort of 'bulk-closed' label
>>> to issues that we close via JIRA filter batch operations (and resolve 
>>> the
>>> issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
>>> makes it easier to audit what was closed, simplifying the process of
>>> identifying and re-opening valid issues caught in our dragnet.
>>>
>>>
>>> On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:
>>>
 I gave up looking through JIRAs a long time ago, so, big respect for
 continuing to try to triage them. I am afraid we're missing a few
 important bug reports in the torrent, but most JIRAs are not
 well-formed, just questions, stale, or simply things that won't be
 added. I do think it's important to reflect that reality, and so I'm
 always in favor of more aggressively closing JIRAs. I think this is
 more standard practice, from projects like TensorFlow/Keras, pandas,
 etc to just automatically drop Issues that don't see activity for N
 days. We won't do that, but, are probably on the other hand far too
 lax in closing them.

 Remember that JIRAs stay searchable and can be reopened, so it's not
 like we lose much information.

 I'd close anything that hasn't had activity in 2 years (?), as a
 start.
 I like the idea of closing things that only affect an EOL release,
 but, many items aren't marked, so may need to cast the net wider.

 I think only then does it make sense to look at bothering to
 reproduce
 or evaluate the 1000s that will still remain.

Re: Resolving all JIRAs affecting EOL releases

[Dongjoon Hyun]

+1, too.

Thank you, Hyukjin!

Bests,
Dongjoon.


On Fri, May 17, 2019 at 9:07 AM Imran Rashid 
wrote:

> +1, thanks for taking this on
>
> On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon  wrote:
>
>> oh, wait. 'Incomplete' can still make sense in this way then.
>> Yes, I am good with 'Incomplete' too.
>>
>> 2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon 님이 작성:
>>
>>> I actually recently used 'Incomplete'  a bit when the JIRA is basically
>>> too poorly formed (like just copying and pasting an error) ...
>>>
>>> I was thinking about 'Unresolved' status or `Auto Closed' too. I double
>>> checked they can be reopen as well after resolution.
>>>
>>> [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
>>> [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]
>>>
>>> 2019년 5월 16일 (목) 오전 11:04, Sean Owen 님이 작성:
>>>
 Agree, anything without an Affected Version should be old enough to
 time out.
 I might use "Incomplete" or something as the status, as we haven't
 otherwise used that. Maybe that's simpler than a label. But, anything like
 that sounds good.

 On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon 
 wrote:

> BTW, affected version became a required field (I don't remember when
> exactly was .. I believe it's around when we work on Spark 2.3):
>
> [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]
>
> So, including all EOL versions and affected versions not specified
> will roughly work.
> Using "Cannot Reproduce" as its status and 'bulk-closed' label makes
> the best sense to me.
>
> Okie. I want to open this roughly for a week before taking an actual
> action for this. If there's no more feedback, I will do as I said ^ next
> week.
>
>
> 2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:
>
>> +1 in favor of some sort of JIRA cleanup.
>>
>> My only request is that we attach some sort of 'bulk-closed' label to
>> issues that we close via JIRA filter batch operations (and resolve the
>> issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
>> makes it easier to audit what was closed, simplifying the process of
>> identifying and re-opening valid issues caught in our dragnet.
>>
>>
>> On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:
>>
>>> I gave up looking through JIRAs a long time ago, so, big respect for
>>> continuing to try to triage them. I am afraid we're missing a few
>>> important bug reports in the torrent, but most JIRAs are not
>>> well-formed, just questions, stale, or simply things that won't be
>>> added. I do think it's important to reflect that reality, and so I'm
>>> always in favor of more aggressively closing JIRAs. I think this is
>>> more standard practice, from projects like TensorFlow/Keras, pandas,
>>> etc to just automatically drop Issues that don't see activity for N
>>> days. We won't do that, but, are probably on the other hand far too
>>> lax in closing them.
>>>
>>> Remember that JIRAs stay searchable and can be reopened, so it's not
>>> like we lose much information.
>>>
>>> I'd close anything that hasn't had activity in 2 years (?), as a
>>> start.
>>> I like the idea of closing things that only affect an EOL release,
>>> but, many items aren't marked, so may need to cast the net wider.
>>>
>>> I think only then does it make sense to look at bothering to
>>> reproduce
>>> or evaluate the 1000s that will still remain.
>>>
>>> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon 
>>> wrote:
>>> >
>>> > Hi all,
>>> >
>>> > I would like to propose to resolve all JIRAs that affects EOL
>>> releases - 2.2 and below. and affected version
>>> > not specified. I was rather against this way and considered this
>>> as last resort in roughly 3 years ago
>>> > when we discussed. Now I think we should go ahead with this. See
>>> below.
>>> >
>>> > I have been talking care of this for so long time almost every day
>>> those 3 years. The number of JIRAs
>>> > keeps increasing and it does never go down. Now the number is
>>> going over 2500 JIRAs.
>>> > Did you guys know? in JIRA, we can only go through page by page up
>>> to 1000 items. So, currently we're even
>>> > having difficulties to go through every JIRA. We should manually
>>> filter out and check each.
>>> > The number is going over the manageable size.
>>> >
>>> > I am not suggesting this without anything actually trying. This is
>>> what we have tried within my visibility:
>>> >
>>> >   1. In roughly 3 years ago, Sean tried to gather committers and
>>> even non-committers people to sort
>>> > out this number. At that time, we were only able to keep this
>>> number as is. After we lost this momentum,
>>> > it kept increasing back.
>>> >   2. At least I scanned _all_ the previous 

Re: Resolving all JIRAs affecting EOL releases

[Imran Rashid]

+1, thanks for taking this on

On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon  wrote:

> oh, wait. 'Incomplete' can still make sense in this way then.
> Yes, I am good with 'Incomplete' too.
>
> 2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon 님이 작성:
>
>> I actually recently used 'Incomplete'  a bit when the JIRA is basically
>> too poorly formed (like just copying and pasting an error) ...
>>
>> I was thinking about 'Unresolved' status or `Auto Closed' too. I double
>> checked they can be reopen as well after resolution.
>>
>> [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
>> [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]
>>
>> 2019년 5월 16일 (목) 오전 11:04, Sean Owen 님이 작성:
>>
>>> Agree, anything without an Affected Version should be old enough to time
>>> out.
>>> I might use "Incomplete" or something as the status, as we haven't
>>> otherwise used that. Maybe that's simpler than a label. But, anything like
>>> that sounds good.
>>>
>>> On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon 
>>> wrote:
>>>
 BTW, affected version became a required field (I don't remember when
 exactly was .. I believe it's around when we work on Spark 2.3):

 [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]

 So, including all EOL versions and affected versions not specified will
 roughly work.
 Using "Cannot Reproduce" as its status and 'bulk-closed' label makes
 the best sense to me.

 Okie. I want to open this roughly for a week before taking an actual
 action for this. If there's no more feedback, I will do as I said ^ next
 week.


 2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:

> +1 in favor of some sort of JIRA cleanup.
>
> My only request is that we attach some sort of 'bulk-closed' label to
> issues that we close via JIRA filter batch operations (and resolve the
> issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
> makes it easier to audit what was closed, simplifying the process of
> identifying and re-opening valid issues caught in our dragnet.
>
>
> On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:
>
>> I gave up looking through JIRAs a long time ago, so, big respect for
>> continuing to try to triage them. I am afraid we're missing a few
>> important bug reports in the torrent, but most JIRAs are not
>> well-formed, just questions, stale, or simply things that won't be
>> added. I do think it's important to reflect that reality, and so I'm
>> always in favor of more aggressively closing JIRAs. I think this is
>> more standard practice, from projects like TensorFlow/Keras, pandas,
>> etc to just automatically drop Issues that don't see activity for N
>> days. We won't do that, but, are probably on the other hand far too
>> lax in closing them.
>>
>> Remember that JIRAs stay searchable and can be reopened, so it's not
>> like we lose much information.
>>
>> I'd close anything that hasn't had activity in 2 years (?), as a
>> start.
>> I like the idea of closing things that only affect an EOL release,
>> but, many items aren't marked, so may need to cast the net wider.
>>
>> I think only then does it make sense to look at bothering to reproduce
>> or evaluate the 1000s that will still remain.
>>
>> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon 
>> wrote:
>> >
>> > Hi all,
>> >
>> > I would like to propose to resolve all JIRAs that affects EOL
>> releases - 2.2 and below. and affected version
>> > not specified. I was rather against this way and considered this as
>> last resort in roughly 3 years ago
>> > when we discussed. Now I think we should go ahead with this. See
>> below.
>> >
>> > I have been talking care of this for so long time almost every day
>> those 3 years. The number of JIRAs
>> > keeps increasing and it does never go down. Now the number is going
>> over 2500 JIRAs.
>> > Did you guys know? in JIRA, we can only go through page by page up
>> to 1000 items. So, currently we're even
>> > having difficulties to go through every JIRA. We should manually
>> filter out and check each.
>> > The number is going over the manageable size.
>> >
>> > I am not suggesting this without anything actually trying. This is
>> what we have tried within my visibility:
>> >
>> >   1. In roughly 3 years ago, Sean tried to gather committers and
>> even non-committers people to sort
>> > out this number. At that time, we were only able to keep this
>> number as is. After we lost this momentum,
>> > it kept increasing back.
>> >   2. At least I scanned _all_ the previous JIRAs at least more than
>> two times and resolved them. Roughly
>> > once a year. The rest of them are mostly obsolete but not
>> enough information to investigate further.
>> >   3. I strictly stick to 

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

I actually recently used 'Incomplete'  a bit when the JIRA is basically too
poorly formed (like just copying and pasting an error) ...

I was thinking about 'Unresolved' status or `Auto Closed' too. I double
checked they can be reopen as well after resolution.

[image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
[image: Screen Shot 2019-05-16 at 10.35.39 AM.png]

2019년 5월 16일 (목) 오전 11:04, Sean Owen 님이 작성:

> Agree, anything without an Affected Version should be old enough to time
> out.
> I might use "Incomplete" or something as the status, as we haven't
> otherwise used that. Maybe that's simpler than a label. But, anything like
> that sounds good.
>
> On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon  wrote:
>
>> BTW, affected version became a required field (I don't remember when
>> exactly was .. I believe it's around when we work on Spark 2.3):
>>
>> [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]
>>
>> So, including all EOL versions and affected versions not specified will
>> roughly work.
>> Using "Cannot Reproduce" as its status and 'bulk-closed' label makes the
>> best sense to me.
>>
>> Okie. I want to open this roughly for a week before taking an actual
>> action for this. If there's no more feedback, I will do as I said ^ next
>> week.
>>
>>
>> 2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:
>>
>>> +1 in favor of some sort of JIRA cleanup.
>>>
>>> My only request is that we attach some sort of 'bulk-closed' label to
>>> issues that we close via JIRA filter batch operations (and resolve the
>>> issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
>>> makes it easier to audit what was closed, simplifying the process of
>>> identifying and re-opening valid issues caught in our dragnet.
>>>
>>>
>>> On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:
>>>
 I gave up looking through JIRAs a long time ago, so, big respect for
 continuing to try to triage them. I am afraid we're missing a few
 important bug reports in the torrent, but most JIRAs are not
 well-formed, just questions, stale, or simply things that won't be
 added. I do think it's important to reflect that reality, and so I'm
 always in favor of more aggressively closing JIRAs. I think this is
 more standard practice, from projects like TensorFlow/Keras, pandas,
 etc to just automatically drop Issues that don't see activity for N
 days. We won't do that, but, are probably on the other hand far too
 lax in closing them.

 Remember that JIRAs stay searchable and can be reopened, so it's not
 like we lose much information.

 I'd close anything that hasn't had activity in 2 years (?), as a start.
 I like the idea of closing things that only affect an EOL release,
 but, many items aren't marked, so may need to cast the net wider.

 I think only then does it make sense to look at bothering to reproduce
 or evaluate the 1000s that will still remain.

 On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon 
 wrote:
 >
 > Hi all,
 >
 > I would like to propose to resolve all JIRAs that affects EOL
 releases - 2.2 and below. and affected version
 > not specified. I was rather against this way and considered this as
 last resort in roughly 3 years ago
 > when we discussed. Now I think we should go ahead with this. See
 below.
 >
 > I have been talking care of this for so long time almost every day
 those 3 years. The number of JIRAs
 > keeps increasing and it does never go down. Now the number is going
 over 2500 JIRAs.
 > Did you guys know? in JIRA, we can only go through page by page up to
 1000 items. So, currently we're even
 > having difficulties to go through every JIRA. We should manually
 filter out and check each.
 > The number is going over the manageable size.
 >
 > I am not suggesting this without anything actually trying. This is
 what we have tried within my visibility:
 >
 >   1. In roughly 3 years ago, Sean tried to gather committers and even
 non-committers people to sort
 > out this number. At that time, we were only able to keep this
 number as is. After we lost this momentum,
 > it kept increasing back.
 >   2. At least I scanned _all_ the previous JIRAs at least more than
 two times and resolved them. Roughly
 > once a year. The rest of them are mostly obsolete but not enough
 information to investigate further.
 >   3. I strictly stick to "Contributing to JIRA Maintenance"
 https://spark.apache.org/contributing.html and
 > resolve JIRAs.
 >   4. Promoting other people to comment on JIRA or actively resolve
 them.
 >
 > One of the facts I realised is the increasing number of committers
 doesn't virtually help this much (although
 > it might be helpful if somebody active in JIRA becomes a committer.)
 >
 > One of the important thing I should note is that, 

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

oh, wait. 'Incomplete' can still make sense in this way then.
Yes, I am good with 'Incomplete' too.

2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon 님이 작성:

> I actually recently used 'Incomplete'  a bit when the JIRA is basically
> too poorly formed (like just copying and pasting an error) ...
>
> I was thinking about 'Unresolved' status or `Auto Closed' too. I double
> checked they can be reopen as well after resolution.
>
> [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
> [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]
>
> 2019년 5월 16일 (목) 오전 11:04, Sean Owen 님이 작성:
>
>> Agree, anything without an Affected Version should be old enough to time
>> out.
>> I might use "Incomplete" or something as the status, as we haven't
>> otherwise used that. Maybe that's simpler than a label. But, anything like
>> that sounds good.
>>
>> On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon  wrote:
>>
>>> BTW, affected version became a required field (I don't remember when
>>> exactly was .. I believe it's around when we work on Spark 2.3):
>>>
>>> [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]
>>>
>>> So, including all EOL versions and affected versions not specified will
>>> roughly work.
>>> Using "Cannot Reproduce" as its status and 'bulk-closed' label makes the
>>> best sense to me.
>>>
>>> Okie. I want to open this roughly for a week before taking an actual
>>> action for this. If there's no more feedback, I will do as I said ^ next
>>> week.
>>>
>>>
>>> 2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:
>>>
 +1 in favor of some sort of JIRA cleanup.

 My only request is that we attach some sort of 'bulk-closed' label to
 issues that we close via JIRA filter batch operations (and resolve the
 issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
 makes it easier to audit what was closed, simplifying the process of
 identifying and re-opening valid issues caught in our dragnet.


 On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:

> I gave up looking through JIRAs a long time ago, so, big respect for
> continuing to try to triage them. I am afraid we're missing a few
> important bug reports in the torrent, but most JIRAs are not
> well-formed, just questions, stale, or simply things that won't be
> added. I do think it's important to reflect that reality, and so I'm
> always in favor of more aggressively closing JIRAs. I think this is
> more standard practice, from projects like TensorFlow/Keras, pandas,
> etc to just automatically drop Issues that don't see activity for N
> days. We won't do that, but, are probably on the other hand far too
> lax in closing them.
>
> Remember that JIRAs stay searchable and can be reopened, so it's not
> like we lose much information.
>
> I'd close anything that hasn't had activity in 2 years (?), as a start.
> I like the idea of closing things that only affect an EOL release,
> but, many items aren't marked, so may need to cast the net wider.
>
> I think only then does it make sense to look at bothering to reproduce
> or evaluate the 1000s that will still remain.
>
> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon 
> wrote:
> >
> > Hi all,
> >
> > I would like to propose to resolve all JIRAs that affects EOL
> releases - 2.2 and below. and affected version
> > not specified. I was rather against this way and considered this as
> last resort in roughly 3 years ago
> > when we discussed. Now I think we should go ahead with this. See
> below.
> >
> > I have been talking care of this for so long time almost every day
> those 3 years. The number of JIRAs
> > keeps increasing and it does never go down. Now the number is going
> over 2500 JIRAs.
> > Did you guys know? in JIRA, we can only go through page by page up
> to 1000 items. So, currently we're even
> > having difficulties to go through every JIRA. We should manually
> filter out and check each.
> > The number is going over the manageable size.
> >
> > I am not suggesting this without anything actually trying. This is
> what we have tried within my visibility:
> >
> >   1. In roughly 3 years ago, Sean tried to gather committers and
> even non-committers people to sort
> > out this number. At that time, we were only able to keep this
> number as is. After we lost this momentum,
> > it kept increasing back.
> >   2. At least I scanned _all_ the previous JIRAs at least more than
> two times and resolved them. Roughly
> > once a year. The rest of them are mostly obsolete but not enough
> information to investigate further.
> >   3. I strictly stick to "Contributing to JIRA Maintenance"
> https://spark.apache.org/contributing.html and
> > resolve JIRAs.
> >   4. Promoting other people to comment on JIRA or actively resolve
> them.
> 

Re: Resolving all JIRAs affecting EOL releases

[Sean Owen]

Agree, anything without an Affected Version should be old enough to time
out.
I might use "Incomplete" or something as the status, as we haven't
otherwise used that. Maybe that's simpler than a label. But, anything like
that sounds good.

On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon  wrote:

> BTW, affected version became a required field (I don't remember when
> exactly was .. I believe it's around when we work on Spark 2.3):
>
> [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]
>
> So, including all EOL versions and affected versions not specified will
> roughly work.
> Using "Cannot Reproduce" as its status and 'bulk-closed' label makes the
> best sense to me.
>
> Okie. I want to open this roughly for a week before taking an actual
> action for this. If there's no more feedback, I will do as I said ^ next
> week.
>
>
> 2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:
>
>> +1 in favor of some sort of JIRA cleanup.
>>
>> My only request is that we attach some sort of 'bulk-closed' label to
>> issues that we close via JIRA filter batch operations (and resolve the
>> issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
>> makes it easier to audit what was closed, simplifying the process of
>> identifying and re-opening valid issues caught in our dragnet.
>>
>>
>> On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:
>>
>>> I gave up looking through JIRAs a long time ago, so, big respect for
>>> continuing to try to triage them. I am afraid we're missing a few
>>> important bug reports in the torrent, but most JIRAs are not
>>> well-formed, just questions, stale, or simply things that won't be
>>> added. I do think it's important to reflect that reality, and so I'm
>>> always in favor of more aggressively closing JIRAs. I think this is
>>> more standard practice, from projects like TensorFlow/Keras, pandas,
>>> etc to just automatically drop Issues that don't see activity for N
>>> days. We won't do that, but, are probably on the other hand far too
>>> lax in closing them.
>>>
>>> Remember that JIRAs stay searchable and can be reopened, so it's not
>>> like we lose much information.
>>>
>>> I'd close anything that hasn't had activity in 2 years (?), as a start.
>>> I like the idea of closing things that only affect an EOL release,
>>> but, many items aren't marked, so may need to cast the net wider.
>>>
>>> I think only then does it make sense to look at bothering to reproduce
>>> or evaluate the 1000s that will still remain.
>>>
>>> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon 
>>> wrote:
>>> >
>>> > Hi all,
>>> >
>>> > I would like to propose to resolve all JIRAs that affects EOL releases
>>> - 2.2 and below. and affected version
>>> > not specified. I was rather against this way and considered this as
>>> last resort in roughly 3 years ago
>>> > when we discussed. Now I think we should go ahead with this. See below.
>>> >
>>> > I have been talking care of this for so long time almost every day
>>> those 3 years. The number of JIRAs
>>> > keeps increasing and it does never go down. Now the number is going
>>> over 2500 JIRAs.
>>> > Did you guys know? in JIRA, we can only go through page by page up to
>>> 1000 items. So, currently we're even
>>> > having difficulties to go through every JIRA. We should manually
>>> filter out and check each.
>>> > The number is going over the manageable size.
>>> >
>>> > I am not suggesting this without anything actually trying. This is
>>> what we have tried within my visibility:
>>> >
>>> >   1. In roughly 3 years ago, Sean tried to gather committers and even
>>> non-committers people to sort
>>> > out this number. At that time, we were only able to keep this
>>> number as is. After we lost this momentum,
>>> > it kept increasing back.
>>> >   2. At least I scanned _all_ the previous JIRAs at least more than
>>> two times and resolved them. Roughly
>>> > once a year. The rest of them are mostly obsolete but not enough
>>> information to investigate further.
>>> >   3. I strictly stick to "Contributing to JIRA Maintenance"
>>> https://spark.apache.org/contributing.html and
>>> > resolve JIRAs.
>>> >   4. Promoting other people to comment on JIRA or actively resolve
>>> them.
>>> >
>>> > One of the facts I realised is the increasing number of committers
>>> doesn't virtually help this much (although
>>> > it might be helpful if somebody active in JIRA becomes a committer.)
>>> >
>>> > One of the important thing I should note is that, it's now almost
>>> pretty difficult to reproduce and test the
>>> > issues found in EOL releases. We should git clone, checkout, build and
>>> test. And then, see if that issue
>>> > still exists in upstream, and fix. This is non-trivial overhead.
>>> >
>>> > Therefore, I would like to propose resolving _all_ the JIRAs that
>>> targets EOL releases - 2.2 and below.
>>> > Please let me know if anyone has some concerns or objections.
>>> >
>>> > Thanks.
>>>
>>> 

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

BTW, affected version became a required field (I don't remember when
exactly was .. I believe it's around when we work on Spark 2.3):

[image: Screen Shot 2019-05-16 at 10.29.50 AM.png]

So, including all EOL versions and affected versions not specified will
roughly work.
Using "Cannot Reproduce" as its status and 'bulk-closed' label makes the
best sense to me.

Okie. I want to open this roughly for a week before taking an actual action
for this. If there's no more feedback, I will do as I said ^ next week.


2019년 5월 15일 (수) 오후 11:33, Josh Rosen 님이 작성:

> +1 in favor of some sort of JIRA cleanup.
>
> My only request is that we attach some sort of 'bulk-closed' label to
> issues that we close via JIRA filter batch operations (and resolve the
> issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
> makes it easier to audit what was closed, simplifying the process of
> identifying and re-opening valid issues caught in our dragnet.
>
>
> On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:
>
>> I gave up looking through JIRAs a long time ago, so, big respect for
>> continuing to try to triage them. I am afraid we're missing a few
>> important bug reports in the torrent, but most JIRAs are not
>> well-formed, just questions, stale, or simply things that won't be
>> added. I do think it's important to reflect that reality, and so I'm
>> always in favor of more aggressively closing JIRAs. I think this is
>> more standard practice, from projects like TensorFlow/Keras, pandas,
>> etc to just automatically drop Issues that don't see activity for N
>> days. We won't do that, but, are probably on the other hand far too
>> lax in closing them.
>>
>> Remember that JIRAs stay searchable and can be reopened, so it's not
>> like we lose much information.
>>
>> I'd close anything that hasn't had activity in 2 years (?), as a start.
>> I like the idea of closing things that only affect an EOL release,
>> but, many items aren't marked, so may need to cast the net wider.
>>
>> I think only then does it make sense to look at bothering to reproduce
>> or evaluate the 1000s that will still remain.
>>
>> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon  wrote:
>> >
>> > Hi all,
>> >
>> > I would like to propose to resolve all JIRAs that affects EOL releases
>> - 2.2 and below. and affected version
>> > not specified. I was rather against this way and considered this as
>> last resort in roughly 3 years ago
>> > when we discussed. Now I think we should go ahead with this. See below.
>> >
>> > I have been talking care of this for so long time almost every day
>> those 3 years. The number of JIRAs
>> > keeps increasing and it does never go down. Now the number is going
>> over 2500 JIRAs.
>> > Did you guys know? in JIRA, we can only go through page by page up to
>> 1000 items. So, currently we're even
>> > having difficulties to go through every JIRA. We should manually filter
>> out and check each.
>> > The number is going over the manageable size.
>> >
>> > I am not suggesting this without anything actually trying. This is what
>> we have tried within my visibility:
>> >
>> >   1. In roughly 3 years ago, Sean tried to gather committers and even
>> non-committers people to sort
>> > out this number. At that time, we were only able to keep this
>> number as is. After we lost this momentum,
>> > it kept increasing back.
>> >   2. At least I scanned _all_ the previous JIRAs at least more than two
>> times and resolved them. Roughly
>> > once a year. The rest of them are mostly obsolete but not enough
>> information to investigate further.
>> >   3. I strictly stick to "Contributing to JIRA Maintenance"
>> https://spark.apache.org/contributing.html and
>> > resolve JIRAs.
>> >   4. Promoting other people to comment on JIRA or actively resolve them.
>> >
>> > One of the facts I realised is the increasing number of committers
>> doesn't virtually help this much (although
>> > it might be helpful if somebody active in JIRA becomes a committer.)
>> >
>> > One of the important thing I should note is that, it's now almost
>> pretty difficult to reproduce and test the
>> > issues found in EOL releases. We should git clone, checkout, build and
>> test. And then, see if that issue
>> > still exists in upstream, and fix. This is non-trivial overhead.
>> >
>> > Therefore, I would like to propose resolving _all_ the JIRAs that
>> targets EOL releases - 2.2 and below.
>> > Please let me know if anyone has some concerns or objections.
>> >
>> > Thanks.
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: Resolving all JIRAs affecting EOL releases

[Josh Rosen]

+1 in favor of some sort of JIRA cleanup.

My only request is that we attach some sort of 'bulk-closed' label to
issues that we close via JIRA filter batch operations (and resolve the
issues as "Timed Out" / "Cannot Reproduce", not "Fixed"). Using a label
makes it easier to audit what was closed, simplifying the process of
identifying and re-opening valid issues caught in our dragnet.


On Wed, May 15, 2019 at 7:19 AM Sean Owen  wrote:

> I gave up looking through JIRAs a long time ago, so, big respect for
> continuing to try to triage them. I am afraid we're missing a few
> important bug reports in the torrent, but most JIRAs are not
> well-formed, just questions, stale, or simply things that won't be
> added. I do think it's important to reflect that reality, and so I'm
> always in favor of more aggressively closing JIRAs. I think this is
> more standard practice, from projects like TensorFlow/Keras, pandas,
> etc to just automatically drop Issues that don't see activity for N
> days. We won't do that, but, are probably on the other hand far too
> lax in closing them.
>
> Remember that JIRAs stay searchable and can be reopened, so it's not
> like we lose much information.
>
> I'd close anything that hasn't had activity in 2 years (?), as a start.
> I like the idea of closing things that only affect an EOL release,
> but, many items aren't marked, so may need to cast the net wider.
>
> I think only then does it make sense to look at bothering to reproduce
> or evaluate the 1000s that will still remain.
>
> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon  wrote:
> >
> > Hi all,
> >
> > I would like to propose to resolve all JIRAs that affects EOL releases -
> 2.2 and below. and affected version
> > not specified. I was rather against this way and considered this as last
> resort in roughly 3 years ago
> > when we discussed. Now I think we should go ahead with this. See below.
> >
> > I have been talking care of this for so long time almost every day those
> 3 years. The number of JIRAs
> > keeps increasing and it does never go down. Now the number is going over
> 2500 JIRAs.
> > Did you guys know? in JIRA, we can only go through page by page up to
> 1000 items. So, currently we're even
> > having difficulties to go through every JIRA. We should manually filter
> out and check each.
> > The number is going over the manageable size.
> >
> > I am not suggesting this without anything actually trying. This is what
> we have tried within my visibility:
> >
> >   1. In roughly 3 years ago, Sean tried to gather committers and even
> non-committers people to sort
> > out this number. At that time, we were only able to keep this number
> as is. After we lost this momentum,
> > it kept increasing back.
> >   2. At least I scanned _all_ the previous JIRAs at least more than two
> times and resolved them. Roughly
> > once a year. The rest of them are mostly obsolete but not enough
> information to investigate further.
> >   3. I strictly stick to "Contributing to JIRA Maintenance"
> https://spark.apache.org/contributing.html and
> > resolve JIRAs.
> >   4. Promoting other people to comment on JIRA or actively resolve them.
> >
> > One of the facts I realised is the increasing number of committers
> doesn't virtually help this much (although
> > it might be helpful if somebody active in JIRA becomes a committer.)
> >
> > One of the important thing I should note is that, it's now almost pretty
> difficult to reproduce and test the
> > issues found in EOL releases. We should git clone, checkout, build and
> test. And then, see if that issue
> > still exists in upstream, and fix. This is non-trivial overhead.
> >
> > Therefore, I would like to propose resolving _all_ the JIRAs that
> targets EOL releases - 2.2 and below.
> > Please let me know if anyone has some concerns or objections.
> >
> > Thanks.
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: Resolving all JIRAs affecting EOL releases

[Sean Owen]

I gave up looking through JIRAs a long time ago, so, big respect for
continuing to try to triage them. I am afraid we're missing a few
important bug reports in the torrent, but most JIRAs are not
well-formed, just questions, stale, or simply things that won't be
added. I do think it's important to reflect that reality, and so I'm
always in favor of more aggressively closing JIRAs. I think this is
more standard practice, from projects like TensorFlow/Keras, pandas,
etc to just automatically drop Issues that don't see activity for N
days. We won't do that, but, are probably on the other hand far too
lax in closing them.

Remember that JIRAs stay searchable and can be reopened, so it's not
like we lose much information.

I'd close anything that hasn't had activity in 2 years (?), as a start.
I like the idea of closing things that only affect an EOL release,
but, many items aren't marked, so may need to cast the net wider.

I think only then does it make sense to look at bothering to reproduce
or evaluate the 1000s that will still remain.

On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon  wrote:
>
> Hi all,
>
> I would like to propose to resolve all JIRAs that affects EOL releases - 2.2 
> and below. and affected version
> not specified. I was rather against this way and considered this as last 
> resort in roughly 3 years ago
> when we discussed. Now I think we should go ahead with this. See below.
>
> I have been talking care of this for so long time almost every day those 3 
> years. The number of JIRAs
> keeps increasing and it does never go down. Now the number is going over 2500 
> JIRAs.
> Did you guys know? in JIRA, we can only go through page by page up to 1000 
> items. So, currently we're even
> having difficulties to go through every JIRA. We should manually filter out 
> and check each.
> The number is going over the manageable size.
>
> I am not suggesting this without anything actually trying. This is what we 
> have tried within my visibility:
>
>   1. In roughly 3 years ago, Sean tried to gather committers and even 
> non-committers people to sort
> out this number. At that time, we were only able to keep this number as 
> is. After we lost this momentum,
> it kept increasing back.
>   2. At least I scanned _all_ the previous JIRAs at least more than two times 
> and resolved them. Roughly
> once a year. The rest of them are mostly obsolete but not enough 
> information to investigate further.
>   3. I strictly stick to "Contributing to JIRA Maintenance" 
> https://spark.apache.org/contributing.html and
> resolve JIRAs.
>   4. Promoting other people to comment on JIRA or actively resolve them.
>
> One of the facts I realised is the increasing number of committers doesn't 
> virtually help this much (although
> it might be helpful if somebody active in JIRA becomes a committer.)
>
> One of the important thing I should note is that, it's now almost pretty 
> difficult to reproduce and test the
> issues found in EOL releases. We should git clone, checkout, build and test. 
> And then, see if that issue
> still exists in upstream, and fix. This is non-trivial overhead.
>
> Therefore, I would like to propose resolving _all_ the JIRAs that targets EOL 
> releases - 2.2 and below.
> Please let me know if anyone has some concerns or objections.
>
> Thanks.

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

Yea, more sophisticated condition is welcome. My only goal is to make it to
a manageable size.

I would go for the option that reduces more tickets - under 1000 OPEN (and
REOPEN) tickets so that we can at least go through in one go without coming
up with a non duplicating filter to go through.

On Wed, 15 May 2019, 19:33 Abdeali Kothari, 
wrote:

> Was thinking that getting an estimated statistic of the number of issues
> that would be closed if this is done would help.
>
> Open issues: 3882 (project = SPARK AND status in (Open, "In Progress",
> Reopened))
> Open + Does not affect 3.0+ = 2795
> Open + Does not affect 2.4+ = 2373
> Open + Does not affect 2.3+ = 1765
> Open + Does not affect 2.2+ = 1322
> Open + Does not affect 2.1+ = 967
> Open + Does not affect 2.0+ = 651
>
> Open + Does not affect 2.0+ + Priority in (Urgent, Blocker, Critical,
> High) [JQL1] = 838
> Open + Does not affect 2.0+ + Priority in (Urgent, Blocker, Critical,
> High, Major) = 206
> Open + Does not affect 2.2+ + Priority not in (Urgent, Blocker, Critical,
> High) [JQL2] = 1303
> Open + Does not affect 2.2+ + Priority not in (Urgent, Blocker, Critical,
> High, Major) = 397
> Open + Does not affect 2.3+ + Priority not in (Urgent, Blocker, Critical,
> High) = 1743
> Open + Does not affect 2.3+ + Priority not in (Urgent, Blocker, Critical,
> High, Major) = 550
>
> Resolving ALL seems a bit overkill to me.
> My current opinion seems like:
>  - Resolving "Open + Does not affect 2.0+" is something that should be
> done, as I doubt anyone would be looking at the 1.x versions anymore (651
> tasks)
>  - Resolving "Open + Does not affect 2.3+ + Priority not in (Urgent,
> Blocker, Critical, High, Major)" may be a good idea (an additional ~1k
> tasks)
> The issues with priority Urgent/Blocker/Critical should be triaged - as it
> may have something important.
>
>
> [JQL1]:
> project = SPARK
>  AND status in (Open, "In Progress", Reopened)
>  AND NOT (affectedVersion in versionMatch("^[2-3].*"))
>  AND priority NOT IN (Urgent, Blocker, Critical, High)
>
> [JQL2]:
> project = SPARK
>  AND status in (Open, "In Progress", Reopened)
>  AND NOT (affectedVersion in versionMatch("^3.*") OR affectedVersion in
> versionMatch("^2.4.*") OR affectedVersion in versionMatch("^2.3.*") OR
> affectedVersion in versionMatch("^2.2.*"))
>  AND priority NOT IN (Urgent, Blocker, Critical, High)
>
>
> On Wed, May 15, 2019, 14:55 Hyukjin Kwon  wrote:
>
>> Hi all,
>>
>> I would like to propose to resolve all JIRAs that affects EOL releases -
>> 2.2 and below. and affected version
>> not specified. I was rather against this way and considered this as last
>> resort in roughly 3 years ago
>> when we discussed. Now I think we should go ahead with this. See below.
>>
>> I have been talking care of this for so long time almost every day those
>> 3 years. The number of JIRAs
>> keeps increasing and it does never go down. Now the number is going over
>> 2500 JIRAs.
>> Did you guys know? in JIRA, we can only go through page by page up to
>> 1000 items. So, currently we're even
>> having difficulties to go through every JIRA. We should manually filter
>> out and check each.
>> The number is going over the manageable size.
>>
>> I am not suggesting this without anything actually trying. This is what
>> we have tried within my visibility:
>>
>>   1. In roughly 3 years ago, Sean tried to gather committers and even
>> non-committers people to sort
>> out this number. At that time, we were only able to keep this number
>> as is. After we lost this momentum,
>> it kept increasing back.
>>   2. At least I scanned _all_ the previous JIRAs at least more than two
>> times and resolved them. Roughly
>> once a year. The rest of them are mostly obsolete but not enough
>> information to investigate further.
>>   3. I strictly stick to "Contributing to JIRA Maintenance"
>> https://spark.apache.org/contributing.html and
>> resolve JIRAs.
>>   4. Promoting other people to comment on JIRA or actively resolve them.
>>
>> One of the facts I realised is the increasing number of committers
>> doesn't virtually help this much (although
>> it might be helpful if somebody active in JIRA becomes a committer.)
>>
>> One of the important thing I should note is that, it's now almost pretty
>> difficult to reproduce and test the
>> issues found in EOL releases. We should git clone, checkout, build and
>> test. And then, see if that issue
>> still exists in upstream, and fix. This is non-trivial overhead.
>>
>> Therefore, I would like to propose resolving _all_ the JIRAs that targets
>> EOL releases - 2.2 and below.
>> Please let me know if anyone has some concerns or objections.
>>
>> Thanks.
>>
>

Re: Resolving all JIRAs affecting EOL releases

[Abdeali Kothari]

Was thinking that getting an estimated statistic of the number of issues
that would be closed if this is done would help.

Open issues: 3882 (project = SPARK AND status in (Open, "In Progress",
Reopened))
Open + Does not affect 3.0+ = 2795
Open + Does not affect 2.4+ = 2373
Open + Does not affect 2.3+ = 1765
Open + Does not affect 2.2+ = 1322
Open + Does not affect 2.1+ = 967
Open + Does not affect 2.0+ = 651

Open + Does not affect 2.0+ + Priority in (Urgent, Blocker, Critical, High)
[JQL1] = 838
Open + Does not affect 2.0+ + Priority in (Urgent, Blocker, Critical, High,
Major) = 206
Open + Does not affect 2.2+ + Priority not in (Urgent, Blocker, Critical,
High) [JQL2] = 1303
Open + Does not affect 2.2+ + Priority not in (Urgent, Blocker, Critical,
High, Major) = 397
Open + Does not affect 2.3+ + Priority not in (Urgent, Blocker, Critical,
High) = 1743
Open + Does not affect 2.3+ + Priority not in (Urgent, Blocker, Critical,
High, Major) = 550

Resolving ALL seems a bit overkill to me.
My current opinion seems like:
 - Resolving "Open + Does not affect 2.0+" is something that should be
done, as I doubt anyone would be looking at the 1.x versions anymore (651
tasks)
 - Resolving "Open + Does not affect 2.3+ + Priority not in (Urgent,
Blocker, Critical, High, Major)" may be a good idea (an additional ~1k
tasks)
The issues with priority Urgent/Blocker/Critical should be triaged - as it
may have something important.


[JQL1]:
project = SPARK
 AND status in (Open, "In Progress", Reopened)
 AND NOT (affectedVersion in versionMatch("^[2-3].*"))
 AND priority NOT IN (Urgent, Blocker, Critical, High)

[JQL2]:
project = SPARK
 AND status in (Open, "In Progress", Reopened)
 AND NOT (affectedVersion in versionMatch("^3.*") OR affectedVersion in
versionMatch("^2.4.*") OR affectedVersion in versionMatch("^2.3.*") OR
affectedVersion in versionMatch("^2.2.*"))
 AND priority NOT IN (Urgent, Blocker, Critical, High)


On Wed, May 15, 2019, 14:55 Hyukjin Kwon  wrote:

> Hi all,
>
> I would like to propose to resolve all JIRAs that affects EOL releases -
> 2.2 and below. and affected version
> not specified. I was rather against this way and considered this as last
> resort in roughly 3 years ago
> when we discussed. Now I think we should go ahead with this. See below.
>
> I have been talking care of this for so long time almost every day those 3
> years. The number of JIRAs
> keeps increasing and it does never go down. Now the number is going over
> 2500 JIRAs.
> Did you guys know? in JIRA, we can only go through page by page up to 1000
> items. So, currently we're even
> having difficulties to go through every JIRA. We should manually filter
> out and check each.
> The number is going over the manageable size.
>
> I am not suggesting this without anything actually trying. This is what we
> have tried within my visibility:
>
>   1. In roughly 3 years ago, Sean tried to gather committers and even
> non-committers people to sort
> out this number. At that time, we were only able to keep this number
> as is. After we lost this momentum,
> it kept increasing back.
>   2. At least I scanned _all_ the previous JIRAs at least more than two
> times and resolved them. Roughly
> once a year. The rest of them are mostly obsolete but not enough
> information to investigate further.
>   3. I strictly stick to "Contributing to JIRA Maintenance"
> https://spark.apache.org/contributing.html and
> resolve JIRAs.
>   4. Promoting other people to comment on JIRA or actively resolve them.
>
> One of the facts I realised is the increasing number of committers doesn't
> virtually help this much (although
> it might be helpful if somebody active in JIRA becomes a committer.)
>
> One of the important thing I should note is that, it's now almost pretty
> difficult to reproduce and test the
> issues found in EOL releases. We should git clone, checkout, build and
> test. And then, see if that issue
> still exists in upstream, and fix. This is non-trivial overhead.
>
> Therefore, I would like to propose resolving _all_ the JIRAs that targets
> EOL releases - 2.2 and below.
> Please let me know if anyone has some concerns or objections.
>
> Thanks.
>

Resolving all JIRAs affecting EOL releases

[Hyukjin Kwon]

Hi all,

I would like to propose to resolve all JIRAs that affects EOL releases -
2.2 and below. and affected version
not specified. I was rather against this way and considered this as last
resort in roughly 3 years ago
when we discussed. Now I think we should go ahead with this. See below.

I have been talking care of this for so long time almost every day those 3
years. The number of JIRAs
keeps increasing and it does never go down. Now the number is going over
2500 JIRAs.
Did you guys know? in JIRA, we can only go through page by page up to 1000
items. So, currently we're even
having difficulties to go through every JIRA. We should manually filter out
and check each.
The number is going over the manageable size.

I am not suggesting this without anything actually trying. This is what we
have tried within my visibility:

  1. In roughly 3 years ago, Sean tried to gather committers and even
non-committers people to sort
out this number. At that time, we were only able to keep this number as
is. After we lost this momentum,
it kept increasing back.
  2. At least I scanned _all_ the previous JIRAs at least more than two
times and resolved them. Roughly
once a year. The rest of them are mostly obsolete but not enough
information to investigate further.
  3. I strictly stick to "Contributing to JIRA Maintenance"
https://spark.apache.org/contributing.html and
resolve JIRAs.
  4. Promoting other people to comment on JIRA or actively resolve them.

One of the facts I realised is the increasing number of committers doesn't
virtually help this much (although
it might be helpful if somebody active in JIRA becomes a committer.)

One of the important thing I should note is that, it's now almost pretty
difficult to reproduce and test the
issues found in EOL releases. We should git clone, checkout, build and
test. And then, see if that issue
still exists in upstream, and fix. This is non-trivial overhead.

Therefore, I would like to propose resolving _all_ the JIRAs that targets
EOL releases - 2.2 and below.
Please let me know if anyone has some concerns or objections.

Thanks.