Re: Resolves too old JIRAs as incomplete
+1.
Thank you, Takeshi.
On Wed, May 19, 2021 at 7:49 PM Hyukjin Kwon wrote:
> Yeah, I wanted to discuss this. I agree since 2.4.x became EOL
>
> 2021년 5월 20일 (목) 오전 10:54, Sean Owen 님이 작성:
>
>> I agree. Such old JIRAs are 99% obsolete. If anyone objects to a
>> particular issue being closed, they can comment and we can reopen. It's a
>> very reversible thing. There is value in keeping JIRA up to date with
>> reality.
>>
>> On Wed, May 19, 2021 at 8:47 PM Takeshi Yamamuro
>> wrote:
>>
>>> Hi, dev,
>>>
>>> As you know, we have too many open JIRAs now:
>>> # of open JIRAs=2698: JQL='project = SPARK AND status in (Open, "In
>>> Progress", Reopened)'
>>>
>>> We've recently released v2.4.8(EOL), so I'd like to bulk-close too old
>>> JIRAs
>>> for making the JIRAs manageable.
>>>
>>> As Hyukjin did the same action two years ago (for details, see:
>>> http://apache-spark-developers-list.1001551.n3.nabble.com/Resolving-all-JIRAs-affecting-EOL-releases-td27838.html),
>>> I'm planning to use a similar JQL below to close them:
>>>
>>> project = SPARK AND status in (Open, "In Progress", Reopened) AND
>>> (affectedVersion = EMPTY OR NOT (affectedVersion in versionMatch("^3.*")))
>>> AND updated <= -52w
>>>
>>> The total number of matched JIRAs is 741.
>>> Or, we might be able to close them more aggressively by removing the
>>> version condition:
>>>
>>> project = SPARK AND status in (Open, "In Progress", Reopened) AND
>>> updated <= -52w
>>>
>>> The matched number is 1484 (almost half of the current open JIRAs).
>>>
>>> If there is no objection, I'd like to do it next week or later.
>>> Any thoughts?
>>>
>>> Bests,
>>> Takeshi
>>> --
>>> ---
>>> Takeshi Yamamuro
>>>
>>
Re: Resolves too old JIRAs as incomplete
Yeah, I wanted to discuss this. I agree since 2.4.x became EOL
2021년 5월 20일 (목) 오전 10:54, Sean Owen 님이 작성:
> I agree. Such old JIRAs are 99% obsolete. If anyone objects to a
> particular issue being closed, they can comment and we can reopen. It's a
> very reversible thing. There is value in keeping JIRA up to date with
> reality.
>
> On Wed, May 19, 2021 at 8:47 PM Takeshi Yamamuro
> wrote:
>
>> Hi, dev,
>>
>> As you know, we have too many open JIRAs now:
>> # of open JIRAs=2698: JQL='project = SPARK AND status in (Open, "In
>> Progress", Reopened)'
>>
>> We've recently released v2.4.8(EOL), so I'd like to bulk-close too old
>> JIRAs
>> for making the JIRAs manageable.
>>
>> As Hyukjin did the same action two years ago (for details, see:
>> http://apache-spark-developers-list.1001551.n3.nabble.com/Resolving-all-JIRAs-affecting-EOL-releases-td27838.html),
>> I'm planning to use a similar JQL below to close them:
>>
>> project = SPARK AND status in (Open, "In Progress", Reopened) AND
>> (affectedVersion = EMPTY OR NOT (affectedVersion in versionMatch("^3.*")))
>> AND updated <= -52w
>>
>> The total number of matched JIRAs is 741.
>> Or, we might be able to close them more aggressively by removing the
>> version condition:
>>
>> project = SPARK AND status in (Open, "In Progress", Reopened) AND updated
>> <= -52w
>>
>> The matched number is 1484 (almost half of the current open JIRAs).
>>
>> If there is no objection, I'd like to do it next week or later.
>> Any thoughts?
>>
>> Bests,
>> Takeshi
>> --
>> ---
>> Takeshi Yamamuro
>>
>
Re: Resolves too old JIRAs as incomplete
I agree. Such old JIRAs are 99% obsolete. If anyone objects to a particular
issue being closed, they can comment and we can reopen. It's a very
reversible thing. There is value in keeping JIRA up to date with reality.
On Wed, May 19, 2021 at 8:47 PM Takeshi Yamamuro
wrote:
> Hi, dev,
>
> As you know, we have too many open JIRAs now:
> # of open JIRAs=2698: JQL='project = SPARK AND status in (Open, "In
> Progress", Reopened)'
>
> We've recently released v2.4.8(EOL), so I'd like to bulk-close too old
> JIRAs
> for making the JIRAs manageable.
>
> As Hyukjin did the same action two years ago (for details, see:
> http://apache-spark-developers-list.1001551.n3.nabble.com/Resolving-all-JIRAs-affecting-EOL-releases-td27838.html),
> I'm planning to use a similar JQL below to close them:
>
> project = SPARK AND status in (Open, "In Progress", Reopened) AND
> (affectedVersion = EMPTY OR NOT (affectedVersion in versionMatch("^3.*")))
> AND updated <= -52w
>
> The total number of matched JIRAs is 741.
> Or, we might be able to close them more aggressively by removing the
> version condition:
>
> project = SPARK AND status in (Open, "In Progress", Reopened) AND updated
> <= -52w
>
> The matched number is 1484 (almost half of the current open JIRAs).
>
> If there is no objection, I'd like to do it next week or later.
> Any thoughts?
>
> Bests,
> Takeshi
> --
> ---
> Takeshi Yamamuro
>
Resolves too old JIRAs as incomplete
Hi, dev,
As you know, we have too many open JIRAs now:
# of open JIRAs=2698: JQL='project = SPARK AND status in (Open, "In
Progress", Reopened)'
We've recently released v2.4.8(EOL), so I'd like to bulk-close too old JIRAs
for making the JIRAs manageable.
As Hyukjin did the same action two years ago (for details, see:
http://apache-spark-developers-list.1001551.n3.nabble.com/Resolving-all-JIRAs-affecting-EOL-releases-td27838.html),
I'm planning to use a similar JQL below to close them:
project = SPARK AND status in (Open, "In Progress", Reopened) AND
(affectedVersion = EMPTY OR NOT (affectedVersion in versionMatch("^3.*")))
AND updated <= -52w
The total number of matched JIRAs is 741.
Or, we might be able to close them more aggressively by removing the
version condition:
project = SPARK AND status in (Open, "In Progress", Reopened) AND updated
<= -52w
The matched number is 1484 (almost half of the current open JIRAs).
If there is no objection, I'd like to do it next week or later.
Any thoughts?
Bests,
Takeshi
--
---
Takeshi Yamamuro
Re: WholeStageCodeGen + DSv2
As it turns out, I also commented on the same Jira (and forgot about it until just now). On Wed, May 19, 2021 at 8:32 AM Shubham Chaurasia wrote: > > Hi, > > I remember creating one for a similar scenario in the past - > https://issues.apache.org/jira/browse/SPARK-29372. > > Thanks, > Shubham > > On Wed, May 19, 2021 at 5:34 PM Takeshi Yamamuro > wrote: >> >> hi, Andrew, >> >> Welcome any improvement proposal for that. >> Could you file an issue in jira first to show us your idea and an example >> query >> to reproduce the issue you described? >> >> Bests, >> Takeshi >> >> On Wed, May 19, 2021 at 11:38 AM Andrew Melo wrote: >>> >>> Hello, >>> >>> When reading a very wide (> 1000 cols) input, WholeStageCodeGen blows >>> past the 64kB source limit and fails. Looking at the generated code, a >>> big part of the code is simply the DSv2 convention that the codegen'd >>> variable names are the same as the columns instead of something more >>> compact like 'c1', 'c2', etc.. >>> >>> Would there be any interest in accepting a patch that shortens these >>> variable names to try and stay under the limit? >>> >>> Thanks >>> Andrew >>> >>> - >>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>> >> >> >> -- >> --- >> Takeshi Yamamuro - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Re: WholeStageCodeGen + DSv2
Hi, I remember creating one for a similar scenario in the past - https://issues.apache.org/jira/browse/SPARK-29372. Thanks, Shubham On Wed, May 19, 2021 at 5:34 PM Takeshi Yamamuro wrote: > hi, Andrew, > > Welcome any improvement proposal for that. > Could you file an issue in jira first to show us your idea and an example > query > to reproduce the issue you described? > > Bests, > Takeshi > > On Wed, May 19, 2021 at 11:38 AM Andrew Melo > wrote: > >> Hello, >> >> When reading a very wide (> 1000 cols) input, WholeStageCodeGen blows >> past the 64kB source limit and fails. Looking at the generated code, a >> big part of the code is simply the DSv2 convention that the codegen'd >> variable names are the same as the columns instead of something more >> compact like 'c1', 'c2', etc.. >> >> Would there be any interest in accepting a patch that shortens these >> variable names to try and stay under the limit? >> >> Thanks >> Andrew >> >> - >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >> > > -- > --- > Takeshi Yamamuro >
Re: WholeStageCodeGen + DSv2
hi, Andrew, Welcome any improvement proposal for that. Could you file an issue in jira first to show us your idea and an example query to reproduce the issue you described? Bests, Takeshi On Wed, May 19, 2021 at 11:38 AM Andrew Melo wrote: > Hello, > > When reading a very wide (> 1000 cols) input, WholeStageCodeGen blows > past the 64kB source limit and fails. Looking at the generated code, a > big part of the code is simply the DSv2 convention that the codegen'd > variable names are the same as the columns instead of something more > compact like 'c1', 'c2', etc.. > > Would there be any interest in accepting a patch that shortens these > variable names to try and stay under the limit? > > Thanks > Andrew > > - > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > -- --- Takeshi Yamamuro
