Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[L. C. Hsieh]

Based on SPIP doc
(https://spark.apache.org/improvement-proposals.html), the vote passes
if at least 3 +1 votes from PMC members and no -1 votes from PMC
members.

Also, the vote should be open for at least 72 hours.

On Thu, Feb 16, 2023 at 10:34 AM Mich Talebzadeh
 wrote:
>
> How many votes are needed for the approval state?
>
> Thanks
>
>
>
>view my Linkedin profile
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, 
> damage or destruction of data or any other property which may arise from 
> relying on this email's technical content is explicitly disclaimed. The 
> author will in no case be liable for any monetary damages arising from such 
> loss, damage or destruction.
>
>
>
>
>
> On Thu, 16 Feb 2023 at 18:19, kazuyuki tanimura  wrote:
>>
>> +1 for myself
>>
>> On Feb 14, 2023, at 10:42 AM, DB Tsai  wrote:
>>
>> +1
>>
>> DB Tsai  |  https://www.dbtsai.com/  |  PGP 42E5B25A8F7A82C1
>>
>> On Feb 14, 2023, at 8:29 AM, Guo Weijie  wrote:
>>
>> +1
>>
>> Yuming Wang  于2023年2月14日周二 15:58写道：
>>>
>>> +1
>>>
>>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo  wrote:

 +1

 On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh  wrote:
>
> +1
>
> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh 
>  wrote:
>>
>> +1 for me
>>
>>
>>view my Linkedin profile
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> Disclaimer: Use it at your own risk. Any and all responsibility for any 
>> loss, damage or destruction of data or any other property which may 
>> arise from relying on this email's technical content is explicitly 
>> disclaimed. The author will in no case be liable for any monetary 
>> damages arising from such loss, damage or destruction.
>>
>>
>>
>>
>>
>> On Mon, 13 Feb 2023 at 23:18, huaxin gao  wrote:
>>>
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun  
>>> wrote:

 +1

 Dongjoon

 On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
 > Hi all,
 >
 > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
 > Read Performance Improvement.
 >
 > The high summary of the SPIP is that it proposes an improvement to 
 > the
 > Parquet reader with lazy materialization which only materializes 
 > (i.e.
 > decompress, de-code, etc...) necessary values. For Spark-SQL filter
 > operations, evaluating the filters first and lazily materializing 
 > only
 > the used values can save computation wastes and improve the read
 > performance.
 >
 > References:
 >
 > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
 > SPIP doc 
 > https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
 > Discussion thread
 > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
 >
 > Please vote on the SPIP for the next 72 hours:
 >
 > [ ] +1: Accept the proposal as an official SPIP
 > [ ] +0
 > [ ] -1: I don’t think this is a good idea because …
 >
 > Thank you!
 >
 > Liang-Chi Hsieh
 >
 > -
 > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
 >
 >

 -
 To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

>>
>>

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[Mich Talebzadeh]

How many votes are needed for the approval state?

Thanks



   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 16 Feb 2023 at 18:19, kazuyuki tanimura  wrote:

> +1 for myself
>
> On Feb 14, 2023, at 10:42 AM, DB Tsai  wrote:
>
> +1
>
> DB Tsai  |  https://www.dbtsai.com/  |  PGP 42E5B25A8F7A82C1
>
> On Feb 14, 2023, at 8:29 AM, Guo Weijie  wrote:
>
> +1
>
> Yuming Wang  于2023年2月14日周二 15:58写道：
>
>> +1
>>
>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo  wrote:
>>
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh  wrote:
>>>
 +1

 On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <
 mich.talebza...@gmail.com> wrote:

> +1 for me
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any loss, damage or destruction of data or any other property which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary damages
> arising from such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Feb 2023 at 23:18, huaxin gao 
> wrote:
>
>> +1
>>
>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun 
>> wrote:
>>
>>> +1
>>>
>>> Dongjoon
>>>
>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>> > Hi all,
>>> >
>>> > I'd like to start the vote for SPIP: Lazy Materialization for
>>> Parquet
>>> > Read Performance Improvement.
>>> >
>>> > The high summary of the SPIP is that it proposes an improvement to
>>> the
>>> > Parquet reader with lazy materialization which only materializes
>>> (i.e.
>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>> > operations, evaluating the filters first and lazily materializing
>>> only
>>> > the used values can save computation wastes and improve the read
>>> > performance.
>>> >
>>> > References:
>>> >
>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>> > SPIP doc
>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>> > Discussion thread
>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>> >
>>> > Please vote on the SPIP for the next 72 hours:
>>> >
>>> > [ ] +1: Accept the proposal as an official SPIP
>>> > [ ] +0
>>> > [ ] -1: I don’t think this is a good idea because …
>>> >
>>> > Thank you!
>>> >
>>> > Liang-Chi Hsieh
>>> >
>>> >
>>> -
>>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>> >
>>> >
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>
>
>

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[kazuyuki tanimura]

+1 for myself

> On Feb 14, 2023, at 10:42 AM, DB Tsai  wrote:
> 
> +1
> 
> DB Tsai  |  https://www.dbtsai.com/  |  PGP 42E5B25A8F7A82C1
> 
>> On Feb 14, 2023, at 8:29 AM, Guo Weijie  wrote:
>> 
>> +1 
>> 
>> Yuming Wang mailto:wgy...@gmail.com>> 于2023年2月14日周二 
>> 15:58写道：
>> +1
>> 
>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo > > wrote:
>> +1
>> 
>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh > > wrote:
>> +1
>> 
>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh > > wrote:
>> +1 for me
>> 
>> 
>>view my Linkedin profile 
>> 
>> 
>>  https://en.everybodywiki.com/Mich_Talebzadeh 
>> 
>>  
>> Disclaimer: Use it at your own risk. Any and all responsibility for any 
>> loss, damage or destruction of data or any other property which may arise 
>> from relying on this email's technical content is explicitly disclaimed. The 
>> author will in no case be liable for any monetary damages arising from such 
>> loss, damage or destruction.
>>  
>> 
>> 
>> On Mon, 13 Feb 2023 at 23:18, huaxin gao > > wrote:
>> +1
>> 
>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun > > wrote:
>> +1
>> 
>> Dongjoon
>> 
>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>> > Hi all,
>> > 
>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>> > Read Performance Improvement.
>> > 
>> > The high summary of the SPIP is that it proposes an improvement to the
>> > Parquet reader with lazy materialization which only materializes (i.e.
>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>> > operations, evaluating the filters first and lazily materializing only
>> > the used values can save computation wastes and improve the read
>> > performance.
>> > 
>> > References:
>> > 
>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256 
>> > 
>> > SPIP doc 
>> > https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>> >  
>> > 
>> > Discussion thread
>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6 
>> > 
>> > 
>> > Please vote on the SPIP for the next 72 hours:
>> > 
>> > [ ] +1: Accept the proposal as an official SPIP
>> > [ ] +0
>> > [ ] -1: I don’t think this is a good idea because …
>> > 
>> > Thank you!
>> > 
>> > Liang-Chi Hsieh
>> > 
>> > -
>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org 
>> > 
>> > 
>> > 
>> 
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org 
>> 
>> 
> 

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[DB Tsai]

+1

DB Tsai  |  https://www.dbtsai.com/  |  PGP 42E5B25A8F7A82C1

> On Feb 14, 2023, at 8:29 AM, Guo Weijie  wrote:
> 
> +1 
> 
> Yuming Wang mailto:wgy...@gmail.com>> 于2023年2月14日周二 
> 15:58写道：
>> +1
>> 
>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo > > wrote:
>>> +1
>>> 
>>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh >> > wrote:
 +1
 
 On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh >>> > wrote:
> +1 for me
> 
> 
>view my Linkedin profile 
> 
> 
>  https://en.everybodywiki.com/Mich_Talebzadeh
> 
>  
> Disclaimer: Use it at your own risk. Any and all responsibility for any 
> loss, damage or destruction of data or any other property which may arise 
> from relying on this email's technical content is explicitly disclaimed. 
> The author will in no case be liable for any monetary damages arising 
> from such loss, damage or destruction.
>  
> 
> 
> On Mon, 13 Feb 2023 at 23:18, huaxin gao  > wrote:
>> +1
>> 
>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun > > wrote:
>>> +1
>>> 
>>> Dongjoon
>>> 
>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>> > Hi all,
>>> > 
>>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>>> > Read Performance Improvement.
>>> > 
>>> > The high summary of the SPIP is that it proposes an improvement to the
>>> > Parquet reader with lazy materialization which only materializes (i.e.
>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>> > operations, evaluating the filters first and lazily materializing only
>>> > the used values can save computation wastes and improve the read
>>> > performance.
>>> > 
>>> > References:
>>> > 
>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>> > SPIP doc 
>>> > https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>> > Discussion thread
>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>> > 
>>> > Please vote on the SPIP for the next 72 hours:
>>> > 
>>> > [ ] +1: Accept the proposal as an official SPIP
>>> > [ ] +0
>>> > [ ] -1: I don’t think this is a good idea because …
>>> > 
>>> > Thank you!
>>> > 
>>> > Liang-Chi Hsieh
>>> > 
>>> > -
>>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org 
>>> > 
>>> > 
>>> > 
>>> 
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org 
>>> 
>>> 

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[Guo Weijie]

+1

Yuming Wang  于2023年2月14日周二 15:58写道：

> +1
>
> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo  wrote:
>
>> +1
>>
>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh  wrote:
>>
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>
 +1 for me



view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Mon, 13 Feb 2023 at 23:18, huaxin gao 
 wrote:

> +1
>
> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun 
> wrote:
>
>> +1
>>
>> Dongjoon
>>
>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>> > Hi all,
>> >
>> > I'd like to start the vote for SPIP: Lazy Materialization for
>> Parquet
>> > Read Performance Improvement.
>> >
>> > The high summary of the SPIP is that it proposes an improvement to
>> the
>> > Parquet reader with lazy materialization which only materializes
>> (i.e.
>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>> > operations, evaluating the filters first and lazily materializing
>> only
>> > the used values can save computation wastes and improve the read
>> > performance.
>> >
>> > References:
>> >
>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>> > SPIP doc
>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>> > Discussion thread
>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>> >
>> > Please vote on the SPIP for the next 72 hours:
>> >
>> > [ ] +1: Accept the proposal as an official SPIP
>> > [ ] +0
>> > [ ] -1: I don’t think this is a good idea because …
>> >
>> > Thank you!
>> >
>> > Liang-Chi Hsieh
>> >
>> >
>> -
>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>> >
>> >
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[Yuming Wang]

+1

On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo  wrote:

> +1
>
> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh  wrote:
>
>> +1
>>
>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>> +1 for me
>>>
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Feb 2023 at 23:18, huaxin gao  wrote:
>>>
 +1

 On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun 
 wrote:

> +1
>
> Dongjoon
>
> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
> > Hi all,
> >
> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
> > Read Performance Improvement.
> >
> > The high summary of the SPIP is that it proposes an improvement to
> the
> > Parquet reader with lazy materialization which only materializes
> (i.e.
> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
> > operations, evaluating the filters first and lazily materializing
> only
> > the used values can save computation wastes and improve the read
> > performance.
> >
> > References:
> >
> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
> > SPIP doc
> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
> > Discussion thread
> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
> >
> > Please vote on the SPIP for the next 72 hours:
> >
> > [ ] +1: Accept the proposal as an official SPIP
> > [ ] +0
> > [ ] -1: I don’t think this is a good idea because …
> >
> > Thank you!
> >
> > Liang-Chi Hsieh
> >
> > -
> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> >
> >
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[Prem Sahoo]

+1

On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh  wrote:

> +1
>
> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh 
> wrote:
>
>> +1 for me
>>
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Feb 2023 at 23:18, huaxin gao  wrote:
>>
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun 
>>> wrote:
>>>
 +1

 Dongjoon

 On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
 > Hi all,
 >
 > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
 > Read Performance Improvement.
 >
 > The high summary of the SPIP is that it proposes an improvement to the
 > Parquet reader with lazy materialization which only materializes (i.e.
 > decompress, de-code, etc...) necessary values. For Spark-SQL filter
 > operations, evaluating the filters first and lazily materializing only
 > the used values can save computation wastes and improve the read
 > performance.
 >
 > References:
 >
 > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
 > SPIP doc
 https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
 > Discussion thread
 > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
 >
 > Please vote on the SPIP for the next 72 hours:
 >
 > [ ] +1: Accept the proposal as an official SPIP
 > [ ] +0
 > [ ] -1: I don’t think this is a good idea because …
 >
 > Thank you!
 >
 > Liang-Chi Hsieh
 >
 > -
 > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
 >
 >

 -
 To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[L. C. Hsieh]

+1

On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh 
wrote:

> +1 for me
>
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Feb 2023 at 23:18, huaxin gao  wrote:
>
>> +1
>>
>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun 
>> wrote:
>>
>>> +1
>>>
>>> Dongjoon
>>>
>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>> > Hi all,
>>> >
>>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>>> > Read Performance Improvement.
>>> >
>>> > The high summary of the SPIP is that it proposes an improvement to the
>>> > Parquet reader with lazy materialization which only materializes (i.e.
>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>> > operations, evaluating the filters first and lazily materializing only
>>> > the used values can save computation wastes and improve the read
>>> > performance.
>>> >
>>> > References:
>>> >
>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>> > SPIP doc
>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>> > Discussion thread
>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>> >
>>> > Please vote on the SPIP for the next 72 hours:
>>> >
>>> > [ ] +1: Accept the proposal as an official SPIP
>>> > [ ] +0
>>> > [ ] -1: I don’t think this is a good idea because …
>>> >
>>> > Thank you!
>>> >
>>> > Liang-Chi Hsieh
>>> >
>>> > -
>>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>> >
>>> >
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[Mich Talebzadeh]

+1 for me



   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 13 Feb 2023 at 23:18, huaxin gao  wrote:

> +1
>
> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun  wrote:
>
>> +1
>>
>> Dongjoon
>>
>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>> > Hi all,
>> >
>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>> > Read Performance Improvement.
>> >
>> > The high summary of the SPIP is that it proposes an improvement to the
>> > Parquet reader with lazy materialization which only materializes (i.e.
>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>> > operations, evaluating the filters first and lazily materializing only
>> > the used values can save computation wastes and improve the read
>> > performance.
>> >
>> > References:
>> >
>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>> > SPIP doc
>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>> > Discussion thread
>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>> >
>> > Please vote on the SPIP for the next 72 hours:
>> >
>> > [ ] +1: Accept the proposal as an official SPIP
>> > [ ] +0
>> > [ ] -1: I don’t think this is a good idea because …
>> >
>> > Thank you!
>> >
>> > Liang-Chi Hsieh
>> >
>> > -
>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>> >
>> >
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[huaxin gao]

+1

On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun  wrote:

> +1
>
> Dongjoon
>
> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
> > Hi all,
> >
> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
> > Read Performance Improvement.
> >
> > The high summary of the SPIP is that it proposes an improvement to the
> > Parquet reader with lazy materialization which only materializes (i.e.
> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
> > operations, evaluating the filters first and lazily materializing only
> > the used values can save computation wastes and improve the read
> > performance.
> >
> > References:
> >
> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
> > SPIP doc
> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
> > Discussion thread
> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
> >
> > Please vote on the SPIP for the next 72 hours:
> >
> > [ ] +1: Accept the proposal as an official SPIP
> > [ ] +0
> > [ ] -1: I don’t think this is a good idea because …
> >
> > Thank you!
> >
> > Liang-Chi Hsieh
> >
> > -
> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> >
> >
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

[Dongjoon Hyun]

+1

Dongjoon

On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
> Hi all,
> 
> I'd like to start the vote for SPIP: Lazy Materialization for Parquet
> Read Performance Improvement.
> 
> The high summary of the SPIP is that it proposes an improvement to the
> Parquet reader with lazy materialization which only materializes (i.e.
> decompress, de-code, etc...) necessary values. For Spark-SQL filter
> operations, evaluating the filters first and lazily materializing only
> the used values can save computation wastes and improve the read
> performance.
> 
> References:
> 
> JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
> SPIP doc 
> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
> Discussion thread
> https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
> 
> Please vote on the SPIP for the next 72 hours:
> 
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
> 
> Thank you!
> 
> Liang-Chi Hsieh
> 
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> 
> 

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org