+1
view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Wed, 1 Feb 2023 at 02:23, huaxin gao <huaxin.ga...@gmail.com> wrote: > +1 > > On Tue, Jan 31, 2023 at 6:10 PM DB Tsai <dbt...@dbtsai.com> wrote: > >> +1 >> >> Sent from my iPhone >> >> On Jan 31, 2023, at 4:16 PM, Yuming Wang <wgy...@gmail.com> wrote: >> >> >> +1. >> >> On Wed, Feb 1, 2023 at 7:42 AM kazuyuki tanimura >> <ktanim...@apple.com.invalid> wrote: >> >>> Great! Much appreciated, Mitch! >>> >>> Kazu >>> >>> On Jan 31, 2023, at 3:07 PM, Mich Talebzadeh <mich.talebza...@gmail.com> >>> wrote: >>> >>> Thanks, Kazu. >>> >>> I followed that template link and indeed as you pointed out it is a >>> common template. If it works then it is what it is. >>> >>> I will be going through your design proposals and hopefully we can >>> review it. >>> >>> Regards, >>> >>> Mich >>> >>> >>> view my Linkedin profile >>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>> >>> >>> https://en.everybodywiki.com/Mich_Talebzadeh >>> >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> >>> >>> >>> >>> On Tue, 31 Jan 2023 at 22:34, kazuyuki tanimura <ktanim...@apple.com> >>> wrote: >>> >>>> Thank you Mich. I followed the instruction at >>>> https://spark.apache.org/improvement-proposals.html and used its >>>> template. >>>> While we are open to revise our design doc, it seems more like you are >>>> proposing the community to change the instruction per se? >>>> >>>> Kazu >>>> >>>> On Jan 31, 2023, at 11:24 AM, Mich Talebzadeh < >>>> mich.talebza...@gmail.com> wrote: >>>> >>>> Hi, >>>> >>>> Thanks for these proposals. good suggestions. Is this style of breaking >>>> down your approach standard? >>>> >>>> My view would be that perhaps it makes more sense to follow the >>>> industry established approach of breaking down >>>> your technical proposal into: >>>> >>>> >>>> 1. Background >>>> 2. Objective >>>> 3. Scope >>>> 4. Constraints >>>> 5. Assumptions >>>> 6. Reporting >>>> 7. Deliverables >>>> 8. Timelines >>>> 9. Appendix >>>> >>>> Your current approach using below >>>> >>>> Q1. What are you trying to do? Articulate your objectives using >>>> absolutely no jargon. What are you trying to achieve? >>>> Q2. What problem is this proposal NOT designed to solve? What issues >>>> the suggested proposal is not going to address >>>> Q3. How is it done today, and what are the limits of current practice? >>>> Q4. What is new in your approach approach and why do you think it will be >>>> successful succeed? >>>> Q5. Who cares? If you are successful, what difference will it make? If >>>> your proposal succeeds, what tangible benefits will it add? >>>> Q6. What are the risks? >>>> Q7. How long will it take? >>>> Q8. What are the midterm and final “exams” to check for success? >>>> >>>> >>>> May not do justice to your proposal. >>>> >>>> HTH >>>> >>>> Mich >>>> >>>> view my Linkedin profile >>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>> >>>> >>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>> >>>> >>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>> any loss, damage or destruction of data or any other property which may >>>> arise from relying on this email's technical content is explicitly >>>> disclaimed. The author will in no case be liable for any monetary damages >>>> arising from such loss, damage or destruction. >>>> >>>> >>>> >>>> >>>> On Tue, 31 Jan 2023 at 17:35, kazuyuki tanimura < >>>> ktanim...@apple.com.invalid> wrote: >>>> >>>>> Hi everyone, >>>>> >>>>> I would like to start a discussion on “Lazy Materialization for >>>>> Parquet Read Performance Improvement" >>>>> >>>>> Chao and I propose a Parquet reader with lazy materialization. For >>>>> Spark-SQL filter operations, evaluating the filters first and lazily >>>>> materializing only the used values can save computation wastes and improve >>>>> the read performance. >>>>> The current implementation of Spark requires the read values to >>>>> materialize (i.e. decompress, de-code, etc...) onto memory first before >>>>> applying the filters even though the filters may eventually throw away >>>>> many >>>>> values. >>>>> >>>>> We made our design doc as follows. >>>>> SPIP Jira: https://issues.apache.org/jira/browse/SPARK-42256 >>>>> SPIP Doc: >>>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME >>>>> >>>>> Liang-Chi was kind enough to shepherd this effort. >>>>> >>>>> Thank you >>>>> Kazu >>>>> >>>> >>>> >>>
