Hi, I thought we already voted to go ahead with this proposal!
view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Mon, 13 Feb 2023 at 20:41, kazuyuki tanimura <ktanim...@apple.com> wrote: > Thank you Liang-Chi! > > Kazu > > On Feb 11, 2023, at 7:12 PM, L. C. Hsieh <vii...@gmail.com> wrote: > > Thanks all for your feedback. > > Given this positive feedback, if there is no other comments/discussion, I > will go to start a vote in the next few days. > > Thank you again! > > On Thu, Feb 2, 2023 at 10:12 AM kazuyuki tanimura < > ktanim...@apple.com.invalid> wrote: > >> Thank you all for +1s and reviewing the SPIP doc. >> >> Kazu >> >> On Feb 1, 2023, at 1:28 AM, Dongjoon Hyun <dongjoon.h...@gmail.com> >> wrote: >> >> +1 >> >> On Wed, Feb 1, 2023 at 12:52 AM Mich Talebzadeh < >> mich.talebza...@gmail.com> wrote: >> >>> +1 >>> >>> >>> view my Linkedin profile >>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>> >>> >>> https://en.everybodywiki.com/Mich_Talebzadeh >>> >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> >>> >>> >>> >>> On Wed, 1 Feb 2023 at 02:23, huaxin gao <huaxin.ga...@gmail.com> wrote: >>> >>>> +1 >>>> >>>> On Tue, Jan 31, 2023 at 6:10 PM DB Tsai <dbt...@dbtsai.com> wrote: >>>> >>>>> +1 >>>>> >>>>> Sent from my iPhone >>>>> >>>>> On Jan 31, 2023, at 4:16 PM, Yuming Wang <wgy...@gmail.com> wrote: >>>>> >>>>> >>>>> +1. >>>>> >>>>> On Wed, Feb 1, 2023 at 7:42 AM kazuyuki tanimura < >>>>> ktanim...@apple.com.invalid> wrote: >>>>> >>>>>> Great! Much appreciated, Mitch! >>>>>> >>>>>> Kazu >>>>>> >>>>>> On Jan 31, 2023, at 3:07 PM, Mich Talebzadeh < >>>>>> mich.talebza...@gmail.com> wrote: >>>>>> >>>>>> Thanks, Kazu. >>>>>> >>>>>> I followed that template link and indeed as you pointed out it is a >>>>>> common template. If it works then it is what it is. >>>>>> >>>>>> I will be going through your design proposals and hopefully we can >>>>>> review it. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Mich >>>>>> >>>>>> >>>>>> view my Linkedin profile >>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>>>> >>>>>> >>>>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>>>> >>>>>> >>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility >>>>>> for any loss, damage or destruction of data or any other property which >>>>>> may >>>>>> arise from relying on this email's technical content is explicitly >>>>>> disclaimed. The author will in no case be liable for any monetary damages >>>>>> arising from such loss, damage or destruction. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, 31 Jan 2023 at 22:34, kazuyuki tanimura <ktanim...@apple.com> >>>>>> wrote: >>>>>> >>>>>>> Thank you Mich. I followed the instruction at >>>>>>> https://spark.apache.org/improvement-proposals.html and used its >>>>>>> template. >>>>>>> While we are open to revise our design doc, it seems more like you >>>>>>> are proposing the community to change the instruction per se? >>>>>>> >>>>>>> Kazu >>>>>>> >>>>>>> On Jan 31, 2023, at 11:24 AM, Mich Talebzadeh < >>>>>>> mich.talebza...@gmail.com> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Thanks for these proposals. good suggestions. Is this style of >>>>>>> breaking down your approach standard? >>>>>>> >>>>>>> My view would be that perhaps it makes more sense to follow the >>>>>>> industry established approach of breaking down >>>>>>> your technical proposal into: >>>>>>> >>>>>>> >>>>>>> 1. Background >>>>>>> 2. Objective >>>>>>> 3. Scope >>>>>>> 4. Constraints >>>>>>> 5. Assumptions >>>>>>> 6. Reporting >>>>>>> 7. Deliverables >>>>>>> 8. Timelines >>>>>>> 9. Appendix >>>>>>> >>>>>>> Your current approach using below >>>>>>> >>>>>>> Q1. What are you trying to do? Articulate your objectives using >>>>>>> absolutely no jargon. What are you trying to achieve? >>>>>>> Q2. What problem is this proposal NOT designed to solve? What >>>>>>> issues the suggested proposal is not going to address >>>>>>> Q3. How is it done today, and what are the limits of current >>>>>>> practice? >>>>>>> Q4. What is new in your approach approach and why do you think it >>>>>>> will be successful succeed? >>>>>>> Q5. Who cares? If you are successful, what difference will it make? >>>>>>> If your proposal succeeds, what tangible benefits will it add? >>>>>>> Q6. What are the risks? >>>>>>> Q7. How long will it take? >>>>>>> Q8. What are the midterm and final “exams” to check for success? >>>>>>> >>>>>>> >>>>>>> May not do justice to your proposal. >>>>>>> >>>>>>> HTH >>>>>>> >>>>>>> Mich >>>>>>> >>>>>>> view my Linkedin profile >>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>>>>> >>>>>>> >>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>>>>> >>>>>>> >>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility >>>>>>> for any loss, damage or destruction of data or any other property which >>>>>>> may >>>>>>> arise from relying on this email's technical content is explicitly >>>>>>> disclaimed. The author will in no case be liable for any monetary >>>>>>> damages >>>>>>> arising from such loss, damage or destruction. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, 31 Jan 2023 at 17:35, kazuyuki tanimura < >>>>>>> ktanim...@apple.com.invalid> wrote: >>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> I would like to start a discussion on “Lazy Materialization for >>>>>>>> Parquet Read Performance Improvement" >>>>>>>> >>>>>>>> Chao and I propose a Parquet reader with lazy materialization. For >>>>>>>> Spark-SQL filter operations, evaluating the filters first and >>>>>>>> lazily materializing only the used values can save computation wastes >>>>>>>> and >>>>>>>> improve the read performance. >>>>>>>> The current implementation of Spark requires the read values to >>>>>>>> materialize (i.e. decompress, de-code, etc...) onto memory first before >>>>>>>> applying the filters even though the filters may eventually throw away >>>>>>>> many >>>>>>>> values. >>>>>>>> >>>>>>>> We made our design doc as follows. >>>>>>>> SPIP Jira: https://issues.apache.org/jira/browse/SPARK-42256 >>>>>>>> SPIP Doc: >>>>>>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME >>>>>>>> >>>>>>>> Liang-Chi was kind enough to shepherd this effort. >>>>>>>> >>>>>>>> Thank you >>>>>>>> Kazu >>>>>>>> >>>>>>> >>>>>>> >>>>>> >> >
