Sure Xiao. Happy Lunar New Year!
On Thu, Feb 3, 2022 at 1:57 PM Xiao Li <gatorsm...@gmail.com> wrote: > Can we extend the voting window to next Wednesday? This week is a holiday > week for the lunar new year. AFAIK, many members in Asia are taking the > whole week off. They might not regularly check the emails. > > Also how about starting a separate email thread starting with [VOTE] ? > > Happy Lunar New Year!!! > > Xiao > > Holden Karau <hol...@pigscanfly.ca> 于2022年2月3日周四 12:28写道： > >> +1 (binding) >> >> On Thu, Feb 3, 2022 at 2:26 PM Erik Krogen <xkro...@apache.org> wrote: >> >>> +1 (non-binding) >>> >>> Really looking forward to having this natively supported by Spark, so >>> that we can get rid of our own hacks to tie in a custom view catalog >>> implementation. I appreciate the care John has put into various parts of >>> the design and believe this will provide a robust and flexible solution to >>> this problem faced by various large-scale Spark users. >>> >>> Thanks John! >>> >>> On Thu, Feb 3, 2022 at 11:22 AM Walaa Eldin Moustafa < >>> wa.moust...@gmail.com> wrote: >>> >>>> +1 >>>> >>>> On Thu, Feb 3, 2022 at 11:19 AM John Zhuge <jzh...@apache.org> wrote: >>>> >>>>> Hi Spark community, >>>>> >>>>> I’d like to restart the vote for the ViewCatalog design proposal (SPIP >>>>> <https://docs.google.com/document/d/1XOxFtloiMuW24iqJ-zJnDzHl2KMxipTjJoxleJFz66A/edit?usp=sharing> >>>>> ). >>>>> >>>>> The proposal is to add a ViewCatalog interface that can be used to >>>>> load, create, alter, and drop views in DataSourceV2. >>>>> >>>>> Please vote on the SPIP in the next 72 hours. Once it is approved, >>>>> I’ll update the PR <https://github.com/apache/spark/pull/28147> for >>>>> review. >>>>> >>>>> [ ] +1: Accept the proposal as an official SPIP >>>>> [ ] +0 >>>>> [ ] -1: I don’t think this is a good idea because … >>>>> >>>>> Thanks! >>>>> >>>>> On Fri, Jun 4, 2021 at 1:46 PM Walaa Eldin Moustafa < >>>>> wa.moust...@gmail.com> wrote: >>>>> >>>>>> Considering the API aspect, the ViewCatalog API sounds like a good >>>>>> idea. A view catalog will enable us to integrate Coral >>>>>> <https://engineering.linkedin.com/blog/2020/coral> (our view SQL >>>>>> translation and management layer) very cleanly to Spark. Currently we can >>>>>> only do it by maintaining our special version of the >>>>>> HiveExternalCatalog. Considering that views can be expanded >>>>>> syntactically without necessarily invoking the analyzer, using a >>>>>> dedicated >>>>>> view API can make performance better if performance is the concern. >>>>>> Further, a catalog can still be both a table and view provider if it >>>>>> chooses to based on this design, so I do not think we necessarily lose >>>>>> the >>>>>> ability of providing both. Looking forward to more discussions on this >>>>>> and >>>>>> making views a powerful tool in Spark. >>>>>> >>>>>> Thanks, >>>>>> Walaa. >>>>>> >>>>>> >>>>>> On Wed, May 26, 2021 at 9:54 AM John Zhuge <jzh...@apache.org> wrote: >>>>>> >>>>>>> Looks like we are running in circles. Should we have an online >>>>>>> meeting to get this sorted out? >>>>>>> >>>>>>> Thanks, >>>>>>> John >>>>>>> >>>>>>> On Wed, May 26, 2021 at 12:01 AM Wenchen Fan <cloud0...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> OK, then I'd vote for TableViewCatalog, because >>>>>>>> 1. This is how Hive catalog works, and we need to migrate Hive >>>>>>>> catalog to the v2 API sooner or later. >>>>>>>> 2. Because of 1, TableViewCatalog is easy to support in the current >>>>>>>> table/view resolution framework. >>>>>>>> 3. It's better to avoid name conflicts between table and views at >>>>>>>> the API level, instead of relying on the catalog implementation. >>>>>>>> 4. Caching invalidation is always a tricky problem. >>>>>>>> >>>>>>>> On Tue, May 25, 2021 at 3:09 AM Ryan Blue <rb...@netflix.com.invalid> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I don't think that it makes sense to discuss a different approach >>>>>>>>> in the PR rather than in the vote. Let's discuss this now since >>>>>>>>> that's the >>>>>>>>> purpose of an SPIP. >>>>>>>>> >>>>>>>>> On Mon, May 24, 2021 at 11:22 AM John Zhuge <jzh...@apache.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi everyone, I’d like to start a vote for the ViewCatalog design >>>>>>>>>> proposal (SPIP). >>>>>>>>>> >>>>>>>>>> The proposal is to add a ViewCatalog interface that can be used >>>>>>>>>> to load, create, alter, and drop views in DataSourceV2. >>>>>>>>>> >>>>>>>>>> The full SPIP doc is here: >>>>>>>>>> https://docs.google.com/document/d/1XOxFtloiMuW24iqJ-zJnDzHl2KMxipTjJoxleJFz66A/edit?usp=sharing >>>>>>>>>> >>>>>>>>>> Please vote on the SPIP in the next 72 hours. Once it is >>>>>>>>>> approved, I’ll update the PR for review. >>>>>>>>>> >>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP >>>>>>>>>> [ ] +0 >>>>>>>>>> [ ] -1: I don’t think this is a good idea because … >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Ryan Blue >>>>>>>>> Software Engineer >>>>>>>>> Netflix >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> John Zhuge >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> John Zhuge >>>>> >>>> -- >> Twitter: https://twitter.com/holdenkarau >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> > -- John Zhuge
