Hello everyone, It's been 12 days since this discussion started on May 31, 2021. After June 6, there are no more new questions arising. So I'd like to summarize the results of this discussion so we can move forward.
Thank you very much to everyone at the Apache IPMC for suggesting changes to our proposal and everyone else who participated in the discussion. We have corrected some sections and content of the proposal during this discussion, such as Orphaned products, Core Developers, Initial Committers, SGA, etc., based on your recommendations. In addition, we had a discussion with our mentors and all former candidates about the composition of Initial Committers. Based on the Apache Way, considering the current status of the Kyuubi community, the contribution of each contributor, the current Initial Committers were adjusted as follows with the consensus of everyone. Kent Yao (y...@apache.org) Fei Wang (cn.feiw...@gmail.com) Ulysses You (ulyssesyo...@gmail.com) Cheng Pan (pan3...@gmail.com) Hongxiang Jiang (jhx1...@gmail.com) Thank you all again. Bests Kent Yao Kent Yao <y...@apache.org> 于2021年6月6日周日 下午10:44写道： > Hi Ming, > > Thanks for your question. > > As mentioned in the proposal, Kyuubi was first developed and applied in > NetEase in 2018. The early scenario we wanted to address was > straightforward - to provide data analysts with a Spark-based analytics > platform. This platform needed to support multi-tenancy [1] and > authentication and authorization [2]. From the project's perspective, it > was just an alternative implementation of Apache Spark Thrift Server. And > for the workloads and users that it serves are not very big in NetEase. So > at that time, I was the only one who was maintaining the project. > > Another reason for this problem should be the limited usage scenario > caused by its architecture. It was implemented based on > *spark.driver.allowMultipleContexts*, which was not good judging from the > dismal feedback after we open-sourced it. Many people have found Kyuubi or > me through GitHub, hoping that Kyuubi could solve a problem they were > having in their scenario. From all of these things, I remember most and > feel sorry for is when they said they were using Kyuubi to help medical > staff fight COVID-19 [9]. Limited by design, there was very little that > could be done at the time. > > Starting with Spark 3.0.0, the Spark community also removed this feature > [3] for Kyuubi's old architecture. This allowed us to rethink and redesign > Kyuubi architecture. A prototype was quickly designed, but we did not start > it in a hurry. As Kyuubi relies heavily on Spark, we decided to actively > and deeply participate in the Spark community beginning around September > 2019 [4][5][6], both to help the Spark community and to verify that the > design of the new Kyuubi architecture [7] was future-proofing enough. In > the process of implementing the new architecture, Zen Jin became a > contributor to the project and contributed several great ideas, which made > Kyuubi's architecture and the scenarios it can cover more general and > comprehensive. This has made Kyuubi's architecture and the scenarios it can > cover more general. This advancement has made Kyuubi more attractive than > older architecture projects. > > As a result, many like-minded people [6] started to gather around the > project and the new architecture, and some of them have already become our > contributors. There are also many people who have encouraged us to > contribute to this project to Apache, which will be very beneficial for > them to be more directly involved in the project. > > In addition, we've learned a lot about community operations while > contributing to other Apache projects like Spark, Submarine, and Hudi. Some > of these communities are new to Apache, and some are ten years out of > school. We also actively participated in some online and offline > conferences or meetups, published technical articles in different media to > promote Kyuubi. > > From our practical experience, we have learned that promoting a project is > not an easy task. We want to donate Kyuubi to the community and bring in > potentially willing people to help the Kyuubi community grow. > > Bests > Kent Yao > > [1] https://issues.apache.org/jira/browse/SPARK-2243 > [2] https://issues.apache.org/jira/browse/SPARK-5159 > [3] https://issues.apache.org/jira/browse/SPARK-26362 > [4] https://github.com/apache/spark/commits?author=yaooqinn > [5] https://github.com/apache/spark/commits?author=turboFei > [6] https://github.com/apache/spark/commits?author=ulysses-you > [7] https://github.com/NetEase/kyuubi/releases/tag/v1.0.0 > [8] > https://github.com/NetEase/kyuubi/graphs/contributors?from=2021-01-01&to=2021-06-06&type=c > [9] https://github.com/NetEase/kyuubi/issues/228 > > Ming Wen <wenm...@apache.org> 于2021年6月6日周日 上午7:37写道： > >> You can see the trend chart from >> >> https://www.apiseven.com/zh/contributor-graph?chart=contributorMonthlyActivity&repo=NetEase/kyuubi >> >> >> Ming Wen <wenm...@apache.org>于2021年6月6日 周日上午7:28写道： >> >> > hi, Kent, >> > >> > From 2018 to 2020, kuyybi is not an active project, most of the time >> there >> > is only one contributor per month. But in 2021, there will be 8 active >> > contributors every month. What is the reason for this? >> > >> > Willem Jiang <willem.ji...@gmail.com>于2021年6月3日 周四下午12:53写道： >> > >> >> I think we can revisit the PPMC part and have more discussion with the >> >> initial committers. >> >> It's a good chance to let them know better about how to run an open >> >> source project in Apache Way. >> >> >> >> >> >> Willem Jiang >> >> >> >> Twitter: willemjiang >> >> Weibo: 姜宁willem >> >> >> >> On Thu, Jun 3, 2021 at 11:52 AM Atri Sharma <a...@apache.org> wrote: >> >> > >> >> > I would like to hear from Kyuubi's mentors on the topic before >> reaching >> >> a >> >> > conclusion. >> >> > >> >> > On Thu, 3 Jun 2021, 05:40 Sheng Wu, <wu.sheng.841...@gmail.com> >> wrote: >> >> > >> >> > > Kent >> >> > > >> >> > > Thanks. >> >> > > The proposal is generally better this time. >> >> > > >> >> > > If your initial committers and your mentors agree, you are ready to >> >> embrace >> >> > > the ASF culture, I am good with that. >> >> > > Just from my experience, your community looks a little too young, >> like >> >> > > Atri's concern, which is reasonable. >> >> > > But I would like to trust you and your mentors' judgment and >> >> determination. >> >> > > >> >> > > Sheng Wu 吴晟 >> >> > > Twitter, wusheng1108 >> >> > > >> >> > > >> >> > > Kent Yao <y...@apache.org> 于2021年6月3日周四 上午12:54写道： >> >> > > >> >> > > > Hi Sheng, >> >> > > > >> >> > > > Truly sorry for the format issue. >> >> > > > >> >> > > > Please trust me I did not mean to hurt anybody and push any >> >> contributor >> >> > > > away. We drafted this proposal together based on the New Podling >> >> > > Proposal. >> >> > > > We never tried to use PPMC and committer to measure whose >> >> contribution is >> >> > > > higher. This mistake has been fixed in the proposal. >> >> > > > >> >> > > > About releases, we will follow >> >> > > > https://www.apache.org/legal/release-policy.html and the guide >> of >> >> our >> >> > > > mentors. We will also take the initiative to learn other >> documents. >> >> > > > >> >> > > > > This is not a block, the point I mentioned this is, you will >> need >> >> more >> >> > > > people. >> >> > > > >> >> > > > We will keep working on this. >> >> > > > >> >> > > > Bests, >> >> > > > >> >> > > > Kent Yao >> >> > > > >> >> > > > >> >> > > > Sheng Wu <wu.sheng.841...@gmail.com> 于2021年6月2日周三 下午10:53写道： >> >> > > > >> >> > > > > Hi Kent >> >> > > > > >> >> > > > > Thanks for replying, but to be honest, your mail format is >> chaos. >> >> > > > > Take a look at bellowing, which is what I saw in the gmail. >> >> > > > > >> >> > > > > This may be an issue of your mail client or some else. >> >> > > > > >> >> > > > > > The current #.2 - 5 top >> >> > > > > > contributors will stay as committers, and they will join the >> >> PPMC >> >> > > > during >> >> > > > > > incubation. >> >> > > > > >> >> > > > > PPMC usually is the default for all initial committers, >> >> considering >> >> > > your >> >> > > > > community is still small and not very active. >> >> > > > > You should invite all people you trusted to join, rather than >> >> pushing >> >> > > > them >> >> > > > > away. >> >> > > > > This is not just a too-high bar issue. Excluding 4 out of 5 is >> >> strange. >> >> > > > > I can't guess what is the root cause, why the other 4 PPMC >> members >> >> > > don't >> >> > > > > touch codes but are trusted completely. Meanwhile, people with >> >> easily >> >> > > > > measurable code contributions are just as committers? >> >> > > > > We(ASF) always remind the community to recognize no-code >> >> contribution, >> >> > > > but >> >> > > > > in this case, it seems the opposite? Don't recognize code >> >> contributors >> >> > > > > enough? >> >> > > > > Please reach your mentors, this seems not a typical ASF style. >> If >> >> you >> >> > > > have >> >> > > > > other people already involved in the ASF, asking them too. >> >> > > > > >> >> > > > > And from your description of the release process, I think you >> >> should >> >> > > talk >> >> > > > > more with your mentors. >> >> > > > > Such as what is a qualified ASF release, what should be >> checked. >> >> > > > > This is not a block, the point I mentioned this is, you will >> need >> >> more >> >> > > > > people. >> >> > > > > >> >> > > > > >> >> > > > > Sheng Wu 吴晟 >> >> > > > > Twitter, wusheng1108 >> >> > > > > >> >> > > > > >> >> > > > > Kent Yao <y...@apache.org> 于2021年6月2日周三 下午10:37写道： >> >> > > > > >> >> > > > > > Hi Sheng, >> >> > > > > > >> >> > > > > > Thanks for your detailed check and valuable comments. They >> are >> >> > > > > > accommodating. I have addressed some of them directly in the >> >> > > proposal. >> >> > > > > > Please also check the table below for some Q&As. >> >> > > > > > QA >> >> > > > > > You listed only 5 core maintainers, but clearly only one has >> >> > > dominated >> >> > > > > > contributions. The distortion and skewness of the >> contributions >> >> are >> >> > > > > mainly >> >> > > > > > caused by the way how we merge PRs changed(mentioned in >> another >> >> email >> >> > > > on >> >> > > > > > the list). >> >> > > > > > I agree with Atri, orphaned is measurable. It is better to >> >> adjust >> >> > > this >> >> > > > > > statement a little. Address the comment to notify the risk in >> >> the >> >> > > > Kyuubi >> >> > > > > > proposal. >> >> > > > > > Also, with this fact, the core maintainers should notice >> >> whether this >> >> > > > > small >> >> > > > > > group of people to make sure your features iteration, bug >> fix, >> >> and >> >> > > > > release >> >> > > > > > process working by following ASF's requirements.This would >> not >> >> be an >> >> > > > easy >> >> > > > > > job. Currently, we learn the release process from some Apache >> >> > > projects, >> >> > > > > > using major.minor.patch for versioning. As we can see, since >> >> Kyuubi >> >> > > > 1.0, >> >> > > > > it >> >> > > > > > takes us 2 ~ 3 weeks for a patch release that contains 5-10 >> bug >> >> fixes >> >> > > > > until >> >> > > > > > EOL. And it takes us about two months to finish a minor >> release >> >> with >> >> > > 3 >> >> > > > ~ >> >> > > > > 5 >> >> > > > > > big improvements and various small features/bug fixes. We >> also >> >> will >> >> > > > start >> >> > > > > > with some release candidates for preview and test before the >> >> final. >> >> > > > This >> >> > > > > is >> >> > > > > > how we develop and release Kyuubi for now, but we do have a >> >> long way >> >> > > to >> >> > > > > go >> >> > > > > > and learn to match the ASF requirement. >> >> > > > > > And your committer list is very strange... 1. Why some of >> them >> >> are >> >> > > PPMC >> >> > > > > and >> >> > > > > > others are committers? Where this hierarchy comes from? The >> >> current >> >> > > > PPMC >> >> > > > > > candidates help us a lot build the Kyuubi project and >> >> community, not >> >> > > > > > directly to the codebase but as mentors(before Apache). Most >> of >> >> them >> >> > > > are >> >> > > > > > from Apache mature TLPs, which have close relationships with >> >> Kyuubi. >> >> > > > They >> >> > > > > > have sufficient Apache Way experience to guide us to become a >> >> more >> >> > > > > > Apache-like project before submitting this proposal in many >> >> ways. We >> >> > > > will >> >> > > > > > continue to promote more contributors to the PPMC/committer >> >> based on >> >> > > > > their >> >> > > > > > code or non-code contributions during incubation. >> >> > > > > > 2. Your core maintainers have 5, which only 4 PPMC? The >> current >> >> #.2 >> >> > > - 5 >> >> > > > > top >> >> > > > > > contributors will stay as committers, and they will join the >> >> PPMC >> >> > > > during >> >> > > > > > incubation. >> >> > > > > > 3. I think only the top 4 have over 400 LOC, why you declared >> >> you >> >> > > have >> >> > > > 5 >> >> > > > > > core developers? I didn't notice that 400 LOC is a bar for >> core >> >> > > > > developer, >> >> > > > > > will change 5 to 4 core developers accordingly >> >> > > > > > I think you don't have enough no-salaried developers, am I >> >> missing >> >> > > > > > anything? Yes. >> >> > > > > > Changed in the proposal, most of the contributors are >> salaried, >> >> and >> >> > > > some >> >> > > > > of >> >> > > > > > them are willing to help in the spare time. Thanks >> >> > > > > > This project belongs to NetEase Inc. clearly, why NO SGA? >> Added >> >> SGA >> >> > > in >> >> > > > > > Proposal. All work here is approved by NetEase and can be >> >> forward. >> >> > > > > > At last, you are asking about >> >> > > > github.com/apache/incubator-kyuubi-shaded >> >> > > > > > repo, >> >> > > > > > what does this represent? I can only see you have only one >> >> source >> >> > > repo >> >> > > > > and >> >> > > > > > a doc repo. Removed this requirement in the proposal, and we >> >> will >> >> > > apply >> >> > > > > it >> >> > > > > > from IPMC if we do need this in the future. >> >> > > > > > ------------------------------ >> >> > > > > > >> >> > > > > > Bests >> >> > > > > > >> >> > > > > > Kent Yao >> >> > > > > > >> >> > > > > > Sheng Wu <wu.sheng.841...@gmail.com> 于2021年6月2日周三 下午5:45写道： >> >> > > > > > >> >> > > > > > > Hi >> >> > > > > > > >> >> > > > > > > I read the contributor list and the proposal >> >> > > > > > > > The risk of the Kyuubi project being abandoned is >> minimal. >> >> Many >> >> > > > > > > organizations are using Kyuubi to build critical big data >> >> pipelines >> >> > > > and >> >> > > > > > > willing to help develop Kyuubi's community if it becomes an >> >> ASF >> >> > > > > project. >> >> > > > > > > >> >> > > > > > > I think the team should evaluate the status again. You >> listed >> >> only >> >> > > 5 >> >> > > > > core >> >> > > > > > > maintainers, but clearly only one has dominated >> >> contributions. I >> >> > > > agree >> >> > > > > > with >> >> > > > > > > Atri, orphaned is measurable. >> >> > > > > > > It is better to adjust this statement a little. Also, with >> >> this >> >> > > fact, >> >> > > > > the >> >> > > > > > > core maintainers should notice whether this small group of >> >> people >> >> > > to >> >> > > > > make >> >> > > > > > > sure your features iteration, bug fix, and release process >> >> working >> >> > > by >> >> > > > > > > following ASF's requirements. >> >> > > > > > > This would not be an easy job. >> >> > > > > > > >> >> > > > > > > And your committer list is very strange >> >> > > > > > > ``` >> >> > > > > > > Kent Yao (y...@apache.org) (PPMC) >> >> > > > > > > Wenchen Fan (wenc...@apache.org) (PPMC) >> >> > > > > > > Wanqiang Ji (j...@apache.org) (PPMC) >> >> > > > > > > Xun Liu (liu...@apache.org) (PPMC) >> >> > > > > > > Fei Wang (cn.feiw...@gmail.com) >> >> > > > > > > Ulysses You (ulyssesyo...@gmail.com) >> >> > > > > > > Cheng Pan (pan3...@gmail.com) >> >> > > > > > > Hongxiang Jiang (jhx1...@gmail.com) >> >> > > > > > > Zen Jin(xinjingziranc...@gmail.com) >> >> > > > > > > ``` >> >> > > > > > > 1. Why some of them are PPMC and others are committers? >> Where >> >> this >> >> > > > > > > hierarchy comes from? >> >> > > > > > > 2. Your core maintainers have 5, which only 4 PPMC? (Even >> >> hierarchy >> >> > > > > makes >> >> > > > > > > sense, most likely not) >> >> > > > > > > 3. I think only the top 4 have over 400 LOC, why you >> declared >> >> you >> >> > > > have >> >> > > > > 5 >> >> > > > > > > core developers? >> >> > > > > > > >> >> > > > > > > Another re-evaluation request is about `Reliance on >> Salaried >> >> > > > > Developers:` >> >> > > > > > > > They are all passionate about the project, and we are >> >> confident >> >> > > > that >> >> > > > > > the >> >> > > > > > > project will continue even if no salaried developers >> >> contribute to >> >> > > > the >> >> > > > > > > project. >> >> > > > > > > >> >> > > > > > > I think you don't have enough no-salaried developers, am I >> >> missing >> >> > > > > > > anything? >> >> > > > > > > >> >> > > > > > > About IP clearance, `Initial Source and Intellectual >> Property >> >> > > > > Submission >> >> > > > > > > Plan` >> >> > > > > > > > As soon as Kyuubi is approved to join Apache Incubator, >> our >> >> > > initial >> >> > > > > > > committers will submit iCLA(s) and CCLA(s). The codebase is >> >> already >> >> > > > > > > licensed under the Apache License 2.0. >> >> > > > > > > >> >> > > > > > > This project belongs to NetEase Inc. clearly, why NO SGA? >> >> > > > > > > >> >> > > > > > > At last, you are asking about ` >> >> > > > > > github.com/apache/incubator-kyuubi-shaded` >> <http://github.com/apache/incubator-kyuubi-shaded> >> >> <http://github.com/apache/incubator-kyuubi-shaded> >> >> > > <http://github.com/apache/incubator-kyuubi-shaded> >> >> > > > <http://github.com/apache/incubator-kyuubi-shaded> >> >> > > > > <http://github.com/apache/incubator-kyuubi-shaded> >> >> > > > > > <http://github.com/apache/incubator-kyuubi-shaded> >> >> > > > > > > <http://github.com/apache/incubator-kyuubi-shaded> >> >> > > > > > > repo, what does this represent? I can only see you have >> only >> >> one >> >> > > > source >> >> > > > > > > repo and a doc repo. >> >> > > > > > > >> >> > > > > > > Sheng Wu 吴晟 >> >> > > > > > > Twitter, wusheng1108 >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > Atri Sharma <a...@apache.org> 于2021年6月2日周三 下午4:49写道： >> >> > > > > > > >> >> > > > > > > > Thanks. >> >> > > > > > > > >> >> > > > > > > > I looked at the link you advised on and if I am reading >> it >> >> > > > correctly, >> >> > > > > > top >> >> > > > > > > > committer has 630ish commits and the next committer has >> 73 >> >> > > commits? >> >> > > > > > > > >> >> > > > > > > > IMO this is an indication of the community not being >> active >> >> > > enough >> >> > > > > and >> >> > > > > > > > hence the risk of project getting orphaned is measurable. >> >> > > > > > > > >> >> > > > > > > > Atri >> >> > > > > > > > >> >> > > > > > > > On Wed, 2 Jun 2021, 06:57 Willem Jiang, < >> >> willem.ji...@gmail.com> >> >> > > > > > wrote: >> >> > > > > > > > >> >> > > > > > > > > Hi Atri Sharma, >> >> > > > > > > > > >> >> > > > > > > > > It's easier to check out the recent contribution of >> >> kyuubi from >> >> > > > > this >> >> > > > > > > > > link[1]. >> >> > > > > > > > > We already identified the one person contribution >> issue of >> >> > > kyuubi >> >> > > > > > when >> >> > > > > > > > > Kent came to us and we kept reminding him to put more >> >> effort >> >> > > into >> >> > > > > > > > > building the community. >> >> > > > > > > > > Last week I had a long F2F conversation with Kent, I >> >> think he >> >> > > > > already >> >> > > > > > > > > knows about it and there have been some improvements in >> >> recent >> >> > > > > months >> >> > > > > > > > > but there is still a long way to go if Kyuubi is >> accepted >> >> as a >> >> > > > > > podling >> >> > > > > > > > > in the Apache Incubator. >> >> > > > > > > > > >> >> > > > > > > > > [1] >> https://github.com/NetEase/kyuubi/graphs/contributors >> >> > > > > > > > > >> >> > > > > > > > > Willem Jiang >> >> > > > > > > > > >> >> > > > > > > > > Twitter: willemjiang >> >> > > > > > > > > Weibo: 姜宁willem >> >> > > > > > > > > >> >> > > > > > > > > On Tue, Jun 1, 2021 at 10:52 PM Atri Sharma < >> >> a...@apache.org> >> >> > > > > wrote: >> >> > > > > > > > > > >> >> > > > > > > > > > Hi, >> >> > > > > > > > > > >> >> > > > > > > > > > I took a look at commit log and seems like all the >> >> commits >> >> > > for >> >> > > > > > last 3 >> >> > > > > > > > > > months have been done by a single person. >> >> > > > > > > > > > >> >> > > > > > > > > > How active are the remaining committers? >> >> > > > > > > > > > >> >> > > > > > > > > > On Mon, 31 May 2021, 12:59 Kent Yao, <y...@apache.org >> > >> >> wrote: >> >> > > > > > > > > > >> >> > > > > > > > > > > Hi Alexander Alten, >> >> > > > > > > > > > > >> >> > > > > > > > > > > Thanks for your interest and +1 for Kyuubi. >> >> > > > > > > > > > > >> >> > > > > > > > > > > Not being very familiar with Dremel, I just took a >> >> quick >> >> > > look >> >> > > > > at >> >> > > > > > > the >> >> > > > > > > > > > > profile of a very similar and exciting project, >> Apache >> >> > > Drill. >> >> > > > > If >> >> > > > > > I >> >> > > > > > > > > > > understand correctly, Kyuubi is built on top of the >> >> compute >> >> > > > > layer >> >> > > > > > > > > > > (Apache Spark), while Apache Drill is built on top >> of >> >> the >> >> > > > > storage >> >> > > > > > > > > > > layer. >> >> > > > > > > > > > > >> >> > > > > > > > > > > Apache Spark has a large user base and usage >> >> scenarios all >> >> > > > over >> >> > > > > > the >> >> > > > > > > > > > > world. When users have some basic Spark programming >> >> and >> >> > > > tuning >> >> > > > > > > > skills, >> >> > > > > > > > > > > things are easy for them. But many times, most >> users >> >> do not >> >> > > > > have >> >> > > > > > > all >> >> > > > > > > > > > > of these abilities when they use Spark directly or >> >> > > > indirectly. >> >> > > > > We >> >> > > > > > > > want >> >> > > > > > > > > > > to build a more easy-to-use platform on top of it >> to >> >> help >> >> > > > these >> >> > > > > > > > users. >> >> > > > > > > > > > > On the one hand, we will work on the interface >> layer >> >> to >> >> > > make >> >> > > > it >> >> > > > > > > easy >> >> > > > > > > > > > > to access (of course, this effort will make our >> >> project >> >> > > look >> >> > > > > very >> >> > > > > > > > > > > similar to other projects, such as HiveServer2). On >> >> the >> >> > > other >> >> > > > > > hand, >> >> > > > > > > > we >> >> > > > > > > > > > > will work under Kyuubi's multi-tenant architecture >> to >> >> > > extend >> >> > > > > our >> >> > > > > > > > > > > on-prem Spark SQL engine(a.k.a Kyuubi engine) to >> make >> >> it >> >> > > more >> >> > > > > > > stable >> >> > > > > > > > > > > and efficient in our scenario. >> >> > > > > > > > > > > >> >> > > > > > > > > > > What's more, we are going to add more on-prem >> Kyuubi >> >> engine >> >> > > > > > types, >> >> > > > > > > > e.g. >> >> > > > > > > > > > > Flink. >> >> > > > > > > > > > > >> >> > > > > > > > > > > Alexander Alten <a...@scalytics.io> 于2021年5月31日周一 >> >> 下午2:35写道： >> >> > > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > Hey, >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > Looks like an interesting project, competition to >> >> > > > Starburst. >> >> > > > > > How >> >> > > > > > > > > does it >> >> > > > > > > > > > > > compare to Apache Dremel? >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > +1 (non-binding) for incubation. >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > Thanks and stay safe, >> >> > > > > > > > > > > > --alex >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > On Mon, May 31, 2021, 08:21 Kent Yao < >> >> y...@apache.org> >> >> > > > wrote: >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > > Dear all, >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > > > We want to propose Kyuubi as a new Apache >> >> Incubator >> >> > > > > project. >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > > > Kyuubi is a distributed multi-tenant Thrift >> >> JDBC/ODBC >> >> > > > > server >> >> > > > > > > for >> >> > > > > > > > > > > > > large-scale data management, processing, and >> >> analytics, >> >> > > > > built >> >> > > > > > > on >> >> > > > > > > > > top >> >> > > > > > > > > > > > > of Apache Spark and designed to support more >> >> engines >> >> > > > (i.e., >> >> > > > > > > > Apache >> >> > > > > > > > > > > > > Flink). We are aiming to make Kyuubi an >> >> > > "out-of-the-box" >> >> > > > > tool >> >> > > > > > > for >> >> > > > > > > > > data >> >> > > > > > > > > > > > > warehouses and data lakes. Kyuubi concentrates >> on >> >> > > > improving >> >> > > > > > the >> >> > > > > > > > > > > > > experience for non-Spark users, making it easy >> >> for them >> >> > > > to >> >> > > > > > use >> >> > > > > > > > > Spark >> >> > > > > > > > > > > > > as a service and focus on business data of >> their >> >> own, >> >> > > not >> >> > > > > > where >> >> > > > > > > > it >> >> > > > > > > > > > > > > stores, how it computes. NetEase has >> open-sourced >> >> > > Kyuubi >> >> > > > > > since >> >> > > > > > > > > 2018, >> >> > > > > > > > > > > > > https://github.com/NetEase/kyuubi. >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > > > Many thanks to Willem Jiang (Champion), Jeff >> >> Zhang, Duo >> >> > > > > > Zhang, >> >> > > > > > > > > Akira >> >> > > > > > > > > > > > > Ajisaka for being the mentors of the Kyuubi >> >> project and >> >> > > > > > helping >> >> > > > > > > > us >> >> > > > > > > > > set >> >> > > > > > > > > > > > > up this proposal. The proposal can be found at >> >> > > > > > > > > > > > > >> >> > > > > > > > > >> >> > > > > >> >> https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal >> >> > > > > > . >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > > > Looking forward to your feedback and thank you >> >> very >> >> > > much. >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > > > Best regards, >> >> > > > > > > > > > > > > Kent Yao >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > > > >> >> > > > > > > > > >> >> > > > > >> >> --------------------------------------------------------------------- >> >> > > > > > > > > > > > > To unsubscribe, e-mail: >> >> > > > > > > general-unsubscr...@incubator.apache.org >> >> > > > > > > > > > > > > For additional commands, e-mail: >> >> > > > > > > > general-h...@incubator.apache.org >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > -- >> >> > > > > > > > > > > >> >> > > > > > > > > > > Kent Yao >> >> > > > > > > > > > > @ Data Science Center, Hangzhou Research Institute, >> >> NetEase >> >> > > > > Corp. >> >> > > > > > > > > > > a spark enthusiast >> >> > > > > > > > > > > kyuubiis a unified multi-tenant JDBC interface for >> >> > > > large-scale >> >> > > > > > data >> >> > > > > > > > > > > processing and analytics, built on top of Apache >> >> Spark. >> >> > > > > > > > > > > >> >> > > > > > > > > > > spark-authorizerA Spark SQL extension which >> provides >> >> SQL >> >> > > > > Standard >> >> > > > > > > > > > > Authorization for Apache Spark. >> >> > > > > > > > > > > spark-postgres A library for reading data from and >> >> > > > transferring >> >> > > > > > > data >> >> > > > > > > > > > > to Postgres / Greenplum with Spark SQL and >> DataFrames, >> >> > > > 10~100x >> >> > > > > > > > faster. >> >> > > > > > > > > > > itatchiA library that brings useful functions from >> >> various >> >> > > > > modern >> >> > > > > > > > > > > database management systems to Apache Spark. >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > >> >> > > >> --------------------------------------------------------------------- >> >> > > > > > > > > > > To unsubscribe, e-mail: >> >> > > > > general-unsubscr...@incubator.apache.org >> >> > > > > > > > > > > For additional commands, e-mail: >> >> > > > > > general-h...@incubator.apache.org >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > > >> >> > > > > >> >> --------------------------------------------------------------------- >> >> > > > > > > > > To unsubscribe, e-mail: >> >> > > general-unsubscr...@incubator.apache.org >> >> > > > > > > > > For additional commands, e-mail: >> >> > > > general-h...@incubator.apache.org >> >> > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > -- >> >> > > > > > >> >> > > > > > *Kent Yao * >> >> > > > > > @ Data Science Center, Hangzhou Research Institute, NetEase >> >> Corp. >> >> > > > > > a spark enthusiast >> >> > > > > > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified >> >> multi-tenant >> >> > > > > JDBC >> >> > > > > > interface for large-scale data processing and analytics, >> built >> >> on top >> >> > > > of >> >> > > > > > Apache >> >> > > > > > Spark <http://spark.apache.org/>. >> >> > > > > > spark-authorizer < >> https://github.com/yaooqinn/spark-authorizer >> >> >A >> >> > > Spark >> >> > > > > SQL >> >> > > > > > extension which provides SQL Standard Authorization for >> Apache >> >> Spark >> >> > > > > > <http://spark.apache.org/>. >> >> > > > > > spark-postgres <https://github.com/yaooqinn/spark-postgres> >> A >> >> > > library >> >> > > > > for >> >> > > > > > reading data from and transferring data to Postgres / >> Greenplum >> >> with >> >> > > > > Spark >> >> > > > > > SQL and DataFrames, 10~100x faster. >> >> > > > > > itatchi <https://github.com/yaooqinn/spark-func-extras>A >> >> library >> >> > > that >> >> > > > > > brings useful functions from various modern database >> management >> >> > > > > > systems to Apache >> >> > > > > > Spark <http://spark.apache.org/>. >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > > >> >> > > > -- >> >> > > > >> >> > > > *Kent Yao * >> >> > > > @ Data Science Center, Hangzhou Research Institute, NetEase Corp. >> >> > > > a spark enthusiast >> >> > > > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified >> >> multi-tenant >> >> > > JDBC >> >> > > > interface for large-scale data processing and analytics, built on >> >> top of >> >> > > > Apache >> >> > > > Spark <http://spark.apache.org/>. >> >> > > > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A >> >> Spark >> >> > > SQL >> >> > > > extension which provides SQL Standard Authorization for Apache >> Spark >> >> > > > <http://spark.apache.org/>. >> >> > > > spark-postgres <https://github.com/yaooqinn/spark-postgres> A >> >> library >> >> > > for >> >> > > > reading data from and transferring data to Postgres / Greenplum >> with >> >> > > Spark >> >> > > > SQL and DataFrames, 10~100x faster. >> >> > > > itatchi <https://github.com/yaooqinn/spark-func-extras>A library >> >> that >> >> > > > brings useful functions from various modern database management >> >> > > > systems to Apache >> >> > > > Spark <http://spark.apache.org/>. >> >> > > > >> >> > > >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> >> For additional commands, e-mail: general-h...@incubator.apache.org >> >> >> >> -- >> > Thanks, >> > Ming Wen, Apache APISIX PMC Chair >> > Twitter: _WenMing >> > >> -- >> Thanks, >> Ming Wen, Apache APISIX PMC Chair >> Twitter: _WenMing >> > > > -- > > *Kent Yao * > @ Data Science Center, Hangzhou Research Institute, NetEase Corp. > a spark enthusiast > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC > interface for large-scale data processing and analytics, built on top of > Apache > Spark <http://spark.apache.org/>. > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark > SQL extension which provides SQL Standard Authorization for Apache Spark > <http://spark.apache.org/>. > spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for > reading data from and transferring data to Postgres / Greenplum with Spark > SQL and DataFrames, 10~100x faster. > itatchi <https://github.com/yaooqinn/spark-func-extras>A library that > brings useful functions from various modern database management systems to > Apache > Spark <http://spark.apache.org/>. > > > -- *Kent Yao * @ Data Science Center, Hangzhou Research Institute, NetEase Corp. a spark enthusiast kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark <http://spark.apache.org/>. spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark SQL extension which provides SQL Standard Authorization for Apache Spark <http://spark.apache.org/>. spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster. itatchi <https://github.com/yaooqinn/spark-func-extras>A library that brings useful functions from various modern database management systems to Apache Spark <http://spark.apache.org/>.
