+1 On Mon, 30 Dec 2019 at 10:24, Gengliang <ltn...@gmail.com> wrote:
> +1 > > On Sun, Dec 29, 2019 at 8:33 PM Wenchen Fan <cloud0...@gmail.com> wrote: > >> +1 for the new thrift server to get rid of the Hive dependencies! >> >> On Mon, Dec 23, 2019 at 7:55 PM Yuming Wang <wgy...@gmail.com> wrote: >> >>> I'm +1 for this SPIP for these two reasons: >>> >>> 1. The current thriftserver has some issues that are not easy to solve, >>> such as: SPARK-28636 <https://issues.apache.org/jira/browse/SPARK-28636> >>> . >>> 2. The difference between the version of ORC we are using and the >>> built-in Hive is using is getting bigger and bigger. We can't ensure that >>> there will be no compatibility issues in the future. If thriftserver does >>> not depend on Hive, it will be much easier to upgrade the built-in Hive in >>> the future. >>> >>> On Sat, Dec 21, 2019 at 9:28 PM angers.zhu <angers....@gmail.com> wrote: >>> >>>> Hi all, >>>> >>>> I have complete a Design doc about how to use and config this new >>>> thrift server, and some design detail about change and impersonation. >>>> >>>> Hope for your suggestions and ideas. >>>> >>>> SPIP DOC : >>>> https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit#heading=h.x97c6tj78zo0 >>>> Design DOC : >>>> https://docs.google.com/document/d/1UKE9QTtHqSZBq0V_vEn54PlWaWPiRAKf_JvcT0skaSo/edit#heading=h.q1ed5q1ldh14 >>>> Thrift server about configurations >>>> https://docs.google.com/document/d/1uI35qJmQO4FKE6pr0h3zetZqww-uI8QsQjxaYY_qb1s/edit?usp=drive_web&ouid=110963191229426834922 >>>> >>>> Best Regards >>>> angers.zhu >>>> angers....@gmail.com >>>> >>>> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D> >>>> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 >>>> >>>> --------- Forwarded Message --------- >>>> From： angers.zhu <angers....@gmail.com> <angers....@gmail.com> >>>> Date： 12/18/2019 22:29 >>>> To： dev-ow...@spark.apache.org <dev-ow...@spark.apache.org> >>>> <dev-ow...@spark.apache.org> >>>> Subject： Re: [VOTE][SPARK-29018][SPIP]:Build spark thrift server based >>>> on protocol v11 >>>> >>>> Add spark-dev group access privilege to google. >>>> angers.zhu >>>> angers....@gmail.com >>>> >>>> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D> >>>> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 >>>> >>>> On 12/18/2019 22:02，Sandeep Katta<sandeep0102.opensou...@gmail.com> >>>> <sandeep0102.opensou...@gmail.com> wrote： >>>> >>>> I couldn't access the doc, please give permission to the spark-dev group >>>> >>>> On Wed, 18 Dec 2019 at 18:05, angers.zhu <angers....@gmail.com> wrote: >>>> >>>>> With the development of Spark and Hive，in current >>>>> sql/hive-thriftserver module, >>>>> >>>>> we need to do a lot of work to solve code conflicts for different >>>>> built-in hive versions. >>>>> >>>>> It's an annoying and unending work in current ways. And these issues >>>>> have limited >>>>> >>>>> our ability and convenience to develop new features for Spark’s thrift >>>>> server. >>>>> >>>>> We suppose to implement a new thrift server and JDBC driver based >>>>> on Hive’s latest v11 >>>>> >>>>> TCLService.thrift thrift protocol. Finally, the new thrift server >>>>> have below feature: >>>>> >>>>> 1. >>>>> >>>>> Build new module spark-service as spark’s thrift server >>>>> 2. >>>>> >>>>> Don't need as much reflection and inherited code as >>>>> `hive-thriftser` modules >>>>> 3. >>>>> >>>>> Support all functions current `sql/hive-thriftserver` support >>>>> 4. >>>>> >>>>> Use all code maintained by spark itself, won’t depend on Hive >>>>> 5. >>>>> >>>>> Support origin functions use spark’s own way, won't limited by >>>>> Hive's code >>>>> 6. >>>>> >>>>> Support running without hive metastore or with hive metastore >>>>> 7. >>>>> >>>>> Support user impersonation by Multi-tenant splited hive >>>>> authentication and DFS authentication >>>>> 8. >>>>> >>>>> Support session hook for with spark’s own code >>>>> 9. >>>>> >>>>> Add a new jdbc driver spark-jdbc, with spark’s own connection url >>>>> “jdbc:spark:<host>:<port>/<db>” >>>>> 10. >>>>> >>>>> Support both hive-jdbc and spark-jdbc client, then we can support >>>>> most clients and BI platform >>>>> >>>>> >>>>> >>>>> https://issues.apache.org/jira/browse/SPARK-29018 >>>>> >>>>> Google Doc: >>>>> https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit# >>>>> >>>>> [ ] +1: Accept the proposal as an official SPIP >>>>> [ ] +0 >>>>> [ ] -1: I don't think this is a good idea because ... >>>>> >>>>> I'll start with my +1 >>>>> angers.zhu >>>>> angers....@gmail.com >>>>> >>>>> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D> >>>>> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 >>>>> >>>>>
