Re: [DISCUSS] Linkis Proposal (a Computation Middleware project)
+1. Linkis is an interesting project that builds a unified layer to decouple the upper layer computation/application and under layer data engines, such as Spark, Presto, Flink, etc. Also it looks already build a community around the project. Good luck! Shuai Di 于2021年7月12日 周一上午11:13写道： > Greetings! > > > We would like to start an open discussion on bringing Linkis ( > https://github.com/WeBankFinTech/Linkis), a computation middleware > project, to the Apache Incubator. > The proposal can be found below and is also listed in the Incubator wiki: > https://cwiki.apache.org/confluence/display/INCUBATOR/LinkisProposal, > thanks @Junping Du for creating the page! > We appreciate anyone who would give guidance or be willing to support us a > an additional mentor. > == > Linkis Proposal > > > =Abstract= > Linkis builds a computation middleware layer to decouple the upper > applications and the underlying data engines, provides standardized > interfaces (REST, JDBC, WebSocket etc.) to easily connect to various > underlying engines (Spark, Presto, Flink, etc.), while enables cross engine > context sharing, unified job& engine governance and orchestration. > Linkis codebase: https://github.com/WeBankFinTech/Linkis > > > =Proposal= > Linkis is designed to solve computation governance problems in complex > distributed environments (typically in a big data platform), where you have > to deal with different types, versions, or clusters of underlying data > engines and hundreds of diversified engine clients at the upper application > layer as well. > Linkis acts as a proxy between the upper applications layer and underlying > engines layer. By abstracting and implementing the 3 common phases of a > job/request for submit, prepare and execute, Linkis is able to facilitate > the connectivity, governance and orchestration capabilities of different > kind of engines like OLAP, OLTP (developing), Streaming, and handle all > these "computation governance" affairs in a standardized reusable way. > We are actively operating the Linkis community and we are looking forward > to increase community activity continuously.We propose to contribute the > Linkis codebase to the Apache Software Foundation. We believe that bringing > Linkis into Apache Software Foundation and following the COMMUNITY-LED > DEVELOPMENT "APACHE WAY" could continuously improve project quality and > community vitality. > > > =Background= > In today's complex and distributed environment, the communication, > coordination and governance of application services have developed mature > solutions from SOA to micro-services, and many practices from ESB to > Service Mesh to decouple different services. > However, things go different while an application service needs to > communicate with the underlying engines. Engines are isolated from each > other, and the client-server tight coupling pattern goes everywhere. Each > and every upper application has to directly connect to and access various > underlying engines in a tightly coupled way, and solves the "computation > governance" problems on its own, including maintaining different client > environments, submiting the job, monitoring job status, fetching the > output, handling large number of concurrent client instances, watching the > bad jobs, adapt to engine version changes, etc. > It lacks a common layer of "computation middleware" between the numerous > upper-layer applications and the countless underlying engines to handle all > these "computation governance" affairs in a standardized reusable way, > that's why we started the Linkis project. > Firstly, Linkis could reduce the complexity of connectivity. Instead of > maintaining a variety of engine client environments, users now only need to > install the Linkis client, or even just HTTP client while using the REST > interface. Routing query to desired clusters could be done by simply > providing a tag. > Secondly, Links provides governance capabilities such as multi-tenancy, > concurrency control, resource management, query validation, privilege > enhancement and auditing. > Meanwhile, Linkis enables orchestration strategies such as routing, > load-balance, active-active and hybrid computation across engines (some > still under development). > > > =Rationale= > Linkis is built on distributed microservice architecture with great > scalability and extendibility. The enhancements of high concurrency and > fault tolerance make it more stable and reliable. It has already supported > many production environments with large number of daily jobs over a long > term. > Linkis's microservices are divided into 3 groups: Computation Governance > Services, Public Enhancement Services, and Microservice Governance Services. > Computation Governance Services(CGS) group is responsible for the core > process of job/request submission, preparation and execution, lifecycle > management, resource management, validation and orchestration. > Public enhancement Services(PES) group
Re: IP clearance officer for accepting Terraform
Thanks Daniel for confirming. In that case I think we don't have any outstanding concerns, all the guidance and advice we've received on this thread has been attempted and resolved. Hi Paul - I think both Daniel and Justin have been answered and we can continue. Kindly review and start a formal "[IP CLEARANCE]" thread on general@incubator as per https://incubator.apache.org/ip-clearance/ip-clearance-template.html#process tracking the IP clearance at https://github.com/apache/cloudstack/issues/5159 Regards. On Mon, Jul 12, 2021 at 1:00 PM Daniel Widdis wrote: > > I have no concerns. I am not a member of the IPMC, just an interested > participant in the conversation. > > On 7/11/21, 11:25 PM, "Rohit Yadav" wrote: > > Hi Daniel, Justin, IPMC, > > Are you happy with the answers to satisfaction? Do you have any other > questions/concerns, or can we continue with the IP clearance vote? > It has been three weeks since this thread, while the Apache CloudStack > PMC has passed the vote to accept donations in April 2021. If there > are any, can you advise by the end of tomorrow? > > Regards. > > On Sat, Jul 10, 2021 at 10:56 AM Rohit Yadav wrote: > > > > Hi Justin, > > > > Yes that's right, there's no 3rd party code in the latest codebase/tags > that is being donated. And yes, the large commits that brought the 3rd party > code in vendor directory have been all removed too. > > > > Regards. > > > > On Sat, 10 Jul, 2021, 6:26 am Justin Mclean, > wrote: > >> > >> Hi, > >> > >> > However, if you compare the changes in above commits against the > >> > repositories being donated the "vendor" directory does not exist now > >> > in both the repositories being donated: > >> > https://github.com/xanzy/go-cloudstack > >> > https://github.com/xanzy/terraform-provider-cloudstack (for example > >> > vendor removed in this commit - > >> > > https://github.com/xanzy/terraform-provider-cloudstack/commit/4db2f701592b5af74376f5b138624bff75763152) > >> > >> So what you are saying is that all of those large commits have been > removed? What I would be concerned about happening is if a large amount of > 3rd party code incorrectly gets an ASF header on it. There no issue with 3rd > party code in the repo but it must be clearly marked, have the correct non > ASF header and its license compatible with the Apache license. Looking at the > repos all headers are ASF ones, is there any 3rd party code in the donated > code? > >> > >> Thanks, > >> Justin > >> > > - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: IP clearance officer for accepting Terraform
I have no concerns. I am not a member of the IPMC, just an interested participant in the conversation. On 7/11/21, 11:25 PM, "Rohit Yadav" wrote: Hi Daniel, Justin, IPMC, Are you happy with the answers to satisfaction? Do you have any other questions/concerns, or can we continue with the IP clearance vote? It has been three weeks since this thread, while the Apache CloudStack PMC has passed the vote to accept donations in April 2021. If there are any, can you advise by the end of tomorrow? Regards. On Sat, Jul 10, 2021 at 10:56 AM Rohit Yadav wrote: > > Hi Justin, > > Yes that's right, there's no 3rd party code in the latest codebase/tags that is being donated. And yes, the large commits that brought the 3rd party code in vendor directory have been all removed too. > > Regards. > > On Sat, 10 Jul, 2021, 6:26 am Justin Mclean, wrote: >> >> Hi, >> >> > However, if you compare the changes in above commits against the >> > repositories being donated the "vendor" directory does not exist now >> > in both the repositories being donated: >> > https://github.com/xanzy/go-cloudstack >> > https://github.com/xanzy/terraform-provider-cloudstack (for example >> > vendor removed in this commit - >> > https://github.com/xanzy/terraform-provider-cloudstack/commit/4db2f701592b5af74376f5b138624bff75763152) >> >> So what you are saying is that all of those large commits have been removed? What I would be concerned about happening is if a large amount of 3rd party code incorrectly gets an ASF header on it. There no issue with 3rd party code in the repo but it must be clearly marked, have the correct non ASF header and its license compatible with the Apache license. Looking at the repos all headers are ASF ones, is there any 3rd party code in the donated code? >> >> Thanks, >> Justin >> - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: IP clearance officer for accepting Terraform
Hi Daniel, Justin, IPMC, Are you happy with the answers to satisfaction? Do you have any other questions/concerns, or can we continue with the IP clearance vote? It has been three weeks since this thread, while the Apache CloudStack PMC has passed the vote to accept donations in April 2021. If there are any, can you advise by the end of tomorrow? Regards. On Sat, Jul 10, 2021 at 10:56 AM Rohit Yadav wrote: > > Hi Justin, > > Yes that's right, there's no 3rd party code in the latest codebase/tags that > is being donated. And yes, the large commits that brought the 3rd party code > in vendor directory have been all removed too. > > Regards. > > On Sat, 10 Jul, 2021, 6:26 am Justin Mclean, wrote: >> >> Hi, >> >> > However, if you compare the changes in above commits against the >> > repositories being donated the "vendor" directory does not exist now >> > in both the repositories being donated: >> > https://github.com/xanzy/go-cloudstack >> > https://github.com/xanzy/terraform-provider-cloudstack (for example >> > vendor removed in this commit - >> > https://github.com/xanzy/terraform-provider-cloudstack/commit/4db2f701592b5af74376f5b138624bff75763152) >> >> So what you are saying is that all of those large commits have been removed? >> What I would be concerned about happening is if a large amount of 3rd party >> code incorrectly gets an ASF header on it. There no issue with 3rd party >> code in the repo but it must be clearly marked, have the correct non ASF >> header and its license compatible with the Apache license. Looking at the >> repos all headers are ASF ones, is there any 3rd party code in the donated >> code? >> >> Thanks, >> Justin >> - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
