I believe this proposal is excellent First of all, there are a few questions that we need clarification:
1. Are we only removing the gRPC configuration of etcd in APISIX, or are we removing the entire gRPC feature of etcd in APISIX? 2. If it's the former, the impact and scope of changes are relatively small. If it's the latter, there will be significant pressure and work for regression testing due to the broad impact. How do we control the scope of these effects? Additionally, any merged PRs that fixed this feature for etcd gRPC until now will need to be extracted for a rollback. 3. Is the 'etcd-adapter' a configuration module? As I understand it, it's essentially a refactoring of the original etcd module, creating a new adapter layer that serves as an intermediary between the existing module and the database for interactions. Our perspective is that maybe we should first hide the original gRPC functionality, or explicitly note that this feature may have blind spots. We also strongly hope that APISIX can interact with etcd through gRPC. Best regards, Fabriceli ________________________________ 发件人: Jinhua Luo <luajit...@gmail.com> 发送时间: 2023年8月3日 17:58 收件人: dev@apisix.apache.org <dev@apisix.apache.org> 主题: [DISCUSS] Proposal: APISIX: remove etcd grpc and conf server ## Background ### etcd grpc In previous versions, apisix used HTTP to connect with etcd's grpc-gateway. But it had the following disadvantages: 1. Each admin resource per worker process, e.g. routes, needs to use a dedicated TCP connection, i.e. each worker process will have more than 10 TCP connections, which increases the load on the etcd server. 2. Some extra features only exist in etcd server, not in grpc-gateway. 3. etcd official may remove grpc-gateway at some date. So we introduce grpc here to replace http. However this module has too many bugs, meanwhile, we will introduce a new conf server (maybe it's etcd-adapter) later, which will act as a real etcd server but supports different db backends, e..g postgresql. So we plan to remove etcd grpc in apisix. On the other hand, HTTP optimizations have been merged into the apisix master branch: 1. fix(etcd): reuse cli and enable keepalive https://github.com/apache/apisix/pull/9420 2. feat(config_etcd): use a single long http connection to watch all resources https://github.com/apache/apisix/pull/9456 After optimization, the HTTP way is as good as the grpc way. ### current conf server This is an internal feature added earlier and is intended to be database backend agnostic. However, the current implementation is a simple pure proxy between DP/CP and etcd. Also, this makes debugging via gdb difficult, since all handlers downstream and upstream will always be triggered. It also obscures configuration. It's worth noting that the decoupled mode doesn't work currently. Since we'll be using the new unified external config server later, we'd rather remove the currently useless dumb version. And after refactoring, all deploy modes will work fine. ### github bug issues https://github.com/apache/apisix/issues/9381 https://github.com/apache/apisix/issues/9347 https://github.com/apache/apisix/issues/8067 https://github.com/apache/apisix/issues/9212 https://github.com/apache/apisix/issues/9007 https://github.com/apache/apisix/issues/9336 https://github.com/apache/apisix/issues/9049 https://github.com/apache/apisix/issues/9582 https://github.com/apache/apisix/issues/9578 https://github.com/apache/apisix/issues/9528 ## Conculsion All in all, we wish to revert the apisix etcd access to 2.x behavior, i.e. connect etcd directly via HTTP, but that doesn't mean we lose grpc and config db variants, instead we will implement a separate conf server to provide these features.
