Agree your idea, for those relatively static data, it’s really not suitable to sync them by etcd. But maybe we can enhance this idea further for a better operationability?
Chao Zhang https://github.com/tokers On November 24, 2020 at 5:36:02 PM, Ming Wen ([email protected]) wrote: hello, dev, I thought about it again. There are several problems with Apache APISIX data plane writing into etcd: 1. The json schema file is only for the dashboard project, and writing into etcd will cause confusion for users who only use Apache APISIX; 2. Writing the data plane directly to etcd is a big security risk; 3. Due to gray-scale release or multi-cluster reasons, the schema of multiple data plane nodes may be inconsistent, which will bring a lot of trouble to the investigation. So, my plan is: 1. Add the `export_schema` command to the CLI of Apache APISIX to generate files in json or yaml format; 2. Manually copy the generated files to the dashboard. what do you think? Thanks, Ming Wen, Apache APISIX PMC Chair Twitter: _WenMing YuanSheng Wang <[email protected]> 于2020年11月12日周四 下午11:56写道： > On Thu, Nov 12, 2020 at 6:06 PM Ming Wen <[email protected]> wrote: > > > Init etcd should be the behavior of the administrator in the control > plane. > > > > We can split the current CLI into data plane CLI and control plane CLI. > > > > The schema writing we are discussing now is the work of the control plane > > CLI. > > > > This is more secure. +1 > > > > > > JunXu Chen <[email protected]>于2020年11月12日 周四下午5:37写道： > > > > > Sure, I know that APISIX should be read-only to ETCD. But we also have > > > `init_etcd`. And when saving json schema to ETCD, only schema can be > > saved, > > > but custom data cannot be saved, similar to a part of `init_etcd`. So I > > > think it's OK. After that, I temporarily can't think of a better way to > > > write the schema. What do you think? > > > > > > On Thu, Nov 12, 2020 at 3:20 PM Ming Wen <[email protected]> wrote: > > > > > > > IMO, APISIX should not write etcd, just only read. > > > > > > > > JunXu Chen <[email protected]>于2020年11月12日 周四上午7:58写道： > > > > > > > > > I think the main logic should be: > > > > > > > > > > When APISIX starts, read the key `/apisix/jsonschema` of ETCD. > > > > > > > > > > If the key does not exist or is empty, then read the JSON schema > from > > > > > APISIX and save to the ETCD key `/apisix/jsonschema`. > > > > > > > > > > If the key already exists and is not empty, nothing will be done. > > > > > > > > > > APISIX provides a command line tool that can be used to manually > > update > > > > the > > > > > JSON schema, like: `apisix sync-schema`, > > > > > users can use this tool to update JSON schema when needed. > > > > > > > > > > > > > > > What do you think ? I also comment one the issue [1]. > > > > > > > > > > Any suggestion is welcome. Thanks. > > > > > > > > > > [1] > > https://github.com/apache/apisix/issues/2700#issuecomment-725730479 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 9, 2020 at 9:33 PM dabue <[email protected]> wrote: > > > > > > > > > > > agree it. > > > > > > > > > > > > > > > > > > > > > > > > [email protected] > > > > > > > > > > > > From: YuanSheng Wang > > > > > > Date: 2020-11-09 18:19 > > > > > > To: [email protected] > > > > > > Subject: Re: Re: [DISCUSS] how to get different type of plugin > > schema > > > > in > > > > > > 2.0 manager-api > > > > > > I think we can create a github issue now, let us implement it. > > > > > > > > > > > > ^_^ > > > > > > > > > > > > On Mon, Nov 9, 2020 at 11:12 AM vincixu <[email protected]> > wrote: > > > > > > > > > > > > > +1 > > > > > > > Data plane and control plane should share data with ETCD > > > > > > > > > > > > > > JunXu Chen <[email protected]> 于2020年11月4日周三 下午1:56写道： > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > On Tue, Nov 3, 2020 at 6:47 PM 刘曦冉 <[email protected]> > > wrote: > > > > > > > > > > > > > > > > > Agreeing to move the generation tool to Apisix, I have a > > > > premature > > > > > > > > > recommendation for dashboard synchronization configuration: > > via > > > > > ETCD. > > > > > > > > > The 2.0 version of dashboard and Apisix share ETCD data, > and > > we > > > > can > > > > > > > > > consider sharing configuration via ETCD as well. > > > > > > > > > This way allows Dashboard to add no additional > configuration > > > > files > > > > > > and > > > > > > > > LuA > > > > > > > > > dependencies, and schema changes are handled entirely by > > Apisix > > > > > > > > > > > > > > > > > > On 2020/11/03 03:11:57 Zhang Chao wrote: > > > > > > > > > > Then the schema compatibility relationship should be > showed > > > > > > > explicitly. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On November 3, 2020 at 10:48:53 AM, YuanSheng Wang ( > > > > > > > > [email protected]) > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Here is the original discuss issue[1] (two more months > ago) > > > > > > > > > > > > > > > > > > > > I think we can move this tool from `apisix-dashboard` to > > > > `apisix` > > > > > > > > first. > > > > > > > > > > Then the dashboard can copy the JSON schema file directly > > > from > > > > > > APISIX > > > > > > > > > > project. > > > > > > > > > > > > > > > > > > > > [1] https://github.com/apache/apisix/issues/2247 > > > > > > > > > > > > > > > > > > > > On Tue, Nov 3, 2020 at 10:08 AM Ming Wen < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > hello, dev, > > > > > > > > > > > This discussion was on github[1] at the beginning, and > > > > because > > > > > of > > > > > > > its > > > > > > > > > > > importance, I synced it to the mailing list. > > > > > > > > > > > > > > > > > > > > > > Because the dashboard project requires schema > > verification > > > > for > > > > > > > > > plug-ins, > > > > > > > > > > > upstreams, etc., and the definition of the schema is > > > written > > > > in > > > > > > the > > > > > > > > Lua > > > > > > > > > > > code of Apache APISIX. Therefore, we need a tool to > > > generate > > > > a > > > > > > json > > > > > > > > > > schema > > > > > > > > > > > file based on the Lua code of Apache APISIX so that it > > can > > > be > > > > > > used > > > > > > > by > > > > > > > > > the > > > > > > > > > > > Go and front-end code of the dashboard project. > > > > > > > > > > > > > > > > > > > > > > The current approach is: > > > > > > > > > > > 1. Integrate this tool in the dashboard project, mock > the > > > > > > > environment > > > > > > > > > and > > > > > > > > > > > dependent libraries of Apache APISIX; > > > > > > > > > > > 2. After Apache APISIX modifies the plug-in schema, the > > > code > > > > of > > > > > > the > > > > > > > > > > > dashboard project needs to be modified synchronously; > > > > > > > > > > > In addition, how do we deal with user-defined plug-ins? > > > > > > > > > > > > > > > > > > > > > > And what I hope to achieve in the end is: > > > > > > > > > > > > > > > > > > > > > > 1. The tool for synchronizing schema has nothing to do > > with > > > > the > > > > > > > > > > > dashboard project. After all, this is a product that > the > > > data > > > > > > plane > > > > > > > > > > > should > > > > > > > > > > > generate; > > > > > > > > > > > 2. When the schema of the data plane changes, there is > no > > > > need > > > > > to > > > > > > > > > modify > > > > > > > > > > > the code of this tool; > > > > > > > > > > > 3. The dashboard project should not depend on Lua. > > > > > > > > > > > > > > > > > > > > > > what do you think? > > > > > > > > > > > > > > > > > > > > > > [1] > > https://github.com/apache/apisix-dashboard/issues/605 > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Ming Wen, Apache APISIX > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > > > > > > > > 刘曦冉 > > > > > > > > > | > > > > > > > > > | > > > > > > > > > 邮箱[email protected] > > > > > > > > > | > > > > > > > > > > > > > > > > > > 签名由 网易邮箱大师 定制 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > *MembPhis* > > > > > > My GitHub: https://github.com/membphis > > > > > > Apache APISIX: https://github.com/apache/apisix > > > > > > > > > > > > > > > -- > > > > Thanks, > > > > Ming Wen, Apache APISIX & Apache SkyWalking > > > > Twitter: _WenMing > > > > > > > > > -- > > Thanks, > > Ming Wen, Apache APISIX & Apache SkyWalking > > Twitter: _WenMing > > > > > -- > > *MembPhis* > My GitHub: https://github.com/membphis > Apache APISIX: https://github.com/apache/apisix >
