LGTM.
Ming Wen <wenm...@apache.org> 于2022年11月27日周日 14:37写道：
>
> please don’t use qq email
>
> 刘维 <levy...@qq.com.invalid>于2022年11月26日 周六23:23写道：
>
> > Background
> >
> > Nowadays, if a plugin needs to store sensitive information, it will design
> > a separate set of encryption mechanism, which causes the problem of
> > repeated development and cumbersome management. Now we consider introducing
> > Data Encryption function, hereinafter referred to as DE, to encrypt the
> > specified information, and any module/plugin that needs to encrypt
> > sensitive information only needs to specify the fields to be encrypted on
> > display, so that transparent encryption/decryption can be performed when
> > saving/reading the information.
> >
> > Benefits
> >
> > Unify the encryption mechanism of APISIX to avoid duplication of
> > development while improving product competitiveness
> >
> > Goals
> >
> > APISIX provides DE functionality to support the use of this capability in
> > plugins and elsewhere to protect sensitive information.
> >
> > The user scenarios are as follows
> >
> >
> > Turn on the DE function in the configuration file
> >
> > apisix:
> > enable_global_data_encryption: true
> >
> > set encrypted fields
> >
> > local schema = {
> > type = "object",
> > properties = {
> > username = { type = "string" },
> > password = { type = "string", encrypted = true },
> > <==== Here
> > },
> > required = {"username", "password"},
> > }
> >
> > Set the plugin
> >
> > curl http://127.0.0.1:9180/apisix/admin/consumers -H 'X-API-KEY:
> > edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
> > {
> > "username": "foo",
> > "plugins": {
> > "basic-auth": {
> > "username": "foo",
> > "password": "bar"
> > <= Sensitive Information
> > }
> > }
> > }'
> >
> > The plugin data read directly from ETCD is encrypted
> >
> > etcdctl get /apisix/consumers/foo
> > {
> > "username":"foo",
> > "update_time":1668584767,
> > "create_time":1668584767,
> > "plugins":{
> > "basic-auth":{
> > "username":"foo",
> >
> > "password":"da61c0083b6d19ef3db2490d0da96a71572da0fa" <=
> > encrypted
> > }
> > }
> > }
> >
> > The data surface is read normally
> >
> > curl http://127.0.0.1:9180/apisix/admin/consumers -H 'X-API-KEY:
> > edd1c9f034335f136f87ad84b625c8f1'
> > {
> > "list": [
> > {
> > "createdIndex": 8627,
> > "modifiedIndex": 8627,
> > "value": {
> > "username": "foo",
> > "update_time": 1668584767,
> > "create_time": 1668584767,
> > "plugins": {
> > "basic-auth": {
> > "username": "foo",
> > "password": "bar"
> > <= Unencrypted
> > }
> > }
> > },
> > "key": "/apisix/consumers/foo"
> > }
> > ],
> > "total": 1
> > }
> > Detailed Design
> >
> >
> > Add a field in the configuration file to enable data encryption
> >
> > apisix:
> > data_encryption:
> > # use `encrypted = {fields}` in plugin schema to enable encryption
> > enable: false
> > # If not set, the default value is `false`.
> > keyring:
> > - edd1c9f0985e76a2
> > # If not set, will save origin value into etcd.
> > - qeddd145sfvddff3
> > # If set this, the keyring should be an array whose elements are
> > string, and the size is also 16, and it will encrypt fields with AES-128-CBC
> >
> > # !!! So do not
> > change it after encryption, it can't decrypt the fields have be saved if
> > you change !!
> >
> > # Only use the first
> > key to encrypt, and decrypt in the order of the array.
> >
> > Specify the fields to be encrypted in the schema of the plugin
> >
> > local consumer_schema = {
> > type = "object",
> > title = "work with consumer object",
> > properties = {
> > username = { type = "string" },
> > password = { type = "string", encrypted = true },
> > <===== here
> > },
> > required = {"username", "password"},
> > }
> >
> > local schema = {
> >
> > }
> >
> > If data encryption is enabled, the corresponding fields of the following
> > plugins will be encrypted
> >
> > jwt-auth：secret，private_key
> > basic-auth：password
> > authz-keycloak：client_secret
> > authz-casdoor：client_secret
> > openid-connect：client_secret
> > hmac-auth: secret_key
> > csrf：key
> > rocketmq-logger：secret_key
> > clickhouse-logger：password
> > error-log-logger：clickhouse.password
> > sls-logger：access_key_secret
> > google-cloud-logging：auth_config.private_key
> > elasticsearch-logger：auth.password
> > tencent-cloud-cls：secret_key
> > kafka-proxy：sasl.password
> >
> > Encryption and Decryption
> >
> >
> >
> > Commonly used algorithms AES, ChaCha20, we use AES128CBC encryption
> > algorithm, the logic is the same as ssl encryption and decryption, that is,
> > only use the first key to encrypt, if it fails to take turns to use all
> > keys to decrypt, consider reusing the ssl encryption and decryption
> > function （https://github.com/apache/apisix/blob/master/apisix/ssl.lua）
> >
> >
> >
> > Currently, only those distributed through the apisix admin interface can
> > be encrypted. Many users have developed their own admin and directly
> > manipulated storage media such as etcd, so they need to implement the
> > encryption function themselves.
> >
> >
> >
> > After the encryption function is enabled, the data that was not encrypted
> > needs to be sent down again through the interface in order to be encrypted,
> > is a refresh interface provided? Consider not providing it for now
> >
> >
> >
> > Combining the above, the original string is used directly after decryption
> > failure
> >
> >
> >
> > Admin now only uses etcd, considering that the first version only supports
> > etcd and only supports encryption of plugin parameters
> >
> >
> >
> > Encryption：
> >
> >
> > Add encryption and decryption to check_single_plugin_schema, note that
> > there are 2 different types of schema within the plugin, namely: schema and
> > consumer_schema
> > local function check_single_plugin_schema(name, plugin_conf, schema_type,
> > skip_disabled_plugin, encrypt_or_decrypt)
> > ...
> > if plugin_obj.check_schema then
> > ...
> > end
> >
> > if schema_type == core.schema.TYPE_CONSUMER then
> > ok, err = encrypt(plugin_conf,
> > plugin_obj.consumer_schema, encrypt_or_decrypt)
> > else
> > ok, err = encrypt(plugin_conf,
> > plugin_obj.schema, encrypt_or_decrypt)
> > end
> >
> > return true
> > end
> > The encryption and decryption function iterates through the corresponding
> > schema, and if it finds the encrypted = true field, it encrypts and
> > decrypts the field in the corresponding configuration
> > local function encrypt(plugin_conf, schema, encrypt_or_decrypt)
> > if schema.type == "object" then
> > for name, obj in pairs(schema.properties) do
> > encrypt(plugin_conf[name], obj,
> > encrypt_or_decrypt)
> > end
> > elseif schema.type = "array" then
> > ...
> > end
> >
> > if schema.encrypted then
> > aes128(plugin_conf)
> > end
> > end
> > Add corresponding options in check_schema and stream_check_schema, now
> > directly called in admin, no need to modify, encrypt_or_decrypt is nil by
> > default, i.e. encrypt
> > local function check_schema(plugins_conf, schema_type,
> > skip_disabled_plugin, encrypt_or_decrypt)
> > for name, plugin_conf in pairs(plugins_conf) do
> > local ok, err =
> > check_single_plugin_schema(name, plugin_conf,
> > schema_type,
> > skip_disabled_plugin, encrypt_or_decrypt)
> > if not ok then
> > return false, err
> > end
> > end
> >
> > return true
> > end
> > plugin_checker and stream_plugin_checker are used for checksumming when
> > reading data from etcd, and are decrypted by default
> > <br class="Apple-interchange-newline"><div></div>
> >
> >
> >
> >
> >
> >
> >
> > function _M.plugin_checker(item, schema_type)
> > if item.plugins then
> > return check_schema(item.plugins, schema_type,
> > true, decrypt) <=== decrypt end
> > return true
> >
> > end
>
> --
> Thanks,
> Ming Wen, Apache APISIX PMC Chair
> Twitter: _WenMing
