monkeyDluffy6017 opened a new issue, #8407:
URL: https://github.com/apache/apisix/issues/8407
### Description
# Background
Nowadays, if a plugin needs to store sensitive information, it will design a
separate set of encryption mechanism, which causes the problem of repeated
development and cumbersome management. Now we consider introducing Data
Encryption function, hereinafter referred to as DE, to encrypt the specified
information, and any module/plugin that needs to encrypt sensitive information
only needs to specify the fields to be encrypted on display, so that
transparent encryption/decryption can be performed when saving/reading the
information.
# Benefits
Unify the encryption mechanism of APISIX to avoid duplication of development
while improving product competitiveness
# Goals
1. Turn on the DE function in the configuration file
apisix:
enable_global_data_encryption: true
2. set encrypted fields
local schema = {
type = "object",
properties = {
username = { type = "string" },
password = { type = "string", encrypted = true }, <==== Here
},
required = {"username", "password"},
}
3. Set the plugin
curl http://127.0.0.1:9180/apisix/admin/consumers -H 'X-API-KEY:
edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"username": "foo",
"plugins": {
"basic-auth": {
"username": "foo",
"password": "bar" <= Sensitive Information
}
}
}'
4. The plugin data read directly from ETCD is encrypted
etcdctl get /apisix/consumers/foo
{
"username":"foo",
"update_time":1668584767,
"create_time":1668584767,
"plugins":{
"basic-auth":{
"username":"foo",
"password":"da61c0083b6d19ef3db2490d0da96a71572da0fa" <=
encrypted
}
}
}
5. The data surface is read normally
curl http://127.0.0.1:9180/apisix/admin/consumers -H 'X-API-KEY:
edd1c9f034335f136f87ad84b625c8f1'
{
"list": [
{
"createdIndex": 8627,
"modifiedIndex": 8627,
"value": {
"username": "foo",
"update_time": 1668584767,
"create_time": 1668584767,
"plugins": {
"basic-auth": {
"username": "foo",
"password": "bar" <= Unencrypted
}
}
},
"key": "/apisix/consumers/foo"
}
],
"total": 1
}
# Detailed Design
1. Add a field in the configuration file to enable data encryption
apisix:
data_encryption: # use `encrypted = {fields}` in plugin
schema to enable encryption
enable: false # If not set, the default value is
`false`.
keyring:
- edd1c9f0985e76a2 # If not set, will save origin value
into etcd.
- qeddd145sfvddff3 # If set this, the keyring should be an
array whose elements are string, and the size is also 16, and it will encrypt
fields with AES-128-CBC
# !!! So do not change it after
encryption, it can't decrypt the fields have be saved if you change !!
# Only use the first key to encrypt, and
decrypt in the order of the array.
2. Specify the fields to be encrypted in the schema of the plugin
local consumer_schema = {
type = "object",
title = "work with consumer object",
properties = {
username = { type = "string" },
password = { type = "string", encrypted = true }, <===== here
},
required = {"username", "password"},
}
local schema = {
}
3. If data encryption is enabled, the corresponding fields of the following
plug-ins will be encrypted
jwt-auth：secret，private_key
basic-auth：password
authz-keycloak：client_secret
authz-casdoor：client_secret
openid-connect：client_secret
hmac-auth: secret_key
csrf：key
rocketmq-logger：secret_key
clickhouse-logger：password
error-log-logger：clickhouse.password
sls-logger：access_key_secret
google-cloud-logging：auth_config.private_key
elasticsearch-logger：auth.password
tencent-cloud-cls：secret_key
kafka-proxy：sasl.password
4. Encryption and Decryption
- Commonly used algorithms AES, ChaCha20, we use AES128CBC encryption
algorithm, the logic is the same as ssl encryption and decryption, that is,
only use the first key to encrypt, if it fails to take turns to use all keys to
decrypt, consider reusing the ssl encryption and decryption function
（https://github.com/apache/apisix/blob/master/apisix/ssl.lua）
- Currently, only those distributed through the apisix admin interface can
be encrypted. Many users have developed their own admin and directly
manipulated storage media such as etcd, so they need to implement the
encryption function themselves.
- After the encryption function is enabled, the data that was not encrypted
needs to be sent down again through the interface in order to be encrypted, is
a refresh interface provided? Consider not providing it for now
- Combining the above, the original string is used directly after decryption
failure
- Admin now only uses etcd, considering that the first version only supports
etcd and only supports encryption of plugin parameters
- Encryption：
Add encryption and decryption to check_single_plugin_schema, note that there
are 2 different types of schema within the plugin, namely: schema and
consumer_schema
local function check_single_plugin_schema(name, plugin_conf, schema_type,
skip_disabled_plugin, encrypt_or_decrypt)
...
if plugin_obj.check_schema then
...
end
if schema_type == core.schema.TYPE_CONSUMER then
ok, err = encrypt(plugin_conf, plugin_obj.consumer_schema,
encrypt_or_decrypt)
else
ok, err = encrypt(plugin_conf, plugin_obj.schema, encrypt_or_decrypt)
end
return true
end
The encryption and decryption function iterates through the corresponding
schema, and if it finds the encrypted = true field, it encrypts and decrypts
the field in the corresponding configuration
local function encrypt(plugin_conf, schema, encrypt_or_decrypt)
if schema.type == "object" then
for name, obj in pairs(schema.properties) do
encrypt(plugin_conf[name], obj, encrypt_or_decrypt)
end
elseif schema.type = "array" then
...
end
if schema.encrypted then
aes128(plugin_conf)
end
end
Add corresponding options in check_schema and stream_check_schema, now
directly called in admin, no need to modify, encrypt_or_decrypt is nil by
default, i.e. encrypt
local function check_schema(plugins_conf, schema_type, skip_disabled_plugin,
encrypt_or_decrypt)
for name, plugin_conf in pairs(plugins_conf) do
local ok, err = check_single_plugin_schema(name, plugin_conf,
schema_type, skip_disabled_plugin, encrypt_or_decrypt)
if not ok then
return false, err
end
end
return true
end
plugin_checker and stream_plugin_checker are used for checksumming when
reading data from etcd, and are decrypted by default
function _M.plugin_checker(item, schema_type)
if item.plugins then
return check_schema(item.plugins, schema_type, true, decrypt) <===
decrypt
end
return true
end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
