matebenyovszky opened a new pull request, #45788:
URL: https://github.com/apache/superset/pull/45788
### SUMMARY
Fixes #45787. Extension Storage values written with `encrypt=True` can never
be read back.
**Cause.** `_enc_type()` returned `EncryptedType(LargeBinary, ...)`, but the
sqlalchemy_utils engines encrypt text:
- **Write:** the engine received `bytes`, so it encrypted `repr(value)`.
- **Read:** `process_result_value()` ran `bytes(decrypted_str)`, which
raises `TypeError: string argument without an encoding`.
The DAO caught the error, logged it as a possible key rotation issue and
returned `None`.
**Change.** `_enc_type()` now returns a small `EncryptedBytesType` subclass:
- **Write:** base64-encodes the payload before encryption.
- **Read:** decodes it after decryption.
- Key derivation, engines and the stored column type are unchanged.
**Compatibility.** Values written by the broken code (7.0.0rc1/rc2) still
read back. Their plaintext is `repr(bytes)`, which is never valid base64
because `'` is outside the alphabet. On a base64 error the type falls back to
`ast.literal_eval` and accepts the result only if it is `bytes`. The plaintext
was encrypted with the server's key, so this does not parse untrusted input.
**Tests.** The existing tests mock `_enc_type`, so nothing round-tripped a
real ciphertext. The new tests use real ciphers through
`ExtensionStorageDAO.set()` / `get_value()`:
- every combination of `aes` and `aes-gcm`, user-scoped and shared, JSON and
non-UTF-8 payloads;
- another user's key cannot decrypt the value;
- a value written by the old code path still reads back.
**Not in this PR:** `re-encrypt-secrets` over `extension_storage` (see
"Additional context" in #45787) is a separate change.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable.
### TESTING INSTRUCTIONS
```bash
pytest tests/unit_tests/extensions/storage
```
On the 7.0.0rc2 image:
| | New tests | Whole storage suite |
|---|---|---|
| **Before** | 10 fail (`TypeError: string argument without an encoding`) |
— |
| **After** | pass | 189 passed |
To check manually, call `ctx.storage.persistent.set(key, value, {encrypt:
true})` from an extension, then `get(key)`. Before the fix it returns `null`
and the server logs `Failed to decrypt extension storage value`; after, it
returns the stored value.
### ADDITIONAL INFORMATION
- [x] Has associated issue: Fixes #45787
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
