[
https://issues.apache.org/jira/browse/UNOMI-938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Serge Huber updated UNOMI-938:
------------------------------
Description:
h2. Background
UNOMI-139 introduces the [ApiKey|https://github.com/apache/unomi/pull/760]
model as part of the
multi-tenant platform. Each Tenant holds a list of ApiKey objects stored in
Elasticsearch/OpenSearch.
The current implementation stores the plaintext key value in the persistence
layer and returns it in
REST responses whenever the Tenant document is fetched.
h2. Problem
h3. 1. Plaintext keys at rest
{{ApiKey.key}} is a plain {{String}} field with no special treatment. Because
{{Tenant extends Item}}
and is serialized to ES by {{CustomObjectMapper}}, the raw key value is stored
unencrypted in every
shard replica, any snapshot, and any backup. A compromised ES cluster directly
yields all active
API key material.
h3. 2. Plaintext keys over the REST layer
{{ GET /tenants/{id} }} returns the full {{Tenant}} document including the
nested {{apiKeys}} list with
every raw key value, accessible to any caller holding the {{ADMINISTRATOR}}
role.
h3. 3. @JsonIgnore is not a safe stopgap
A naive fix of annotating {{ApiKey.key}} with {{@JsonIgnore}} was evaluated and
*rejected* because
Jackson's {{@JsonIgnore}} suppresses *both* serialization (Java → JSON) *and*
deserialization
(JSON → Java) on the *same* ObjectMapper instance. Unomi uses the *same*
{{CustomObjectMapper}}
for both ES persistence and the REST layer, so:
* Keys written by the migration script (raw JSON, bypasses Jackson) have a
{{key}} field in ES.
* When Java reads that document back, Jackson ignores the field → {{getKey()}}
returns {{null}}.
* Any programmatically-created key is not stored in ES at all.
* Authentication based on {{ApiKey.getKey()}} silently breaks.
h3. 4. Root cause of the mapper conflict
Unomi correctly uses separate ObjectMapper instances for different concerns:
||Mapper||Class||Used for||
|Persistence|{{CustomObjectMapper}} + {{OSCustomObjectMapper}}|ES/OS document
read/write|
|REST|CXF JAX-RS mapper ({{RestServer}})|HTTP response serialization|
|GraphQL|GraphQL servlet mapper|GraphQL response serialization|
The {{OSCustomObjectMapper}} already uses the MixIn pattern ({{OSItemMixIn}},
{{OSEventMixIn}})
to add persistence-specific behaviour. The REST mapper does not yet have a
corresponding MixIn for
{{ApiKey}}.
h2. Proposed Solution
h3. Phase 1 — Separate storage fields from presentation
Modify {{ApiKey}} to hold:
* {{keyHash}} — PBKDF2WithHmacSHA512 hash of the plaintext key, stored in ES,
*never* in REST responses.
* {{maskedKey}} — Display-safe identifier, e.g. {{"unomi_v1_****E5F2"}}, safe
to return in all contexts.
* {{key}} — Removed from the persistent entity entirely. Lives only in
{{ApiKeyCreationResult}} (a transient DTO).
{code:java}
// ApiKey.java (simplified diff)
- private String key; // plaintext — removed from entity
+ private String keyHash; // PBKDF2 hash — ES only
+ private String maskedKey; // "unomi_v1_****E5F2" — safe everywhere
{code}
h3. Phase 2 — Scope serialization per mapper using MixIns
Add {{ApiKeyRestMixIn}} to suppress {{keyHash}} from REST/GraphQL responses:
{code:java}
// new file: rest/src/main/java/.../ApiKeyRestMixIn.java
public abstract class ApiKeyRestMixIn {
@JsonIgnore abstract String getKeyHash();
@JsonIgnore abstract void setKeyHash(String keyHash);
}
{code}
Register in {{RestServer}} (one line):
{code:java}
mapper.addMixIn(ApiKey.class, ApiKeyRestMixIn.class);
{code}
The ES mapper ({{OSCustomObjectMapper}}) has no MixIn for {{ApiKey}}, so
{{keyHash}} is serialized
and deserialized normally — round-trips are preserved.
h3. Phase 3 — One-time creation response DTO
{code:java}
// ApiKeyCreationResult.java
public class ApiKeyCreationResult {
private final ApiKey apiKey; // metadata + maskedKey, no plaintext
private final String key; // plaintext — returned once, discarded
server-side
}
{code}
h3. Phase 4 — Hash service (PBKDF2WithHmacSHA512)
New {{ApiKeyHashService}} in {{services-common}}:
{code:java}
// Self-describing stored format: "iterations:base64(salt):base64(hash)"
// Iterations: 600 000 (OWASP 2023 recommendation for PBKDF2-HMAC-SHA512)
// Salt: 16 bytes (SecureRandom)
// Hash length: 256 bits
public String hash(String plaintext) { ... }
public boolean verify(String plaintext, String storedHash) {
// MessageDigest.isEqual() for constant-time comparison — prevents timing
attacks
}
{code}
Rationale for PBKDF2WithHmacSHA512 over alternatives:
* *Argon2id* is theoretically stronger but requires an external dependency
({{argon2-jvm}}) which complicates OSGi/Karaf bundle wiring.
* *bcrypt* caps input at 72 bytes; API keys are longer.
* *PBKDF2WithHmacSHA512* is built into the JDK (no new dependencies), NIST SP
800-132
compliant, and at 600 000 iterations with a 16-byte random salt provides
computationally infeasible reversal even with a GPU cluster.
h3. Phase 5 — Key format
{noformat}
Format : unomi_v1_{64-char-uppercase-hex}
Example :
unomi_v1_C606D77D1D219509637A82C062BCD17F13D6DF1501702DC396D4A12D63D4E5F2
Masked : unomi_v1_****E5F2 (prefix + **** + last 4 chars)
{noformat}
The {{unomi_v1_}} prefix enables fast rejection of obviously invalid inputs
before
the expensive hash computation and allows future algorithm migration via
version bump.
h3. Phase 6 — Authentication path update
{code:java}
// In KarafSecurityService or a new ApiKeyAuthenticator:
public boolean authenticate(String tenantId, String incomingKey) {
Tenant tenant = tenantService.getTenant(tenantId);
return tenant.getActiveApiKeys().stream()
.anyMatch(k -> apiKeyHashService.verify(incomingKey, k.getKeyHash()));
}
{code}
h3. Phase 7 — Migration script update
The migration script {{migrate-3.1.0-10-tenantInitialization.groovy}} currently
stores the
generated plaintext in {{key}}. It should be updated to:
# Compute {{keyHash}} using a migration-accessible hash utility
# Store {{keyHash}} and {{maskedKey}} in the ES document
# Remove {{key}} from the indexed JSON
The script already prints generated key values to the console and writes them to
{{${karaf.data}/migration/secrets/}} (24-hour TTL). No change to that mechanism
—
the plaintext is already surfaced once to the operator.
h2. Implementation Plan
|| # || Task || File(s) || Notes ||
| 1 | Remove {{key}} field; add {{keyHash}} + {{maskedKey}} |
{{api/.../ApiKey.java}} | Breaking change — see compatibility note |
| 2 | Update {{Tenant.java}} helpers to return {{ApiKey}} objects not
{{String}} keys | {{api/.../Tenant.java}} | {{getActivePrivateKey()}} /
{{getActivePublicKey()}} |
| 3 | Add {{ApiKeyCreationResult}} DTO | {{api/.../ApiKeyCreationResult.java}}
| New file |
| 4 | Add {{ApiKeyHashService}} interface |
{{api/.../services/ApiKeyHashService.java}} | New file |
| 5 | Implement {{ApiKeyHashServiceImpl}} (PBKDF2) |
{{services-common/.../ApiKeyHashServiceImpl.java}} | New file |
| 6 | Add {{ApiKeyRestMixIn}} | {{rest/.../ApiKeyRestMixIn.java}} | New file |
| 7 | Register MixIn in {{RestServer}} | {{rest/.../RestServer.java}} | +1 line
|
| 8 | Update {{TenantService.generateApiKey()}} |
{{services/.../TenantServiceImpl.java}} | Returns {{ApiKeyCreationResult}} |
| 9 | Update REST endpoint for key creation/rotation |
{{rest/.../TenantEndpoint.java}} | Response type changes |
| 10 | Update authentication to use hash comparison |
{{services-common/.../KarafSecurityService.java}} | Replace {{getKey()}} with
{{verify()}} |
| 11 | Update migration script |
{{tools/.../migrate-3.1.0-10-tenantInitialization.groovy}} | Hash on the Groovy
side |
| 12 | Add {{ApiKeyHashServiceImplTest}} | {{services-common/src/test/...}} |
Hash/verify round-trip, timing attack |
h2. Backward Compatibility
Existing ES documents written by the 3.1.0 migration script contain a {{key}}
field (plaintext).
When this change is deployed, those documents will be read by Jackson but
{{key}} is no longer
on the entity — Jackson's default {{FAIL_ON_UNKNOWN_PROPERTIES = false}} means
the field is
silently ignored. No migration is needed to *read* existing data.
However, existing keys stored as {{key}} (plaintext) will have {{keyHash =
null}} after this
change deploys. A one-time re-hashing step is needed:
* Add a Karaf shell command {{unomi:rehash-api-keys}} that reads all {{Tenant}}
documents,
computes {{keyHash}} from the stored plaintext {{key}} (if present), and
removes {{key}}.
* Or include as migration step {{migrate-3.1.0-20-hashApiKeys.groovy}}.
h2. Security Considerations
* The PBKDF2 verify path uses {{MessageDigest.isEqual()}} (constant-time) to
prevent timing attacks.
* The {{keyHash}} field is suppressed from REST and GraphQL via MixIn — not
just annotations —
so it cannot leak through any future ObjectMapper copy that forgets to
re-apply annotations.
* The {{maskedKey}} (last 4 chars) must be unique enough for identification but
not enough for
brute-force. 4 hex chars = 65 536 possibilities — acceptable for display
only, not security.
* Key rotation must invalidate the old {{keyHash}} atomically with writing the
new one.
* The plaintext is held in a Java {{String}} for the duration of the creation
request.
Consider {{char[]}} + explicit zeroing for highest-security deployments
(limited in practice
because the JVM GC can move objects).
h2. Acceptance Criteria
* {{ApiKey}} has no {{key}} field; {{keyHash}} is stored in ES and never
appears in any REST or
GraphQL response.
* {{ApiKeyCreationResult}} returns the plaintext key exactly once; subsequent
{{GET /tenants/{id}}}
show only {{maskedKey}}.
* {{ApiKeyHashService.verify()}} passes timing-attack resistance test (response
time does not
differ between valid and invalid keys by more than JVM noise).
* Authentication succeeds with a correct key and fails with an incorrect key or
a key belonging
to a different tenant.
* {{OSCustomObjectMapper}} round-trips {{keyHash}} through ES without loss.
* {{ApiKeyRestMixIn}} suppresses {{keyHash}} in REST responses.
* All existing ITs pass after the re-hashing migration step.
h2. Definition of Done
* All items in the implementation plan are complete.
* Unit tests for {{ApiKeyHashServiceImpl}} cover: hash, verify, wrong key,
wrong hash format,
null inputs, and confirm no timing leak.
* Integration test covers: key generation → storage → authentication → rotation.
* {{migrate-3.1.0-20-hashApiKeys.groovy}} (or {{unomi:rehash-api-keys}} shell
command) is
documented in the migration guide.
* Security review confirms no plaintext in ES snapshots or REST responses.
was:
h2. Background
UNOMI-139 introduces the [ApiKey|https://github.com/apache/unomi/pull/760]
model as part of the
multi-tenant platform. Each Tenant holds a list of ApiKey objects stored in
Elasticsearch/OpenSearch.
The current implementation stores the plaintext key value in the persistence
layer and returns it in
REST responses whenever the Tenant document is fetched.
h2. Problem
h3. 1. Plaintext keys at rest
{{ApiKey.key}} is a plain {{String}} field with no special treatment. Because
{{Tenant extends Item}}
and is serialized to ES by {{CustomObjectMapper}}, the raw key value is stored
unencrypted in every
shard replica, any snapshot, and any backup. A compromised ES cluster directly
yields all active
API key material.
h3. 2. Plaintext keys over the REST layer
{{GET /tenants/{id}}} returns the full {{Tenant}} document including the nested
{{apiKeys}} list with
every raw key value, accessible to any caller holding the {{ADMINISTRATOR}}
role.
h3. 3. @JsonIgnore is not a safe stopgap
A naive fix of annotating {{ApiKey.key}} with {{@JsonIgnore}} was evaluated and
*rejected* because
Jackson's {{@JsonIgnore}} suppresses *both* serialization (Java → JSON) *and*
deserialization
(JSON → Java) on the *same* ObjectMapper instance. Unomi uses the *same*
{{CustomObjectMapper}}
for both ES persistence and the REST layer, so:
* Keys written by the migration script (raw JSON, bypasses Jackson) have a
{{key}} field in ES.
* When Java reads that document back, Jackson ignores the field → {{getKey()}}
returns {{null}}.
* Any programmatically-created key is not stored in ES at all.
* Authentication based on {{ApiKey.getKey()}} silently breaks.
h3. 4. Root cause of the mapper conflict
Unomi correctly uses separate ObjectMapper instances for different concerns:
||Mapper||Class||Used for||
|Persistence|{{CustomObjectMapper}} + {{OSCustomObjectMapper}}|ES/OS document
read/write|
|REST|CXF JAX-RS mapper ({{RestServer}})|HTTP response serialization|
|GraphQL|GraphQL servlet mapper|GraphQL response serialization|
The {{OSCustomObjectMapper}} already uses the MixIn pattern ({{OSItemMixIn}},
{{OSEventMixIn}})
to add persistence-specific behaviour. The REST mapper does not yet have a
corresponding MixIn for
{{ApiKey}}.
h2. Proposed Solution
h3. Phase 1 — Separate storage fields from presentation
Modify {{ApiKey}} to hold:
* {{keyHash}} — PBKDF2WithHmacSHA512 hash of the plaintext key, stored in ES,
*never* in REST responses.
* {{maskedKey}} — Display-safe identifier, e.g. {{"unomi_v1_****E5F2"}}, safe
to return in all contexts.
* {{key}} — Removed from the persistent entity entirely. Lives only in
{{ApiKeyCreationResult}} (a transient DTO).
{code:java}
// ApiKey.java (simplified diff)
- private String key; // plaintext — removed from entity
+ private String keyHash; // PBKDF2 hash — ES only
+ private String maskedKey; // "unomi_v1_****E5F2" — safe everywhere
{code}
h3. Phase 2 — Scope serialization per mapper using MixIns
Add {{ApiKeyRestMixIn}} to suppress {{keyHash}} from REST/GraphQL responses:
{code:java}
// new file: rest/src/main/java/.../ApiKeyRestMixIn.java
public abstract class ApiKeyRestMixIn {
@JsonIgnore abstract String getKeyHash();
@JsonIgnore abstract void setKeyHash(String keyHash);
}
{code}
Register in {{RestServer}} (one line):
{code:java}
mapper.addMixIn(ApiKey.class, ApiKeyRestMixIn.class);
{code}
The ES mapper ({{OSCustomObjectMapper}}) has no MixIn for {{ApiKey}}, so
{{keyHash}} is serialized
and deserialized normally — round-trips are preserved.
h3. Phase 3 — One-time creation response DTO
{code:java}
// ApiKeyCreationResult.java
public class ApiKeyCreationResult {
private final ApiKey apiKey; // metadata + maskedKey, no plaintext
private final String key; // plaintext — returned once, discarded
server-side
}
{code}
h3. Phase 4 — Hash service (PBKDF2WithHmacSHA512)
New {{ApiKeyHashService}} in {{services-common}}:
{code:java}
// Self-describing stored format: "iterations:base64(salt):base64(hash)"
// Iterations: 600 000 (OWASP 2023 recommendation for PBKDF2-HMAC-SHA512)
// Salt: 16 bytes (SecureRandom)
// Hash length: 256 bits
public String hash(String plaintext) { ... }
public boolean verify(String plaintext, String storedHash) {
// MessageDigest.isEqual() for constant-time comparison — prevents timing
attacks
}
{code}
Rationale for PBKDF2WithHmacSHA512 over alternatives:
* *Argon2id* is theoretically stronger but requires an external dependency
({{argon2-jvm}}) which complicates OSGi/Karaf bundle wiring.
* *bcrypt* caps input at 72 bytes; API keys are longer.
* *PBKDF2WithHmacSHA512* is built into the JDK (no new dependencies), NIST SP
800-132
compliant, and at 600 000 iterations with a 16-byte random salt provides
computationally infeasible reversal even with a GPU cluster.
h3. Phase 5 — Key format
{noformat}
Format : unomi_v1_{64-char-uppercase-hex}
Example :
unomi_v1_C606D77D1D219509637A82C062BCD17F13D6DF1501702DC396D4A12D63D4E5F2
Masked : unomi_v1_****E5F2 (prefix + **** + last 4 chars)
{noformat}
The {{unomi_v1_}} prefix enables fast rejection of obviously invalid inputs
before
the expensive hash computation and allows future algorithm migration via
version bump.
h3. Phase 6 — Authentication path update
{code:java}
// In KarafSecurityService or a new ApiKeyAuthenticator:
public boolean authenticate(String tenantId, String incomingKey) {
Tenant tenant = tenantService.getTenant(tenantId);
return tenant.getActiveApiKeys().stream()
.anyMatch(k -> apiKeyHashService.verify(incomingKey, k.getKeyHash()));
}
{code}
h3. Phase 7 — Migration script update
The migration script {{migrate-3.1.0-10-tenantInitialization.groovy}} currently
stores the
generated plaintext in {{key}}. It should be updated to:
# Compute {{keyHash}} using a migration-accessible hash utility
# Store {{keyHash}} and {{maskedKey}} in the ES document
# Remove {{key}} from the indexed JSON
The script already prints generated key values to the console and writes them to
{{${karaf.data}/migration/secrets/}} (24-hour TTL). No change to that mechanism
—
the plaintext is already surfaced once to the operator.
h2. Implementation Plan
|| # || Task || File(s) || Notes ||
| 1 | Remove {{key}} field; add {{keyHash}} + {{maskedKey}} |
{{api/.../ApiKey.java}} | Breaking change — see compatibility note |
| 2 | Update {{Tenant.java}} helpers to return {{ApiKey}} objects not
{{String}} keys | {{api/.../Tenant.java}} | {{getActivePrivateKey()}} /
{{getActivePublicKey()}} |
| 3 | Add {{ApiKeyCreationResult}} DTO | {{api/.../ApiKeyCreationResult.java}}
| New file |
| 4 | Add {{ApiKeyHashService}} interface |
{{api/.../services/ApiKeyHashService.java}} | New file |
| 5 | Implement {{ApiKeyHashServiceImpl}} (PBKDF2) |
{{services-common/.../ApiKeyHashServiceImpl.java}} | New file |
| 6 | Add {{ApiKeyRestMixIn}} | {{rest/.../ApiKeyRestMixIn.java}} | New file |
| 7 | Register MixIn in {{RestServer}} | {{rest/.../RestServer.java}} | +1 line
|
| 8 | Update {{TenantService.generateApiKey()}} |
{{services/.../TenantServiceImpl.java}} | Returns {{ApiKeyCreationResult}} |
| 9 | Update REST endpoint for key creation/rotation |
{{rest/.../TenantEndpoint.java}} | Response type changes |
| 10 | Update authentication to use hash comparison |
{{services-common/.../KarafSecurityService.java}} | Replace {{getKey()}} with
{{verify()}} |
| 11 | Update migration script |
{{tools/.../migrate-3.1.0-10-tenantInitialization.groovy}} | Hash on the Groovy
side |
| 12 | Add {{ApiKeyHashServiceImplTest}} | {{services-common/src/test/...}} |
Hash/verify round-trip, timing attack |
h2. Backward Compatibility
Existing ES documents written by the 3.1.0 migration script contain a {{key}}
field (plaintext).
When this change is deployed, those documents will be read by Jackson but
{{key}} is no longer
on the entity — Jackson's default {{FAIL_ON_UNKNOWN_PROPERTIES = false}} means
the field is
silently ignored. No migration is needed to *read* existing data.
However, existing keys stored as {{key}} (plaintext) will have {{keyHash =
null}} after this
change deploys. A one-time re-hashing step is needed:
* Add a Karaf shell command {{unomi:rehash-api-keys}} that reads all {{Tenant}}
documents,
computes {{keyHash}} from the stored plaintext {{key}} (if present), and
removes {{key}}.
* Or include as migration step {{migrate-3.1.0-20-hashApiKeys.groovy}}.
h2. Security Considerations
* The PBKDF2 verify path uses {{MessageDigest.isEqual()}} (constant-time) to
prevent timing attacks.
* The {{keyHash}} field is suppressed from REST and GraphQL via MixIn — not
just annotations —
so it cannot leak through any future ObjectMapper copy that forgets to
re-apply annotations.
* The {{maskedKey}} (last 4 chars) must be unique enough for identification but
not enough for
brute-force. 4 hex chars = 65 536 possibilities — acceptable for display
only, not security.
* Key rotation must invalidate the old {{keyHash}} atomically with writing the
new one.
* The plaintext is held in a Java {{String}} for the duration of the creation
request.
Consider {{char[]}} + explicit zeroing for highest-security deployments
(limited in practice
because the JVM GC can move objects).
h2. Acceptance Criteria
* {{ApiKey}} has no {{key}} field; {{keyHash}} is stored in ES and never
appears in any REST or
GraphQL response.
* {{ApiKeyCreationResult}} returns the plaintext key exactly once; subsequent
{{GET /tenants/{id}}}
show only {{maskedKey}}.
* {{ApiKeyHashService.verify()}} passes timing-attack resistance test (response
time does not
differ between valid and invalid keys by more than JVM noise).
* Authentication succeeds with a correct key and fails with an incorrect key or
a key belonging
to a different tenant.
* {{OSCustomObjectMapper}} round-trips {{keyHash}} through ES without loss.
* {{ApiKeyRestMixIn}} suppresses {{keyHash}} in REST responses.
* All existing ITs pass after the re-hashing migration step.
h2. Definition of Done
* All items in the implementation plan are complete.
* Unit tests for {{ApiKeyHashServiceImpl}} cover: hash, verify, wrong key,
wrong hash format,
null inputs, and confirm no timing leak.
* Integration test covers: key generation → storage → authentication → rotation.
* {{migrate-3.1.0-20-hashApiKeys.groovy}} (or {{unomi:rehash-api-keys}} shell
command) is
documented in the migration guide.
* Security review confirms no plaintext in ES snapshots or REST responses.
> Hash API keys at rest using PBKDF2 + MixIn-scoped serialization
> ---------------------------------------------------------------
>
> Key: UNOMI-938
> URL: https://issues.apache.org/jira/browse/UNOMI-938
> Project: Apache Unomi
> Issue Type: Sub-task
> Components: unomi(-core)
> Affects Versions: unomi-3.1.0
> Reporter: Serge Huber
> Assignee: Serge Huber
> Priority: Major
> Fix For: unomi-3.1.0
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> h2. Background
> UNOMI-139 introduces the [ApiKey|https://github.com/apache/unomi/pull/760]
> model as part of the
> multi-tenant platform. Each Tenant holds a list of ApiKey objects stored in
> Elasticsearch/OpenSearch.
> The current implementation stores the plaintext key value in the persistence
> layer and returns it in
> REST responses whenever the Tenant document is fetched.
> h2. Problem
> h3. 1. Plaintext keys at rest
> {{ApiKey.key}} is a plain {{String}} field with no special treatment. Because
> {{Tenant extends Item}}
> and is serialized to ES by {{CustomObjectMapper}}, the raw key value is
> stored unencrypted in every
> shard replica, any snapshot, and any backup. A compromised ES cluster
> directly yields all active
> API key material.
> h3. 2. Plaintext keys over the REST layer
> {{ GET /tenants/{id} }} returns the full {{Tenant}} document including the
> nested {{apiKeys}} list with
> every raw key value, accessible to any caller holding the {{ADMINISTRATOR}}
> role.
> h3. 3. @JsonIgnore is not a safe stopgap
> A naive fix of annotating {{ApiKey.key}} with {{@JsonIgnore}} was evaluated
> and *rejected* because
> Jackson's {{@JsonIgnore}} suppresses *both* serialization (Java → JSON) *and*
> deserialization
> (JSON → Java) on the *same* ObjectMapper instance. Unomi uses the *same*
> {{CustomObjectMapper}}
> for both ES persistence and the REST layer, so:
> * Keys written by the migration script (raw JSON, bypasses Jackson) have a
> {{key}} field in ES.
> * When Java reads that document back, Jackson ignores the field →
> {{getKey()}} returns {{null}}.
> * Any programmatically-created key is not stored in ES at all.
> * Authentication based on {{ApiKey.getKey()}} silently breaks.
> h3. 4. Root cause of the mapper conflict
> Unomi correctly uses separate ObjectMapper instances for different concerns:
> ||Mapper||Class||Used for||
> |Persistence|{{CustomObjectMapper}} + {{OSCustomObjectMapper}}|ES/OS document
> read/write|
> |REST|CXF JAX-RS mapper ({{RestServer}})|HTTP response serialization|
> |GraphQL|GraphQL servlet mapper|GraphQL response serialization|
> The {{OSCustomObjectMapper}} already uses the MixIn pattern ({{OSItemMixIn}},
> {{OSEventMixIn}})
> to add persistence-specific behaviour. The REST mapper does not yet have a
> corresponding MixIn for
> {{ApiKey}}.
> h2. Proposed Solution
> h3. Phase 1 — Separate storage fields from presentation
> Modify {{ApiKey}} to hold:
> * {{keyHash}} — PBKDF2WithHmacSHA512 hash of the plaintext key, stored in ES,
> *never* in REST responses.
> * {{maskedKey}} — Display-safe identifier, e.g. {{"unomi_v1_****E5F2"}}, safe
> to return in all contexts.
> * {{key}} — Removed from the persistent entity entirely. Lives only in
> {{ApiKeyCreationResult}} (a transient DTO).
> {code:java}
> // ApiKey.java (simplified diff)
> - private String key; // plaintext — removed from entity
> + private String keyHash; // PBKDF2 hash — ES only
> + private String maskedKey; // "unomi_v1_****E5F2" — safe everywhere
> {code}
> h3. Phase 2 — Scope serialization per mapper using MixIns
> Add {{ApiKeyRestMixIn}} to suppress {{keyHash}} from REST/GraphQL responses:
> {code:java}
> // new file: rest/src/main/java/.../ApiKeyRestMixIn.java
> public abstract class ApiKeyRestMixIn {
> @JsonIgnore abstract String getKeyHash();
> @JsonIgnore abstract void setKeyHash(String keyHash);
> }
> {code}
> Register in {{RestServer}} (one line):
> {code:java}
> mapper.addMixIn(ApiKey.class, ApiKeyRestMixIn.class);
> {code}
> The ES mapper ({{OSCustomObjectMapper}}) has no MixIn for {{ApiKey}}, so
> {{keyHash}} is serialized
> and deserialized normally — round-trips are preserved.
> h3. Phase 3 — One-time creation response DTO
> {code:java}
> // ApiKeyCreationResult.java
> public class ApiKeyCreationResult {
> private final ApiKey apiKey; // metadata + maskedKey, no plaintext
> private final String key; // plaintext — returned once, discarded
> server-side
> }
> {code}
> h3. Phase 4 — Hash service (PBKDF2WithHmacSHA512)
> New {{ApiKeyHashService}} in {{services-common}}:
> {code:java}
> // Self-describing stored format: "iterations:base64(salt):base64(hash)"
> // Iterations: 600 000 (OWASP 2023 recommendation for PBKDF2-HMAC-SHA512)
> // Salt: 16 bytes (SecureRandom)
> // Hash length: 256 bits
> public String hash(String plaintext) { ... }
> public boolean verify(String plaintext, String storedHash) {
> // MessageDigest.isEqual() for constant-time comparison — prevents timing
> attacks
> }
> {code}
> Rationale for PBKDF2WithHmacSHA512 over alternatives:
> * *Argon2id* is theoretically stronger but requires an external dependency
> ({{argon2-jvm}}) which complicates OSGi/Karaf bundle wiring.
> * *bcrypt* caps input at 72 bytes; API keys are longer.
> * *PBKDF2WithHmacSHA512* is built into the JDK (no new dependencies), NIST SP
> 800-132
> compliant, and at 600 000 iterations with a 16-byte random salt provides
> computationally infeasible reversal even with a GPU cluster.
> h3. Phase 5 — Key format
> {noformat}
> Format : unomi_v1_{64-char-uppercase-hex}
> Example :
> unomi_v1_C606D77D1D219509637A82C062BCD17F13D6DF1501702DC396D4A12D63D4E5F2
> Masked : unomi_v1_****E5F2 (prefix + **** + last 4 chars)
> {noformat}
> The {{unomi_v1_}} prefix enables fast rejection of obviously invalid inputs
> before
> the expensive hash computation and allows future algorithm migration via
> version bump.
> h3. Phase 6 — Authentication path update
> {code:java}
> // In KarafSecurityService or a new ApiKeyAuthenticator:
> public boolean authenticate(String tenantId, String incomingKey) {
> Tenant tenant = tenantService.getTenant(tenantId);
> return tenant.getActiveApiKeys().stream()
> .anyMatch(k -> apiKeyHashService.verify(incomingKey, k.getKeyHash()));
> }
> {code}
> h3. Phase 7 — Migration script update
> The migration script {{migrate-3.1.0-10-tenantInitialization.groovy}}
> currently stores the
> generated plaintext in {{key}}. It should be updated to:
> # Compute {{keyHash}} using a migration-accessible hash utility
> # Store {{keyHash}} and {{maskedKey}} in the ES document
> # Remove {{key}} from the indexed JSON
> The script already prints generated key values to the console and writes them
> to
> {{${karaf.data}/migration/secrets/}} (24-hour TTL). No change to that
> mechanism —
> the plaintext is already surfaced once to the operator.
> h2. Implementation Plan
> || # || Task || File(s) || Notes ||
> | 1 | Remove {{key}} field; add {{keyHash}} + {{maskedKey}} |
> {{api/.../ApiKey.java}} | Breaking change — see compatibility note |
> | 2 | Update {{Tenant.java}} helpers to return {{ApiKey}} objects not
> {{String}} keys | {{api/.../Tenant.java}} | {{getActivePrivateKey()}} /
> {{getActivePublicKey()}} |
> | 3 | Add {{ApiKeyCreationResult}} DTO |
> {{api/.../ApiKeyCreationResult.java}} | New file |
> | 4 | Add {{ApiKeyHashService}} interface |
> {{api/.../services/ApiKeyHashService.java}} | New file |
> | 5 | Implement {{ApiKeyHashServiceImpl}} (PBKDF2) |
> {{services-common/.../ApiKeyHashServiceImpl.java}} | New file |
> | 6 | Add {{ApiKeyRestMixIn}} | {{rest/.../ApiKeyRestMixIn.java}} | New file |
> | 7 | Register MixIn in {{RestServer}} | {{rest/.../RestServer.java}} | +1
> line |
> | 8 | Update {{TenantService.generateApiKey()}} |
> {{services/.../TenantServiceImpl.java}} | Returns {{ApiKeyCreationResult}} |
> | 9 | Update REST endpoint for key creation/rotation |
> {{rest/.../TenantEndpoint.java}} | Response type changes |
> | 10 | Update authentication to use hash comparison |
> {{services-common/.../KarafSecurityService.java}} | Replace {{getKey()}} with
> {{verify()}} |
> | 11 | Update migration script |
> {{tools/.../migrate-3.1.0-10-tenantInitialization.groovy}} | Hash on the
> Groovy side |
> | 12 | Add {{ApiKeyHashServiceImplTest}} | {{services-common/src/test/...}} |
> Hash/verify round-trip, timing attack |
> h2. Backward Compatibility
> Existing ES documents written by the 3.1.0 migration script contain a {{key}}
> field (plaintext).
> When this change is deployed, those documents will be read by Jackson but
> {{key}} is no longer
> on the entity — Jackson's default {{FAIL_ON_UNKNOWN_PROPERTIES = false}}
> means the field is
> silently ignored. No migration is needed to *read* existing data.
> However, existing keys stored as {{key}} (plaintext) will have {{keyHash =
> null}} after this
> change deploys. A one-time re-hashing step is needed:
> * Add a Karaf shell command {{unomi:rehash-api-keys}} that reads all
> {{Tenant}} documents,
> computes {{keyHash}} from the stored plaintext {{key}} (if present), and
> removes {{key}}.
> * Or include as migration step {{migrate-3.1.0-20-hashApiKeys.groovy}}.
> h2. Security Considerations
> * The PBKDF2 verify path uses {{MessageDigest.isEqual()}} (constant-time) to
> prevent timing attacks.
> * The {{keyHash}} field is suppressed from REST and GraphQL via MixIn — not
> just annotations —
> so it cannot leak through any future ObjectMapper copy that forgets to
> re-apply annotations.
> * The {{maskedKey}} (last 4 chars) must be unique enough for identification
> but not enough for
> brute-force. 4 hex chars = 65 536 possibilities — acceptable for display
> only, not security.
> * Key rotation must invalidate the old {{keyHash}} atomically with writing
> the new one.
> * The plaintext is held in a Java {{String}} for the duration of the creation
> request.
> Consider {{char[]}} + explicit zeroing for highest-security deployments
> (limited in practice
> because the JVM GC can move objects).
> h2. Acceptance Criteria
> * {{ApiKey}} has no {{key}} field; {{keyHash}} is stored in ES and never
> appears in any REST or
> GraphQL response.
> * {{ApiKeyCreationResult}} returns the plaintext key exactly once; subsequent
> {{GET /tenants/{id}}}
> show only {{maskedKey}}.
> * {{ApiKeyHashService.verify()}} passes timing-attack resistance test
> (response time does not
> differ between valid and invalid keys by more than JVM noise).
> * Authentication succeeds with a correct key and fails with an incorrect key
> or a key belonging
> to a different tenant.
> * {{OSCustomObjectMapper}} round-trips {{keyHash}} through ES without loss.
> * {{ApiKeyRestMixIn}} suppresses {{keyHash}} in REST responses.
> * All existing ITs pass after the re-hashing migration step.
> h2. Definition of Done
> * All items in the implementation plan are complete.
> * Unit tests for {{ApiKeyHashServiceImpl}} cover: hash, verify, wrong key,
> wrong hash format,
> null inputs, and confirm no timing leak.
> * Integration test covers: key generation → storage → authentication →
> rotation.
> * {{migrate-3.1.0-20-hashApiKeys.groovy}} (or {{unomi:rehash-api-keys}} shell
> command) is
> documented in the migration guide.
> * Security review confirms no plaintext in ES snapshots or REST responses.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
