mattcasters opened a new issue, #9099:
URL: https://github.com/apache/hop/issues/9099
### What would you like to happen?
Add a keystore plugin that keeps one master key in the operating-system
secret store and exposes named secrets through a variable resolver. Logging
into the operating-system account that stored the key makes those secrets
available to every Hop process running as that user.
This gives Hop GUI, `hop-run`, and pipelines one place to resolve a secret,
and it gives Git a credential that survives a restart. Today the token typed in
**Add project → From version control** is used only for that clone
(`UsernamePasswordCredentialsProvider` with username `x-oauth-basic`) and is
then discarded. Push and pull start with no credentials and open a
username/password dialog. The answer is kept in memory on that Git session
until Hop exits. Nothing is written to a store Hop can read the next time.
#### Architecture
The operating-system store holds a single master key. Named secrets are AES2
ciphertext in a file under the Hop config folder. `hop-config.json` stays the
registry of projects and environment files and does not hold the key or the
secrets. Project files do not hold them either.
```mermaid
flowchart TB
subgraph osAccount [Operating-system account]
Store["OS secret store<br/>Windows Credential Manager<br/>macOS
Keychain<br/>Linux Secret Service"]
end
subgraph hopProcess [Hop process running as that account]
Plugin[Keystore plugin]
File["Named secrets<br/>HOP_CONFIG_FOLDER/keystore.json<br/>AES2
ciphertext"]
Resolver["Variable resolver<br/>#{hop-keystore:name}"]
Encoder[AES2 password encoder]
Dialog[Configuration perspective]
Git[Git clone, push and pull]
Runtime[Pipelines, workflows, metadata]
end
Store -->|master key| Plugin
Plugin --> File
Plugin --> Encoder
Dialog --> Plugin
Resolver --> Plugin
Git --> Resolver
Runtime --> Resolver
Runtime --> Encoder
KeyVar["HOP_AES_ENCODER_KEY"] -->|wins| Encoder
KeyFile["HOP_AES_ENCODER_KEY_FILE"] -->|wins| Encoder
```
Master-key lookup, first hit wins:
1. `HOP_AES_ENCODER_KEY`
2. `HOP_AES_ENCODER_KEY_FILE`
3. The operating-system store, service name `Apache Hop`
`AesEncoderKeyUtil` already implements the first two. The plugin adds the
third, so password fields and environment values that are already stored as
`AES2` ciphertext unlock with the same login. The default `Encrypted ` encoder
is obfuscation and is not a fallback for this store.
```mermaid
sequenceDiagram
participant User
participant Git as Git clone or push
participant Resolver as hop-keystore resolver
participant File as keystore.json
participant OS as OS secret store
User->>Git: https://github.com/org/repo.git
Git->>Resolver: #{hop-keystore:git/github.com}
Resolver->>OS: read master key
OS-->>Resolver: master key of the logged-in account
Resolver->>File: read AES2 ciphertext for git/github.com
File-->>Resolver: ciphertext
Resolver-->>Git: token
Git->>User: clone or push, no password dialog
```
When the name is missing, Git keeps the token field and the
username/password dialog it has today. A **Remember for this host** action
writes `git/<host>` through the keystore dialog's save path.
#### Requirements
**Plugin and resolver**
* New plugin, with a `@VariableResolverPlugin`. Expression:
`#{hop-keystore:<name>}`, for example `#{hop-keystore:git/github.com}`.
* Names are chosen by the user. `git/<host>` is the convention the Git
plugin uses. It is not a special case inside the resolver.
* A missing name, a locked store, or no store leaves the expression
unresolved and logs one line. The line does not contain the secret or the
master key. An empty string is not a successful resolution.
* The same resolver works in Hop GUI, `hop-run`, Hop Server, and Hop Web.
There is no separate Hop login that unlocks it.
**Master key**
* Generate a random key the first time Hop GUI needs one and the OS store is
available. Store it under the service name `Apache Hop`.
* Windows Credential Manager opens with the Windows login. No second prompt.
* macOS may ask once whether this Hop binary may read the keychain item.
After the user allows it, later launches in that account succeed.
* Linux uses the secret service unlocked by a graphical login. An SSH
session, a service account, or a container usually has no secret service.
Resolution then depends on `HOP_AES_ENCODER_KEY` or `HOP_AES_ENCODER_KEY_FILE`,
which is the path Hop Server, Hop Web, and containers already use.
* The account that matters is the operating-system user that owns the Hop
process. Another OS user on the same machine cannot read the key. A person
signing into Hop Web in a browser does not unlock the store. The account that
started the JVM does.
* Any other process running as that same user can read the entry. That is
the boundary of Credential Manager, Keychain, and the secret service.
* Copying `hop-config.json` or a project to another machine does not copy
the master key.
**Named secrets**
* Store them in `${HOP_CONFIG_FOLDER}/keystore.json`, readable and writable
by the user only. Each entry has a name, an optional username, and an AES2
ciphertext. The username is not secret. The master key is not in this file.
* Edit the list from a dialog in the configuration perspective: add,
replace, and delete. The feature is not file-only.
* The value field accepts a token. Saving encrypts it with the master key.
The dialog never shows the stored ciphertext as something to edit by hand.
**Git**
* Clone, push, and pull look up `git/<host>` from the remote URL before
asking for a password. The same entry is used at checkout, before any project
or lifecycle environment exists.
* Remembering a token from the clone dialog or the username/password dialog
creates or replaces that entry.
* The username stored on the entry is sent with the token. For GitHub the
existing `x-oauth-basic` form remains valid when the username is empty.
* The token is not written into the remote URL, `.git/config`,
`project-config.json`, or `hop-config.json`.
* A lifecycle environment can override the user entry for the enabled
session, for a deploy token on a build agent. The environment file holds a
reference such as `#{hop-keystore:...}` or `#{vault:...}`, or an `AES2` value.
Checkout does not require an environment.
**Existing secrets**
* With no explicit AES key variable or key file, the AES2 encoder uses the
OS master key. Database passwords and environment secrets already saved as
`AES2` then open for the logged-in account.
* A server that sets `HOP_AES_ENCODER_KEY_FILE` keeps using that file and
does not need the OS store.
#### Implementation notes
* `java.security.KeyStore` is a file format such as PKCS12. It is not the
operating-system store.
* Hop already depends on JNA. Put a small Hop interface in front of the
native calls so the backend can be replaced.
[`com.github.javakeyring:java-keyring`](https://central.sonatype.com/artifact/com.github.javakeyring/java-keyring)
1.0.4 (BSD-3, August 2023) talks to Credential Manager, Keychain, and the
secret service and is a candidate behind that interface. Microsoft
`credential-secure-storage` was archived in March 2025 and should not be added.
* Document the desktop first-run and the server key-file fallback in the
user manual.
### Issue Priority
Priority: 2
### Issue Component
Component: Hop Gui
Component: API
Component: Hop Config
Component: Hop Server
Component: Hop Web
Component: Documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
