Hello Gil, To be precise about where the boundary sits: today, the lookup key (e.g. jdbc-password-lookup="mysql-ofbiz") lives as a plain attribute directly in entityengine.xml, the same place jdbc-username and jdbc-driver already live. EntityConfig.getJdbcPassword() reads that attribute and hands the key to the SecretProvider SPI, which is the only thing that ever calls out to the vault, fetches the real value, caches it, and refreshes it.
If Typesafe Config (PR #1285) lands in the future, that key could instead live in a HOCON override file rather than the XML attribute - but the division of labor stays the same: Typesafe Config would only ever hold which key to use, never the secret itself. The SPI would still be the only piece that resolves that key into an actual value via the vault. So in that future setup, the SPI would simply be reading the lookup key from Typesafe Config instead of from XML - everything downstream of that (the vault call, caching, rotation) is unchanged. These were my thoughts. Please let me know if you require additional information on this. Thank you! -- Kind Regards, Ashish Vijaywargiya Vice President of Operations *HotWax Systems* *Enterprise open source experts* cell: +91-9893479711 http://www.hotwaxsystems.com On Mon, Jun 22, 2026 at 10:09 PM gil.portenseigne < [email protected]> wrote: > Hi Ashish, > > Thanks for the clarification. I don't think I was positioning the two > approaches as competing either. I was rather thinking about different > scales and operational models. > > My point was that, in more basic deployments, retrieving secrets > directly from the application makes a lot of sense and provides valuable > capabilities. In larger infrastructures, secret management is often > delegated to the platform itself, which exposes credentials to > applications through environment variables or mounted files. > > I am not sure I fully understood how you envision combining the two > approaches. Could you clarify this part? From my understanding, Typesafe > Config with environment variable and the SecretProvider SPI seem to > overlap somewhat in terms of configuration value resolution, so I would > be interested to better understand the interaction between both. > > Regarding live secret invalidation, I agree that this is a strong > advantage in simpler architectures where restarting an application can > be disruptive. In Kubernetes-based environments, however, rolling > restarts can generally be performed transparently without service > interruption, so the operational benefit may depend on the deployment > model. > > Looking forward to reading more about it, and thanks again for taking > the time to explain your design. > > Thanks, > > Gil > Le 22/06/26 05:04, Ashish Vijaywargiya a écrit : > > Hello Gil, > > > > I looked into PR #1285 (OFBIZ-12401 & related issues like OFBIZ-9498, > > OFBIZ-10407) more closely, and I don't think the two are in conflict — > they > > sit at different layers: > > > > Here is my take on your points: > > > > 1. Infra-level secret retrieval vs. the Lightbend Typesafe Config > initiative > > > > - The Typesafe/Lightbend Config work is about how OFBiz assembles its > > configuration > > — HOCON files layered as overrides on top of the existing XML config, > with > > native ${VAR} / ${?VAR} substitution from the process environment. If a > > secret is exported as an env var by something external (Vault Agent > > sidecar, External Secrets Operator, a CI/CD pipeline, etc.) before the > JVM > > starts, that layer can pick it up cleanly. The actual fetching and > > lifecycle of the secret happens entirely outside OFBiz. > > > > - What I've built is the opposite direction: OFBiz itself talks to the > > vault via a small SPI at the moment a value is actually needed (DB > password > > resolution, a SystemProperty lookup), with a cache TTL and fallback > chain — so > > it works even without any sidecar/operator infra in place, and it can > pick > > up a rotated secret without a restart. > > > > Rather than positioning these as competing approaches, I'd suggest a > > concrete convergence point: a HOCON override file could set > > jdbc-password-lookup = "jdbc-password.mysql-ofbiz" (i.e., use Typesafe > > Config purely as the override mechanism for which lookup key to use), > while > > the SecretProvider SPI still does the actual vault call, caching, and > > rotation handling underneath. That way Typesafe Config stays focused on > > "how config is assembled," and the secret-manager plugins stay focused on > > "how credential-shaped values in that config get resolved and refreshed" > — > > instead of the two solving the same problem twice. > > > > 2. Live secret invalidation > > > > That's the one capability that's hard to get from a pure > > env-var/restart-based setup. Most vault rotation policies (HashiCorp > > Vault's lease TTLs, AWS Secrets Manager's automatic rotation, etc.) > expect > > consumers to re-fetch on a schedule, not to have credentials baked into a > > process that only refreshes on restart. With the cache-TTL-driven SPI, a > > rotated DB password or gateway key in the vault becomes visible to a > > running OFBiz instance on the next refresh — no coordinated restart, no > > brief window where the app is holding a stale credential the vault has > > already invalidated. > > > > Thanks again for digging in. I feel that my work on Secret Managers/Vault > > support in Apache OFBiz is worth merging — it's backward-compatible, > > opt-in, and the only approach here that gives live secret rotation > without > > a restart. > > > > Please let me know your thoughts whenever you can. > > > > =============== > > I was referring to Kubernetes secret operators, such as > > https://developer.hashicorp.com/vault/docs/deploy/kubernetes/vso > > > > In my view, in a project configuration that includes: > > * The client for whom the implementation is being carried out > > * The integrator who customizes and develops the solution > > * The operations team: which manages and monitors the infrastructure > > > > Some of the secrets must be managed solely by the OPS team, without > > being shared with the integrator; so I thought that secret management > > should take place outside of OFBiz. > > > > But not all project follow that structure :) > > =============== > > > > I am thinking about your recent email and will get back once I have > > additional thoughts. Thank you for sharing. > > > > -- > > Kind Regards, > > Ashish Vijaywargiya > > Vice President of Operations > > *HotWax Systems* > > *Enterprise open source experts* > > http://www.hotwaxsystems.com > > > > > > > > On Fri, Jun 19, 2026 at 8:16 PM gil.portenseigne via dev < > > [email protected]> wrote: > > > > > Hello Ashish, > > > > > > I tried to have a look about your contribution, that a quite big one > ;). > > > > > > I just wanted to point that if you intend to merge this weekend it is > > > quite short for people to have a look :). > > > > > > First I had some concerns about the fact that secret management, in my > > > experience, is an infra domain subject. There are tools which goals are > > > to retrieve and share those secrets into application environment. > > > > > > With the other initiative of integrating lighbend typesafe config [1], > > > that could allow another way of configuring OFBiz app, with environment > > > variables support for secret. > > > > > > That do not go against your proposal, but I guess that both can serve > > > the same purpose. Do you have some thought about that ? > > > > > > I feel like the EncryptValue value feature is for simple production > grade > > > architecture, for example an OFBiz instance hosted in one server. > > > Or maybe, since I am not familiar with PCI-DSS, SOC2, etc., those > > > present some specific requirement that make config encryption > mandatory ? > > > > > > The plus I see from this implementation is the secret invalidation > > > without restarting the application, that is quite nice. > > > > > > I will try to go deeper soon, looking forward to reading you and > others! > > > > > > Thanks, > > > > > > Gil > > > > > > [1] https://github.com/apache/ofbiz-framework/pull/1285 > > > > > > Le 17/06/26 11:43, Ashish Vijaywargiya a écrit : > > > > Hello Dev Members, > > > > > > > > I'd like to propose merging the changes from the following branches: > > > > > > > > "various-secret-manager" branch - > > > > > > > > https://github.com/ashishvijaywargiya/ofbiz-framework/tree/various-secret-manager > > > > > > > > "plugins-various-secret-manager" branch - > > > > > > > > https://github.com/ashishvijaywargiya/ofbiz-plugins/tree/plugins-various-secret-manager > > > > > > > > into trunk. > > > > > > > > I shared a detailed summary of the changes in my earlier email. In > short, > > > > it adds support for pluggable secret managers, a webtools "Encrypt > Value" > > > > screen with CSV bulk upload, and a configurable secret-value marker. > It > > > > will be an important feature for the Apache OFBiz project for > > > > compliance-sensitive deployments(PCI-DSS, SOC2, etc.). > > > > > > > > The Secret Manager functionality is disabled by default, with the > > > existing > > > > passwords.properties/ENC(...) behavior used as the fallback when no > > > > provider is configured. This means existing OFBiz installations will > > > > continue to work exactly as before, with no changes required and no > > > > breaking changes to current functionality. > > > > > > > > Unless there are concerns, I'm planning to merge these changes into > trunk > > > > over the coming weekend. Please take a look at the branch before > then and > > > > share any feedback or concerns. > > > > > > > > Here are the PRs for your kind reference: > > > > > > > > https://github.com/apache/ofbiz-framework/pull/1367 > > > > > > > > https://github.com/apache/ofbiz-plugins/pull/291 > > > > > > > > Thank you in advance for your help and review. > > > > > > > > I will also share the User Manual/ DIY(Do It Yourself) document for > > > Secret > > > > Managers setup on the OFBiz Wiki sometime in the next week. > > > > > > > > -- > > > > Kind Regards, > > > > Ashish Vijaywargiya > > > > Vice President of Operations > > > > *HotWax Systems* > > > > *Enterprise open source experts* > > > > http://www.hotwaxsystems.com > > > > > > > > > > > > On Tue, Jun 16, 2026 at 8:58 PM Ashish Vijaywargiya < > > > > [email protected]> wrote: > > > > > > > > > Hello Dev Members, > > > > > > > > > > I have updated the Secret Manager/Vault support in Apache OFBiz, > and > > > the > > > > > following changes have been made in my forked version. > > > > > > > > > > Webtools Secret Manager Screen (EncryptValue) > > > > > ---------------------------------------------- > > > > > > > > > > 1. Single-entry form with dual storage targets. > > > > > > > > > > The EncryptValue screen supports two storage targets via a radio > > > button: > > > > > SystemProperty entity and passwords.properties. For the > SystemProperty > > > > > target, Resource ID and Property ID are required; Lookup Key is > > > optional > > > > > and, when provided, wires the record to a remote secret provider. > For > > > the > > > > > passwords.properties target, two sub-cases are supported: > > > > > > > > > > Case A (Lookup Key only): writes jdbc-password.<key>=ENC(...) to > > > > > passwords.properties, covering the entityengine.xml > > > jdbc-password-lookup > > > > > use case. > > > > > > > > > > Case B (all four fields): writes <key>=ENC(...) to > > > passwords.properties, > > > > > sets <propertyId>=LOOKUP(<key>) in the source .properties file on > > > disk, and > > > > > flushes the OFBiz property cache — no server restart needed. > > > > > > > > > > 2. Dynamic labels based on target selection. > > > > > > > > > > Field labels update dynamically when the target radio button is > > > switched. > > > > > Selecting SystemProperty shows "System Resource ID > (SystemProperty)" > > > and > > > > > "System Property ID (SystemProperty)"; selecting > passwords.properties > > > shows > > > > > "Resource ID (Property File)" and "Property ID (Property File)". > This > > > > > avoids confusion about which entity or file a field refers to. > > > > > > > > > > 3. Form field persistence on validation errors. > > > > > > > > > > When a submission fails validation, the radio button selection, > > > > > systemResourceId, systemPropertyId, and lookupKey fields are > > > pre-populated > > > > > with the values the user entered. The secretValue field is > > > intentionally > > > > > cleared on error so the plain-text password is never re-rendered > in the > > > > > page. > > > > > > > > > > 4. Stale lookup key cleanup. > > > > > > > > > > When Case B is used and the user changes the lookupKey for a > property > > > that > > > > > already had a LOOKUP(...) reference in the source .properties file, > > > the old > > > > > key's entry is automatically removed from passwords.properties > before > > > the > > > > > new one is written, preventing orphaned encrypted entries. > > > > > > > > > > 5. CSV bulk upload with security pre-scan. > > > > > > > > > > A CSV upload path allows multiple secrets to be created in one > > > operation > > > > > (target, systemResourceId, systemPropertyId, lookupKey, secretValue > > > > > columns). The entire file is validated before any writes occur — > the > > > file > > > > > is rejected as a whole if any row contains unknown targets, invalid > > > > > identifier characters (only letters, digits, dots, hyphens, and > > > underscores > > > > > are allowed in identifier fields), path traversal sequences (..), > or > > > if the > > > > > file exceeds 512 KB or 500 rows. The whole-file content scan uses > > > OFBiz's > > > > > existing SecuredUpload.isValidTextContent() sandboxing mechanism to > > > block > > > > > null bytes and control characters at the Unicode code-point level. > > > > > > > > > > 6. Input validation on form and CSV upload. > > > > > > > > > > Server-side guards reject a LOOKUP(...) marker typed into the > lookupKey > > > > > field and an ENC(...) prefix typed into the secretValue field, on > both > > > the > > > > > single-entry form and the CSV bulk-upload path. A matching > client-side > > > > > JavaScript validator highlights the offending field and shows an > inline > > > > > error before the form is even submitted, preventing accidental > misuse. > > > > > > > > > > Marker and Resolver Changes > > > > > ---------------------------- > > > > > > > > > > 7. Configurable secret marker (default: LOOKUP). > > > > > > > > > > The marker used to identify secret references in property values > > > > > (previously hardcoded as SECRET) is now configurable via > > > > > secret.value.marker in security.properties. The default has been > > > changed to > > > > > LOOKUP, which aligns naturally with the existing > systemPropertyLookup > > > field > > > > > name in the SystemProperty entity and the jdbc-password-lookup > > > attribute in > > > > > entityengine.xml. Admins can change it to any uppercase word if it > > > collides > > > > > with an existing property value. > > > > > > > > > > 8. Plain key in SystemProperty.systemPropertyLookup. > > > > > > > > > > Previously, the field stored a wrapped value like > LOOKUP(smtp-key). It > > > now > > > > > stores the plain key (smtp-key) directly, since the field name > already > > > > > implies lookup semantics. EntityUtilProperties now calls the new > > > > > SecretValueResolver.resolveKey(String) method to resolve the raw > key > > > > > without needing a marker wrapper. Marker-based resolution > > > (LOOKUP(key)) in > > > > > .properties files remains unchanged. > > > > > > > > > > 9. Marker config moved to security.properties. > > > > > > > > > > secret.value.marker and secret.cache.ttl.seconds are now > documented in > > > > > security.properties rather than general.properties, which is the > more > > > > > appropriate home for secret-management configuration. > > > > > > > > > > 10. Fixed marker coupling in SecretManagerServices. > > > > > > > > > > All hardcoded "SECRET(" string literals in the writer and parser > have > > > been > > > > > replaced with SecretValueResolver.MARKER_NAME (now public), > ensuring > > > the > > > > > writer and resolver always stay in sync when an admin changes the > > > > > configured marker. > > > > > > > > > > 11. SecretValueResolver refactored. > > > > > > > > > > Cache and provider logic extracted into a new public > resolveKey(String > > > > > key) method; resolve(String) now delegates to it after parsing the > > > marker. > > > > > This eliminates duplicated cache logic and gives callers a clean > API > > > for > > > > > direct raw-key resolution without constructing a synthetic > LOOKUP(...) > > > > > wrapper. > > > > > > > > > > If you wish to review the code changes, please review the URL > below: > > > > > > > > > > > > > > https://github.com/ashishvijaywargiya/ofbiz-framework/tree/various-secret-manager > > > > > > > > > > Thank you. > > > > > > > > > > PS: I noticed that my previous email had formatting issues. > Sharing it > > > > > again. > > > > > > > > > > -- > > > > > Kind Regards, > > > > > Ashish Vijaywargiya > > > > > Vice President of Operations > > > > > *HotWax Systems* > > > > > *Enterprise open source experts* > > > > > http://www.hotwaxsystems.com > > > > > > > > > > > > > > > > > > > > On Mon, Jun 15, 2026 at 5:35 PM Ashish Vijaywargiya < > > > > > [email protected]> wrote: > > > > > > > > > >> Dear Dev Community, > > > > >> > > > > >> Over the past few days, I've been working on adding optional > Secret > > > > >> Manager/Vault integration to Apache OFBiz, and I'd like to share > it > > > with > > > > >> the Dev list for feedback before taking it further. > > > > >> > > > > >> The goal is simple: today, DB passwords, JWT/login keys, and > various > > > > >> gateway credentials mostly live as plain text in .properties > files or > > > in > > > > >> the SystemProperty entity, which is a real concern for production > and > > > > >> compliance-sensitive deployments (PCI-DSS, SOC2, etc.). This work > > > gives > > > > >> adopters a way to back those values with an external secret > manager of > > > > >> their choice, while leaving everything else untouched. > > > > >> > > > > >> To make this happen, I've added six provider plugins, each > > > implementing a > > > > >> small SecretProvider SPI: AWS Secrets Manager, GCP(Google Cloud > > > Platform) > > > > >> Secret Manager, Azure Key Vault, HashiCorp Vault by IBM, > 1Password, > > > and > > > > >> Bitwarden Secrets Manager. > > > > >> > > > > >> On the framework side, the key additions are a > > > > >> SecretProvider/SecretProviderFactory/FallbackSecretProvider setup > in > > > > >> framework/base, a ConfigCryptoUtil class for local AES-256-GCM > > > encryption > > > > >> of values as ENC(...) keyed by OFBIZ_MASTER_KEY, and a > > > SecretValueResolver > > > > >> that resolves any SECRET(key) reference in a property file or > > > > >> SystemProperty row through the configured provider, with a > > > configurable > > > > >> cache TTL. > > > > >> > > > > >> entityengine.xml now supports jdbc-password-lookup on inline-jdbc > > > > >> elements, which is resolved via SecretProviderFactory at startup, > so > > > the DB > > > > >> password never needs to appear in entityengine.xml at all. > Previously, > > > > >> passwords.properties only supported plain-text values; now those > same > > > > >> entries can be encrypted at rest using a new generateDBPassword > gradle > > > > >> task, which prompts for the value via masked stdin and writes out > the > > > > >> ENC(...) form, so no secret ever needs to be typed on the command > > > line or > > > > >> land in shell history. > > > > >> > > > > >> SystemProperty also has a new systemPropertyLookup field so > individual > > > > >> properties can opt into secret-manager resolution. > > > > >> > > > > >> I've also added unit tests (e.g. SecretValueResolverTest) covering > > > plain > > > > >> values, ENC(...) values, cached lookups, and provider > failure/fallback > > > > >> behavior, plus a generateDBPassword gradle task with the same > > > masked-prompt > > > > >> approach for setting up the local DB password. > > > > >> > > > > >> On the impact to existing implementations/configurations, this is > > > fully > > > > >> backward-compatible and opt-in only: > > > > >> > > > > >> 1) passwords.properties / entityengine.xml – Existing plain-text > > > > >> jdbc-password values continue to work exactly as before. Sites can > > > > >> optionally migrate to ENC(...) (AES-256-GCM, keyed by > > > OFBIZ_MASTER_KEY, > > > > >> generated via generateEncryptedSecret) or to a > jdbc-password-lookup > > > key > > > > >> resolved via a secret manager, but if they do nothing, behavior is > > > > >> unchanged. > > > > >> > > > > >> 2) SystemProperty – The new systemPropertyLookup field is > additive. > > > > >> Existing rows with only systemPropertyValue (plain or ENC(...)) > > > behave as > > > > >> they do today. A site only sees secret-manager resolution if it > > > explicitly > > > > >> sets systemPropertyLookup on a row. > > > > >> > > > > >> 3) Secret managers – All six provider plugins (AWS, Azure, > HashiCorp, > > > > >> GCP, 1Password, Bitwarden) are optional. None are enabled by > default, > > > with > > > > >> no new mandatory dependencies in core. You can enable one > > > plugin/provider > > > > >> at a time by changing values in the respective plugin's > > > ofbiz-component.xml > > > > >> file. > > > > >> > > > > >> 4) Fallback chain – Lookup → remote secret manager (if > configured) → > > > > >> local ENC(...)/plain value. If the remote provider is unreachable > or > > > > >> unconfigured, OFBiz falls back to the existing file/entity-based > > > value, > > > > >> resulting in no new failure mode for sites that do not adopt this > > > > >> functionality. > > > > >> > > > > >> 5) Testing – The new resolver, crypto utility, and provider > fallback > > > > >> logic are all covered by unit tests, so existing config-loading > paths > > > > >> remain verified alongside the new optional ones. > > > > >> > > > > >> Here is the source code for your kind review: > > > > >> > > > > >> Framework related changes: > > > > >> > > > > >> > > > > https://github.com/ashishvijaywargiya/ofbiz-framework/tree/various-secret-manager > > > > >> > > > > >> Plugins related changes: > > > > >> > > > > >> > > > > https://github.com/ashishvijaywargiya/ofbiz-plugins/tree/plugins-various-secret-manager > > > > >> > > > > >> Further work: > > > > >> > > > > >> 1) I am working on adding an Admin Screen in webtools where a > user can > > > > >> create secrets either in the SystemProperty entity or the > > > password.properties > > > > >> file. I will provide an option where a user can create multiple > > > secrets > > > > >> in a CSV file and upload them in bulk. The AES algorithm will > encrypt > > > > >> the password and put it either in SystemProperty or > > > password.properties > > > > >> file. > > > > >> > > > > >> 2) Another item on the roadmap is secret key/password rotation > > > handling > > > > >> from the remote service, with the corresponding values updated in > > > > >> passwords.properties or the relevant SystemProperty records. Many > > > > >> third-party services expect support for secret key rotation at > regular > > > > >> intervals for security reasons. For now, we won't provide support > for > > > > >> database password rotation in the Secret Manager, since that would > > > also > > > > >> require handling of the user password rotation on the database > side > > > itself > > > > >> (MySQL, PostgreSQL, etc.). > > > > >> > > > > >> 3) Merge the generateDBPassword and generateEncryptedSecret Gradle > > > tasks > > > > >> and make them one, i.e generateEncryptedSecret. > > > > >> > > > > >> Please let me know your thoughts/feedback on this work. I will > > > quickly accommodate > > > > >> your thoughts/ideas in the current implementation. > > > > >> > > > > >> I'm planning to commit the current work to the Apache OFBiz > > > > >> project(trunk) in the next few days. > > > > >> > > > > >> Thank you. > > > > >> > > > > >> -- > > > > >> Kind Regards, > > > > >> Ashish Vijaywargiya > > > > >> Vice President of Operations > > > > >> *HotWax Systems* > > > > >> *Enterprise open source experts* > > > > >> http://www.hotwaxsystems.com > > > > >> > > > > >> > > > >
