Hello Dev Members, Quick update on the structured logging work I shared yesterday - the implementation is finalized now and I've tested it end to end.
1. json.logs.enabled and json.logs.template switches added in framework/base/config/debug.properties, default off, default is the bundled ECS template. 2. build.gradle reads both and passes them as JVM system properties automatically - plain ./gradlew ofbiz picks them up, no extra flags needed. 3. log4j2.xml has one new opt-in JSON appender writing ECS formatted lines to runtime/logs/ofbiz-json.log alongside the existing text logs, nothing else changes. 4. Correlation fields (requestId, visitId, userLoginId) now come from a new CorrelationValve in framework/catalina, running as a Tomcat valve ahead of the filter chain - ControlFilter stays untouched. 5. Added CorrelationFieldProvider, a ServiceLoader based extension point - any component can add extra fields like security group or permissions with zero framework changes. 6. All providers get merged; a provider overwriting a core field is allowed but logged as a warning, and a failing provider is caught and skipped without breaking the request. 7. Tested live on a running OFBiz instance - anonymous request, session/visit, and login correlation, plus the provider merge/overwrite/failure cases, all confirmed from real log output. 8. Tested against two separate observability stacks to confirm the output is genuinely portable - Elastic (Elasticsearch + Kibana + Filebeat) and OpenSearch (OpenSearch + OpenSearch Dashboards + Fluent Bit). 9. Both ingested the same ofbiz-json.log with no changes, and I got a working dashboard (log volume by level plus a live search) with real-time data in each. 10. Also confirmed the template placeholder is genuinely swappable at runtime through debug.properties, tested with a different schema, not just the default ECS one. Here are the screenshots of watching the ofbiz logs generated in ECS/JSON format using following stacks (Grafana + Elastic + OpenSearch): Grafana-Loki-Promtail(shared yesterday) https://drive.google.com/drive/folders/1PD0BLKa4weAXEf16aegEC2DAfKLcgZhw?usp=sharing Elasticsearch-Kibana-Filebeat https://drive.google.com/drive/folders/1OvKrQYZZxN4GPlQA9rR6tzU54IGjv_Mf?usp=sharing OpenSearch-OpenSearchDashboards-FluentBit https://drive.google.com/drive/folders/1cIrnEyPS91u7GR38hdOPwt8JixFhHwdP?usp=drive_link I will share the PR for this work soon. Thank you. -- Kind Regards, Ashish Vijaywargiya Vice President of Operations *HotWax Systems* *Enterprise open source experts* http://www.hotwaxsystems.com On Wed, Jul 8, 2026 at 9:01 PM Ashish Vijaywargiya < [email protected]> wrote: > Hello OFBiz Dev Members, > > Here I am sharing my notes on which I've been working through: opt-in > structured (ECS - Elastic Common Schema) JSON logging on top of the > existing log4j2.xml. > > 1. Today's OFBiz logs are plain text, so every downstream consumer needs > its own regex/grok parser before the data is even queryable - that's the > core problem this addresses. > > 2. Added a JSON appender to the existing > framework/base/config/log4j2.xml, writing ECS (Elastic Common Schema) > formatted lines to runtime/logs/ofbiz-json.log, alongside the current > console/ofbiz.log/error.log output - nothing existing changes shape or > content. > > 3. It's fully opt-in and off by default: a single json.logs.enabled=false > switch in framework/base/config/debug.properties. Flip it to true and the > default ./gradlew ofbiz command picks it up on its own - no extra flag to > remember or document. > > 4. ECS was chosen as the default (and only bundled) schema because it's an > industry standard that Elastic, Grafana, and most log shippers (Promtail, > Fluent Bit, Filebeat) already understand out of the box, with no custom > mapping needed. > > 5. log4j2.xml exposes a jsonLogTemplateUri placeholder (defaulting to the > bundled ECS template), overridable via > -Dofbiz.json.logs.template=classpath:your-template.json - so a > Splunk/CloudWatch/anything-else template can live in a separate custom > component and be swapped in with zero framework changes. > > 6. Every log line also carries requestId, visitId, and userLoginId > (populated in ControlFilter via Log4j2's ThreadContext/MDC), so all lines > from one HTTP request or one user visit can be pulled with a single query - > correlation the current plain-text logs don't support at all. > > 7. JSON-over-file also fits containerized/Kubernetes deployments better > than relying solely on local rolling text files, which assume persistent, > non-ephemeral disk. > > 8. The whole thing lives inside the existing log4j2.xml behind a > Select/SystemPropertyArbiter condition. > > 9. How to test it: Setup a local observability/ stack (Grafana + Loki + > Promtail via docker-compose) to try this out end-to-end. > > I completed the first pass of the above notes. Here are the screenshots of > watching the ofbiz logs generated in ECS/JSON format using Grafana + Loki > + Promtail. > > > https://drive.google.com/file/d/1SJRqFKNT0dtJ5mBP2fsC7YSKY-BP97Wo/view?usp=drive_link > > > https://drive.google.com/file/d/1J6RlrjnQawFFwtu4YO8V2HzlbEJD5hZ-/view?usp=drive_link > > > https://drive.google.com/file/d/1AKsdcGg-N1WL7k2uClUdilQoOu2P3MLL/view?usp=drive_link > > > https://drive.google.com/file/d/181f5kSNwfCieA_z79Z2IEjw0O7PgIPWX/view?usp=drive_link > > Here are the reference documents that I considered: > > - Elastic Common Schema (ECS) spec: > https://www.elastic.co/guide/en/ecs/current/index.html > > - ECS field reference: > https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html > > - ECS GitHub repo: https://github.com/elastic/ecs > > - Log4j2 JsonTemplateLayout (the mechanism used here, includes the bundled > ECS template): > https://logging.apache.org/log4j/2.x/manual/json-template-layout.html > > I will be sharing a PR soon for everyone's reference. > > Please let me know your thoughts on this feature. > Thank you. > > -- > Kind Regards, > Ashish Vijaywargiya > Vice President of Operations > *HotWax Systems* > *Enterprise open source experts* > http://www.hotwaxsystems.com > >
