Hi Ashish,
it looks interesting and i can see this potentially beeing very usefull.
I definitly would stick to keeping it an opt-in feature on top of the
existing log4j instead of a replacement.
One question I am wondering about is how querying will work.
Will your PR include some type of interface for querying the logs?
Best regards,
Lukas
Am 09.07.26 um 09:36 schrieb Ashish Vijaywargiya:
Hello Dev Members,
Quick update on the structured logging work I shared yesterday - the
implementation is finalized now and I've tested it end to end.
1. json.logs.enabled and json.logs.template switches added in
framework/base/config/debug.properties, default off, default is the bundled
ECS template.
2. build.gradle reads both and passes them as JVM system properties
automatically - plain ./gradlew ofbiz picks them up, no extra flags needed.
3. log4j2.xml has one new opt-in JSON appender writing ECS formatted lines
to runtime/logs/ofbiz-json.log alongside the existing text logs, nothing
else changes.
4. Correlation fields (requestId, visitId, userLoginId) now come from a new
CorrelationValve in framework/catalina, running as a Tomcat valve ahead of
the filter chain - ControlFilter stays untouched.
5. Added CorrelationFieldProvider, a ServiceLoader based extension point -
any component can add extra fields like security group or permissions with
zero framework changes.
6. All providers get merged; a provider overwriting a core field is allowed
but logged as a warning, and a failing provider is caught and skipped
without breaking the request.
7. Tested live on a running OFBiz instance - anonymous request,
session/visit, and login correlation, plus the provider
merge/overwrite/failure cases, all confirmed from real log output.
8. Tested against two separate observability stacks to confirm the output
is genuinely portable - Elastic (Elasticsearch + Kibana + Filebeat) and
OpenSearch (OpenSearch + OpenSearch Dashboards + Fluent Bit).
9. Both ingested the same ofbiz-json.log with no changes, and I got a
working dashboard (log volume by level plus a live search) with real-time
data in each.
10. Also confirmed the template placeholder is genuinely swappable at
runtime through debug.properties, tested with a different schema, not just
the default ECS one.
Here are the screenshots of watching the ofbiz logs generated in ECS/JSON
format using following stacks (Grafana + Elastic + OpenSearch):
Grafana-Loki-Promtail(shared yesterday)
https://drive.google.com/drive/folders/1PD0BLKa4weAXEf16aegEC2DAfKLcgZhw?usp=sharing
Elasticsearch-Kibana-Filebeat
https://drive.google.com/drive/folders/1OvKrQYZZxN4GPlQA9rR6tzU54IGjv_Mf?usp=sharing
OpenSearch-OpenSearchDashboards-FluentBit
https://drive.google.com/drive/folders/1cIrnEyPS91u7GR38hdOPwt8JixFhHwdP?usp=drive_link
I will share the PR for this work soon.
Thank you.
--
Kind Regards,
Ashish Vijaywargiya
Vice President of Operations
*HotWax Systems*
*Enterprise open source experts*
http://www.hotwaxsystems.com
On Wed, Jul 8, 2026 at 9:01 PM Ashish Vijaywargiya <
[email protected]> wrote:
Hello OFBiz Dev Members,
Here I am sharing my notes on which I've been working through: opt-in
structured (ECS - Elastic Common Schema) JSON logging on top of the
existing log4j2.xml.
1. Today's OFBiz logs are plain text, so every downstream consumer needs
its own regex/grok parser before the data is even queryable - that's the
core problem this addresses.
2. Added a JSON appender to the existing
framework/base/config/log4j2.xml, writing ECS (Elastic Common Schema)
formatted lines to runtime/logs/ofbiz-json.log, alongside the current
console/ofbiz.log/error.log output - nothing existing changes shape or
content.
3. It's fully opt-in and off by default: a single json.logs.enabled=false
switch in framework/base/config/debug.properties. Flip it to true and the
default ./gradlew ofbiz command picks it up on its own - no extra flag to
remember or document.
4. ECS was chosen as the default (and only bundled) schema because it's an
industry standard that Elastic, Grafana, and most log shippers (Promtail,
Fluent Bit, Filebeat) already understand out of the box, with no custom
mapping needed.
5. log4j2.xml exposes a jsonLogTemplateUri placeholder (defaulting to the
bundled ECS template), overridable via
-Dofbiz.json.logs.template=classpath:your-template.json - so a
Splunk/CloudWatch/anything-else template can live in a separate custom
component and be swapped in with zero framework changes.
6. Every log line also carries requestId, visitId, and userLoginId
(populated in ControlFilter via Log4j2's ThreadContext/MDC), so all lines
from one HTTP request or one user visit can be pulled with a single query -
correlation the current plain-text logs don't support at all.
7. JSON-over-file also fits containerized/Kubernetes deployments better
than relying solely on local rolling text files, which assume persistent,
non-ephemeral disk.
8. The whole thing lives inside the existing log4j2.xml behind a
Select/SystemPropertyArbiter condition.
9. How to test it: Setup a local observability/ stack (Grafana + Loki +
Promtail via docker-compose) to try this out end-to-end.
I completed the first pass of the above notes. Here are the screenshots of
watching the ofbiz logs generated in ECS/JSON format using Grafana + Loki
+ Promtail.
https://drive.google.com/file/d/1SJRqFKNT0dtJ5mBP2fsC7YSKY-BP97Wo/view?usp=drive_link
https://drive.google.com/file/d/1J6RlrjnQawFFwtu4YO8V2HzlbEJD5hZ-/view?usp=drive_link
https://drive.google.com/file/d/1AKsdcGg-N1WL7k2uClUdilQoOu2P3MLL/view?usp=drive_link
https://drive.google.com/file/d/181f5kSNwfCieA_z79Z2IEjw0O7PgIPWX/view?usp=drive_link
Here are the reference documents that I considered:
- Elastic Common Schema (ECS) spec:
https://www.elastic.co/guide/en/ecs/current/index.html
- ECS field reference:
https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html
- ECS GitHub repo: https://github.com/elastic/ecs
- Log4j2 JsonTemplateLayout (the mechanism used here, includes the bundled
ECS template):
https://logging.apache.org/log4j/2.x/manual/json-template-layout.html
I will be sharing a PR soon for everyone's reference.
Please let me know your thoughts on this feature.
Thank you.
--
Kind Regards,
Ashish Vijaywargiya
Vice President of Operations
*HotWax Systems*
*Enterprise open source experts*
http://www.hotwaxsystems.com
--
Lukas Finster
Softwareentwickler & Berater
ecomify GmbH, Stralsunder Straße 63, 33605 Bielefeld
Fon: +49 521 448157-90 | Fax: +49 521 448157-99 | www.ecomify.de
Court Registration: Amtsgericht Bielefeld, HRB 41683 | CEO: Martin Becker,
Michael Brohl
