[jira] [Updated] (UNOMI-918) Migrate from custom.system.properties to Karaf Native Environment Variable and System Property Override System

2025-11-27 Thread Serge Huber (Jira)


 [ 
https://issues.apache.org/jira/browse/UNOMI-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Serge Huber updated UNOMI-918:
--
Description: 
h2. Summary

Plan and execute migration from Unomi's custom {{custom.system.properties}} 
file to Karaf's native environment variable and system property override 
mechanism. This will simplify configuration management, align with Karaf best 
practices, and eliminate the need for a custom configuration loading mechanism.
h2. Current State
h3. How custom.system.properties Works

Apache Unomi currently uses a centralized configuration file 
{{$KARAF_HOME/etc/custom.system.properties}} that:
 # Contains ~203 system properties with environment variable references using 
the pattern: {{{}org.apache.unomi.{*}=$\{env:UNOMI_{*}:-defaultValue{
 # Uses Karaf's {{${optionals}=unomi.custom.system.properties}} mechanism to 
load user overrides from {{$KARAF_HOME/etc/unomi.custom.system.properties}}
 # Provides variable substitution in {{.cfg}} files using 
{{{}$\{org.apache.unomi.*{ syntax
 # Is documented as the primary way to configure Unomi via environment variables

h3. Current Environment Variable Pattern

All environment variables follow the pattern: UNOMI_CATEGORY_PROPERTY_NAME

*Examples:*
* {{UNOMI_HTTP_PORT}} → {{org.osgi.service.http.port}}
* {{UNOMI_ELASTICSEARCH_ADDRESSES}} → 
{{org.apache.unomi.elasticsearch.addresses}}
* {{UNOMI_CLUSTER_NODEID}} → {{org.apache.unomi.cluster.nodeId}}

h3. User Customization Methods

Users can override configuration in two ways:

# *{{unomi.custom.system.properties}} file* — Create 
{{$KARAF_HOME/etc/unomi.custom.system.properties}} with property overrides
# *Environment variables* — Set {{UNOMI_*}} environment variables

h3. Documentation References

The following documentation sections reference {{{}custom.system.properties{}}}:

* {{manual/src/main/asciidoc/configuration.adoc}} — Main configuration 
documentation
* {{manual/src/main/asciidoc/clustering.adoc}} — Cluster configuration
* {{manual/src/main/asciidoc/profile-import-export.adoc}} — Router configuration
* {{manual/src/main/asciidoc/connectors/salesforce-connector.adoc}} — Connector 
configuration
* {{CODING_GUIDELINES.md}} — Developer guidelines

h2. Target State: Karaf Native System

h3. How Karaf's Native Override System Works

Apache Karaf provides a built-in mechanism to override configuration properties 
using environment variables or system properties:

*Environment Variable Pattern:* {{PID_PROPERTY}} (uppercase, dots become 
underscores)
*System Property Pattern:* {{pid.property}} (lowercase, dots remain)

*Example:*
* Configuration PID: {{org.apache.karaf.shell}}
* Property name: {{sshPort}}
* Environment variable: {{ORG_APACHE_KARAF_SHELL_SSHPORT}}
* System property: {{-Dorg.apache.karaf.shell.sshPort}}

h3. Benefits of Migration

# *Simpler architecture* — No custom configuration loading mechanism needed
# *Standard Karaf approach* — Aligns with Karaf best practices
# *Better integration* — Works seamlessly with Karaf's 
{{KarafConfigurationPlugin}}
# *Reduced maintenance* — Less custom code to maintain
# *Consistent with Karaf ecosystem* — Other Karaf-based projects use this 
approach

h3. Challenges

# *Breaking change* — All environment variable names will change (requires 
major version release)
# *Migration effort* — Users must update their environment variables, 
Docker/Kubernetes deployments, and CI/CD pipelines
# *Documentation updates* — Extensive documentation changes required across 
multiple files

h2. Migration Strategy

h3. Phase 1: Preparation (Non-Breaking)

# *Create comprehensive environment variable mapping table* — Document all 203+ 
variables
# *Create migration script/tool* — Help users convert their environment 
variables
# *Add deprecation warnings* — Log warnings when old {{UNOMI_*}} variables are 
detected (if possible)

h3. Phase 2: Implementation (Breaking Change - Major Version)

# *Remove {{custom.system.properties}} file and 
{{${optionals}=unomi.custom.system.properties}} mechanism*
# *Update {{.cfg}} files* — Remove {{{}$\{org.apache.unomi.*{ variable 
substitutions where appropriate
# *Update code* — Remove any custom loading logic for 
{{custom.system.properties}}
# *Update {{BaseIT.java}}* — Replace {{editConfigurationFilePut}} calls with 
Karaf native system properties

h3. Phase 3: Cleanup

# *Update examples* — All examples in documentation and code
# *Remove migration script* — After sufficient time has passed

h2. Environment Variable Mapping

h3. Current vs. New Naming Convention

*Current Pattern:* UNOMI_CATEGORY_PROPERTY
*New Pattern (Karaf Native):* {{PID_PROPERTY}} where:
 * PID = Configuration Persistent ID (e.g., {{{}org.apache.unomi.cluster{}}})
 * Property = Property name (e.g., {{{}nodeId{}}})
 * Dots ({{{}.{}}}) become underscores ({{{}_{}}})
 * All uppercase

h3. Example Mappings
||Current Environment Variable||New Environment Variab

[jira] [Updated] (UNOMI-918) Migrate from custom.system.properties to Karaf Native Environment Variable and System Property Override System

2025-11-27 Thread Serge Huber (Jira)


 [ 
https://issues.apache.org/jira/browse/UNOMI-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Serge Huber updated UNOMI-918:
--
Description: 
h2. Summary

Plan and execute migration from Unomi's custom {{custom.system.properties}} 
file to Karaf's native environment variable and system property override 
mechanism. This will simplify configuration management, align with Karaf best 
practices, and eliminate the need for a custom configuration loading mechanism.
h2. Current State
h3. How custom.system.properties Works

Apache Unomi currently uses a centralized configuration file 
{{$KARAF_HOME/etc/custom.system.properties}} that:
 # Contains ~203 system properties with environment variable references using 
the pattern: {{{}org.apache.unomi.{*}=$\{env:UNOMI_{*}:-defaultValue{
 # Uses Karaf's {{${optionals}=unomi.custom.system.properties}} mechanism to 
load user overrides from {{$KARAF_HOME/etc/unomi.custom.system.properties}}
 # Provides variable substitution in {{.cfg}} files using 
{{{}$\{org.apache.unomi.*{ syntax
 # Is documented as the primary way to configure Unomi via environment variables

h3. Current Environment Variable Pattern

All environment variables follow the pattern: 
\{{UNOMI_{CATEGORY}_\{PROPERTY_NAME}}}

*Examples:*
* {{UNOMI_HTTP_PORT}} → {{org.osgi.service.http.port}}
* {{UNOMI_ELASTICSEARCH_ADDRESSES}} → 
{{org.apache.unomi.elasticsearch.addresses}}
* {{UNOMI_CLUSTER_NODEID}} → {{org.apache.unomi.cluster.nodeId}}

h3. User Customization Methods

Users can override configuration in two ways:

# *{{unomi.custom.system.properties}} file* — Create 
{{$KARAF_HOME/etc/unomi.custom.system.properties}} with property overrides
# *Environment variables* — Set {{UNOMI_*}} environment variables

h3. Documentation References

The following documentation sections reference {{{}custom.system.properties{}}}:

* {{manual/src/main/asciidoc/configuration.adoc}} — Main configuration 
documentation
* {{manual/src/main/asciidoc/clustering.adoc}} — Cluster configuration
* {{manual/src/main/asciidoc/profile-import-export.adoc}} — Router configuration
* {{manual/src/main/asciidoc/connectors/salesforce-connector.adoc}} — Connector 
configuration
* {{CODING_GUIDELINES.md}} — Developer guidelines

h2. Target State: Karaf Native System

h3. How Karaf's Native Override System Works

Apache Karaf provides a built-in mechanism to override configuration properties 
using environment variables or system properties:

*Environment Variable Pattern:* {{PID_PROPERTY}} (uppercase, dots become 
underscores)
*System Property Pattern:* {{pid.property}} (lowercase, dots remain)

*Example:*
* Configuration PID: {{org.apache.karaf.shell}}
* Property name: {{sshPort}}
* Environment variable: {{ORG_APACHE_KARAF_SHELL_SSHPORT}}
* System property: {{-Dorg.apache.karaf.shell.sshPort}}

h3. Benefits of Migration

# *Simpler architecture* — No custom configuration loading mechanism needed
# *Standard Karaf approach* — Aligns with Karaf best practices
# *Better integration* — Works seamlessly with Karaf's 
{{KarafConfigurationPlugin}}
# *Reduced maintenance* — Less custom code to maintain
# *Consistent with Karaf ecosystem* — Other Karaf-based projects use this 
approach

h3. Challenges

# *Breaking change* — All environment variable names will change (requires 
major version release)
# *Migration effort* — Users must update their environment variables, 
Docker/Kubernetes deployments, and CI/CD pipelines
# *Documentation updates* — Extensive documentation changes required across 
multiple files

h2. Migration Strategy

h3. Phase 1: Preparation (Non-Breaking)

# *Create comprehensive environment variable mapping table* — Document all 203+ 
variables
# *Create migration script/tool* — Help users convert their environment 
variables
# *Add deprecation warnings* — Log warnings when old {{UNOMI_*}} variables are 
detected (if possible)

h3. Phase 2: Implementation (Breaking Change - Major Version)

# *Remove {{custom.system.properties}} file and 
{{${optionals}=unomi.custom.system.properties}} mechanism*
# *Update {{.cfg}} files* — Remove {{{}$\{org.apache.unomi.*{ variable 
substitutions where appropriate
# *Update code* — Remove any custom loading logic for 
{{custom.system.properties}}
# *Update {{BaseIT.java}}* — Replace {{editConfigurationFilePut}} calls with 
Karaf native system properties

h3. Phase 3: Cleanup

# *Update examples* — All examples in documentation and code
# *Remove migration script* — After sufficient time has passed

h2. Environment Variable Mapping

h3. Current vs. New Naming Convention

*Current Pattern:* \{{UNOMI_{CATEGORY}

_

{PROPERTY}

}}
*New Pattern (Karaf Native):* {{PID_PROPERTY}} where:
 * PID = Configuration Persistent ID (e.g., {{{}org.apache.unomi.cluster{}}})
 * Property = Property name (e.g., {{{}nodeId{}}})
 * Dots ({{{}.{}}}) become underscores ({{{}_{}}})
 * All uppercase

h3. Example Mappings
||Current Environment Variab