Serge Huber created UNOMI-880:
---------------------------------
Summary: Unified Multi-Tenant Caching Service with Predefined Item
Support
Key: UNOMI-880
URL: https://issues.apache.org/jira/browse/UNOMI-880
Project: Apache Unomi
Issue Type: Sub-task
Components: unomi(-core)
Affects Versions: unomi-3.0.0
Reporter: Serge Huber
Fix For: unomi-3.0.0
h2. Problem Statement
The current codebase has multiple services implementing their own caching logic
and predefined item loading mechanisms, leading to:
* Code duplication across services
* Inconsistent caching behaviors
* Redundant predefined item loading logic
* Lack of centralized cache management
* No unified monitoring of cache performance
h2. Solution
We have implemented a new unified caching architecture with the following
components:
h3. Core Components
* *MultiTypeCacheService*: Interface defining the contract for multi-tenant
caching operations
* *AbstractMultiTypeCachingService*: Base implementation providing common
caching functionality
* *CacheableTypeConfig*: Configuration class for defining cacheable types
* New Karaf shell commands for cache management
h3. Key Features
* Multi-tenant support with inheritance from system tenant
* Automatic predefined item loading from bundles
* Periodic cache refresh from persistence
* Cache statistics and monitoring
* Tenant-aware cache operations
* Bundle lifecycle management
h2. Implementation Details
h3. Converted Services
The following services have been migrated to use the new caching system:
* *DefinitionsServiceImpl*: Caching for condition types, action types, value
types, and property merge strategies
* *SegmentServiceImpl*: Caching for segments and scoring definitions
h3. Services Remaining to be Converted
Based on codebase analysis, the following services should be converted next:
* *ProfileServiceImpl*: Currently has custom implementation for personas and
property types
* *RulesServiceImpl*: Has custom predefined rules loading
* *GoalsServiceImpl*: Manages predefined goals and campaigns
* *EventServiceImpl*: Handles predefined event types
h3. Cache Management Commands
New Karaf shell commands have been added ({{CacheCommands.java}}):
{noformat}
unomi:cache --stats # Display cache statistics
unomi:cache --reset # Reset statistics
unomi:cache --type <type> # Filter by type
unomi:cache --tenant <tenantId> # Filter by tenant
unomi:cache --clear # Clear cache for tenant
unomi:cache --inspect # Inspect cache contents
unomi:cache --watch <interval> # Watch cache statistics
unomi:cache --view --id <id> # View specific cache entry
unomi:cache --remove --id <id> # Remove cache entry
{noformat}
h3. Benefits
h4. Code Reduction
* Eliminated ~2000 lines of duplicated caching code
* Centralized predefined item loading logic
* Unified bundle lifecycle handling
h4. Performance Improvements
* Efficient tenant-aware caching
* Optimized cache refresh scheduling
* Reduced memory usage through shared implementation
h4. Maintainability
* Consistent caching behavior across services
* Centralized cache monitoring
* Simplified service implementations
* Type-safe cache operations
h4. Operational Benefits
* Real-time cache monitoring
* Cache performance metrics
* Easy cache management through shell commands
* Improved debugging capabilities
h2. Technical Implementation
h3. Cache Configuration
{noformat:java}
CacheableTypeConfig<T> config = new CacheableTypeConfig<>(
type.class, // Type class
itemType, // Type identifier
metaInfPath, // Predefined items path
inheritFromSystem, // System tenant inheritance
requiresRefresh, // Periodic refresh
refreshInterval, // Refresh frequency
idExtractor // ID extraction function
);
{noformat}
h3. Service Migration Steps
# Extend {{AbstractMultiTypeCachingService}}
# Implement {{getTypeConfigs()}} method
# Remove custom caching code
# Update bundle lifecycle handlers
# Add cache statistics support
h2. Testing Strategy
h3. Unit Tests
* Cache operations
* Tenant inheritance
* Statistics collection
* Configuration validation
h3. Integration Tests
* Bundle lifecycle
* Predefined item loading
* Cache refresh
* Multi-tenant scenarios
h3. Performance Tests
* Cache hit rates
* Memory usage
* Refresh timing
* Concurrent access
h2. Migration Plan
h3. Phase 1 (Completed)
* Core implementation
* Definitions service migration
* Segment service migration
* Shell commands
h3. Phase 2 (Planned)
* Profile service migration
* Rules service migration
* Goals service migration
* Event service migration
h3. Phase 3 (Future)
* Additional service migrations
* Performance optimizations
* Extended monitoring
h2. Success Metrics
h3. Code Quality
* Reduced code duplication
* Improved test coverage
* Consistent implementation patterns
h3. Performance
* Cache hit rates > 90%
* Reduced memory usage
* Faster service operations
h3. Operational
* Improved monitoring capabilities
* Easier cache management
* Better debugging support
h2. Risks and Mitigation
h3. Migration Complexity
* Gradual service migration
* Comprehensive testing
* Rollback procedures
h3. Performance Impact
* Cache warm-up strategies
* Configurable refresh intervals
* Performance monitoring
h3. Data Consistency
* Tenant isolation checks
* Inheritance validation
* Cache invalidation controls
h3. Related Issues
* {color:#707070}Add links to related issues here{color}
h3. Attachments
* {color:#707070}Add any relevant diagrams or documentation{color}
{panel:title=Comments}
Please review the proposal and provide feedback on:
* Migration priority for remaining services
* Additional services that could benefit from the caching system
* Performance monitoring requirements
* Testing coverage expectations
{panel}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
