[
https://issues.apache.org/jira/browse/UNOMI-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Serge Huber reassigned UNOMI-880:
---------------------------------
Assignee: Serge Huber
> Unified Multi-Tenant Caching Service with Predefined Item Support
> -----------------------------------------------------------------
>
> Key: UNOMI-880
> URL: https://issues.apache.org/jira/browse/UNOMI-880
> Project: Apache Unomi
> Issue Type: Sub-task
> Components: unomi(-core)
> Affects Versions: unomi-3.0.0
> Reporter: Serge Huber
> Assignee: Serge Huber
> Priority: Major
> Fix For: unomi-3.0.0
>
>
> h2. Problem Statement
> The current codebase has multiple services implementing their own caching
> logic and predefined item loading mechanisms, leading to:
> * Code duplication across services
> * Inconsistent caching behaviors
> * Redundant predefined item loading logic
> * Lack of centralized cache management
> * No unified monitoring of cache performance
> h2. Solution
> We have implemented a new unified caching architecture with the following
> components:
> h3. Core Components
> * *MultiTypeCacheService*: Interface defining the contract for multi-tenant
> caching operations
> * *AbstractMultiTypeCachingService*: Base implementation providing common
> caching functionality
> * *CacheableTypeConfig*: Configuration class for defining cacheable types
> * New Karaf shell commands for cache management
> h3. Key Features
> * Multi-tenant support with inheritance from system tenant
> * Automatic predefined item loading from bundles
> * Periodic cache refresh from persistence
> * Cache statistics and monitoring
> * Tenant-aware cache operations
> * Bundle lifecycle management
> h2. Implementation Details
> h3. Converted Services
> The following services have been migrated to use the new caching system:
> * *DefinitionsServiceImpl*: Caching for condition types, action types, value
> types, and property merge strategies
> * *SegmentServiceImpl*: Caching for segments and scoring definitions
> h3. Services Remaining to be Converted
> Based on codebase analysis, the following services should be converted next:
> * *ProfileServiceImpl*: Currently has custom implementation for personas and
> property types
> * *RulesServiceImpl*: Has custom predefined rules loading
> * *GoalsServiceImpl*: Manages predefined goals and campaigns
> * *EventServiceImpl*: Handles predefined event types
> h3. Cache Management Commands
> New Karaf shell commands have been added ({{CacheCommands.java}}):
> {noformat}
> unomi:cache --stats # Display cache statistics
> unomi:cache --reset # Reset statistics
> unomi:cache --type <type> # Filter by type
> unomi:cache --tenant <tenantId> # Filter by tenant
> unomi:cache --clear # Clear cache for tenant
> unomi:cache --inspect # Inspect cache contents
> unomi:cache --watch <interval> # Watch cache statistics
> unomi:cache --view --id <id> # View specific cache entry
> unomi:cache --remove --id <id> # Remove cache entry
> {noformat}
> h3. Benefits
> h4. Code Reduction
> * Eliminated ~2000 lines of duplicated caching code
> * Centralized predefined item loading logic
> * Unified bundle lifecycle handling
> h4. Performance Improvements
> * Efficient tenant-aware caching
> * Optimized cache refresh scheduling
> * Reduced memory usage through shared implementation
> h4. Maintainability
> * Consistent caching behavior across services
> * Centralized cache monitoring
> * Simplified service implementations
> * Type-safe cache operations
> h4. Operational Benefits
> * Real-time cache monitoring
> * Cache performance metrics
> * Easy cache management through shell commands
> * Improved debugging capabilities
> h2. Technical Implementation
> h3. Cache Configuration
> {noformat:java}
> CacheableTypeConfig<T> config = new CacheableTypeConfig<>(
> type.class, // Type class
> itemType, // Type identifier
> metaInfPath, // Predefined items path
> inheritFromSystem, // System tenant inheritance
> requiresRefresh, // Periodic refresh
> refreshInterval, // Refresh frequency
> idExtractor // ID extraction function
> );
> {noformat}
> h3. Service Migration Steps
> # Extend {{AbstractMultiTypeCachingService}}
> # Implement {{getTypeConfigs()}} method
> # Remove custom caching code
> # Update bundle lifecycle handlers
> # Add cache statistics support
> h2. Testing Strategy
> h3. Unit Tests
> * Cache operations
> * Tenant inheritance
> * Statistics collection
> * Configuration validation
> h3. Integration Tests
> * Bundle lifecycle
> * Predefined item loading
> * Cache refresh
> * Multi-tenant scenarios
> h3. Performance Tests
> * Cache hit rates
> * Memory usage
> * Refresh timing
> * Concurrent access
> h2. Migration Plan
> h3. Phase 1 (Completed)
> * Core implementation
> * Definitions service migration
> * Segment service migration
> * Shell commands
> h3. Phase 2 (Planned)
> * Profile service migration
> * Rules service migration
> * Goals service migration
> * Event service migration
> h3. Phase 3 (Future)
> * Additional service migrations
> * Performance optimizations
> * Extended monitoring
> h2. Success Metrics
> h3. Code Quality
> * Reduced code duplication
> * Improved test coverage
> * Consistent implementation patterns
> h3. Performance
> * Cache hit rates > 90%
> * Reduced memory usage
> * Faster service operations
> h3. Operational
> * Improved monitoring capabilities
> * Easier cache management
> * Better debugging support
> h2. Risks and Mitigation
> h3. Migration Complexity
> * Gradual service migration
> * Comprehensive testing
> * Rollback procedures
> h3. Performance Impact
> * Cache warm-up strategies
> * Configurable refresh intervals
> * Performance monitoring
> h3. Data Consistency
> * Tenant isolation checks
> * Inheritance validation
> * Cache invalidation controls
> h3. Related Issues
> * {color:#707070}Add links to related issues here{color}
> h3. Attachments
> * {color:#707070}Add any relevant diagrams or documentation{color}
> {panel:title=Comments}
> Please review the proposal and provide feedback on:
> * Migration priority for remaining services
> * Additional services that could benefit from the caching system
> * Performance monitoring requirements
> * Testing coverage expectations
> {panel}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
