kaxil opened a new issue, #51545: URL: https://github.com/apache/airflow/issues/51545
With server-client separation, we need to determine what should be shared vs duplicated vs kept separate. Should the Providers with executors be separated into providers to that they can be included on Server side? ## Categories of Potentially Shared Components ### Core Interfaces and Protocols - **`BaseSecretsBackend`**: Core interface for secrets backend implementations - **`AirflowConfigParser`**: Configuration access interface - **`airflow.serialization`**: DAG serialization/deserialization interface ### Exception Hierarchies - **AirflowException**: Base exception class - **AirflowSkipException**: Task skipping exceptions - **AirflowSensorTimeout**: Sensor-specific exceptions - **Common error types**: Used by both server and SDK ### Core Utilities - **Module loading**: `import_string`, `import_module_safely` - **Trigger rules**: `TriggerRule` enum and logic - **Date/time utilities**: Timezone handling, date parsing - **String utilities**: Template processing, formatting ### Serialization Components - **Schema definitions**: JSON schemas for DAG representation - **Schema versioning**: Version management and compatibility - **Validation logic**: Schema validation and error handling ### Configuration Components - **Configuration interfaces**: Protocol definitions for config access - **Default values**: Shared configuration defaults - **Validation logic**: Configuration validation rules - **Environment handling**: Environment variable processing ## Shared Package Strategy Options ### Option 1: Single Shared Package (`airflow-protocols` / `airflow-commons`) - **Approach**: All shared components in one package - **Pros**: Simple dependency management, single source of truth - **Cons**: Package may become large, mixed concerns - **Use case**: Clean separation with minimal package proliferation ### Option 2: No Shared Package (Duplication) - **Approach**: Each package implements its own versions - **Pros**: Complete independence, no circular dependencies - **Cons**: Code duplication, maintenance overhead - **Use case**: When separation is more important than efficiency ## Key Questions to Answer - **How many packages is optimal?** Trade-off between focus and complexity - **What naming convention is clearest?** For users and developers - **How to handle backward compatibility?** Migration path for existing users - **What about provider dependencies?** Which packages should providers depend on? - **Testing strategy?** How to test package combinations efficiently ## Definition of Done - All package structure options evaluated - Dependency and testing implications understood - Clear recommendation with migration path -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
