shahar1 opened a new issue, #70296: URL: https://github.com/apache/airflow/issues/70296
### Background Template fields are rendered **after** an operator's constructor runs. Any logic applied to a template-field parameter inside `__init__` — validation, type checks, transformation, string interpolation — therefore operates on the un-rendered Jinja expression, not the real value. This is documented in [Creating a custom Operator](https://airflow.apache.org/docs/apache-airflow/stable/howto/custom-operator.html#templating) and [contributing-docs/05_pull_requests.rst](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst), and it causes real bugs: #69813 (`SSHRemoteJobOperator` validated a templated `remote_base_dir` in `__init__`, so cleanup validation failed for any custom base) is a recent example. The `validate-operators-init` prek hook previously only verified that template fields are *assigned* verbatim at the top level of `__init__`; it could not see validation calls, conditionals, or transformations (and it did not cover sensors, `AwsBaseOperator[...]` subclasses, or `aws_template_fields(...)`-based classes at all). The hook has been extended to detect any non-sanctioned read of a template field in `__init__` (PR to follow). ### The exemption ratchet Existing violations are listed in [`scripts/ci/prek/validate_operators_init_exemptions.txt`](https://github.com/apache/airflow/blob/main/scripts/ci/prek/validate_operators_init_exemptions.txt) as `path::ClassName` entries so the hook can enforce the rule on new code immediately. - A PR that fixes a class **must remove its entry** in the same PR — the hook fails on stale exemptions, so this cannot be forgotten. - New violations cannot be added: only files listed in the exemption file are suppressed. - This issue is done when the exemption file is empty. **How to fix a class:** move the validation / transformation from `__init__` into `execute()` (or the first method that runs after rendering), keep `__init__` down to plain `self.field = field` assignments (defaulting via `field or default` is fine). Don't forget the corresponding tests. Please do **not** open sub-issues for individual entries — comment here or just open a PR referencing this issue. ### Current exemption list (snapshot) The authoritative list is the exemptions file; this table is the snapshot at the time the check was introduced. **83 classes** across 22 providers. | Provider | Class | File | Finding | |---|---|---|---| | amazon | `AppflowBaseOperator` | [appflow.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/appflow.py) | logic in `__init__` | | amazon | `AwsToAwsBaseOperator` | [base.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/transfers/base.py) | logic in `__init__` | | amazon | `BedrockCreateKnowledgeBaseOperator` | [bedrock.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/bedrock.py) | logic in `__init__` | | amazon | `BedrockRaGOperator` | [bedrock.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/bedrock.py) | transformed assignment, logic in `__init__`, missing assignment | | amazon | `DataSyncOperator` | [datasync.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/datasync.py) | logic in `__init__` | | amazon | `DmsModifyTaskOperator` | [dms.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/dms.py) | logic in `__init__` | | amazon | `DmsStartReplicationOperator` | [dms.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/dms.py) | logic in `__init__` | | amazon | `EcsRunTaskOperator` | [ecs.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/ecs.py) | logic in `__init__` | | amazon | `EmrAddStepsOperator` | [emr.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/emr.py) | logic in `__init__` | | amazon | `GCSToS3Operator` | [gcs_to_s3.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/transfers/gcs_to_s3.py) | logic in `__init__` | | amazon | `GlueDataQualityOperator` | [glue.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/glue.py) | transformed assignment, logic in `__init__`, missing assignment | | amazon | `MongoToS3Operator` | [mongo_to_s3.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/transfers/mongo_to_s3.py) | logic in `__init__` | | amazon | `NeptuneStartDbClusterOperator` | [neptune.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/neptune.py) | transformed assignment, missing assignment | | amazon | `NeptuneStopDbClusterOperator` | [neptune.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/neptune.py) | transformed assignment, missing assignment | | amazon | `S3DeleteObjectsOperator` | [s3.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/s3.py) | logic in `__init__` | | amazon | `S3ToRedshiftOperator` | [s3_to_redshift.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/transfers/s3_to_redshift.py) | logic in `__init__` | | amazon | `SageMakerCreateNotebookOperator` | [sagemaker.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/sagemaker.py) | logic in `__init__` | | amazon | `SageMakerProcessingOperator` | [sagemaker.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/sagemaker.py) | logic in `__init__` | | amazon | `StepFunctionStartExecutionOperator` | [step_function.py](https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/operators/step_function.py) | transformed assignment, missing assignment | | anthropic | `AnthropicAgentSessionOperator` | [agent.py](https://github.com/apache/airflow/blob/main/providers/anthropic/src/airflow/providers/anthropic/operators/agent.py) | logic in `__init__` | | apache/hive | `HivePartitionSensor` | [hive_partition.py](https://github.com/apache/airflow/blob/main/providers/apache/hive/src/airflow/providers/apache/hive/sensors/hive_partition.py) | logic in `__init__` | | apache/hive | `NamedHivePartitionSensor` | [named_hive_partition.py](https://github.com/apache/airflow/blob/main/providers/apache/hive/src/airflow/providers/apache/hive/sensors/named_hive_partition.py) | logic in `__init__` | | apache/kafka | `ProduceToTopicOperator` | [produce.py](https://github.com/apache/airflow/blob/main/providers/apache/kafka/src/airflow/providers/apache/kafka/operators/produce.py) | logic in `__init__` | | apache/spark | `SparkSubmitOperator` | [spark_submit.py](https://github.com/apache/airflow/blob/main/providers/apache/spark/src/airflow/providers/apache/spark/operators/spark_submit.py) | logic in `__init__` | | cncf/kubernetes | `KubernetesInstallKueueOperator` | [kueue.py](https://github.com/apache/airflow/blob/main/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/kueue.py) | logic in `__init__` | | cncf/kubernetes | `KubernetesPodOperator` | [pod.py](https://github.com/apache/airflow/blob/main/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/pod.py) | logic in `__init__` | | cncf/kubernetes | `KubernetesResourceBaseOperator` | [resource.py](https://github.com/apache/airflow/blob/main/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/resource.py) | logic in `__init__` | | cohere | `CohereEmbeddingOperator` | [embedding.py](https://github.com/apache/airflow/blob/main/providers/cohere/src/airflow/providers/cohere/operators/embedding.py) | logic in `__init__` | | common/ai | `AgentOperator` | [agent.py](https://github.com/apache/airflow/blob/main/providers/common/ai/src/airflow/providers/common/ai/operators/agent.py) | logic in `__init__` | | common/ai | `DocumentLoaderOperator` | [document_loader.py](https://github.com/apache/airflow/blob/main/providers/common/ai/src/airflow/providers/common/ai/operators/document_loader.py) | logic in `__init__` | | databricks | `DatabricksCopyIntoOperator` | [databricks_sql.py](https://github.com/apache/airflow/blob/main/providers/databricks/src/airflow/providers/databricks/operators/databricks_sql.py) | logic in `__init__` | | databricks | `DatabricksReposCreateOperator` | [databricks_repos.py](https://github.com/apache/airflow/blob/main/providers/databricks/src/airflow/providers/databricks/operators/databricks_repos.py) | logic in `__init__` | | databricks | `DatabricksReposDeleteOperator` | [databricks_repos.py](https://github.com/apache/airflow/blob/main/providers/databricks/src/airflow/providers/databricks/operators/databricks_repos.py) | logic in `__init__` | | databricks | `DatabricksReposUpdateOperator` | [databricks_repos.py](https://github.com/apache/airflow/blob/main/providers/databricks/src/airflow/providers/databricks/operators/databricks_repos.py) | logic in `__init__` | | databricks | `DatabricksSQLStatementsSensor` | [databricks.py](https://github.com/apache/airflow/blob/main/providers/databricks/src/airflow/providers/databricks/sensors/databricks.py) | logic in `__init__` | | dbt/cloud | `DbtCloudGetJobRunArtifactOperator` | [dbt.py](https://github.com/apache/airflow/blob/main/providers/dbt/cloud/src/airflow/providers/dbt/cloud/operators/dbt.py) | logic in `__init__` | | docker | `DockerOperator` | [docker.py](https://github.com/apache/airflow/blob/main/providers/docker/src/airflow/providers/docker/operators/docker.py) | logic in `__init__` | | google | `AzureFileShareToGCSOperator` | [azure_fileshare_to_gcs.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/transfers/azure_fileshare_to_gcs.py) | logic in `__init__` | | google | `BigQueryDataTransferServiceTransferRunSensor` | [bigquery_dts.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/sensors/bigquery_dts.py) | transformed assignment, logic in `__init__`, missing assignment | | google | `BigQueryInsertJobOperator` | [bigquery.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/bigquery.py) | logic in `__init__` | | google | `BigQueryToMsSqlOperator` | [bigquery_to_mssql.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/transfers/bigquery_to_mssql.py) | logic in `__init__` | | google | `CloudBatchSubmitJobOperator` | [cloud_batch.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/cloud_batch.py) | logic in `__init__` | | google | `CloudBuildCreateBuildOperator` | [cloud_build.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/cloud_build.py) | logic in `__init__` | | google | `CloudComposerExternalTaskSensor` | [cloud_composer.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/sensors/cloud_composer.py) | logic in `__init__` | | google | `CloudDataTransferServiceCreateJobOperator` | [cloud_storage_transfer_service.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/cloud_storage_transfer_service.py) | logic in `__init__` | | google | `CloudFunctionDeployFunctionOperator` | [functions.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/functions.py) | logic in `__init__` | | google | `ComputeEngineCopyInstanceTemplateOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineDeleteInstanceGroupManagerOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineDeleteInstanceOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineDeleteInstanceTemplateOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineInsertInstanceFromTemplateOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineInsertInstanceGroupManagerOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineInsertInstanceOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineInsertInstanceTemplateOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineInstanceGroupUpdateManagerTemplateOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `ComputeEngineSetMachineTypeOperator` | [compute.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/compute.py) | logic in `__init__` | | google | `DataprocCreateClusterOperator` | [dataproc.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/dataproc.py) | logic in `__init__` | | google | `DataprocSubmitJobOperator` | [dataproc.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/dataproc.py) | logic in `__init__` | | google | `GCSDeleteObjectsOperator` | [gcs.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/gcs.py) | logic in `__init__` | | google | `GCSFileTransformOperator` | [gcs.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/gcs.py) | logic in `__init__` | | google | `GCSListObjectsOperator` | [gcs.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/gcs.py) | logic in `__init__` | | google | `GCSToBigQueryOperator` | [gcs_to_bigquery.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py) | logic in `__init__` | | google | `GCSToGCSOperator` | [gcs_to_gcs.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/transfers/gcs_to_gcs.py) | logic in `__init__` | | google | `GCSToLocalFilesystemOperator` | [gcs_to_local.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/transfers/gcs_to_local.py) | logic in `__init__` | | google | `GenAIGeminiCreateBatchJobOperator` | [gen_ai.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/gen_ai.py) | logic in `__init__` | | google | `GenAIGeminiCreateEmbeddingsBatchJobOperator` | [gen_ai.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/cloud/operators/gen_ai.py) | logic in `__init__` | | google | `GoogleCampaignManagerDeleteReportOperator` | [campaign_manager.py](https://github.com/apache/airflow/blob/main/providers/google/src/airflow/providers/google/marketing_platform/operators/campaign_manager.py) | logic in `__init__` | | microsoft/azure | `AzureVirtualMachineStateSensor` | [compute.py](https://github.com/apache/airflow/blob/main/providers/microsoft/azure/src/airflow/providers/microsoft/azure/sensors/compute.py) | logic in `__init__` | | microsoft/azure | `GCSToAzureBlobStorageOperator` | [gcs_to_wasb.py](https://github.com/apache/airflow/blob/main/providers/microsoft/azure/src/airflow/providers/microsoft/azure/transfers/gcs_to_wasb.py) | logic in `__init__` | | microsoft/azure | `OracleToAzureDataLakeOperator` | [oracle_to_azure_data_lake.py](https://github.com/apache/airflow/blob/main/providers/microsoft/azure/src/airflow/providers/microsoft/azure/transfers/oracle_to_azure_data_lake.py) | logic in `__init__` | | microsoft/psrp | `PsrpOperator` | [psrp.py](https://github.com/apache/airflow/blob/main/providers/microsoft/psrp/src/airflow/providers/microsoft/psrp/operators/psrp.py) | logic in `__init__` | | neo4j | `Neo4jOperator` | [neo4j.py](https://github.com/apache/airflow/blob/main/providers/neo4j/src/airflow/providers/neo4j/operators/neo4j.py) | logic in `__init__` | | oracle | `OracleToOracleOperator` | [oracle_to_oracle.py](https://github.com/apache/airflow/blob/main/providers/oracle/src/airflow/providers/oracle/transfers/oracle_to_oracle.py) | logic in `__init__` | | papermill | `PapermillOperator` | [papermill.py](https://github.com/apache/airflow/blob/main/providers/papermill/src/airflow/providers/papermill/operators/papermill.py) | logic in `__init__` | | snowflake | `SnowparkContainerJobOperator` | [snowpark_containers.py](https://github.com/apache/airflow/blob/main/providers/snowflake/src/airflow/providers/snowflake/operators/snowpark_containers.py) | logic in `__init__` | | ssh | `SSHOperator` | [ssh.py](https://github.com/apache/airflow/blob/main/providers/ssh/src/airflow/providers/ssh/operators/ssh.py) | logic in `__init__` | | ssh | `SSHRemoteJobOperator` | [ssh_remote_job.py](https://github.com/apache/airflow/blob/main/providers/ssh/src/airflow/providers/ssh/operators/ssh_remote_job.py) | logic in `__init__` | | standard | `BashOperator` | [bash.py](https://github.com/apache/airflow/blob/main/providers/standard/src/airflow/providers/standard/operators/bash.py) | logic in `__init__` | | standard | `DateTimeSensor` | [date_time.py](https://github.com/apache/airflow/blob/main/providers/standard/src/airflow/providers/standard/sensors/date_time.py) | logic in `__init__`, missing assignment | | standard | `HITLOperator` | [hitl.py](https://github.com/apache/airflow/blob/main/providers/standard/src/airflow/providers/standard/operators/hitl.py) | logic in `__init__` | | standard | `TriggerDagRunOperator` | [trigger_dagrun.py](https://github.com/apache/airflow/blob/main/providers/standard/src/airflow/providers/standard/operators/trigger_dagrun.py) | logic in `__init__` | | teradata | `TeradataToTeradataOperator` | [teradata_to_teradata.py](https://github.com/apache/airflow/blob/main/providers/teradata/src/airflow/providers/teradata/transfers/teradata_to_teradata.py) | logic in `__init__` | | weaviate | `WeaviateIngestOperator` | [weaviate.py](https://github.com/apache/airflow/blob/main/providers/weaviate/src/airflow/providers/weaviate/operators/weaviate.py) | logic in `__init__` | --- Drafted-by: Claude Code (Opus 4.8) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
