pavansai1997 opened a new pull request, #76203:
URL: https://github.com/apache/airflow/pull/76203
`SparkResources.convert_resources` (used by `SparkKubernetesOperator` when
the spec contains
`container_resources`) only recognised memory limits containing `G` or a
lowercase `m`. Any other
value was left as a string and then divided by 1.4, so common values such as
the Kubernetes-style
`512Mi` or the Spark-style `2g` made the task fail with:
```
TypeError: unsupported operand type(s) for /: 'str' and 'float'
```
The memory limit is now parsed once for both driver and executor:
- `G`, `Gi` and `m` are converted exactly as before (`2Gi` still becomes
`1462m`).
- `Mi`, `M` and `g` are now converted as well.
- Values that cannot be converted, such as `2Ti` or a bare number, raise a
`ValueError` that
names the value and the supported units instead of the `TypeError` above.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Code Puppy
Generated-by: Code Puppy following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
