bearomorphism opened a new pull request, #59349:
URL: https://github.com/apache/spark/pull/59349
### What changes were proposed in this pull request?
- Declare `dot()` on `pyspark.mllib.linalg.Vector`, following the base
class's existing `NotImplementedError` pattern.
- Add compatible overloads to `Vector`, `DenseVector`, and `SparseVector`:
vector/list/tuple/range operands return `np.float64`, while NumPy/SciPy
operands may return a scalar or array.
- Remove five redundant `type: ignore[attr-defined]` comments from
classification and regression callers.
- Add a focused typing regression and register the MLlib typing-test
directory in `dev/lint-python`.
The concrete dot-product implementations are unchanged. The separate `.size`
declaration fix is not included.
### Why are the changes needed?
Both concrete vector classes implement `dot()`, but the base `Vector` does
not declare it. Valid calls through a `Vector` reference therefore require
attribute-error suppressions.
The existing concrete return annotations also describe only scalar results,
although matrix operands can produce arrays. The overloads expose the shared
method without incorrectly treating every result as a scalar.
### Does this PR introduce _any_ user-facing change?
Static typing changes only: type checkers recognize `Vector.dot()` and
distinguish scalar-only operand types from potentially array-valued operands.
Dense and sparse dot-product runtime behavior is unchanged.
### How was this patch tested?
Added one focused typing regression checking return-type inference through
the base and concrete vector classes.
The following checks passed:
```sh
mypy --config-file python/mypy.ini --follow-imports=silent
python/pyspark/mllib
ruff check python/pyspark/mllib/linalg/__init__.py \
python/pyspark/mllib/classification.py python/pyspark/mllib/regression.py
ruff format --check python/pyspark/mllib/linalg/__init__.py \
python/pyspark/mllib/classification.py python/pyspark/mllib/regression.py
bash -n dev/lint-python
git diff --check upstream/master...HEAD
```
Also passed 93 existing doctest examples (89 in `mllib.linalg` and four in
`LinearRegressionModelBase`) using NumPy's legacy `1.13` output formatting,
plus the existing `VectorTests.test_dot` method body without JVM fixtures. No
JVM-backed test suites were run.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Pi 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
