zero323 commented on a change in pull request #34273:
URL: https://github.com/apache/spark/pull/34273#discussion_r728758011
##########
File path: examples/src/main/python/als.py
##########
@@ -87,15 +87,15 @@ def update(i, mat, ratings):
for i in range(ITERATIONS):
ms = sc.parallelize(range(M), partitions) \
.map(lambda x: update(x, usb.value, Rb.value)) \
- .collect()
+ .collect() # type: ignore[assignment]
# collect() returns a list, so array ends up being
# a 3-d array, we take the first 2 dims for the matrix
- ms = matrix(np.array(ms)[:, :, 0])
+ ms = matrix(np.array(ms)[:, :, 0]) # type: ignore[assignment]
msb = sc.broadcast(ms)
us = sc.parallelize(range(U), partitions) \
.map(lambda x: update(x, msb.value, Rb.value.T)) \
- .collect()
+ .collect() # type: ignore[assignment]
Review comment:
Yes, we can (or at least we can reduce the number `ignores` to minimum).
The question is, how to do it:
- We can rewrite examples [as I've done in other
place](https://github.com/apache/spark/pull/34273/files/846cdc0e200d9ab8cc009b0d3c39122116f9d6fd#diff-da9c07a08d3b20939b5456cd79104717d94ad6909074213f56109a6d92467a2b)
- We can use less strict checker default (not fond of that)
If we're going to rewrite (here it is primarily a matter of using different
names I guess), how far should we go? Type everything? Type only when types are
necessary to make mypy happy? If the latter one, we'd have to leak typing
import into rendered examples. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
