This is an automated email from the ASF dual-hosted git repository.
Yicong-Huang pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new e2e3b99cdbf2 [SPARK-58177][GRAPHX] Fix wrong operand in SVD++
aggregateMessages combiner
e2e3b99cdbf2 is described below
commit e2e3b99cdbf2f256467a3e4e6f89933486a1ce60
Author: Matthew B. <[email protected]>
AuthorDate: Mon Jul 20 22:32:02 2026 +0000
[SPARK-58177][GRAPHX] Fix wrong operand in SVD++ aggregateMessages combiner
### What changes were proposed in this pull request?
Fix the Phase-2 `aggregateMessages` combiner in `SVDPlusPlus.run`, which
seeded the `_2` accumulator from the wrong operand:
```diff
- val out2 = g2._2.clone()
+ val out2 = g1._2.clone()
BLAS.nativeBLAS.daxpy(out2.length, 1.0, g2._2, 1, out2, 1)
```
### Why are the changes needed?
The next line adds `g2` on top, so the result should be `g1 + g2`. Because
the code started with `g2` instead of `g1`, it computed `g2 + g2` and discarded
`g1`. Any node with two or more edges ended up with the wrong value. The
matching code just below already does it the right way (`g1 + g2`).
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Verified with a unit test (UT) that feeds two concrete updates through the
combiner and checks the merged `_2` value, plus the existing suite: `build/sbt
"graphx/testOnly org.apache.spark.graphx.lib.SVDPlusPlusSuite"` passes. With
the faulty code, the output does not match the expected sum; with the fix, it
does:
| | `_2` value |
|---|---|
| Input `g1._2` | `[3, 4]` |
| Input `g2._2` | `[30, 40]` |
| Expected (`g1._2 + g2._2`) | `[33, 44]` |
| Faulty code output | `[60, 80]` (wrong: `2 * g2._2`, drops `g1._2`) |
| Corrected code output | `[33, 44]` (correct) |
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #57314 from Ma77Ball/SPARK-58177.
Authored-by: Matthew B. <[email protected]>
Signed-off-by: Yicong-Huang <[email protected]>
(cherry picked from commit b5e242ea01bb362a3e67dd5c33289a5d4394bc7b)
Signed-off-by: Yicong-Huang <[email protected]>
---
graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala
b/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala
index 8764c9854c53..7b282ec6d24e 100644
--- a/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala
+++ b/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala
@@ -156,7 +156,7 @@ object SVDPlusPlus {
{
val out1 = g1._1.clone()
BLAS.nativeBLAS.daxpy(out1.length, 1.0, g2._1, 1, out1, 1)
- val out2 = g2._2.clone()
+ val out2 = g1._2.clone()
BLAS.nativeBLAS.daxpy(out2.length, 1.0, g2._2, 1, out2, 1)
(out1, out2, g1._3 + g2._3)
})
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
