Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/14156
@srowen OK I close the pr for now if I found better way to optimize it I
will reopen it, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14156
I get though sounds like there is not necessarily any such optimization now
and actually not sure there can be. It could even be slower; it introduces an
extra copy. It is somewhat harder to
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/14156
@srowen yeah, the function supplied here called cannot be turned into SIMD
instructions but I think it can do some parallelization optimization on large
matrix, for example we can split the
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14156
That's the question indeed. I'm not sure because the function that's
supplied could be anything. I don't see how it could automatically be converted
to a vectorized operation automatically.
---
If
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/14156
yeah, currently it seems to make a little overhead (do a copy), but I think
it will take advantage of breeze optimization, in the future, e.g, SIMD
instructions or something ?
---
If your
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14156
I see, this copies x to y then modifies y in place. OK. Is that more
efficient? it seems like extra work, but does the transform method make up for
it? just seeing if this has actually been observed
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/14156
@srowen
The := operator in BDM is simply copy one BDM to another, and it is widely
used in breeze source, e.g, we can check DenseMatrix.copy function in Breeze:
it first use
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14156
Is there reasonable evidence this speeds things up? just want to make sure
this does not make it slower. Help me understand the := operator? I don't
recognize how it's helping compute y as a
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/14156
cc @srowen thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14156
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14156
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62172/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14156
**[Test build #62172 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62172/consoleFull)**
for PR 14156 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14156
**[Test build #62172 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62172/consoleFull)**
for PR 14156 at commit
13 matches
Mail list logo
