[
https://issues.apache.org/jira/browse/SPARK-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623704#comment-14623704
]
Yanbo Liang edited comment on SPARK-9003 at 7/12/15 9:36 AM:
-------------------------------------------------------------
Yes, I agree that this is not supposed to become yet another vector/matrix
libaray. But I think map/update function is important enough to become the
interface of vector just like foreachActive which is supported at present.
I can also provide an example which may be benefit of these function.
For example:
val originalPrediction = Vectors.dense(Array(1, 2, 3))
val expected = Vectors.dense(Array(10, 20, 30))
In some cases, we can use "~==" to compare two Vector/Matrix which is defined
in org.apache.spark.mllib.util.TestingUtils.
So currently we can only code as following:
val prediction = Vectors.dense(originalPrediction.toArray.map(x => x*10))
assert(prediction ~== expected absTol 0.01, "prediction error")
If we support map/update for Vector, we can code as:
assert(originalPrediction.map(x => x*10) ~== expected absTol 0.01, "prediction
error")
However, MLlib/Matrix has already supported map/update/foreachActive function,
and we can compare two Matrices use ~== effortless.
was (Author: yanboliang):
Yes, I agree that this is not supposed to become yet another vector/matrix
libaray. But I think map/update function is important enough to become the
interface of vector just like the foreachActive which is supported at present.
I can also provide an example which may be benefit of these function.
For example:
val originalPrediction = Vectors.dense(Array(1, 2, 3))
val expected = Vectors.dense(Array(10, 20, 30))
In some cases, we can use "~==" to compare two Vector/Matrix which is defined
in org.apache.spark.mllib.util.TestingUtils.
So currently we can only code as following:
val prediction = Vectors.dense(originalPrediction.toArray.map(x => x*10))
assert(prediction ~== expected absTol 0.01, "prediction error")
If we support map/update for Vector, we can code as:
assert(originalPrediction.map(x => x*10) ~== expected absTol 0.01, "prediction
error")
However, MLlib/Matrix has already supported map/update/foreachActive function,
and we can compare two Matrices use ~== effortless.
> Add map/update function to MLlib/Vector
> ---------------------------------------
>
> Key: SPARK-9003
> URL: https://issues.apache.org/jira/browse/SPARK-9003
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: Yanbo Liang
> Priority: Minor
>
> MLlib/Vector only support foreachActive function and is short of map/update
> which is inconvenience for some Vector operations.
> For example:
> val a = Vectors.dense(...)
> If we want to compute math.log for each elements of a and get Vector as
> return value, we can only code as:
> val b = Vectors.dense(a.toArray.map(math.log))
> or we can use "toBreeze" and "fromBreeze" make transformation with breeze API.
> The code snippet is not elegant, we want it can implement:
> val c = a.map(math.log)
> Also currently MLlib/Matrix has implemented map/update/foreachActive
> function. I think Vector should also has map/update.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
