[
https://issues.apache.org/jira/browse/MAHOUT-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582736#comment-14582736
]
ASF GitHub Bot commented on MAHOUT-1660:
----------------------------------------
Github user dlyubimov commented on a diff in the pull request:
https://github.com/apache/mahout/pull/135#discussion_r32282538
--- Diff:
spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AtB.scala ---
@@ -27,92 +32,330 @@ import org.apache.spark.SparkContext._
import org.apache.log4j.Logger
import org.apache.mahout.math.drm.logical.OpAtB
+import scala.collection.mutable.ArrayBuffer
+
object AtB {
- private val log = Logger.getLogger(AtB.getClass)
+ private final implicit val log = getLog(AtB.getClass)
+ def atb[A: ClassTag](operator: OpAtB[A], srcA: DrmRddInput[A], srcB:
DrmRddInput[A]): DrmRddInput[Int] = {
+ atb_nograph_mmul(operator, srcA, srcB, operator.A.partitioningTag ==
operator.B.partitioningTag)
+ }
/**
* The logic for computing A'B is pretty much map-side generation of
partial outer product blocks
* over co-grouped rows of A and B. If A and B are identically
partitioned, we can just directly
* zip all the rows. Otherwise, we need to inner-join them first.
+ *
*/
- def atb_nograph[A: ClassTag](
- operator: OpAtB[A],
- srcA: DrmRddInput[A],
- srcB: DrmRddInput[A],
- zippable:Boolean = false
- ): DrmRddInput[Int] = {
+ @deprecated("slow, will remove", since = "0.10.2")
--- End diff --
it's replaced. remove is the plan. experiments show it's decisively losing
in current spark implementations. May change though with future spark
versions. one never can be sure.
On Thu, Jun 11, 2015 at 5:09 PM, Andrew Musselman <[email protected]>
wrote:
> In spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AtB.scala
> <https://github.com/apache/mahout/pull/135#discussion_r32281497>:
>
> > */
> > - def atb_nograph[A: ClassTag](
> > - operator: OpAtB[A],
> > - srcA: DrmRddInput[A],
> > - srcB: DrmRddInput[A],
> > - zippable:Boolean = false
> > - ): DrmRddInput[Int] = {
> > + @deprecated("slow, will remove", since = "0.10.2")
>
> Do we have a plan to improve/remove?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/mahout/pull/135/files#r32281497>.
>
> Hadoop1HDFSUtil.readDRMHEader should be taking Hadoop conf
> ----------------------------------------------------------
>
> Key: MAHOUT-1660
> URL: https://issues.apache.org/jira/browse/MAHOUT-1660
> Project: Mahout
> Issue Type: Bug
> Components: spark
> Affects Versions: 0.10.0
> Reporter: Suneel Marthi
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.10.2
>
>
> Hadoop1HDFSUtil.readDRMHEader should be taking Hadoop configuration from
> Context and not ignore it
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
