Great! This is my plan for the summer too: to fix matrix operations on MRQL.
I noticed that you use adjacent list representation for matrices.
This may not be optimal for some operations.
Eg, transpose will need one map-reduce.
If you use a list of (i,j,value) tuples, transpose can be done using
just one map.
Some other systems use chunking, tiling, etc.
MRQL has code for matrix multiplication that uses Valiant's algorithm.
It is done using just one map-reduce (the mapper must replicate some
data), instead of the naive approach that uses two map-reduces (1 for
join and 1 for group-by).
This is described in the MRQL file src/MapReduce/GroupByJoinPlan.java
I need to fix the optimizer to generate plans with this operations. My
ultimate goal is to have matrix factorization working with near optimal
performance
in both map-reduce and BSP modes.
Leonidas
On 06/03/2013 06:17 PM, Karthik Kambatla wrote:
While this is not directly work on MRQL, have been working on cleaning up
my prior matrix implementations in MR now hosted at
https://github.com/kambatla/gramr. The intent is to port these to MRQL post
mavenization.
On Mon, Jun 3, 2013 at 4:14 PM, Karthik Kambatla <[email protected]> wrote:
Unfortunately, there was no activity the last 2 weeks.
I have been looking into mavenization on/off, will hopefully have
something to share this weekend. Will post an update either way.
On Mon, Jun 3, 2013 at 4:00 PM, Edward J. Yoon <[email protected]>wrote:
Unfortunately, there was no activity the last 2 weeks. We should discuss
this in the private email list.
Sorry for my inactive.
I'm mentoring 3 students working on Hama for Google Summer of Code
projects, and preparing release Hama 0.6.2. So, I'll be tied up for a
while.
On Tue, Jun 4, 2013 at 4:19 AM, Leonidas Fegaras <[email protected]>
wrote:
Alan,
Thanks for reminding us.
I will do the first draft.
Unfortunately, there was no activity the last 2 weeks. We should discuss
this in the private email list.
Leonidas
On Jun 2, 2013, at 12:06 AM, Alan Cabrera wrote:
Time for another Incubator report! Any volunteers?
Regards,
Alan
Begin forwarded message:
From: Marvin <[email protected]>
Subject: Incubator PMC/Board report for Jun 2013 ([ppmc])
Date: May 31, 2013 5:13:21 PM PDT
Dear podling,
This email was sent by an automated system on behalf of the Apache
Incubator PMC.
It is an initial reminder to give you plenty of time to prepare your
quarterly
board report.
The board meeting is scheduled for Wed, 19 June 2013, 10:30:00:00 PST.
The report
for your podling will form a part of the Incubator PMC report. The
Incubator PMC
requires your report to be submitted 2 weeks before the board
meeting, to
allow
sufficient time for review and submission (Wed, Jun 5th).
Please submit your report with sufficient time to allow the incubator
PMC, and
subsequently board members to review and digest. Again, the very
latest
you
should submit your report is 2 weeks prior to the board meeting.
Thanks,
The Apache Incubator PMC
Submitting your Report
----------------------
Your report should contain the following:
* Your project name
* A brief description of your project, which assumes no knowledge of
the
project
or necessarily of its field
* A list of the three most important issues to address in the move
towards
graduation.
* Any issues that the Incubator PMC or ASF Board might wish/need to be
aware of
* How has the community developed since the last report
* How has the project developed since the last report.
This should be appended to the Incubator Wiki page at:
http://wiki.apache.org/incubator/June2013
Note: This manually populated. You may need to wait a little before
this
page is
created from a template.
Mentors
-------
Mentors should review reports for their project(s) and sign them off
on
the
Incubator wiki page. Signing off reports shows that you are following
the
project - projects that are not signed may raise alarms for the
Incubator
PMC.
Incubator PMC
--
Best Regards, Edward J. Yoon
@eddieyoon
.
