Re: Oryx + Spark mllib

2014-10-18 Thread Rajiv Abraham
Oryx 2 seems to be geared for Spark https://github.com/OryxProject/oryx 2014-10-18 11:46 GMT-04:00 Debasish Das : > Hi, > > Is someone working on a project on integrating Oryx model serving layer > with Spark ? Models will be built using either Streaming data / Batch data > in HDFS and cross val

Response to archived question 'Spark and Scala Worksheet'

2014-09-12 Thread Rajiv Abraham
Hi, This is a response to an archived email about how to run Spark in a Scala worksheet in the Scala IDE. http://mail-archives.apache.org/mod_mbox/spark-user/201401.mbox/%3ccaauywg8a+mjqwhtgytz0lumntlgfwa-noxtopeyadeq+gws...@mail.gmail.com%3E I know it's a bit late :) but here is how I do it. ht

Re: Junit spark tests

2014-09-12 Thread Rajiv Abraham
Hi Sudershan, That's interesting. I don't have an answer to your question but considering the functional nature of Spark, I have hardly had to use mock objects(maybe you could inform us of your use case). Mock object 'expectations' are in 'most' cases implementation of 'Tell, Don't ask' principle

Re: Need info on Spark's Communication/Networking layer...

2014-08-13 Thread Rajiv Abraham
Hi Aniket, Perhaps this video will help: https://www.youtube.com/watch?v=HG2Yd-3r4-M&list=PLTPXxbhUt-YWGNTaDj6HSjnHMxiTD1HCR&index=1 You can see other upto date videos and slides here at : http://spark-summit.org/2014/training Best regards, Rajiv 2014-08-13 19:36 GMT-04:00 aniketadnaik : > Hi,

Interested in contributing to GraphX in Python

2014-08-01 Thread Rajiv Abraham
Hi, I just saw Ankur's GraphX presentation and it looks very exciting! I would like to contribute to a Python version of GraphX. I checked out JIRA and Github but I did not find much info. - Are there limitations currently to port GraphX in Python? (e.g. Maybe the Python Spark RDD API is incomplet