spark jobs input/output http request

2015-05-06 Thread Saurabh Gupta
I have setup an AWS EMR based cluster, where in I am being able to run my spark queries quite ok. The next part of my work is to run the queries coming in from a webclient and show the results at it. The Spark queries as i Know I can only run from my EMR, and they don't return instantly with any

spark 1.3.1

2015-05-04 Thread Saurabh Gupta
HI, I am trying to build a example code given at https://spark.apache.org/docs/latest/sql-programming-guide.html#interoperating-with-rdds code is: // Import factory methods provided by DataType.import org.apache.spark.sql.types.DataType;// Import StructType and StructFieldimport

Re: spark 1.3.1

2015-05-04 Thread Saurabh Gupta
I am really new to this but what should I look into maven logs? I have tried mvn package -X -e SHould I show the full trace? On Mon, May 4, 2015 at 6:54 PM, Driesprong, Fokko fo...@driesprong.frl wrote: Hi Saurabh, Did you check the log of maven? 2015-05-04 15:17 GMT+02:00 Saurabh Gupta

Re: spark with standalone HBase

2015-04-30 Thread Saurabh Gupta
I am using hbase -0.94.8. On Wed, Apr 29, 2015 at 11:56 PM, Ted Yu yuzhih...@gmail.com wrote: Can you enable HBase DEBUG logging in log4j.properties so that we can have more clue ? What hbase release are you using ? Cheers On Wed, Apr 29, 2015 at 4:27 AM, Saurabh Gupta saurabh.gu

Re: spark with standalone HBase

2015-04-30 Thread Saurabh Gupta
(SparkSubmit.scala:111) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) On Thu, Apr 30, 2015 at 1:25 PM, Saurabh Gupta saurabh.gu...@semusi.com wrote: I am using hbase -0.94.8. On Wed, Apr 29, 2015 at 11:56 PM, Ted Yu yuzhih...@gmail.com wrote: Can you enable HBase DEBUG logging

spark with standalone HBase

2015-04-29 Thread Saurabh Gupta
Hi, I am working with standalone HBase. And I want to execute HBaseTest.scala (in scala examples) . I have created a test table with three rows and I just want to get the count using HBaseTest.scala I am getting this issue: 15/04/29 11:17:10 INFO BlockManagerMaster: Registered BlockManager