Boosting spark.yarn.executor.memoryOverhead

2015-08-11 Thread Eric Bless
Previously I was getting a failure which included the message Container killed by YARN for exceeding memory limits. 2.1 GB of 2 GB physical memory used. Consider boosting spark.yarn.executor.memoryOverhead. So I attempted the following - spark-submit --jars examples.jar latest_msmtdt_by

Re: Problems getting expected results from hbase_inputformat.py

2015-08-10 Thread Eric Bless
row3'}) Just to be clear, you refer to "Spark update these two scripts recently.". What two scripts were you referencing? On Friday, August 7, 2015 7:59 PM, gen tang wrote: Hi, In fact, Pyspark use org.apache.spark.examples.pythonconverters(./examples/sr

Problems getting expected results from hbase_inputformat.py

2015-08-07 Thread Eric Bless
I’m having some difficulty getting the desired results fromthe Spark Python example hbase_inputformat.py. I’m running with CDH5.4, hbaseVersion 1.0.0, Spark v 1.3.0 Using Python version 2.6.6   I followed the example to create a test HBase table. Here’sthe data from the table I created – hbase(m