SPARK CREATING EXTERNAL TABLE

Hi All, I am facing different problem using spark, i am using spark-sql. below are the details, sqlcontext.sql("""create external table location '/' as select * from XXX""" ) this is my query table success fully done but in hive command describe formatted showing MANAGETABLE,

SPARK PERFORMANCE TUNING

Hi all, how to increase spark performance ,i am using pyspark. cluster info : Total memory :600gb Cores:96 command : spark-submit --master yarn-client --executor-memory 10G --num-executors 50 --executor-cores 2 --driver-memory 10g --queue thequeue please help on this -- Thanks

increase spark performance

Hi all, how to increase spark performance , cluster info : total memory :600gb cores -- Thanks K.Trinadh Ph-7348826118

Spark job failing with Adjusted frame length exceeds 2147483647: 2222367317 - discarded

Hi Team, I am encountering the below error while running a spark job.here i am mentioning spark submit command for reference. I request you to suggest me to avoid the below error and to let the program to run with out errors. CDH version: 5.6.0 Spark version: 1.5.0 spark-submit --deploy-mode