Re: separate spark and hive

2016-11-16 Thread Ricardo Almeida
gt; >> *From:* Mendelson, Assaf >> *Sent:* Tuesday, November 15, 2016 10:11 AM >> *To:* 'rxin [via Apache Spark Developers List]' >> *Subject:* RE: separate spark and hive >> >> >> >> Spark shell (and pyspark) by default create the spa

Re: separate spark and hive

2016-11-15 Thread Herman van Hövell tot Westerflier
documentation at http://spark.apache.org/docs/latest/configuration.html > has no mentioning of hive at all) > > Assaf. > > > > *From:* Mendelson, Assaf > *Sent:* Tuesday, November 15, 2016 10:11 AM > *To:* 'rxin [via Apache Spark Developers List]' > *Subject:*

RE: separate spark and hive

2016-11-15 Thread assaf.mendelson
no mentioning of hive at all) Assaf. From: Mendelson, Assaf Sent: Tuesday, November 15, 2016 10:11 AM To: 'rxin [via Apache Spark Developers List]' Subject: RE: separate spark and hive Spark shell (and pyspark) by default create the spark session with hive support (also true when the

RE: separate spark and hive

2016-11-15 Thread assaf.mendelson
] [mailto:ml-node+s1001551n19882...@n3.nabble.com] Sent: Tuesday, November 15, 2016 9:46 AM To: Mendelson, Assaf Subject: Re: separate spark and hive If you just start a SparkSession without calling enableHiveSupport it actually won't use the Hive catalog support. On Mon, Nov 14, 2016 at 11:

Re: separate spark and hive

2016-11-14 Thread Reynold Xin
ither a simple > configuration or even the default and that if there is any missing > functionality it should be documented. > > Assaf. > > > > > > *From:* Reynold Xin [mailto:r...@databricks.com] > *Sent:* Tuesday, November 15, 2016 9:31 AM > *To:* Mendelson, Assaf

RE: separate spark and hive

2016-11-14 Thread Mendelson, Assaf
@spark.apache.org Subject: Re: separate spark and hive I agree with the high level idea, and thus SPARK-15691<https://issues.apache.org/jira/browse/SPARK-15691>. In reality, it's a huge amount of work to create & maintain a custom catalog. It might actually make sense to do, but it j

Re: separate spark and hive

2016-11-14 Thread Reynold Xin
I agree with the high level idea, and thus SPARK-15691 . In reality, it's a huge amount of work to create & maintain a custom catalog. It might actually make sense to do, but it just seems a lot of work to do right now and it'd take a toll on inte