Apache Spark 2.3 and Apache ORC 1.4 finally

Hi, All. Today, Apache Spark starts to use Apache ORC 1.4 as a `native` ORC implementation. SPARK-20728 Make OrcFileFormat configurable between `sql/hive` and `sql/core`. - https://github.com/apache/spark/commit/326f1d6728a7734c228d8bfaa69442a1c7b92e9b Thank you so much for all your supports for

Re: Leveraging S3 select

On 29 Nov 2017, at 21:45, Lalwani, Jayesh mailto:jayesh.lalw...@capitalone.com>> wrote: AWS announced at re:Invent that they are launching S3 Select. This can allow Spark to push down predicates to S3, rather than read the entire file in memory. Are there any plans to update Spark to use S3 S

Re: How to persistent database/table created in sparkSession

Try with `SparkSession.builder().enableHiveSupport` ? On Tue, Dec 5, 2017 at 3:22 PM, 163 wrote: > Hi, > How can I persistent database/table created in spark application? > > object TestPersistentDB { > def main(args:Array[String]): Unit = { > val spark = SparkSession