spark git commit: [SPARK-19891][SS] Await Batch Lock notified on stream execution exit

Repository: spark Updated Branches: refs/heads/master 5949e6c44 -> 501b71119 [SPARK-19891][SS] Await Batch Lock notified on stream execution exit ## What changes were proposed in this pull request? We need to notify the await batch lock when the stream exits early e.g., when an exception has

spark git commit: [SPARK-19891][SS] Await Batch Lock notified on stream execution exit

Repository: spark Updated Branches: refs/heads/branch-2.1 a59cc369f -> f0d50fd54 [SPARK-19891][SS] Await Batch Lock notified on stream execution exit ## What changes were proposed in this pull request? We need to notify the await batch lock when the stream exits early e.g., when an exception

spark git commit: [SPARK-19008][SQL] Improve performance of Dataset.map by eliminating boxing/unboxing

Repository: spark Updated Branches: refs/heads/master 82138e09b -> 5949e6c44 [SPARK-19008][SQL] Improve performance of Dataset.map by eliminating boxing/unboxing ## What changes were proposed in this pull request? This PR improve performance of Dataset.map() for primitive types by removing

spark git commit: [SPARK-19886] Fix reportDataLoss if statement in SS KafkaSource

Repository: spark Updated Branches: refs/heads/branch-2.1 ffe65b065 -> a59cc369f [SPARK-19886] Fix reportDataLoss if statement in SS KafkaSource ## What changes were proposed in this pull request? Fix the `throw new IllegalStateException` if statement part. ## How is this patch tested Regre

spark git commit: [SPARK-19886] Fix reportDataLoss if statement in SS KafkaSource

Repository: spark Updated Branches: refs/heads/master f79371ad8 -> 82138e09b [SPARK-19886] Fix reportDataLoss if statement in SS KafkaSource ## What changes were proposed in this pull request? Fix the `throw new IllegalStateException` if statement part. ## How is this patch tested Regressio

spark git commit: [SPARK-19611][SQL] Introduce configurable table schema inference

Repository: spark Updated Branches: refs/heads/master cabe1df86 -> f79371ad8 [SPARK-19611][SQL] Introduce configurable table schema inference ## Summary of changes Add a new configuration option that allows Spark SQL to infer a case-sensitive schema from a Hive Metastore table's data files w

spark git commit: [SPARK-12334][SQL][PYSPARK] Support read from multiple input paths for orc file in DataFrameReader.orc

Repository: spark Updated Branches: refs/heads/master 30b18e693 -> cabe1df86 [SPARK-12334][SQL][PYSPARK] Support read from multiple input paths for orc file in DataFrameReader.orc Beside the issue in spark api, also fix 2 minor issues in pyspark - support read from multiple input paths for or

spark git commit: [SPARK-19861][SS] watermark should not be a negative time.

Repository: spark Updated Branches: refs/heads/branch-2.1 2a76e2420 -> ffe65b065 [SPARK-19861][SS] watermark should not be a negative time. ## What changes were proposed in this pull request? `watermark` should not be negative. This behavior is invalid, check it before real run. ## How was

spark git commit: [SPARK-19861][SS] watermark should not be a negative time.

Repository: spark Updated Branches: refs/heads/master 40da4d181 -> 30b18e693 [SPARK-19861][SS] watermark should not be a negative time. ## What changes were proposed in this pull request? `watermark` should not be negative. This behavior is invalid, check it before real run. ## How was this

spark git commit: [SPARK-19715][STRUCTURED STREAMING] Option to Strip Paths in FileSource

Repository: spark Updated Branches: refs/heads/master 3232e54f2 -> 40da4d181 [SPARK-19715][STRUCTURED STREAMING] Option to Strip Paths in FileSource ## What changes were proposed in this pull request? Today, we compare the whole path when deciding if a file is new in the FileSource for struc

spark git commit: [SPARK-19793] Use clock.getTimeMillis when mark task as finished in TaskSetManager.

Repository: spark Updated Branches: refs/heads/master b60b9fc10 -> 3232e54f2 [SPARK-19793] Use clock.getTimeMillis when mark task as finished in TaskSetManager. ## What changes were proposed in this pull request? TaskSetManager is now using `System.getCurrentTimeMillis` when mark task as fi

spark git commit: [SPARK-19757][CORE] DriverEndpoint#makeOffers race against CoarseGrainedSchedulerBackend#killExecutors

Repository: spark Updated Branches: refs/heads/master 206030bd1 -> b60b9fc10 [SPARK-19757][CORE] DriverEndpoint#makeOffers race against CoarseGrainedSchedulerBackend#killExecutors ## What changes were proposed in this pull request? While some executors are being killed due to idleness, if som

spark git commit: [SPARK-19561][SQL] add int case handling for TimestampType

Repository: spark Updated Branches: refs/heads/branch-2.1 0c140c168 -> 2a76e2420 [SPARK-19561][SQL] add int case handling for TimestampType ## What changes were proposed in this pull request? Add handling of input of type `Int` for dataType `TimestampType` to `EvaluatePython.scala`. Py4J ser

spark git commit: [SPARK-19561][SQL] add int case handling for TimestampType

Repository: spark Updated Branches: refs/heads/master 274973d2a -> 206030bd1 [SPARK-19561][SQL] add int case handling for TimestampType ## What changes were proposed in this pull request? Add handling of input of type `Int` for dataType `TimestampType` to `EvaluatePython.scala`. Py4J seriali

spark git commit: [SPARK-19763][SQL] qualified external datasource table location stored in catalog

Repository: spark Updated Branches: refs/heads/master eeb1d6db8 -> 274973d2a [SPARK-19763][SQL] qualified external datasource table location stored in catalog ## What changes were proposed in this pull request? If we create a external datasource table with a non-qualified location , we shou