spark git commit: [SPARK-24861][SS][TEST] create corrected temp directories in RateSourceSuite

Repository: spark Updated Branches: refs/heads/master 7db81ac8a -> 1462b1766 [SPARK-24861][SS][TEST] create corrected temp directories in RateSourceSuite ## What changes were proposed in this pull request? `RateSourceSuite` may leave garbage files under `sql/core/dummy`, we should use a

spark git commit: [SPARK-24195][CORE] Ignore the files with "local" scheme in SparkContext.addFile

Repository: spark Updated Branches: refs/heads/master 7e847646d -> 7db81ac8a [SPARK-24195][CORE] Ignore the files with "local" scheme in SparkContext.addFile ## What changes were proposed in this pull request? In Spark "local" scheme means resources are already on the driver/executor nodes,

spark git commit: [SPARK-24307][CORE] Support reading remote cached partitions > 2gb

Repository: spark Updated Branches: refs/heads/master 67e108daa -> 7e847646d [SPARK-24307][CORE] Support reading remote cached partitions > 2gb (1) Netty's ByteBuf cannot support data > 2gb. So to transfer data from a ChunkedByteBuffer over the network, we use a custom version of FileRegion

svn commit: r28240 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_19_16_01-67e108d-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Thu Jul 19 23:15:58 2018 New Revision: 28240 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_19_16_01-67e108d docs [This commit notification would consist of 1468 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24846][SQL] Made hashCode ExprId independent of jvmId

Repository: spark Updated Branches: refs/heads/master b3d88ac02 -> 67e108daa [SPARK-24846][SQL] Made hashCode ExprId independent of jvmId ## What changes were proposed in this pull request? Made ExprId hashCode independent of jvmId to make canonicalization independent of JVM, by overriding

spark git commit: [SPARK-22187][SS] Update unsaferow format for saved state in flatMapGroupsWithState to allow timeouts with deleted state

Repository: spark Updated Branches: refs/heads/master 8d707b060 -> b3d88ac02 [SPARK-22187][SS] Update unsaferow format for saved state in flatMapGroupsWithState to allow timeouts with deleted state ## What changes were proposed in this pull request? Currently, the group state of

spark git commit: [SPARK-24858][SQL] Avoid unnecessary parquet footer reads

Repository: spark Updated Branches: refs/heads/master 8b7d4f842 -> 6a9a058e0 [SPARK-24858][SQL] Avoid unnecessary parquet footer reads ## What changes were proposed in this pull request? Currently the same Parquet footer is read twice in the function `buildReaderWithPartitionValues` of

svn commit: r28223 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_19_04_02-8b7d4f8-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Thu Jul 19 11:22:25 2018 New Revision: 28223 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_19_04_02-8b7d4f8 docs [This commit notification would consist of 1468 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r28214 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_19_00_01-d05a926-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Thu Jul 19 07:17:53 2018 New Revision: 28214 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_19_00_01-d05a926 docs [This commit notification would consist of 1468 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24717][SS] Split out max retain version of state for memory in HDFSBackedStateStoreProvider

Repository: spark Updated Branches: refs/heads/master d05a926e7 -> 8b7d4f842 [SPARK-24717][SS] Split out max retain version of state for memory in HDFSBackedStateStoreProvider ## What changes were proposed in this pull request? This patch proposes breaking down configuration of retaining