Repository: spark
Updated Branches:
refs/heads/master 38391c9aa -> ff8dcc1d4
[SPARK-25235][SHELL] Merge the REPL code in Scala 2.11 and 2.12 branches
## What changes were proposed in this pull request?
Using some reflection tricks to merge Scala 2.11 and 2.12 codebase.
## How was this patch
Author: pwendell
Date: Wed Aug 29 03:16:01 2018
New Revision: 29001
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_28_20_02-38391c9 docs
[This commit notification would consist of 1478 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master 68ec207a3 -> 38391c9aa
[SPARK-25253][PYSPARK] Refactor local connection & auth code
This eliminates some duplication in the code to connect to a server on
localhost to talk directly to the jvm. Also it gives consistent ipv6 and error
Repository: spark
Updated Branches:
refs/heads/master 32c8a3d7b -> 68ec207a3
[SPARK-25260][SQL] Fix namespace handling in SchemaConverters.toAvroType
## What changes were proposed in this pull request?
`toAvroType` converts spark data type to avro schema. It always appends the
record name
Repository: spark
Updated Branches:
refs/heads/master bbbf81469 -> 32c8a3d7b
[MINOR] Avoid code duplication for nullable in Higher Order function
## What changes were proposed in this pull request?
Most of `HigherOrderFunction`s have the same `nullable` definition, ie. they
are nullable
Repository: spark
Updated Branches:
refs/heads/master 103854028 -> bbbf81469
[SPARK-22357][CORE] SparkContext.binaryFiles ignore minPartitions parameter
## What changes were proposed in this pull request?
Fix the issue that minPartitions was not used in the method. This is a simple
fix and I
Author: pwendell
Date: Tue Aug 28 23:15:56 2018
New Revision: 29000
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_28_16_01-1038540 docs
[This commit notification would consist of 1478 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master 7ad18ee9f -> 103854028
[SPARK-25212][SQL] Support Filter in ConvertToLocalRelation
## What changes were proposed in this pull request?
Support Filter in ConvertToLocalRelation, similar to how Project works.
Additionally, in Optimizer, run
Author: pwendell
Date: Tue Aug 28 21:15:27 2018
New Revision: 28998
Log:
Apache Spark 2.3.3-SNAPSHOT-2018_08_28_14_01-306e881 docs
[This commit notification would consist of 1443 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master aff8f15c1 -> 7ad18ee9f
[SPARK-25004][CORE] Add spark.executor.pyspark.memory limit.
## What changes were proposed in this pull request?
This adds `spark.executor.pyspark.memory` to configure Python's address space
limit,
Author: pwendell
Date: Tue Aug 28 19:16:55 2018
New Revision: 28997
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_28_12_01-aff8f15 docs
[This commit notification would consist of 1478 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master 4e3f3cebe -> aff8f15c1
[SPARK-25240][SQL] Fix for a deadlock in RECOVER PARTITIONS
## What changes were proposed in this pull request?
In the PR, I propose to not perform recursive parallel listening of files in
the `scanPartitions`
Repository: spark
Updated Branches:
refs/heads/branch-2.3 8db935f97 -> 306e881b6
[SPARK-24704][WEBUI] Fix the order of stages in the DAG graph
## What changes were proposed in this pull request?
Before:
Repository: spark
Updated Branches:
refs/heads/master de46df549 -> 4e3f3cebe
[SPARK-23679][YARN] Setting RM_HA_URLS for AmIpFilter to avoid redirect failure
in YARN mode
## What changes were proposed in this pull request?
YARN `AmIpFilter` adds a new parameter "RM_HA_URLS" to support RM HA,
Repository: spark
Updated Branches:
refs/heads/master 1149c4efb -> de46df549
[SPARK-23997][SQL] Configurable maximum number of buckets
## What changes were proposed in this pull request?
This PR implements the possibility of the user to override the maximum number
of buckets when saving to a
Repository: spark
Updated Branches:
refs/heads/master 592e3a42c -> 1149c4efb
[SPARK-25005][SS] Support non-consecutive offsets for Kafka
## What changes were proposed in this pull request?
As the user uses Kafka transactions to write data, the offsets in Kafka will be
non-consecutive. It
Repository: spark
Updated Branches:
refs/heads/master 8198ea501 -> 592e3a42c
[SPARK-25218][CORE] Fix potential resource leaks in TransportServer and
SocketAuthHelper
## What changes were proposed in this pull request?
Make sure TransportServer and SocketAuthHelper close the resources for
17 matches
Mail list logo