Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r212701416
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -114,6 +114,10 @@ package object config {
.checkValue(_ >= 0,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r212499194
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,20 @@ private[spark] object PythonEvalType {
*/
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r211763717
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/BaseYarnClusterSuite.scala
---
@@ -161,6 +162,11 @@ abstract class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r211763465
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,20 @@ private[spark] object PythonEvalType {
*/
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r211748871
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/BaseYarnClusterSuite.scala
---
@@ -161,6 +162,11 @@ abstract class
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r211750476
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,20 @@ private[spark] object PythonEvalType {
*/
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r211752765
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -91,6 +91,13 @@ private[spark] class Client(
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r210317086
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala
---
@@ -69,7 +67,7 @@ case class
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r210274748
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala
---
@@ -69,7 +67,7 @@ case class
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r210271735
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -91,6 +91,13 @@ private[spark] class Client(
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209992878
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,20 @@ private[spark] object PythonEvalType {
*/
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209823163
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,20 @@ private[spark] object PythonEvalType {
*/
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209771361
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/BaseYarnClusterSuite.scala
---
@@ -179,17 +185,23 @@ abstract class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209707560
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -137,13 +135,12 @@ case class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209707290
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,20 @@ private[spark] object PythonEvalType {
*/
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209703555
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209703452
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -91,6 +91,13 @@ private[spark] class Client(
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209702097
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209374683
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user rezasafi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209364194
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -137,13 +135,12 @@ case class
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209330276
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -91,6 +91,13 @@ private[spark] class Client(
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209329689
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209331503
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -137,13 +135,12 @@ case class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209209021
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,20 @@ private[spark] object PythonEvalType {
*/
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r209209726
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -137,13 +135,12 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208449418
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -137,13 +135,12 @@ case class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208339172
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208325737
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208310790
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,26 @@ private[spark] object PythonEvalType {
*/
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208302473
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,26 @@ private[spark] object PythonEvalType {
*/
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208120759
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,26 @@ private[spark] object PythonEvalType {
*/
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208091782
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,26 @@ private[spark] object PythonEvalType {
*/
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208051279
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,26 @@ private[spark] object PythonEvalType {
*/
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207747975
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207726897
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207726903
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -133,10 +133,17 @@ private[yarn] class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207722434
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207722412
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207722402
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -133,10 +133,17 @@ private[yarn] class
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716854
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716877
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -133,10 +133,17 @@ private[yarn] class
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716893
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -133,10 +133,17 @@ private[yarn] class
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716813
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207692117
--- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala
---
@@ -51,6 +52,17 @@ private[spark] class PythonRDD(
val bufferSize =
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207690569
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -81,6 +82,17 @@ case class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207637946
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207636504
--- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala
---
@@ -51,6 +52,17 @@ private[spark] class PythonRDD(
val bufferSize =
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207635841
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -81,6 +82,17 @@ case class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207631295
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -333,7 +340,7 @@ private[spark] class Client(
val
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207630342
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -114,6 +114,10 @@ package object config {
.checkValue(_ >= 0,
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207629997
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207596709
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -333,7 +340,7 @@ private[spark] class Client(
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207595892
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207597325
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala
---
@@ -89,6 +91,16 @@ abstract class EvalPythonExec(udfs:
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207601410
--- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala
---
@@ -51,6 +52,17 @@ private[spark] class PythonRDD(
val bufferSize =
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207597180
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -81,6 +82,17 @@ case class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207414776
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -114,6 +114,10 @@ package object config {
.checkValue(_ >=
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207409267
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -333,7 +340,7 @@ private[spark] class Client(
val
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207409771
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_PYTHON are correctly
GitHub user rdblue opened a pull request:
https://github.com/apache/spark/pull/21977
SPARK-25004: Add spark.executor.pyspark.memory limit.
## What changes were proposed in this pull request?
This adds `spark.executor.pyspark.memory` to configure Python's address
space
60 matches
Mail list logo
