Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r213060868
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +188,73 @@ private[spark] abstract class BasePythonRunner[IN,
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r213050049
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +188,73 @@ private[spark] abstract class
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r213043068
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +188,73 @@ private[spark] abstract class BasePythonRunner[IN,
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r213032992
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +188,73 @@ private[spark] abstract class BasePythonRunner[IN,
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r212762076
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +188,73 @@ private[spark] abstract class BasePythonRunner[IN,
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r212755510
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +188,73 @@ private[spark] abstract class BasePythonRunner[IN,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r212168597
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +99,143 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22085
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211683369
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +99,126 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211460405
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -20,15 +20,16 @@ package org.apache.spark.api.python
import
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211360719
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +99,126 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211356245
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +99,126 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211359959
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -381,6 +465,20 @@ private[spark] abstract class BasePythonRunner[IN,
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211358615
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +190,61 @@ private[spark] abstract class BasePythonRunner[IN,
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211356840
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -76,6 +77,15 @@ private[spark] abstract class BasePythonRunner[IN, OUT](
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211355022
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -20,15 +20,16 @@ package org.apache.spark.api.python
import
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211359028
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +190,61 @@ private[spark] abstract class BasePythonRunner[IN,
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211358743
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +190,61 @@ private[spark] abstract class BasePythonRunner[IN,
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211357983
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +190,61 @@ private[spark] abstract class BasePythonRunner[IN,
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r211182337
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +99,124 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r210148558
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209974729
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209862546
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +95,92 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209862336
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +95,92 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209853276
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +95,92 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209846054
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +95,92 @@ def getLocalProperty(self, key):
Get a local property set upstream in the driver,
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209846015
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +95,92 @@ def getLocalProperty(self, key):
Get a local property set upstream in the driver,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209835246
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +95,92 @@ def getLocalProperty(self, key):
Get a local property set upstream in the
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209834646
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209833972
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209830941
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class BasePythonRunner[IN,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209491244
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209491060
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209490553
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209476191
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209473946
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class BasePythonRunner[IN,
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209473919
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +96,33 @@ def getLocalProperty(self, key):
Get a local property set upstream in the driver,
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209473887
--- Diff: python/pyspark/taskcontext.py ---
@@ -95,3 +96,33 @@ def getLocalProperty(self, key):
Get a local property set upstream in the driver,
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209464591
--- Diff: python/pyspark/worker.py ---
@@ -275,6 +280,10 @@ def main(infile, outfile):
shuffle.DiskBytesSpilled = 0
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209464569
--- Diff: python/pyspark/worker.py ---
@@ -261,6 +263,9 @@ def main(infile, outfile):
# initialize global state
taskContext =
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209464679
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class BasePythonRunner[IN,
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209464515
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -180,7 +183,42 @@ private[spark] abstract class BasePythonRunner[IN,
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22085#discussion_r209464621
--- Diff: python/pyspark/taskcontext.py ---
@@ -29,6 +29,7 @@ class TaskContext(object):
"""
_taskContext = None
+
GitHub user jiangxb1987 opened a pull request:
https://github.com/apache/spark/pull/22085
[SPARK-25095][PySpark] Python support for BarrierTaskContext
## What changes were proposed in this pull request?
Add method `barrier()` and `getTaskInfos()` in python TaskContext,
44 matches
Mail list logo
