Github user mkhaitman commented on the pull request:
https://github.com/apache/spark/pull/7532#issuecomment-133127541
Been testing this merged PR specifically and it looks great so far :). One
thing I noticed however is that the TaskSchedulerImpl throws and error once an
executor has
Github user mkhaitman commented on the pull request:
https://github.com/apache/spark/pull/1977#issuecomment-86640421
Tried this out with 1.3-rc3 and was getting FetchFailedExceptions while
performing a join between two RDDs:
org.apache.spark.shuffle.FetchFailedException
Github user mkhaitman commented on the pull request:
https://github.com/apache/spark/pull/1977#issuecomment-86688264
@davies Sorry, I deleted the comment though you still received the
notification. Think it was just a fluke since it didn't happen the second time.
Sorry about that! So
Github user mkhaitman commented on a diff in the pull request:
https://github.com/apache/spark/pull/1977#discussion_r27223983
--- Diff: python/pyspark/shuffle.py ---
@@ -244,72 +258,57 @@ def _next_limit(self):
def mergeValues(self, iterator):
Combine
Github user mkhaitman commented on a diff in the pull request:
https://github.com/apache/spark/pull/1977#discussion_r27223759
--- Diff: python/pyspark/shuffle.py ---
@@ -244,72 +258,57 @@ def _next_limit(self):
def mergeValues(self, iterator):
Combine
Github user mkhaitman commented on the pull request:
https://github.com/apache/spark/pull/1977#issuecomment-86236377
This PR looks amazing! I'm going to test this out tomorrow with 1.3-rc3 and
report back with some findings. I started taking a stab initially at trying to
improve
