Repository: spark Updated Branches: refs/heads/master a2016b4bc -> fdf63f124
[SPARK-8339] [PYSPARK] integer division for python 3 Itertools islice requires an integer for the stop argument. Switching to integer division here prevents a ValueError when vs is evaluated above. davies This is my original work, and I license it to the project. Author: Kevin Conor <[email protected]> Closes #6794 from kconor/kconor-patch-1 and squashes the following commits: da5e700 [Kevin Conor] Integer division for batch size Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fdf63f12 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/fdf63f12 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/fdf63f12 Branch: refs/heads/master Commit: fdf63f12490c674cc1877ddf7b70343c4fd6f4f1 Parents: a2016b4 Author: Kevin Conor <[email protected]> Authored: Fri Jun 19 00:12:20 2015 -0700 Committer: Davies Liu <[email protected]> Committed: Fri Jun 19 00:12:20 2015 -0700 ---------------------------------------------------------------------- python/pyspark/serializers.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/fdf63f12/python/pyspark/serializers.py ---------------------------------------------------------------------- diff --git a/python/pyspark/serializers.py b/python/pyspark/serializers.py index d8cdcda..7f9d0a3 100644 --- a/python/pyspark/serializers.py +++ b/python/pyspark/serializers.py @@ -272,7 +272,7 @@ class AutoBatchedSerializer(BatchedSerializer): if size < best: batch *= 2 elif size > best * 10 and batch > 1: - batch /= 2 + batch //= 2 def __repr__(self): return "AutoBatchedSerializer(%s)" % self.serializer --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
