Repository: spark Updated Branches: refs/heads/branch-1.4 f48f3a2e2 -> 164b9d32e
[SPARK-8339] [PYSPARK] integer division for python 3 Itertools islice requires an integer for the stop argument. Switching to integer division here prevents a ValueError when vs is evaluated above. davies This is my original work, and I license it to the project. Author: Kevin Conor <[email protected]> Closes #6794 from kconor/kconor-patch-1 and squashes the following commits: da5e700 [Kevin Conor] Integer division for batch size (cherry picked from commit fdf63f12490c674cc1877ddf7b70343c4fd6f4f1) Signed-off-by: Davies Liu <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/164b9d32 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/164b9d32 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/164b9d32 Branch: refs/heads/branch-1.4 Commit: 164b9d32e764b2a67b372a3d685b57c4bbeccbfa Parents: f48f3a2 Author: Kevin Conor <[email protected]> Authored: Fri Jun 19 00:12:20 2015 -0700 Committer: Davies Liu <[email protected]> Committed: Fri Jun 19 00:12:43 2015 -0700 ---------------------------------------------------------------------- python/pyspark/serializers.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/164b9d32/python/pyspark/serializers.py ---------------------------------------------------------------------- diff --git a/python/pyspark/serializers.py b/python/pyspark/serializers.py index d8cdcda..7f9d0a3 100644 --- a/python/pyspark/serializers.py +++ b/python/pyspark/serializers.py @@ -272,7 +272,7 @@ class AutoBatchedSerializer(BatchedSerializer): if size < best: batch *= 2 elif size > best * 10 and batch > 1: - batch /= 2 + batch //= 2 def __repr__(self): return "AutoBatchedSerializer(%s)" % self.serializer --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
