Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/6230#discussion_r30537653
--- Diff: python/pyspark/sql/context.py ---
@@ -122,6 +122,26 @@ def udf(self):
"""Returns a :class:`UDFRegistration` for UDF registration."""
return UDFRegistration(self)
+ def range(self, start, end, step=1, numPartitions=None):
+ """
+ Create a :class:`DataFrame` with single LongType column named `id`,
+ containing elements in a range from `start` to `end` (exclusive)
with
+ step value `step`.
+
+ :param start: the start value
+ :param end: the end value (exclusive)
+ :param step: the incremental step (default: 1)
+ :param numPartitions: the number of partitions of the DataFrame
+ :return: A new DataFrame
+
+ >>> sqlContext.range(1, 7, 2).collect()
+ [Row(id=1), Row(id=3), Row(id=5)]
+ """
+ if numPartitions is None:
+ numPartitions = self._sc.defaultParallelism
+ jdf = self._ssql_ctx.range(int(start), int(end), int(step),
int(numPartitions))
--- End diff --
If the start or end is invalid, you will get an exception anyway. By
converting them in Python, we will got an exception in Python way (failed to
converted into `int`), not a Py4j exception (failed to find a method to call),
the later is much harder to understand for most of users.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
