Github user ueshin closed the pull request at:
https://github.com/apache/spark/pull/19147
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138831553
--- Diff: python/pyspark/sql/functions.py ---
@@ -2111,6 +2126,53 @@ def wrapper(*args):
return wrapper
+def _udf(f, returnType,
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138831092
--- Diff: python/pyspark/sql/functions.py ---
@@ -2111,6 +2126,53 @@ def wrapper(*args):
return wrapper
+def _udf(f, returnType,
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138830622
--- Diff: python/pyspark/serializers.py ---
@@ -573,6 +573,39 @@ def __repr__(self):
return "UTF8Deserializer(%s)" % self.use_unicode
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138215300
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138093529
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala
---
@@ -62,6 +62,7 @@ import
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138012166
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala
---
@@ -62,6 +62,7 @@ import org.apache.spark.util.Utils
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138010327
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala
---
@@ -62,6 +62,7 @@ import org.apache.spark.util.Utils
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138005735
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138003592
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala
---
@@ -62,6 +62,7 @@ import org.apache.spark.util.Utils
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r138003254
--- Diff: python/pyspark/sql/tests.py ---
@@ -3122,6 +3124,147 @@ def test_filtered_frame(self):
self.assertTrue(pdf.empty)
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137945184
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala
---
@@ -62,6 +62,7 @@ import org.apache.spark.util.Utils
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137945084
--- Diff: python/pyspark/sql/tests.py ---
@@ -3122,6 +3124,147 @@ def test_filtered_frame(self):
self.assertTrue(pdf.empty)
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137929007
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137707828
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137568590
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137568255
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137507456
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137507341
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137287302
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/19147#discussion_r137286685
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/VectorizedPythonRunner.scala
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to
GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/19147
[WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Python
## What changes were proposed in this pull request?
This pr introduces vectorized UDFs in Python.
Note that this pr should
22 matches
Mail list logo
