Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r144931295
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/api/python/PythonSQLUtils.scala ---
@@ -29,4 +32,19 @@ private[sql] object PythonSQLUtils {
def listBuiltinFunctionInfos(): Array[ExpressionInfo] = {
FunctionRegistry.functionSet.flatMap(f =>
FunctionRegistry.builtin.lookupFunction(f)).toArray
}
+
+ /**
+ * Python Callable function to convert ArrowPayloads into a
[[DataFrame]].
+ *
+ * @param payloadRDD A JavaRDD of ArrowPayloads.
+ * @param schemaString JSON Formatted Schema for ArrowPayloads.
+ * @param sqlContext The active [[SQLContext]].
+ * @return The converted [[DataFrame]].
+ */
+ def arrowPayloadToDataFrame(
+ payloadRDD: JavaRDD[Array[Byte]],
+ schemaString: String,
+ sqlContext: SQLContext): DataFrame = {
--- End diff --
oh man, good catch! I don't know how that happened :\
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
