alexeykudinkin commented on a change in pull request #5181:
URL: https://github.com/apache/hudi/pull/5181#discussion_r840747304
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/HoodieUnsafeRDDUtils.scala
##########
@@ -15,18 +16,23 @@
* limitations under the License.
*/
-package org.apache.spark
+package org.apache.spark.sql
import org.apache.hudi.HoodieUnsafeRDD
import org.apache.spark.rdd.RDD
import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.types.StructType
import org.apache.spark.util.MutablePair
/**
* Suite of utilities helping in handling instances of [[HoodieUnsafeRDD]]
*/
object HoodieUnsafeRDDUtils {
+ // TODO scala-doc
+ def createDataFrame(spark: SparkSession, rdd: RDD[InternalRow], structType:
StructType): DataFrame =
Review comment:
This will be used in the ColumnStatsIndexSupport after tests are
stabilized (in general it's more performant counterpart of `createDataFrame`
working on `Row`s, since this one doesn't require deserialization to Java types)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
