Github user ash211 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3882#discussion_r22428199
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala ---
@@ -269,6 +269,43 @@ class SQLContext(@transient val sparkContext:
SparkContext)
path, ScalaReflection.attributesFor[A], allowExisting, conf, this))
}
+
+ /**
+ * :: Experimental ::
+ * Creates an empty parquet file with the provided schema. The parquet
file thus created
+ * can be registered as a table, which can then be used as the target of
future
+ * `insertInto` operations.
+ *
+ * {{{
+ * val sqlContext = new SQLContext(...)
+ * import sqlContext._
+ *
+ * val schema = StructType(List(StructField("name",
StringType),StructField("age", IntegerType)))
+ * createParquetFile(schema,
"path/to/file.parquet").registerTempTable("people")
+ * sql("INSERT INTO people SELECT 'michael', 29")
+ * }}}
+ *
+ * @param schema StructType describing the records to be stored in the
Parquet file.
+ * @param path The path where the directory containing parquet metadata
should be created.
+ * Data inserted into this table will also be stored at this
location.
+ * @param allowExisting When false, an exception will be thrown if this
directory already exists.
+ * @param conf A Hadoop configuration object that can be used to specify
options to the parquet
+ * output format.
+ *
+ * @group userf
+ */
+ @Experimental
+ def createParquetFile(
--- End diff --
I kind of think createEmptyParquetFile would be a better name for this
method, since most Parquet files have data I'd think
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
