Currently Spark SQL doesn't support customizing schema name and
metadata. May I know why these two matters in your use case? Some
Parquet data models, like parquet-avro, do support it, while some others
don't (e.g. parquet-hive).
Cheng
On 9/21/15 7:13 AM, Borisa Zivkovic wrote:
Hi,
I am trying to figure out how to write parquet metadata when
persisting DataFrames to parquet using Spark (1.4.1)
I could not find a way to change schema name (which seems to be
hardcoded to root) and also how to add data to key/value metadata in
parquet footer.
org.apache.parquet.hadoop.metadata.FileMetaData#getKeyValueMetaData
org.apache.parquet.schema.Type#getName
thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
