zhoukang created SPARK-23692:
--------------------------------
Summary: Print metadata of files when infer schema failed
Key: SPARK-23692
URL: https://issues.apache.org/jira/browse/SPARK-23692
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.3.0
Reporter: zhoukang
A trivial modify.
Currently, when we had no input files to infer schema,we will throw below
exception.
For some users it may be misleading.If we can print files' metadata it will be
more clearer.
{code:java}
Caused by: org.apache.spark.sql.AnalysisException: Unable to infer schema for
Parquet. It must be specified manually.;
at
org.apache.spark.sql.execution.datasources.DataSource$$anonfun$8.apply(DataSource.scala:189)
at
org.apache.spark.sql.execution.datasources.DataSource$$anonfun$8.apply(DataSource.scala:189)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.execution.datasources.DataSource.org$apache$spark$sql$execution$datasources$DataSource$$getOrInferFileFormatSchema(DataSource.scala:188)
at
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:387)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:152)
at
org.apache.spark.sql.DataFrameReader.parquet(DataFrameReader.scala:441)
at
org.apache.spark.sql.DataFrameReader.parquet(DataFrameReader.scala:425)
at
com.xiaomi.matrix.pipeline.jobspark.importer.MatrixAdEventDailyImportJob.<init>(MatrixAdEventDailyImportJob.scala:18)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
