Thanks yana!
I will try it!
> 在 2015年1月16日,20:51,yana <yana.kadiy...@gmail.com
> <mailto:yana.kadiy...@gmail.com>> 写道:
>
> I think you might need to set
> spark.sql.hive.convertMetastoreParquet to false if I understand that flag
> correctly
>
> Sent on the new Sprint Network from my Samsung Galaxy S®4.
>
>
> -------- Original message --------
> From: Xiaoyu Wang
> Date:01/16/2015 5:09 AM (GMT-05:00)
> To: user@spark.apache.org <mailto:user@spark.apache.org>
> Subject: Why custom parquet format hive table execute "ParquetTableScan"
> physical plan, not "HiveTableScan"?
>
> Hi all!
>
> In the Spark SQL1.2.0.
> I create a hive table with custom parquet inputformat and outputformat.
> like this :
> CREATE TABLE test(
> id string,
> msg string)
> CLUSTERED BY (
> id)
> SORTED BY (
> id ASC)
> INTO 10 BUCKETS
> ROW FORMAT SERDE
> 'com.a.MyParquetHiveSerDe'
> STORED AS INPUTFORMAT
> 'com.a.MyParquetInputFormat'
> OUTPUTFORMAT
> 'com.a.MyParquetOutputFormat';
>
> And the spark shell see the plan of "select * from test" is :
>
> [== Physical Plan ==]
> [!OutputFaker [id#5,msg#6]]
> [ ParquetTableScan [id#12,msg#13], (ParquetRelation
> hdfs://hadoop/user/hive/warehouse/test.db/test
> <hdfs://hadoop/user/hive/warehouse/test.db/test>, Some(Configuration:
> core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml,
> yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml),
> org.apache.spark.sql.hive.HiveContext@6d15a113, []), []]
>
> Not HiveTableScan!!!
> So it dosn't execute my custom inputformat!
> Why? How can it execute my custom inputformat?
>
> Thanks!