LantaoJin edited a comment on issue #26499: [SPARK-29869][SQL] Fix assertion error in HiveMetastoreCatalog#convertToLogicalRelation URL: https://github.com/apache/spark/pull/26499#issuecomment-554207146 Since this issue table has a nest data path: LOCATION is `/path/tablename/dt=20191113`. Data files are under `/path/tablename/dt=20191113/dt=20191113/xxx.parquet`. By default, Spark can not recursively lookup the data files. So the UT I added can not return correct result. But there is no assertion error now. I open #26525 to enable recursively lookup in SQL. But once recursively lookup enabled, the inferPartitioning will be disabled in current implementation. So this problem here can not be reproduced. So should we trim out the inferring partition cols for Hive table, anyhow? Or just throws a `CanNotConvertException` to handle it in upper layer?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
