Sujith Chacko created SPARK-27403: ------------------------------------- Summary: Failed to update the table size automatically even though spark.sql.statistics.size.autoUpdate.enabled is set as rue Key: SPARK-27403 URL: https://issues.apache.org/jira/browse/SPARK-27403 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.4.1 Environment: system shall able to update the table stats automatically if user sets spark.sql.statistics.size.autoUpdate.enabled as true , Currently this property doesnt have any significance even if it is enabled or disabled.
Please follow the below steps to reproduce the issue scala> spark.sql("create table x222 (name string,age int) stored as parquet") 19/04/07 23:41:56 WARN HiveMetaStore: Location: file:/D:/spark-2.4.1-bin-hadoop2.7/bin/spark-warehouse/table1 specified for non-external table:table1 -chgrp: 'HTIPL-23270\None' does not match expected pattern for group Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH... res0: org.apache.spark.sql.DataFrame = [] scala> spark.sql("insert into table1 select 'a',29") res2: org.apache.spark.sql.DataFrame = [] scala> spark.sql("desc extended table1").show(false) +----------------------------+--------------------------------------------------------------+-------+ |col_name |data_type |comment| +----------------------------+--------------------------------------------------------------+-------+ |name |string |null | |age |int |null | | | | | |# Detailed Table Information| | | |Database |default | | |Table |table1 | | |Owner |Administrator | | |Created Time |Sun Apr 07 23:41:56 IST 2019 | | |Last Access |Thu Jan 01 05:30:00 IST 1970 | | |Created By |Spark 2.4.1 | | |Type |MANAGED | | |Provider |hive | | |Table Properties |[transient_lastDdlTime=1554660716] | | |Location |file:/D:/spark-2.4.1-bin-hadoop2.7/bin/spark-warehouse/table1 | | |Serde Library |org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe | | |InputFormat |org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat | | |OutputFormat |org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat| | |Storage Properties |[serialization.format=1] | | |Partition Provider |Catalog | | +----------------------------+--------------------------------------------------------------+-------+ Reporter: Sujith Chacko -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org