cloud-fan commented on a change in pull request #24715:
[SPARK-25474][SQL][DOCS] Update the docs for spark.sql.statistics.fallBackToHdfs
URL: https://github.com/apache/spark/pull/24715#discussion_r318409103
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1230,6 +1224,16 @@ object SQLConf {
.bytesConf(ByteUnit.BYTE)
.createWithDefault(Long.MaxValue)
+ val ENABLE_FALL_BACK_TO_HDFS_FOR_STATS =
buildConf("spark.sql.statistics.fallBackToHdfs")
+ .doc("When true, it will fall back to HDFS if the table statistics are not
available from " +
+ "table metadata. This is useful in determining if a table is small
enough to use auto " +
+ "broadcast joins. This flag is effective only if it is non-partitioned
Hive table. " +
Review comment:
`only for non-partitioned Hive tables`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
