dongjoon-hyun edited a comment on issue #25460: [SPARK-25474][SQL][FOLLOW-UP] 
fallback to hdfs when relation table stats is not available
URL: https://github.com/apache/spark/pull/25460#issuecomment-521892398
 
 
   To @shahidki31 . 
   
   @maropu and @cloud-fan meant the corner case when the table size is equal to 
the user configuration value (not 8.0EB). Let say we set the configuration to 
1GB and we have a static table T1 whose size happens to be 1GB. In that case, 
every query on that tables might invoke this function. Although it's a very 
special case, but it's a regression.
   
   So, @cloud-fan and @maropu suggested to close this PR and proceed with 
#24715 .
   
   I'm +1 for that suggestion because that is the correct way.
   
   I know that you are worrying that #24715 doesn't resolve 8.0EB issue. 
However, that should be covered by your UTs in the previous PR. In the worst 
case, some of your code might be reverted. However, your test cases should 
survive there. It's your contribution. I believe @wangyum 's PR will pass your 
existing test cases in addition to his new test code. That's the way we make 
Apache Spark stronger.
   
   How do you think about this, @shahidki31 ? It's a way of collaboration.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to