[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog
HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718330774 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog
HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718331516 Additionally, if the pattern is normal in Spark codebase I think we should revisit - if users configure something (A) and Spark decides to fail back (B), it must be only case where there's no functional difference between A and B (e.g. whole stage codegen failback might be OK as it should ideally only have difference on performance). Otherwise Spark is silently breaking the intention. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog
HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718331516 If the pattern is normal in Spark codebase I think we should revisit - if users configure something (A) and Spark decides to fail back (B), it must be only case where there's no behavioral difference between A and B (e.g. whole stage codegen failback might be OK as it should ideally only have difference on performance). Otherwise Spark is silently breaking the intention. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog
HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718273215 Whether previous behavior is making sense or not looks to be a major argument; I think silently ignoring the user's intention never makes sense. Setting the config reflects their intention, and end users will get annoyed if the intention is silently disregarded in any case. If they do the query in spark-shell they'll get the error log message so they might be wondering why and try to figure out immediately, but if they run the query in application they'll indicate it later (if they have good monitoring tool then earlier but not then probably delayed to when they encounter other issue) and the bad things are already occurred. We had examples on the harm about previous behavior, but we've not made clear the bright side of the previous behavior. Without enumerating the cases of bright side of the previous behavior I wouldn't agree about the statement this is not a bug but design call. Probably worth to revive original discussion thread to discuss this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog
HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718273215 Whether previous behavior is making sense or not looks to be a major argument; I think silently ignoring the user's intention never makes sense. Setting the config reflects their intention, and end users will get annoyed if the intention is silently disregarded in any case. If they do the query in spark-shell they'll get the error log message so they might be wondering why and try to figure out immediately, but if they run the query in application they'll indicate it later (if they have good monitoring tool then earlier but not then probably delayed to when they encounter other issue) and things are already occurred. We had examples on the harm about previous behavior, but we've not made clear the bright side of the previous behavior. Without enumerating the cases of bright side of the previous behavior I wouldn't agree about the statement this is not a bug but design call. Probably worth to revive original discussion thread to discuss this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog
HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-717716130 Isn't it declared as "incorrect" behavior in discussion thread in dev. mailing list? If that's not a bug what exactly we fix for master branch? I think it's a kind of serious bug as Spark "ignores" the end users' intention and silently fails back. Suppose the custom session catalog does audit then it silently skips auditing and continue to do the operation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
