jfsii commented on PR #4197: URL: https://github.com/apache/hive/pull/4197#issuecomment-1496625307
Re: @TuroczyX When it is re-thrown it will bubble up to the user as an exception during execution. It would be treated as other thrift exceptions would be - such as if the HMS server went down or the call timed out. It is not a good choice to swallow here, because a missing write notification will affect various systems such as maybe event notification and replication. It is much preferable to fail and make it visible so a user can take action (either by checking HMS logs, making sure HMS connectivity is good, etc) rather than having to figure out why the notification log is missing entries long after the fact. I feel like this is an oversight when attempting the fallback mechanism rather than a purposeful choice to ignore these exceptions. The reason I found this is due to the Impala catalogd implementation not implementing add_write_notification_log_in_batch which causes it to throw a different TApplicationException and thus the add_write_notification_log_in_batch is failed silently. Since catalogd also relies on the notification log/events, depending on timing of events, the metadata cache would end up containing incorrect metadata because it didn't realize a partition was updated. This could be fixed on the catalogd side of things to throw a UNKNOWN_METHOD or WRONG_METHOD_NAME exception, but it is still entirely possible for a TApplicationException of a different type to still be thrown. I've seen "UNKNOWN" in the past, but there also exists INVALID_MESSAGE_TYPE/BAD_SEQUENCE_ID/MISSING_RESULT which could very well show up depending on network conditions and/or issues during the processing on the HMS side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
