Hello Everyone, Wanted to bring up some work I am doing in Hive to improve how we detect backwards incompatible changes to core APIs (HIVE-17129 <https://issues.apache.org/jira/browse/HIVE-17129> and HIVE-17130 <https://issues.apache.org/jira/browse/HIVE-17130>).
Hive contains InterfaceAudience and InterfaceStability annotations to mark specific Java classes as Public APIs. The goal of these annotations to help mark what parts of Hive can be considered part of the Hive API. The annotations were introduced a while ago, but have only been applied to a few Java classes. I've filed HIVE-17129 <https://issues.apache.org/jira/browse/HIVE-17129> to update the usage of these annotations so they apply to more parts of Hive, such as the SerDe and UDF APIs. If anyone has any comments or suggestions on what should and shouldn't be considered an API, feel free to comment on the JIRA. One major advantage of using these annotations is that it allows us to use tools such as the Java API Compliance Checker <https://github.com/lvc/japi-compliance-checker>. This tool checks for backwards incompatible changes to Java classes tagged with a specific annotation. It is used by a number of other Apache projects such as Hadoop, HBase, Kudu, and Spark (Spark uses a Scala equivalent). The tool provides us with an automated way of detecting backwards incompatible changes to a given API. We could setup a dedicated Jenkins job to run the tool, or even build it into the pre-commit checks. I've filed HIVE-17130 <https://issues.apache.org/jira/browse/HIVE-17130> to track this work. Wanted to surface this on the dev list in case anyone has comments on what should be considered a Public API and what shouldn't be. Thanks! --Sahil Takiar