moulimukherjee opened a new pull request #470: WIP: Not throw exception if table is not found URL: https://github.com/apache/incubator-iceberg/pull/470 The call stack from create table looks like the following ```at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:74) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreCatalog.createTable(BaseMetastoreCatalog.java:70) at org.apache.iceberg.catalog.Catalog.createTable(Catalog.java:84) ``` Now if the `doRefresh` throws a `NoSuchTableException` which it should when a new table does not exist yet, then the job would throw this exception and would crash. On the other hand say `doRefresh` swallows the exception, then the call stack basically becomes a loop since `shouldRefresh` is not set to `false` and looks something like ```at com.stripe.mugatu.iceberg.custom_metastore.StripeTableOperations.doRefresh(StripeIcebergSource.scala:71) at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:74) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:83) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:83) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:83) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:83) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:83) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:83) at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:58) at org.apache.iceberg.BaseMetastoreCatalog.createTable(BaseMetastoreCatalog.java:70) ``` @rdblue @aokolnychyi thoughts?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
