idrismike commented on issue #6787: URL: https://github.com/apache/hudi/issues/6787#issuecomment-1273452370
In this relevant discussion, I have encountered yet another problem. First of all, We have data already written using hudi `0.9.0`. And we are using spark `hive metastore` to read the data. **Note** The metadata is disabled (by default as in documentation) and hence no `table/.hoodie/metadata` folder exists. Reading using metastore works fine. Now, We upgraded to hudi `0.12.0`, and tried to read the data using `metastore` - and get a continuous warning (see screenshot) where it tries to refresh the table after few seconds, and throws warnings that it cannot find the metadata folder.  These warnings are shown continuously and no advancements made. As i see from the comments of @yihua , the reader by default does/should not use metadata - then why do we get these unlimited continuous refreshes and warnings, and it never advances. Is this confiugraiton `DEFAULT_METADATA_ENABLE_FOR_READERS` only used for spark DataSourceOptions? How can metadata be disabled for spark's `hive metastore`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
