YannByron edited a comment on pull request #4083: URL: https://github.com/apache/hudi/pull/4083#issuecomment-1001894512
@dongkelun @xushiyan I'm sorry of not supporting this pr to solve the problems that `set the start query time of the table` and `query incrementally`. Some points we should think about: 1. as this pr, just work for spark-sql. What about Spark DataFrame Write? We should support both. 2. after adding `database` config, no matter get the `database` value by using a individual config like `hoodie.datasource.write.database.name` or parsing from the existing `hoodie.datasource.write.table.name`/`hoodie.table.name` when enable `hoodie.sql.uses.database.table.name`, we'll have four related options: `hoodie.datasource.hive_sync.table`, `hoodie.datasource.hive_sync.database` and the two mentioned above. Then, user have to learn these. Can we combine and simplify these? IMO, Hudi with a mountain of configs already has a high threshold of use. We should choose some solutions which balance the functionality and use experience as far as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
