Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-12 Thread Timo Walther
Thanks for the feedback everyone. If there are no objections, I would like to continue with the voting now. We can still discuss class names or package locations during the implementation. But as far as I can see everyone agrees about the general proposition of this FLIP to improve the schema

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-10 Thread Timo Walther
Hi Jark, I don't think many users use WatermarkSpec. UniqueConstraint could cause some confusion but this mostly affects catalog or connector implementers. After deprecating the old APIs it should be obvious when an outdated interface is used. I'm fine with using a different name, do we have

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-09 Thread Jark Wu
Hi Timo, 1) I'm fine with `Column`, but are we going to introduce new interfaces for `UniqueConstraint` and `WatermarkSpec`? If we want to introduce a new stack, it would be better to have a different name, otherwise, it's easy to use a wrong class for users. Best, Jark On Wed, 10 Feb 2021 at

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-09 Thread Rui Li
I see. Makes sense to me. Thanks Timo for the detailed explanation! On Tue, Feb 9, 2021 at 9:48 PM Timo Walther wrote: > Hi Rui, > > 1. It depends whether you would like to declare (unresolved) or use > (resolved) a schema. In catalogs and APIs, people would actually like to > declare a schema.

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-09 Thread Timo Walther
Hi Rui, 1. It depends whether you would like to declare (unresolved) or use (resolved) a schema. In catalogs and APIs, people would actually like to declare a schema. Because the schema might reference objects from other catalogs etc. However, whenever the schema comes out of the framework it

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-09 Thread Rui Li
Hi Timo, Thanks for the FLIP. It looks good to me overall. I have two questions. 1. When should we use a resolved schema and when to use an unresolved one? 2. The FLIP mentions only resolved tables/views can be stored into a catalog. Does that mean the getTable method should also return a

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-09 Thread Timo Walther
Hi Jark, thanks for your feedback. Let me answer some of your comments: 1) Since we decided to build an entire new stack, we can also introduce better names for columns, constraints, and watermark spec. My goal was to shorten the names during this refactoring. Therefore, `TableSchema`

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-09 Thread Jark Wu
Hi Timo, The messy TableSchema confuses many developers. It's great to see we can finally come up with a clean interface hierarchy and still backward compatible. Thanks for preparing the nice FLIP. It looks good to me. I have some minor comments: 1) Should `ResolvedSchema#getColumn(int)`

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-08 Thread Dawid Wysakowicz
Hi Timo, From my perspective the proposed changes look good. I agree it is an important step towards FLIP-129 and FLIP-136. Personally I feel comfortable voting on the document. Best, Dawid On 05/02/2021 16:09, Timo Walther wrote: > Hi everyone, > > you might have seen that we discussed a

[DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-05 Thread Timo Walther
Hi everyone, you might have seen that we discussed a better schema API in past as part of FLIP-129 and FLIP-136. We also discussed this topic during different releases: https://issues.apache.org/jira/browse/FLINK-17793 Jark and I had an offline discussion how we can finally fix this