I have a question about this part: After we support Paimon/hudi in the future, different forms will have different characteristics and unique functions. For example, paimon has a consumer streaming consumption meter mode[1]
In the Rest API design, can we do some isolation such as adding the /paimon prefix In terms of code, I do not recommend that the code completely couple the current iceberg code, otherwise it will bring huge maintenance costs 1. paimon consumer: https://paimon.apache.org/docs/master/flink/sql-query/#consumer-id Congxian Qiu <qcx978132...@gmail.com> 于2024年7月31日周三 14:51写道： > Thanks for the feedback, #2635 has been added to the umbrella issue. and > welcome anyone interested in this and contributing. thanks. > > Best, > Congxian > > > Paul Lam <paullin3...@gmail.com> 于2024年7月26日周五 17:11写道： > > > Hi Congxian, > > > > Thanks for driving the proposal. As a heavy user of REST API, I’m a big > +1 > > for it. > > > > I filed a performance issue about table partitions earlier[1], please > > kindly group it under the umbrella issue. > > > > [1] https://github.com/apache/amoro/issues/2635 > > > > Best, > > Paul Lam > > > > > 2024年7月26日 16:32，Congxian Qiu <qcx978132...@gmail.com> 写道： > > > > > > Hi > > > I've crated the umbrella issue[1] for the improvement, > > > > > > [1] https://github.com/apache/amoro/issues/3064 > > > > > > Best, > > > Congxian > > > > > > > > > Congxian Qiu <qcx978132...@gmail.com> 于2024年7月22日周一 09:17写道： > > > > > >> Thanks all for the feedback. will create issues to track these > > >> improvements. and paste the parent issue here once created. > > >> > > >> Best, > > >> Congxian > > >> > > >> > > >> Jinsong Zhou <jinsongz...@apache.org> 于2024年7月15日周一 16:54写道： > > >> > > >>> Hi, > > >>> > > >>> Thanks for bringing up this discussion. > > >>> > > >>> Based on the examples from the documentation, the current causes of > > slow > > >>> API responses include: > > >>> 1. Slow database query: Some database tables may retain a large > amount > > of > > >>> data, but they may not have been properly designed with appropriate > > >>> indexes > > >>> to speed up queries on these tables. > > >>> 2. Frequently table loading: Some operations result in frequent table > > >>> loading, which is a time-consuming operation to load tables from > > storage. > > >>> 3. Unreasonable implementation: The implementation of some methods > > >>> introduces unnecessary and costly operations. > > >>> > > >>> However, fixing these issues for all APIs one by one is > time-consuming > > and > > >>> tedious work. Nevertheless, I am more than willing to participate in > > it. > > >>> > > >>> Best, > > >>> Jinsong > > >>> > > >>> On Fri, Jul 12, 2024 at 11:27 AM Xavier Bai <x...@apache.org> wrote: > > >>> > > >>>> Thank you for posting this proposal, some queries are indeed slower > > and > > >>> we > > >>>> can start by optimising the query overhead of the database first > > >>>> > > >>>> Congxian Qiu <qcx978132...@gmail.com> 于2024年7月11日周四 18:40写道： > > >>>> > > >>>>> Hi devs, > > >>>>> We have encountered some problems with Rest API access not > working > > >>>>> efficiently when using Amoro recently, made a collation, and > > suggested > > >>>> some > > >>>>> possible solutions in the doc[1], please let me know what you think > > >>> about > > >>>>> it, thanks. > > >>>>> > > >>>>> The problem is summarised below: > > >>>>> 1. Amoro reads too many rows of data(some of which we do not need) > > >>> each > > >>>>> time it accesses the DB, which results in slow access. > > >>>>> 2. Amoro needs to access the external Catalog(e.g. HiveMetaStore) > > >>>> (multiple > > >>>>> times), resulting in slow access. > > >>>>> > > >>>>> [1] https://docs.qq.com/doc/DQU9sZ2RsdmRYSE1V > > >>>>> > > >>>>> Best, > > >>>>> Congxian > > >>>>> > > >>>> > > >>> > > >> > > > > > -- Best ConradJam
