Hi Congxian, Thanks for driving the proposal. As a heavy user of REST API, I’m a big +1 for it.
I filed a performance issue about table partitions earlier[1], please kindly group it under the umbrella issue. [1] https://github.com/apache/amoro/issues/2635 Best, Paul Lam > 2024年7月26日 16:32，Congxian Qiu <qcx978132...@gmail.com> 写道： > > Hi > I've crated the umbrella issue[1] for the improvement, > > [1] https://github.com/apache/amoro/issues/3064 > > Best, > Congxian > > > Congxian Qiu <qcx978132...@gmail.com> 于2024年7月22日周一 09:17写道： > >> Thanks all for the feedback. will create issues to track these >> improvements. and paste the parent issue here once created. >> >> Best, >> Congxian >> >> >> Jinsong Zhou <jinsongz...@apache.org> 于2024年7月15日周一 16:54写道： >> >>> Hi, >>> >>> Thanks for bringing up this discussion. >>> >>> Based on the examples from the documentation, the current causes of slow >>> API responses include: >>> 1. Slow database query: Some database tables may retain a large amount of >>> data, but they may not have been properly designed with appropriate >>> indexes >>> to speed up queries on these tables. >>> 2. Frequently table loading: Some operations result in frequent table >>> loading, which is a time-consuming operation to load tables from storage. >>> 3. Unreasonable implementation: The implementation of some methods >>> introduces unnecessary and costly operations. >>> >>> However, fixing these issues for all APIs one by one is time-consuming and >>> tedious work. Nevertheless, I am more than willing to participate in it. >>> >>> Best, >>> Jinsong >>> >>> On Fri, Jul 12, 2024 at 11:27 AM Xavier Bai <x...@apache.org> wrote: >>> >>>> Thank you for posting this proposal, some queries are indeed slower and >>> we >>>> can start by optimising the query overhead of the database first >>>> >>>> Congxian Qiu <qcx978132...@gmail.com> 于2024年7月11日周四 18:40写道： >>>> >>>>> Hi devs, >>>>> We have encountered some problems with Rest API access not working >>>>> efficiently when using Amoro recently, made a collation, and suggested >>>> some >>>>> possible solutions in the doc[1], please let me know what you think >>> about >>>>> it, thanks. >>>>> >>>>> The problem is summarised below: >>>>> 1. Amoro reads too many rows of data(some of which we do not need) >>> each >>>>> time it accesses the DB, which results in slow access. >>>>> 2. Amoro needs to access the external Catalog(e.g. HiveMetaStore) >>>> (multiple >>>>> times), resulting in slow access. >>>>> >>>>> [1] https://docs.qq.com/doc/DQU9sZ2RsdmRYSE1V >>>>> >>>>> Best, >>>>> Congxian >>>>> >>>> >>> >>
