Hi Shreelekhya, +1 for the feature. It was pending for a long time, thanks for working on it.
I have added comments in the doc, please check Regards, Akash R Nilugal On Wed, Mar 23, 2022 at 9:31 AM Kunal Kapoor <kunalkapoor...@gmail.com> wrote: > Hi Shreelekhya, > +1 for this feature, > > I could not understand how you would be able to handle compatibility, for > the old tables the SQL doesn't have to be rewritten, while for new tables > it has to be. So how will you decide this? > Same scenario for old table load > > Also, Need to benchmark the impact on query performance for a large number > of segments > > On Tue, Mar 22, 2022 at 10:28 AM Indhumathi <indhumuthumurug...@apache.org > > > wrote: > > > Hello Shreelekhya, > > > > Please find the comments inline in the design document link shared. > > > > Regards, > > Indhumathi M > > > > On 2022/03/17 16:09:10 Shreelekhya Gampa wrote: > > > Hi everyone, > > > > > > Currently, MV in carbon is supported as incremental or full refresh > load > > > based on the type of query. Whenever MV is created with Average > > aggregate, > > > a full refresh is done meaning it reloads the whole MV for any newly > > added > > > segments. This will slow down the loading. With an incremental data > load > > of > > > Average, only the segments that are newly added can be loaded to the > MV. > > > With the Average function, incremental loading doesn't work directly as > > the > > > average of avg(col) in MV won't give correct results. To achieve > > > proper results we need the sum and count of the columns. > > > > > > Following is the link to the design document. Please let me know > > > your thoughts about the same. > > > > > > https://docs.google.com/document/d/1kPEMCX50FLZcmyzm6kcIQtUH9KXWDIqh-Hco7NkTp80/edit?usp=sharing > > > > > > > > > Thanks, > > > Shreelekhya Gampa > > > > > >
