Hi Shreelekhya, +1 for this feature, I could not understand how you would be able to handle compatibility, for the old tables the SQL doesn't have to be rewritten, while for new tables it has to be. So how will you decide this? Same scenario for old table load
Also, Need to benchmark the impact on query performance for a large number of segments On Tue, Mar 22, 2022 at 10:28 AM Indhumathi <indhumuthumurug...@apache.org> wrote: > Hello Shreelekhya, > > Please find the comments inline in the design document link shared. > > Regards, > Indhumathi M > > On 2022/03/17 16:09:10 Shreelekhya Gampa wrote: > > Hi everyone, > > > > Currently, MV in carbon is supported as incremental or full refresh load > > based on the type of query. Whenever MV is created with Average > aggregate, > > a full refresh is done meaning it reloads the whole MV for any newly > added > > segments. This will slow down the loading. With an incremental data load > of > > Average, only the segments that are newly added can be loaded to the MV. > > With the Average function, incremental loading doesn't work directly as > the > > average of avg(col) in MV won't give correct results. To achieve > > proper results we need the sum and count of the columns. > > > > Following is the link to the design document. Please let me know > > your thoughts about the same. > > > https://docs.google.com/document/d/1kPEMCX50FLZcmyzm6kcIQtUH9KXWDIqh-Hco7NkTp80/edit?usp=sharing > > > > > > Thanks, > > Shreelekhya Gampa > > >
