Never used BRIN. I think in general DB of Airflow is rather small under most circumstances (comparing to the databases I used to work with) - and I think it could be a premature optimisation to switch type of index without an actual evidence it is a problem of any sort and that we should use it. Especially that it is Postgres-only - if find out that is worth fixing in Postgres (because there is a serious performance improvement we can get) then I think we must find a way how to make similar improvements in MySQL. There are some existing mysql installations out there.
J. On Tue, Dec 3, 2019 at 1:15 AM Kamil Breguła <kamil.breg...@polidea.com> wrote: > Hello, > > Has anyone tried to introduce BRIN indexes into the database schema? > > BRIN index is a great solution when we have a table in which the data is > arranged chronologically by date or sequence number. It provides similar > benefits to horizontal partitioning or sharding but without needing to > explicitly declare partitions. BRIN-based indexes consume significantly > less memory than B-tree indexes. It is supported by PostgreSQL > > In one project, where I have over 10 million records in one single table, > rows looup took over 5 minutes. After introduce BRIN indexes, the time has > dropped to less than 0.5 seconds. I personally love it. > > At first glance, this database structure perfectly matches our use case. > > Best regards, > Kamil Breguła > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
