Hello, I'm not sure if I should have posted this to pgsql-advocacy, but this is being developed so I posted here.
Does anyone know if this development come to open source Postgres, or only to the cloud services of Microsoft and Google? (I wonder this will become another reason that Postgres won't incorporate optimizer hint feature.) Data systems that learn to be better http://news.mit.edu/2020/mit-data-systems-learn-be-better-tsunami-bao-0810 [Quote] -------------------------------------------------- As a first step toward this vision, Kraska and colleagues developed Tsunami and Bao. Tsunami uses machine learning to automatically re-organize a dataset’s storage layout based on the types of queries that its users make. Tests show that it can run queries up to 10 times faster than state-of-the-art systems. What’s more, its datasets can be organized via a series of "learned indexes" that are up to 100 times smaller than the indexes used in traditional systems. Bao, meanwhile, focuses on improving the efficiency of query optimization through machine learning. ... Traditional query optimizers take years to build, are very hard to maintain, and, most importantly, do not learn from their mistakes. Bao is the first learning-based approach to query optimization that has been fully integrated into the popular database management system PostgreSQL. Lead author Ryan Marcus, a postdoc in Kraska’s group, says that Bao produces query plans that run up to 50 percent faster than those created by the PostgreSQL optimizer, meaning that it could help to significantly reduce the cost of cloud services, like Amazon’s Redshift, that are based on PostgreSQL. Kraska says that in contrast to other learning-based approaches to query optimization, Bao learns much faster and can outperform open-source and commercial optimizers with as little as one hour of training time.In the future, his team aims to integrate Bao into cloud systems to improve resource utilization in environments where disk, RAM, and CPU time are scarce resources. ... The work was done as part of the Data System and AI Lab (DSAIL@CSAIL), which is sponsored by Intel, Google, Microsoft, and the U.S. National Science Foundation. -------------------------------------------------- Regards Takayuki Tsunakawa
