Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Peter Geoghegan
C-Tree? Peter Geoghegan (Sent from my phone)

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Heikki Linnakangas
On 09/04/2019 23:24, Ashwin Agrawal wrote: BTW, can I express a small measure of disappointment that the name for the thing under discussion on this thread chose to be called "zedstore"?  That seems to invite confusion with "zheap", especially in parts of the world where the last

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Ashwin Agrawal
i and I have been hacking recently for few weeks to implement > >>> in-core columnar storage for PostgreSQL. Here's the design and initial > >>> implementation of Zedstore, compressed in-core columnar storage (table > >>> access method). Attaching the patch and link

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Ashwin Agrawal
On Tue, Apr 9, 2019 at 11:29 AM Robert Haas wrote: > On Tue, Apr 9, 2019 at 11:51 AM Alvaro Herrera > wrote: > > This is not surprising, considering that columnar store is precisely the > > reason for starting the work on table AMs. > > > > We should certainly look into integrating some sort of

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Robert Haas
On Tue, Apr 9, 2019 at 11:51 AM Alvaro Herrera wrote: > This is not surprising, considering that columnar store is precisely the > reason for starting the work on table AMs. > > We should certainly look into integrating some sort of columnar storage > in mainline. Not sure which of zedstore or

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Konstantin Knizhnik
On 09.04.2019 19:19, Heikki Linnakangas wrote: On 09/04/2019 18:00, Konstantin Knizhnik wrote: Looks like the original problem was caused by internal postgres compressor: I have not configured Postgres to use lz4. When I configured Postgres --with-lz4, data was correctly inserted in zedstore

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Heikki Linnakangas
On 09/04/2019 18:00, Konstantin Knizhnik wrote: Looks like the original problem was caused by internal postgres compressor: I have not configured Postgres to use lz4. When I configured Postgres --with-lz4, data was correctly inserted in zedstore table, but looks it is not compressed at all:

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Konstantin Knizhnik
of Zedstore, compressed in-core columnar storage (table access method). Attaching the patch and link to github branch [1] to follow along. Thank you for publishing this patch. IMHO Postgres is really missing normal support of columnar store Yep. and table access method API is the best way of integrating

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Alvaro Herrera
On 2019-Apr-09, Konstantin Knizhnik wrote: > On 09.04.2019 3:27, Ashwin Agrawal wrote: > > Heikki and I have been hacking recently for few weeks to implement > > in-core columnar storage for PostgreSQL. Here's the design and initial > > implementation of Zedstore, compre

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Konstantin Knizhnik
On 09.04.2019 18:08, Heikki Linnakangas wrote: On 09/04/2019 18:00, Konstantin Knizhnik wrote: On 09.04.2019 17:09, Konstantin Knizhnik wrote: standard Postgres heap and my VOPS extension. As test data I used TPC-H benchmark (actually only one lineitem table generated with tpch-dbgen

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Heikki Linnakangas
On 09/04/2019 18:00, Konstantin Knizhnik wrote: On 09.04.2019 17:09, Konstantin Knizhnik wrote: standard Postgres heap and my VOPS extension. As test data I used TPC-H benchmark (actually only one lineitem table generated with tpch-dbgen utility with scale factor 10 (~8Gb database). I attached

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Konstantin Knizhnik
On 09.04.2019 17:09, Konstantin Knizhnik wrote: Hi, On 09.04.2019 3:27, Ashwin Agrawal wrote: Heikki and I have been hacking recently for few weeks to implement in-core columnar storage for PostgreSQL. Here's the design and initial implementation of Zedstore, compressed in-core columnar

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Konstantin Knizhnik
Hi, On 09.04.2019 3:27, Ashwin Agrawal wrote: Heikki and I have been hacking recently for few weeks to implement in-core columnar storage for PostgreSQL. Here's the design and initial implementation of Zedstore, compressed in-core columnar storage (table access method). Attaching the patch

Re: Zedstore - compressed in-core columnar storage

2019-04-08 Thread Ashwin Agrawal
implementation of Zedstore, compressed in-core columnar storage (table > > access method). > > That's very cool. > > > > Motivations / Objectives > > > > * Performance improvement for queries selecting subset of columns > > (reduced IO). > > * Reduced on-

Re: Zedstore - compressed in-core columnar storage

2019-04-08 Thread Andres Freund
Hi, On 2019-04-08 17:27:05 -0700, Ashwin Agrawal wrote: > Heikki and I have been hacking recently for few weeks to implement > in-core columnar storage for PostgreSQL. Here's the design and initial > implementation of Zedstore, compressed in-core columnar storage (table > access meth

<    1   2