[GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-08-01 Thread Jeff Janes
On Thursday, July 31, 2014, Larry White ljw1...@gmail.com javascript:_e(%7B%7D,'cvml','ljw1...@gmail.com'); wrote: Hi, I'm running an experiment on 9.4 beta 2. I put 275,000 identical JSON files into a table using JSONB (one per row). Each raw text file is 251K in size, so the total

Re: [GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-08-01 Thread Larry White
On Fri, Aug 1, 2014 at 2:20 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Thursday, July 31, 2014, Larry White ljw1...@gmail.com wrote: Hi, I'm running an experiment on 9.4 beta 2. I put 275,000 identical JSON files into a table using JSONB (one per row). Each raw text file is 251K in

Re: [GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-08-01 Thread Larry White
There is no TOAST compression on JSON or JSONB data in 9.4 beta 2. I'm not sure about other versions. I'm also not sure if this is a bug or by design, but if it is by design, I think the documentation should be updated. Here is a summary of my results inserting 10,000 highly compressible JSON

Re: [GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-08-01 Thread Jeff Janes
On Thu, Jul 31, 2014 at 11:36 PM, Larry White ljw1...@gmail.com wrote: On Fri, Aug 1, 2014 at 2:20 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Thursday, July 31, 2014, Larry White ljw1...@gmail.com wrote: Hi, I'm running an experiment on 9.4 beta 2. I put 275,000 identical JSON files

Re: [GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-08-01 Thread Larry White
Jeff, Thank you for your help. This is a Postgres bug, but I don't think I'd have figured it out without your help. What is happening is that if PG can, after compression, put the entire 'document' into one row/page in the toast table it does. However, if the document is too big to fit in one

Re: [GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-08-01 Thread Larry White
Reported as bug #11109. On Fri, Aug 1, 2014 at 1:46 PM, Larry White ljw1...@gmail.com wrote: Jeff, Thank you for your help. This is a Postgres bug, but I don't think I'd have figured it out without your help. What is happening is that if PG can, after compression, put the entire

[GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-07-31 Thread Larry White
Hi, I'm running an experiment on 9.4 beta 2. I put 275,000 identical JSON files into a table using JSONB (one per row). Each raw text file is 251K in size, so the total uncompressed is 69GB. The column storage is set to EXTENDED. There are other toastable columns in the table, but none have

Re: [GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-07-31 Thread Adrian Klaver
On 07/31/2014 01:44 PM, Larry White wrote: Hi, I'm running an experiment on 9.4 beta 2. I put 275,000 identical JSON files into a table using JSONB (one per row). Each raw text file is 251K in size, so the total uncompressed is 69GB. The column storage is set to EXTENDED. There are other

Re: [GENERAL] Very Limited Toast Compression on JSONB (9.4 beta 2)

2014-07-31 Thread Larry White
Yes. It was EXTENDED. As a further test, I dropped the table and rebuilt it, explicitly changing the EXTENDED designation to EXTERNAL and got exactly the same size TOAST table. So there was no compression at all with storage set to EXTENDED. On Thu, Jul 31, 2014 at 11:51 PM, Adrian Klaver