Re: [GENERAL] JSONB performance enhancement for 9.6

2016-05-01 Thread Tom Smith
No, it is within the individual json object storage. In a way, it would be part of query plan, but strictly for the individual json object storage structure, it is not necessarily an "index" one possible(but primitive) implementation could be like having multiple "segments" in the storage, all keys

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-05-01 Thread Oleg Bartunov
On Sun, May 1, 2016 at 6:46 AM, Tom Smith wrote: > Hello: > > I'd like to bring this JSONB performance issue again. > Below is a link of MySQL way of storing/retrieving Json key/value > > https://dev.mysql.com/doc/refman/5.7/en/json.html > > Instead of providing column indexing(like GIN for JSONB

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-04-30 Thread Tom Smith
Hello: I'd like to bring this JSONB performance issue again. Below is a link of MySQL way of storing/retrieving Json key/value https://dev.mysql.com/doc/refman/5.7/en/json.html Instead of providing column indexing(like GIN for JSONB in Postgresql). it provides only internal data structure level

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-02-08 Thread Álvaro Hernández Tortosa
On 20/01/16 13:32, Bill Moran wrote: On Tue, 19 Jan 2016 23:53:19 -0300 Alvaro Herrera wrote: Bill Moran wrote: As far as a current solution: my solution would be to decompose the JSON into an optimized table. I.e.: CREATE TABLE store1 ( id SERIAL PRIMARY KEY, data JSONB ); CREATE TA

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-02-08 Thread Tom Smith
Using JSON/JSONB type in postgresql is usually due to the use case that the keys (top level included) can not be predefined. this is the major difference between NoSQL/Document and RDBMS. Why would TOAST have to be used? Can some speciailly structured "raw" files be used outside current databa

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-01-20 Thread Bill Moran
On Tue, 19 Jan 2016 23:53:19 -0300 Alvaro Herrera wrote: > Bill Moran wrote: > > > As far as a current solution: my solution would be to decompose the > > JSON into an optimized table. I.e.: > > > > CREATE TABLE store1 ( > > id SERIAL PRIMARY KEY, > > data JSONB > > ); > > > > CREATE TABLE s

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-01-20 Thread Dorian Hoxha
Is there any database that actually supports what the original poster wanted ? The only thing that I know that's similar is bigtable/hbase/hypertable wide column store. The way it works is: break the lexicographically sorted rows into blocks of compressed XXKB, and then keeps an index on the start

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-01-20 Thread Oleg Bartunov
On Wed, Jan 20, 2016 at 4:51 AM, Bruce Momjian wrote: > On Mon, Jan 11, 2016 at 09:01:03PM -0500, Tom Smith wrote: > > Hi, > > > > Congrats on the official release of 9.5 > > > > And I'd like bring up the issue again about if 9.6 would address the > jsonb > > performance issue > > with large numb

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-01-19 Thread Alvaro Herrera
Bill Moran wrote: > As far as a current solution: my solution would be to decompose the > JSON into an optimized table. I.e.: > > CREATE TABLE store1 ( > id SERIAL PRIMARY KEY, > data JSONB > ); > > CREATE TABLE store2 ( > id INT NOT NULL REFERENCES store1(id), > top_level_key VARCHAR(1024),

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-01-19 Thread Bruce Momjian
On Mon, Jan 11, 2016 at 09:01:03PM -0500, Tom Smith wrote: > Hi, > > Congrats on the official release of 9.5 > > And I'd like bring up the issue again about if 9.6 would address the jsonb > performance issue > with large number of top level keys. > It is true that it does not have to use JSON for

Re: [GENERAL] JSONB performance enhancement for 9.6

2016-01-11 Thread Tom Smith
Hi, Congrats on the official release of 9.5 And I'd like bring up the issue again about if 9.6 would address the jsonb performance issue with large number of top level keys. It is true that it does not have to use JSON format. it is about serialization and fast retrieval of dynamic tree structure

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-12-03 Thread Jim Nasby
On 11/29/15 9:30 AM, Arthur Silva wrote: Is this correct? I'm fairly sure jsonb supports lazily parsing objects and each object level is actually searched using binary search. The problem is there's no support for loading just part of a TOASTed field. Even if that existed, we'd still need a wa

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Tom Smith
Hi: The goal is fast retrieval of a a field value with a row when the row is already picked, one scenario is download a particular field value (if exists) of all rows in the table. It is actually a very common use case of exporting data of several user selected fields. The performance is extremel

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Tom Lane
Bill Moran writes: > Tom Smith wrote: >> Is there a plan for 9.6 to resolve the issue of very slow >> query/retrieval of jsonb fields when there are large number (maybe >> several thousands) of top level keys. Currently, if I save a large >> json document with top level keys of thousands and que

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Arthur Silva
Is this correct? I'm fairly sure jsonb supports lazily parsing objects and each object level is actually searched using binary search. Em 29/11/2015 11:25 AM, "Tom Smith" escreveu: > Hi, Thanks for everyone's response. > > The issue is not just compression, but lack of "indexing" or > "segmentati

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Bill Moran
On Sun, 29 Nov 2015 08:24:12 -0500 Tom Smith wrote: > Hi, Thanks for everyone's response. > > The issue is not just compression, but lack of "indexing" or "segmentation" > when a > single doc has, say 2000 top level keys (with multiple levels of subkeys). > right now, if I query for one key, t

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Tom Smith
Unfortunately, the keys can not be predefined or fixed. it is a doc, the reason jsonb is used. It works well for small docs with small number of keys. but really slow with large number of keys. If this issue is resolved, I think Postgresql would be an absolutely superior choice over MongoDB.for do

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Tom Smith
Hi, Thanks for everyone's response. The issue is not just compression, but lack of "indexing" or "segmentation" when a single doc has, say 2000 top level keys (with multiple levels of subkeys). right now, if I query for one key, the whole doc has to be first uncompressed and loaded and then sear

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Thomas Kellerer
Tom Smith schrieb am 29.11.2015 um 03:27: Hello: Is there a plan for 9.6 to resolve the issue of very slow query/retrieval of jsonb fields when there are large number (maybe several thousands) of top level keys. Currently, if I save a large json document with top level keys of thousands and qu

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Francisco Olarte
Hi: On Sun, Nov 29, 2015 at 1:09 PM, Bill Moran wrote: > On Sat, 28 Nov 2015 21:27:51 -0500 >> Currently, if I save a large json document with top level keys of thousands ** LARGE ** > The default storage for a JSONB field is EXTENDED. Switching it to > EXTERNAL will disable compression. You'll ha

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-29 Thread Bill Moran
On Sat, 28 Nov 2015 21:27:51 -0500 Tom Smith wrote: > > Is there a plan for 9.6 to resolve the issue of very slow query/retrieval > of jsonb fields > when there are large number (maybe several thousands) of top level keys. > Currently, if I save a large json document with top level keys of thousa

Re: [GENERAL] JSONB performance enhancement for 9.6

2015-11-28 Thread John R Pierce
On 11/28/2015 6:27 PM, Tom Smith wrote: Is there a plan for 9.6 to resolve the issue of very slow query/retrieval of jsonb fields when there are large number (maybe several thousands) of top level keys. Currently, if I save a large json document with top level keys of thousands and query/retrie

[GENERAL] JSONB performance enhancement for 9.6

2015-11-28 Thread Tom Smith
Hello: Is there a plan for 9.6 to resolve the issue of very slow query/retrieval of jsonb fields when there are large number (maybe several thousands) of top level keys. Currently, if I save a large json document with top level keys of thousands and query/retrieve field values, the whole document