Re: [HACKERS] jsonb and nested hstore & VODKA

2014-03-31 Thread Josh Berkus
On 03/31/2014 09:34 AM, Robert Haas wrote: > On Fri, Mar 14, 2014 at 9:17 PM, Josh Berkus wrote: >> Precisely. Hence, the Russian plans for VODKA. > > Have these plans been shared publicly somewhere? Got a link? > Nothing other than the pgCon proposal. Presumably we'll know at pgCon, unless

Re: [HACKERS] jsonb and nested hstore

2014-03-31 Thread Robert Haas
On Fri, Mar 14, 2014 at 9:17 PM, Josh Berkus wrote: > On 03/14/2014 06:44 PM, Tomas Vondra wrote: >> Stupid question - so if I have a json like this: > > Not a stupid question, actually. In fact, I expect to answer it 400 or > 500 times over the lifespan of 9.4. > >> { "a" : { "b" : "c"}} >> >

Re: [HACKERS] jsonb and nested hstore

2014-03-23 Thread Peter Geoghegan
On Sun, Mar 23, 2014 at 11:10 AM, Tomas Vondra wrote: > Keeping jsonb_ops as the default seems better / safer to me. That's what I did. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/ma

Re: [HACKERS] jsonb and nested hstore

2014-03-23 Thread Tomas Vondra
On 21.3.2014 08:23, Peter Geoghegan wrote: > On Thu, Mar 13, 2014 at 3:39 PM, Peter Geoghegan wrote: >> On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark wrote: >>> It does sound like the main question here is which opclass should >>> be the default. From the discussion there's a jsonb_hash_ops >>> whi

Re: [HACKERS] jsonb and nested hstore

2014-03-21 Thread Greg Stark
On Fri, Mar 21, 2014 at 7:23 AM, Peter Geoghegan wrote: > I must admit that I'm coming around to the view that jsonb_hash_ops > would make a better default. Its performance is superb, and I think > there's a strong case to be made for that more than making up for it > not supporting all indexable

Re: [HACKERS] jsonb and nested hstore

2014-03-21 Thread Peter Geoghegan
On Thu, Mar 13, 2014 at 3:39 PM, Peter Geoghegan wrote: > On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark wrote: >> It does sound like the main question here is which opclass should be >> the default. From the discussion there's a jsonb_hash_ops which works >> on all input values but supports fewer o

Re: [HACKERS] jsonb and nested hstore

2014-03-20 Thread Peter Geoghegan
On Thu, Mar 20, 2014 at 5:32 AM, Alexander Korotkov wrote: > Besides implementation, what the idea was here? For me, it's impossible to > skip any single element, because it's possible for query to include only > this element. If we skip that element, we can't answer corresponding query > no more.

Re: [HACKERS] jsonb and nested hstore

2014-03-20 Thread Alexander Korotkov
I've noticed two commits on github. commit b8199ee3c2506ab81b47a0b440363fc90c0d6956 Author: Peter Geoghegan Date: Wed Mar 19 02:02:16 2014 -0700 For jsonb_hash_ops, hash less By limiting the GIN entries to the least-nested level, the delicious.com sample JSON dataset index shrinks

Re: [HACKERS] jsonb and nested hstore

2014-03-15 Thread Tomas Vondra
On 15.3.2014 02:15, Peter Geoghegan wrote: > On Fri, Mar 14, 2014 at 5:10 PM, Tomas Vondra wrote: >> I'm on commit a3115f0d, which is just 2 days old, so I suppose this was >> not fixed yet. > > Try merging the feature branch now, which will get you commit 16923d, > which you're missing. That was

Re: [HACKERS] jsonb and nested hstore

2014-03-15 Thread Tomas Vondra
On 15.3.2014 06:40, Peter Geoghegan wrote: > On Fri, Mar 14, 2014 at 6:44 PM, Tomas Vondra wrote: >> Well, depends on how you define useful. With the sample dataset >> 'delicious' (see Peter's post) I can do this: >> >>SELECT doc FROM delicious >> WHERE doc @> '{"title_detail" : {"va

Re: [HACKERS] jsonb and nested hstore

2014-03-15 Thread Greg Stark
On Sat, Mar 15, 2014 at 1:44 AM, Tomas Vondra wrote: > Because otherwise I don't understand how the index could be used for > queries with @> '{"a" : {"b" : "c"}}' conditions (i.e. path "[a,b]" with > value "c"). Hm, some experimentation here shows it does indeed work for queries like this and w

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
On Fri, Mar 14, 2014 at 6:44 PM, Tomas Vondra wrote: > Well, depends on how you define useful. With the sample dataset > 'delicious' (see Peter's post) I can do this: > >SELECT doc FROM delicious > WHERE doc @> '{"title_detail" : {"value" : "TheaterMania"}}'; > > with arbitrary paths

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Josh Berkus
On 03/14/2014 06:44 PM, Tomas Vondra wrote: > Stupid question - so if I have a json like this: Not a stupid question, actually. In fact, I expect to answer it 400 or 500 times over the lifespan of 9.4. > { "a" : { "b" : "c"}} > > the GIN code indexes {"b" : "c"} as a single value? And then t

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 15.3.2014 02:03, Greg Stark wrote: > On Fri, Mar 14, 2014 at 9:21 PM, Tomas Vondra wrote: >> I'm not awfully familiar with the GIN code, but based on Alexander's >> feedback I presume fixing the GIN length limit (or rather removing it, >> as it's a feature, not a bug) is quite straightforward.

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
On Fri, Mar 14, 2014 at 5:10 PM, Tomas Vondra wrote: > I'm on commit a3115f0d, which is just 2 days old, so I suppose this was > not fixed yet. Try merging the feature branch now, which will get you commit 16923d, which you're missing. That was an open item for a while, which I only got around to

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Greg Stark
On Fri, Mar 14, 2014 at 9:21 PM, Tomas Vondra wrote: > I'm not awfully familiar with the GIN code, but based on Alexander's > feedback I presume fixing the GIN length limit (or rather removing it, > as it's a feature, not a bug) is quite straightforward. Why not to at > least consider that for 9.4

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Andres Freund
On 2014-03-14 22:21:18 +0100, Tomas Vondra wrote: > Don't get me wrong - I'm aware it's quite late in the last commitfest, > and if it's deemed unacceptable / endandering 9.4 release, I'm not going > to say a word. But if it's a simple patch ... IMNSHO there's no bloody chance for such an addition

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 14.3.2014 23:06, Peter Geoghegan wrote: > For the benefit of anyone that would like to try the patch out, I make > available a custom format dump of some delicious sample data. I can > query the sample data as follows on my local installation: > > [local]/jsondata=# select count(*) from delicio

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 14.3.2014 22:54, Peter Geoghegan wrote: > On Fri, Mar 14, 2014 at 2:21 PM, Tomas Vondra wrote: >> I'm not awfully familiar with the GIN code, but based on Alexander's >> feedback I presume fixing the GIN length limit (or rather removing it, >> as it's a feature, not a bug) is quite straightforw

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
For the benefit of anyone that would like to try the patch out, I make available a custom format dump of some delicious sample data. I can query the sample data as follows on my local installation: [local]/jsondata=# select count(*) from delicious ; count - 1079399 (1 row) [local]/json

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
On Fri, Mar 14, 2014 at 2:21 PM, Tomas Vondra wrote: > I'm not awfully familiar with the GIN code, but based on Alexander's > feedback I presume fixing the GIN length limit (or rather removing it, > as it's a feature, not a bug) is quite straightforward. Why not to at > least consider that for 9.4

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 14.3.2014 20:18, Josh Berkus wrote: > On 03/14/2014 04:52 AM, Oleg Bartunov wrote: >> VODKA index will have no lenght limitation. > > Yeah, so I think we go with what we have, and tell people "if you're > hitting these length issues, wait for 9.5, where they will be > fixed." VODKA may be gre

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Josh Berkus
On 03/14/2014 12:45 PM, Oleg Bartunov wrote: > 9.5 may too optimistic :) Nonsense, you, Teodor and Alexander are geniuses. It can't possibly take you more than a year. ;-) -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postg

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Gavin Flower
On 15/03/14 08:45, Oleg Bartunov wrote: 9.5 may too optimistic :) On Fri, Mar 14, 2014 at 11:18 PM, Josh Berkus wrote: On 03/14/2014 04:52 AM, Oleg Bartunov wrote: VODKA index will have no lenght limitation. Yeah, so I think we go with what we have, and tell people "if you're hitting these l

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Oleg Bartunov
9.5 may too optimistic :) On Fri, Mar 14, 2014 at 11:18 PM, Josh Berkus wrote: > On 03/14/2014 04:52 AM, Oleg Bartunov wrote: >> VODKA index will have no lenght limitation. > > Yeah, so I think we go with what we have, and tell people "if you're > hitting these length issues, wait for 9.5, where

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Josh Berkus
On 03/14/2014 04:52 AM, Oleg Bartunov wrote: > VODKA index will have no lenght limitation. Yeah, so I think we go with what we have, and tell people "if you're hitting these length issues, wait for 9.5, where they will be fixed." -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com --

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Oleg Bartunov
VODKA index will have no lenght limitation. On Fri, Mar 14, 2014 at 3:07 PM, Tomas Vondra wrote: > On 13 Březen 2014, 23:39, Peter Geoghegan wrote: >> On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark wrote: >>> It does sound like the main question here is which opclass should be >>> the default. From

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 13 Březen 2014, 23:39, Peter Geoghegan wrote: > On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark wrote: >> It does sound like the main question here is which opclass should be >> the default. From the discussion there's a jsonb_hash_ops which works >> on all input values but supports fewer operators

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Peter Geoghegan
On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark wrote: > It does sound like the main question here is which opclass should be > the default. From the discussion there's a jsonb_hash_ops which works > on all input values but supports fewer operators and a jsonb_ops which > supports more operators but c

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Merlin Moncure
On Mon, Mar 10, 2014 at 4:18 AM, Peter Geoghegan wrote: > * Extensive additional documentation. References to the very new JSON > RFC. I think that this revision is in general a lot more coherent, and > I found that reflecting on what idiomatic usage should look like while > writing the documentat

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Tomas Vondra
On 13.3.2014 13:28, Oleg Bartunov wrote: > On Thu, Mar 13, 2014 at 4:21 PM, Alexander Korotkov > wrote: >> On Thu, Mar 13, 2014 at 1:21 PM, Greg Stark wrote: >>> >>> Well these are just normal gin and gist indexes. If we want to come up >>> with new index operator classess we can still do that an

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Andrew Dunstan
On 03/13/2014 10:49 AM, Greg Stark wrote: Another question. Is Peter's branch up to date with jsonb_populate_record() ? From discussions on list it sounds like the plan was to get rid of the use_json_as_text argument but his patch still has it. Yes, we're not changing that, and some people lik

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
Another question. Is Peter's branch up to date with jsonb_populate_record() ? From discussions on list it sounds like the plan was to get rid of the use_json_as_text argument but his patch still has it. (Tangentially, I wonder if it wouldn't be possible to make this a plain cast. I'm not sure but

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
On Thu, Mar 13, 2014 at 1:08 PM, Andrew Dunstan wrote: > ->> returns dequoted text if the value it points to is a plain string. If > it's not doing that then that's a bug. Sorry, I must have gotten confused between various tests. It does seem to be doing that. -- greg -- Sent via pgsql-hack

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Andrew Dunstan
On 03/13/2014 08:42 AM, Greg Stark wrote: Fwiw the jsonb data doesn't actually seem to be any smaller than text json on this data set (this is avg(pg_column_size(col)) and I checked, they're both using the same amount of toast space) jsonb | json ---+--- 813.5 | 716.3 (1 row) Tha

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Andrew Dunstan
On 03/13/2014 06:53 AM, Greg Stark wrote: I also find it awkward that col->>'prop' returns the json representation of the property. If it's text that means it's double-quoted. I would think that a user storing text in a json property would want a way to pull out the text that json property repr

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
Fwiw the jsonb data doesn't actually seem to be any smaller than text json on this data set (this is avg(pg_column_size(col)) and I checked, they're both using the same amount of toast space) jsonb | json ---+--- 813.5 | 716.3 (1 row) It's still more than 7x faster in cpu costs though:

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Oleg Bartunov
On Thu, Mar 13, 2014 at 4:21 PM, Alexander Korotkov wrote: > On Thu, Mar 13, 2014 at 1:21 PM, Greg Stark wrote: >> >> Well these are just normal gin and gist indexes. If we want to come up >> with new index operator classess we can still do that and keep the old >> ones if necessary. Even that se

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Alexander Korotkov
On Thu, Mar 13, 2014 at 1:21 PM, Greg Stark wrote: > Well these are just normal gin and gist indexes. If we want to come up > with new index operator classess we can still do that and keep the old > ones if necessary. Even that seems pretty unlikely from past experience. > > I'm actually pretty s

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
Fwiw I have a few questions -- but beware, I'm a complete neophyte when it comes to jsonb style document databases so these are more likely to represent misconceptions on my part than problems with jsonb. I naively though a gin index on a jsonb would help with queries like WHERE col->'prop' = 'val

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
On Thu, Mar 13, 2014 at 6:15 AM, Bruce Momjian wrote: > On Wed, Mar 12, 2014 at 01:58:14PM -0700, Peter Geoghegan wrote: >> The use case you describe here doesn't sound like something similar to >> full text search. It sounds like something identical. >> >> In any case, let's focus on what we have

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Bruce Momjian
On Wed, Mar 12, 2014 at 01:58:14PM -0700, Peter Geoghegan wrote: > The use case you describe here doesn't sound like something similar to > full text search. It sounds like something identical. > > In any case, let's focus on what we have right now. I think that the > indexing facilities proposed

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 22:43, Peter Geoghegan wrote: > On Wed, Mar 12, 2014 at 2:30 PM, Tomas Vondra wrote: >> I think that's unfounded assumption. Many users actually have very >> little control over the documents or queries - a nice example may be the >> mail archive, with headers stored in a hstore/jsonb

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Stephen Frost
* Tomas Vondra (t...@fuzzy.cz) wrote: > So I think it's quite difficult to give simple and exact explanation in > the docs, other than "there are limits, but it's difficult to say when > you hit them". Arrays have more-or-less the same issue... Thanks, Stephen signature

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 21:55, Josh Berkus wrote: > Andrew, Peter: > > Just so I'm clear on the limits here, lemme make sure I understand this: > > a) GIN indexing is limited to ~~1500chars The exact message I get is this: ERROR: index row size 1944 exceeds maximum 1352 for index "tmp_idx" so it's 1352B.

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 2:30 PM, Tomas Vondra wrote: > I think that's unfounded assumption. Many users actually have very > little control over the documents or queries - a nice example may be the > mail archive, with headers stored in a hstore/jsonb. I have absolutely > no control over the header

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 21:58, Peter Geoghegan wrote: > > The use case you describe here doesn't sound like something similar to > full text search. It sounds like something identical. I think this very depends on the definition of full text search. > In any case, let's focus on what we have right now. I t

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 20:40, Peter Geoghegan wrote: > On Wed, Mar 12, 2014 at 6:20 AM, Tomas Vondra wrote: >> I'm still not sure how would that look. Does that mean I'd have to >> create multiple GIN indexes - one for each possible key or >> something like that? Can you give an example? > > It could mean

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Andrew Dunstan
On 03/12/2014 04:58 PM, Peter Geoghegan wrote: In any case, let's focus on what we have right now. I think that the indexing facilities proposed here are solid. In any case they do not preclude working on better indexing strategies as the need emerges. I quite agree, didn't mean to suggest o

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 1:37 PM, Andrew Dunstan wrote: > One major use case for using treeish data types in the first place is that > you don't know when you're designing the database exactly what shape the > data will be. If you don't know that, then how are you supposed to know what > in it will

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Josh Berkus
Andrew, Peter: Just so I'm clear on the limits here, lemme make sure I understand this: a) GIN indexing is limited to ~~1500chars b) The "value", which includes everything other than the top level set of keys, is one item as far as GIN is concerned. Therefore: we are limited to indexing JSON wh

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Andrew Dunstan
On 03/12/2014 04:10 PM, Peter Geoghegan wrote: On Wed, Mar 12, 2014 at 11:57 AM, Oleg Bartunov wrote: Also, GiST index is faster for create/update operations. I really hope we will improve jsonb indexing in the next one-two releases. For now I'd suggest people index expressional indexes to ind

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Oleg Bartunov
On Thu, Mar 13, 2014 at 12:10 AM, Peter Geoghegan wrote: > On Wed, Mar 12, 2014 at 11:57 AM, Oleg Bartunov wrote: >> Also, GiST index is faster for create/update operations. I really hope we >> will >> improve jsonb indexing in the next one-two releases. For now I'd suggest >> people >> index e

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 11:57 AM, Oleg Bartunov wrote: > Also, GiST index is faster for create/update operations. I really hope we will > improve jsonb indexing in the next one-two releases. For now I'd suggest > people > index expressional indexes to index just interesting keys or use GiST. Whe

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 6:20 AM, Tomas Vondra wrote: > I'm still not sure how would that look. Does that mean I'd have to create > multiple GIN indexes - one for each possible key or something like that? > Can you give an example? It could mean that you're obliged to create multiple indexes, yes.

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Oleg Bartunov
Also, GiST index is faster for create/update operations. I really hope we will improve jsonb indexing in the next one-two releases. For now I'd suggest people index expressional indexes to index just interesting keys or use GiST. On Wed, Mar 12, 2014 at 5:15 PM, Tomas Vondra wrote: > On 12 Březen

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12 Březen 2014, 0:51, Peter Geoghegan wrote: > On Tue, Mar 11, 2014 at 4:41 PM, Peter Geoghegan wrote: >> I think that in practice the >> general recommendation will be that when indexing at the "top level", >> use jsonb_hash_ops. When indexing nested items, use the more flexible >> default GIN

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12 Březen 2014, 0:41, Peter Geoghegan wrote: > On Tue, Mar 11, 2014 at 3:58 PM, Tomas Vondra wrote: >> ERROR: index row size 1416 exceeds maximum 1352 for index "gin_idx" > > All index AMs have similar restrictions. Yes, I know and I have no problem with restrictions in general. You may run

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Peter Geoghegan
On Tue, Mar 11, 2014 at 4:41 PM, Peter Geoghegan wrote: > I think that in practice the > general recommendation will be that when indexing at the "top level", > use jsonb_hash_ops. When indexing nested items, use the more flexible > default GIN opclass. That seems like a pretty smart trade-off to

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Peter Geoghegan
On Tue, Mar 11, 2014 at 3:58 PM, Tomas Vondra wrote: > ERROR: index row size 1416 exceeds maximum 1352 for index "gin_idx" All index AMs have similar restrictions. > A good example of such header is "dkim-signature" which basically > contains the whole message digitally signed with DKIM. The

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Tomas Vondra
Hi, I've spent a few hours stress-testing this a bit - loading a mail archive with ~1M of messages (with headers stored in a jsonb column) and then doing queries on that. Good news - no crashes or any such issues so far. The queries that I ran manually seem to return sane results. The only proble

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Alexander Korotkov
On Tue, Mar 11, 2014 at 5:19 AM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 4:19 AM, Alexander Korotkov > wrote: > > Here it is. > > So it looks like what you have here is analogous to the other problems > that I fixed with both GiST and GIN. That isn't surprising, and this > does fix my t

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 4:19 AM, Alexander Korotkov wrote: > Here it is. So it looks like what you have here is analogous to the other problems that I fixed with both GiST and GIN. That isn't surprising, and this does fix my test-case. I'm not terribly happy about the lack of explanation for the

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Andrew Dunstan
On 03/10/2014 10:50 AM, Andrew Dunstan wrote: Thanks for your work on this. It's just occurred to me that we'll need to add hstore_to_jsonb functions and a cast to match the hstore_to_json functions and cast. That should be fairly simple - I'll work on that. It need not hold up progress wi

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Andrew Dunstan
On 03/10/2014 05:18 AM, Peter Geoghegan wrote: On Fri, Mar 7, 2014 at 9:00 AM, Bruce Momjian wrote: OK, it sounds like the adjustments are minimal, like not using the high-order bit. Attached patch is a refinement of the work of Oleg, Teodor and Andrew. Revisions are mostly my own, although A

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 3:04 PM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 3:47 AM, Alexander Korotkov > wrote: > > Fix is attached. > > Could you post a patch with regression tests, please? > Here it is. -- With best regards, Alexander Korotkov. jsonb-hash-ops-fix-2.patch Descrip

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:47 AM, Alexander Korotkov wrote: > Fix is attached. Could you post a patch with regression tests, please? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpr

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 2:19 PM, Peter Geoghegan wrote: > On Mon, Mar 10, 2014 at 3:00 AM, Alexander Korotkov > wrote: > > I din't get comment about "leftmost" element. There is absolutely no > > distinguish between array elements. All elements are extracted into same > > keys independent of the

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:21 AM, Peter Geoghegan wrote: > Sorry, I realize now that that must be incorrect. Still, please take a > look at the commit linked to. To be clear, I mean that my explanation of why this was missed before was incorrect, not my contention that it's a problem right now (fo

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:19 AM, Peter Geoghegan wrote: > I don't have time to post that at the moment, but offhand I *think* > your confusion may be due to the fact that the json_hash_ops opclass > (as I call it) was previously consistent with the behavior of the > other GIN opclass (the default)

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:00 AM, Alexander Korotkov wrote: > I din't get comment about "leftmost" element. There is absolutely no > distinguish between array elements. All elements are extracted into same > keys independent of their indexes. It seems to have no change since I wrote > hstore_hash_o

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 1:18 PM, Peter Geoghegan wrote: > * The jsonb_hash_ops non-default GIN opclass is broken. It has its own > idiosyncratic notion of what constitutes containment, that sees it > only return, say, jsonb arrays that have a matching string as their > "leftmost" element (if we a

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread David E. Wheeler
On Mar 6, 2014, at 1:51 AM, Peter Geoghegan wrote: >> It's true for perl. Syntax of hstore is close to hash/array syntax and it's >> easy serialize/deserialize hstore to/from perl. Syntax of hstore was >> inspired by perl. > > I understand that. There is a module on CPAN called Pg::hstore that >

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Bruce Momjian
On Fri, Mar 7, 2014 at 11:57:48AM -0500, Andrew Dunstan wrote: > >If they can be done for 9.4, great, if not, we have to decide if these > >suboptimal cases are enough for us to delay the data type until 9.5. I > >don't know the answer, but I have to ask the question. > > > > AFAIK, there is no

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Andrew Dunstan
On 03/07/2014 11:45 AM, Bruce Momjian wrote: On Fri, Mar 7, 2014 at 11:35:41AM -0500, Andrew Dunstan wrote: IIRC The sacrifice was one bit in the header (i.e. in the first int after the varlena header). We could now repurpose that (for example if we ever decided to use a new format). Oleg and

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Bruce Momjian
On Fri, Mar 7, 2014 at 11:35:41AM -0500, Andrew Dunstan wrote: > IIRC The sacrifice was one bit in the header (i.e. in the first int > after the varlena header). We could now repurpose that (for example > if we ever decided to use a new format). > > Oleg and Teodor made most of the adjustments on

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Andrew Dunstan
On 03/06/2014 11:33 PM, Bruce Momjian wrote: On Thu, Mar 6, 2014 at 09:50:56PM +0400, Oleg Bartunov wrote: Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to the flagship feature (jsonb)

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Merlin Moncure
On Thu, Mar 6, 2014 at 10:33 PM, Bruce Momjian wrote: > On Thu, Mar 6, 2014 at 09:50:56PM +0400, Oleg Bartunov wrote: >> Hi there, >> >> Looks like consensus is done. I and Teodor are not happy with it, but >> what we can do :) One thing I want to do is to reserve our >> contribution to the fl

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Bruce Momjian
On Thu, Mar 6, 2014 at 04:33:08PM +0100, Ronan Dunklau wrote: > > I'm not sure what the constraints of json that you might want to break > > are. Perhaps you'd like to specify. > > I haven't followed the whole thread, but json is really restrictive on the > supported types: a hierarchical hstore

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Bruce Momjian
On Thu, Mar 6, 2014 at 09:50:56PM +0400, Oleg Bartunov wrote: > Hi there, > > Looks like consensus is done. I and Teodor are not happy with it, but > what we can do :) One thing I want to do is to reserve our > contribution to the flagship feature (jsonb), particularly, "binary > storage for n

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Josh Berkus
On 03/06/2014 12:58 PM, Daniele Varrazzo wrote: > On Thu, Mar 6, 2014 at 6:54 PM, Josh Berkus wrote: >> >> The actual storage upgrade of hstore-->hstore2 is fairly painless from >> the user perspective; they don't have to do anything. The problem is >> that the input/output strings are different,

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Daniele Varrazzo
On Thu, Mar 6, 2014 at 9:10 PM, Peter Geoghegan wrote: > On Thu, Mar 6, 2014 at 12:58 PM, Daniele Varrazzo > wrote: >> Hello, psycopg developer here. Not following the entire thread as it's >> quite articulated and not of my direct interest (nor comprehension). >> But if you throw at me a few tes

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Peter Geoghegan
On Thu, Mar 6, 2014 at 12:58 PM, Daniele Varrazzo wrote: > Hello, psycopg developer here. Not following the entire thread as it's > quite articulated and not of my direct interest (nor comprehension). > But if you throw at me a few test cases I can make sure psycopg can > parse them much before hs

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Daniele Varrazzo
On Thu, Mar 6, 2014 at 6:54 PM, Josh Berkus wrote: > > The actual storage upgrade of hstore-->hstore2 is fairly painless from > the user perspective; they don't have to do anything. The problem is > that the input/output strings are different, something which I didn't > think to check for (and Pe

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
On Thu, Mar 6, 2014 at 10:54 PM, Josh Berkus wrote: g? > > On 03/06/2014 09:50 AM, Oleg Bartunov wrote:> Looks like consensus is > done. I and Teodor are not happy with it, but >> what we can do :) One thing I want to do is to reserve our >> contribution to the flagship feature (jsonb), particu

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Josh Berkus
On 03/06/2014 07:00 AM, Bruce Momjian wrote: > What can we do to help people migrate to an hstore type that supports > data types? Is there a function we can give them to flag possible > problem data, or give them some function to format things the old way > for migrations, etc. If they are going

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
I meant in Release Notes for 9.4 On Thu, Mar 6, 2014 at 10:26 PM, Andrew Dunstan wrote: > > > > > > On 03/06/2014 12:50 PM, Oleg Bartunov wrote: >> >> Hi there, >> >> Looks like consensus is done. I and Teodor are not happy with it, but >> what we can do :) One thing I want to do is to reserve

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Andrew Dunstan
On 03/06/2014 12:50 PM, Oleg Bartunov wrote: Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to the flagship feature (jsonb), particularly, "binary storage for nested structures and in

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to the flagship feature (jsonb), particularly, "binary storage for nested structures and indexing. Their work was sponsored by Engine Yard". A

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Merlin Moncure
On Thu, Mar 6, 2014 at 11:28 AM, Heikki Linnakangas wrote: > So here my opinion on what we should do: > > 1. Forget about hstore2 > 2. Add GIN and GIST operator classes to jsonb, if they're ready for commit > pretty darn soon. If not, punt them to next release. For #2, would we maintain the hstor

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Heikki Linnakangas
On 03/06/2014 05:46 PM, Tom Lane wrote: Magnus Hagander writes: However, if the new hstore type (compatible with the old one) is the wrapper around jsonb, rather than the other way around, I don't see any problem with it at all. Most future users are almost certainly going to use the json inter

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Andrew Dunstan
On 03/06/2014 10:46 AM, Tom Lane wrote: Magnus Hagander writes: However, if the new hstore type (compatible with the old one) is the wrapper around jsonb, rather than the other way around, I don't see any problem with it at all. Most future users are almost certainly going to use the json inte

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Tom Lane
Magnus Hagander writes: > However, if the new hstore type (compatible with the old one) is the > wrapper around jsonb, rather than the other way around, I don't see any > problem with it at all. Most future users are almost certainly going to use > the json interfaces, but we don't want to leave u

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Magnus Hagander
On Thu, Mar 6, 2014 at 4:25 PM, Tom Lane wrote: > Bruce Momjian writes: > > OK, just to summarize: > > > JSONB and everything it shares with hstore will be in core > > hstore-specific code stays in contrib > > hstore contrib will create an hstore type to call contrib and core >

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Ronan Dunklau
Le jeudi 6 mars 2014 09:33:18 Andrew Dunstan a écrit : > On 03/06/2014 08:16 AM, Oleg Bartunov wrote: > > On Thu, Mar 6, 2014 at 12:43 PM, Peter Geoghegan wrote: > >> On Thu, Mar 6, 2014 at 12:23 AM, Teodor Sigaev wrote: > >>> That's possible to introduce GUC variable for i/o functions which will

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Tom Lane
Bruce Momjian writes: > OK, just to summarize: > JSONB and everything it shares with hstore will be in core > hstore-specific code stays in contrib > hstore contrib will create an hstore type to call contrib and core code > 9.4 hstore has some differences from pre-9.4 I'v

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Bruce Momjian
On Thu, Mar 6, 2014 at 09:33:18AM -0500, Andrew Dunstan wrote: > I hear you, and largely agree, as long as the compatibility issue is > solved. If it's not, I think inventing a new hstore2 type is > probably a lousy way to go. > > For good or ill, the world has pretty much settled on wanting to u

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Andrew Dunstan
On 03/06/2014 08:16 AM, Oleg Bartunov wrote: On Thu, Mar 6, 2014 at 12:43 PM, Peter Geoghegan wrote: On Thu, Mar 6, 2014 at 12:23 AM, Teodor Sigaev wrote: That's possible to introduce GUC variable for i/o functions which will control old "bug-to-bug" behavior. IMHO, this is much better optio

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
On Thu, Mar 6, 2014 at 12:43 PM, Peter Geoghegan wrote: > On Thu, Mar 6, 2014 at 12:23 AM, Teodor Sigaev wrote: >> That's possible to introduce GUC variable for i/o functions which will >> control old "bug-to-bug" behavior. IMHO, this is much better option that >> stopping hstore development or s

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Peter Geoghegan
On Thu, Mar 6, 2014 at 1:32 AM, Teodor Sigaev wrote: > It's true for perl. Syntax of hstore is close to hash/array syntax and it's > easy serialize/deserialize hstore to/from perl. Syntax of hstore was > inspired by perl. I understand that. There is a module on CPAN called Pg::hstore that will do

  1   2   3   4   5   >