Re: [HACKERS] jsonb and nested hstore

2014-03-31 Thread Robert Haas
On Fri, Mar 14, 2014 at 9:17 PM, Josh Berkus j...@agliodbs.com wrote: On 03/14/2014 06:44 PM, Tomas Vondra wrote: Stupid question - so if I have a json like this: Not a stupid question, actually. In fact, I expect to answer it 400 or 500 times over the lifespan of 9.4. { a : { b : c}}

Re: [HACKERS] jsonb and nested hstore VODKA

2014-03-31 Thread Josh Berkus
On 03/31/2014 09:34 AM, Robert Haas wrote: On Fri, Mar 14, 2014 at 9:17 PM, Josh Berkus j...@agliodbs.com wrote: Precisely. Hence, the Russian plans for VODKA. Have these plans been shared publicly somewhere? Got a link? Nothing other than the pgCon proposal. Presumably we'll know at

Re: [HACKERS] jsonb and nested hstore

2014-03-23 Thread Tomas Vondra
On 21.3.2014 08:23, Peter Geoghegan wrote: On Thu, Mar 13, 2014 at 3:39 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark st...@mit.edu wrote: It does sound like the main question here is which opclass should be the default. From the discussion there's a

Re: [HACKERS] jsonb and nested hstore

2014-03-23 Thread Peter Geoghegan
On Sun, Mar 23, 2014 at 11:10 AM, Tomas Vondra t...@fuzzy.cz wrote: Keeping jsonb_ops as the default seems better / safer to me. That's what I did. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] jsonb and nested hstore

2014-03-21 Thread Peter Geoghegan
On Thu, Mar 13, 2014 at 3:39 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark st...@mit.edu wrote: It does sound like the main question here is which opclass should be the default. From the discussion there's a jsonb_hash_ops which works on all input

Re: [HACKERS] jsonb and nested hstore

2014-03-21 Thread Greg Stark
On Fri, Mar 21, 2014 at 7:23 AM, Peter Geoghegan p...@heroku.com wrote: I must admit that I'm coming around to the view that jsonb_hash_ops would make a better default. Its performance is superb, and I think there's a strong case to be made for that more than making up for it not supporting

Re: [HACKERS] jsonb and nested hstore

2014-03-20 Thread Alexander Korotkov
I've noticed two commits on github. commit b8199ee3c2506ab81b47a0b440363fc90c0d6956 Author: Peter Geoghegan p...@heroku.com Date: Wed Mar 19 02:02:16 2014 -0700 For jsonb_hash_ops, hash less By limiting the GIN entries to the least-nested level, the delicious.com sample JSON

Re: [HACKERS] jsonb and nested hstore

2014-03-20 Thread Peter Geoghegan
On Thu, Mar 20, 2014 at 5:32 AM, Alexander Korotkov aekorot...@gmail.com wrote: Besides implementation, what the idea was here? For me, it's impossible to skip any single element, because it's possible for query to include only this element. If we skip that element, we can't answer

Re: [HACKERS] jsonb and nested hstore

2014-03-15 Thread Greg Stark
On Sat, Mar 15, 2014 at 1:44 AM, Tomas Vondra t...@fuzzy.cz wrote: Because otherwise I don't understand how the index could be used for queries with @ '{a : {b : c}}' conditions (i.e. path [a,b] with value c). Hm, some experimentation here shows it does indeed work for queries like this and

Re: [HACKERS] jsonb and nested hstore

2014-03-15 Thread Tomas Vondra
On 15.3.2014 06:40, Peter Geoghegan wrote: On Fri, Mar 14, 2014 at 6:44 PM, Tomas Vondra t...@fuzzy.cz wrote: Well, depends on how you define useful. With the sample dataset 'delicious' (see Peter's post) I can do this: SELECT doc FROM delicious WHERE doc @ '{title_detail :

Re: [HACKERS] jsonb and nested hstore

2014-03-15 Thread Tomas Vondra
On 15.3.2014 02:15, Peter Geoghegan wrote: On Fri, Mar 14, 2014 at 5:10 PM, Tomas Vondra t...@fuzzy.cz wrote: I'm on commit a3115f0d, which is just 2 days old, so I suppose this was not fixed yet. Try merging the feature branch now, which will get you commit 16923d, which you're missing.

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 13 Březen 2014, 23:39, Peter Geoghegan wrote: On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark st...@mit.edu wrote: It does sound like the main question here is which opclass should be the default. From the discussion there's a jsonb_hash_ops which works on all input values but supports fewer

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Oleg Bartunov
VODKA index will have no lenght limitation. On Fri, Mar 14, 2014 at 3:07 PM, Tomas Vondra t...@fuzzy.cz wrote: On 13 Březen 2014, 23:39, Peter Geoghegan wrote: On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark st...@mit.edu wrote: It does sound like the main question here is which opclass should be

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Josh Berkus
On 03/14/2014 04:52 AM, Oleg Bartunov wrote: VODKA index will have no lenght limitation. Yeah, so I think we go with what we have, and tell people if you're hitting these length issues, wait for 9.5, where they will be fixed. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com --

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Oleg Bartunov
9.5 may too optimistic :) On Fri, Mar 14, 2014 at 11:18 PM, Josh Berkus j...@agliodbs.com wrote: On 03/14/2014 04:52 AM, Oleg Bartunov wrote: VODKA index will have no lenght limitation. Yeah, so I think we go with what we have, and tell people if you're hitting these length issues, wait for

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Gavin Flower
On 15/03/14 08:45, Oleg Bartunov wrote: 9.5 may too optimistic :) On Fri, Mar 14, 2014 at 11:18 PM, Josh Berkus j...@agliodbs.com wrote: On 03/14/2014 04:52 AM, Oleg Bartunov wrote: VODKA index will have no lenght limitation. Yeah, so I think we go with what we have, and tell people if

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Josh Berkus
On 03/14/2014 12:45 PM, Oleg Bartunov wrote: 9.5 may too optimistic :) Nonsense, you, Teodor and Alexander are geniuses. It can't possibly take you more than a year. ;-) -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 14.3.2014 20:18, Josh Berkus wrote: On 03/14/2014 04:52 AM, Oleg Bartunov wrote: VODKA index will have no lenght limitation. Yeah, so I think we go with what we have, and tell people if you're hitting these length issues, wait for 9.5, where they will be fixed. VODKA may be great, but

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
On Fri, Mar 14, 2014 at 2:21 PM, Tomas Vondra t...@fuzzy.cz wrote: I'm not awfully familiar with the GIN code, but based on Alexander's feedback I presume fixing the GIN length limit (or rather removing it, as it's a feature, not a bug) is quite straightforward. Why not to at least consider

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
For the benefit of anyone that would like to try the patch out, I make available a custom format dump of some delicious sample data. I can query the sample data as follows on my local installation: [local]/jsondata=# select count(*) from delicious ; count - 1079399 (1 row)

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 14.3.2014 22:54, Peter Geoghegan wrote: On Fri, Mar 14, 2014 at 2:21 PM, Tomas Vondra t...@fuzzy.cz wrote: I'm not awfully familiar with the GIN code, but based on Alexander's feedback I presume fixing the GIN length limit (or rather removing it, as it's a feature, not a bug) is quite

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 14.3.2014 23:06, Peter Geoghegan wrote: For the benefit of anyone that would like to try the patch out, I make available a custom format dump of some delicious sample data. I can query the sample data as follows on my local installation: [local]/jsondata=# select count(*) from delicious ;

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Andres Freund
On 2014-03-14 22:21:18 +0100, Tomas Vondra wrote: Don't get me wrong - I'm aware it's quite late in the last commitfest, and if it's deemed unacceptable / endandering 9.4 release, I'm not going to say a word. But if it's a simple patch ... IMNSHO there's no bloody chance for such an addition

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Greg Stark
On Fri, Mar 14, 2014 at 9:21 PM, Tomas Vondra t...@fuzzy.cz wrote: I'm not awfully familiar with the GIN code, but based on Alexander's feedback I presume fixing the GIN length limit (or rather removing it, as it's a feature, not a bug) is quite straightforward. Why not to at least consider

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
On Fri, Mar 14, 2014 at 5:10 PM, Tomas Vondra t...@fuzzy.cz wrote: I'm on commit a3115f0d, which is just 2 days old, so I suppose this was not fixed yet. Try merging the feature branch now, which will get you commit 16923d, which you're missing. That was an open item for a while, which I only

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Tomas Vondra
On 15.3.2014 02:03, Greg Stark wrote: On Fri, Mar 14, 2014 at 9:21 PM, Tomas Vondra t...@fuzzy.cz wrote: I'm not awfully familiar with the GIN code, but based on Alexander's feedback I presume fixing the GIN length limit (or rather removing it, as it's a feature, not a bug) is quite

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Josh Berkus
On 03/14/2014 06:44 PM, Tomas Vondra wrote: Stupid question - so if I have a json like this: Not a stupid question, actually. In fact, I expect to answer it 400 or 500 times over the lifespan of 9.4. { a : { b : c}} the GIN code indexes {b : c} as a single value? And then takes c and

Re: [HACKERS] jsonb and nested hstore

2014-03-14 Thread Peter Geoghegan
On Fri, Mar 14, 2014 at 6:44 PM, Tomas Vondra t...@fuzzy.cz wrote: Well, depends on how you define useful. With the sample dataset 'delicious' (see Peter's post) I can do this: SELECT doc FROM delicious WHERE doc @ '{title_detail : {value : TheaterMania}}'; with arbitrary

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Bruce Momjian
On Wed, Mar 12, 2014 at 01:58:14PM -0700, Peter Geoghegan wrote: The use case you describe here doesn't sound like something similar to full text search. It sounds like something identical. In any case, let's focus on what we have right now. I think that the indexing facilities proposed here

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
On Thu, Mar 13, 2014 at 6:15 AM, Bruce Momjian br...@momjian.us wrote: On Wed, Mar 12, 2014 at 01:58:14PM -0700, Peter Geoghegan wrote: The use case you describe here doesn't sound like something similar to full text search. It sounds like something identical. In any case, let's focus on what

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
Fwiw I have a few questions -- but beware, I'm a complete neophyte when it comes to jsonb style document databases so these are more likely to represent misconceptions on my part than problems with jsonb. I naively though a gin index on a jsonb would help with queries like WHERE col-'prop' =

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Alexander Korotkov
On Thu, Mar 13, 2014 at 1:21 PM, Greg Stark st...@mit.edu wrote: Well these are just normal gin and gist indexes. If we want to come up with new index operator classess we can still do that and keep the old ones if necessary. Even that seems pretty unlikely from past experience. I'm actually

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Oleg Bartunov
On Thu, Mar 13, 2014 at 4:21 PM, Alexander Korotkov aekorot...@gmail.com wrote: On Thu, Mar 13, 2014 at 1:21 PM, Greg Stark st...@mit.edu wrote: Well these are just normal gin and gist indexes. If we want to come up with new index operator classess we can still do that and keep the old ones

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
Fwiw the jsonb data doesn't actually seem to be any smaller than text json on this data set (this is avg(pg_column_size(col)) and I checked, they're both using the same amount of toast space) jsonb | json ---+--- 813.5 | 716.3 (1 row) It's still more than 7x faster in cpu costs though:

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Andrew Dunstan
On 03/13/2014 06:53 AM, Greg Stark wrote: I also find it awkward that col-'prop' returns the json representation of the property. If it's text that means it's double-quoted. I would think that a user storing text in a json property would want a way to pull out the text that json property

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Andrew Dunstan
On 03/13/2014 08:42 AM, Greg Stark wrote: Fwiw the jsonb data doesn't actually seem to be any smaller than text json on this data set (this is avg(pg_column_size(col)) and I checked, they're both using the same amount of toast space) jsonb | json ---+--- 813.5 | 716.3 (1 row)

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
On Thu, Mar 13, 2014 at 1:08 PM, Andrew Dunstan and...@dunslane.net wrote: - returns dequoted text if the value it points to is a plain string. If it's not doing that then that's a bug. Sorry, I must have gotten confused between various tests. It does seem to be doing that. -- greg --

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Greg Stark
Another question. Is Peter's branch up to date with jsonb_populate_record() ? From discussions on list it sounds like the plan was to get rid of the use_json_as_text argument but his patch still has it. (Tangentially, I wonder if it wouldn't be possible to make this a plain cast. I'm not sure but

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Andrew Dunstan
On 03/13/2014 10:49 AM, Greg Stark wrote: Another question. Is Peter's branch up to date with jsonb_populate_record() ? From discussions on list it sounds like the plan was to get rid of the use_json_as_text argument but his patch still has it. Yes, we're not changing that, and some people

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Tomas Vondra
On 13.3.2014 13:28, Oleg Bartunov wrote: On Thu, Mar 13, 2014 at 4:21 PM, Alexander Korotkov aekorot...@gmail.com wrote: On Thu, Mar 13, 2014 at 1:21 PM, Greg Stark st...@mit.edu wrote: Well these are just normal gin and gist indexes. If we want to come up with new index operator classess we

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Merlin Moncure
On Mon, Mar 10, 2014 at 4:18 AM, Peter Geoghegan p...@heroku.com wrote: * Extensive additional documentation. References to the very new JSON RFC. I think that this revision is in general a lot more coherent, and I found that reflecting on what idiomatic usage should look like while writing

Re: [HACKERS] jsonb and nested hstore

2014-03-13 Thread Peter Geoghegan
On Thu, Mar 13, 2014 at 2:21 AM, Greg Stark st...@mit.edu wrote: It does sound like the main question here is which opclass should be the default. From the discussion there's a jsonb_hash_ops which works on all input values but supports fewer operators and a jsonb_ops which supports more

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12 Březen 2014, 0:41, Peter Geoghegan wrote: On Tue, Mar 11, 2014 at 3:58 PM, Tomas Vondra t...@fuzzy.cz wrote: ERROR: index row size 1416 exceeds maximum 1352 for index gin_idx All index AMs have similar restrictions. Yes, I know and I have no problem with restrictions in general. You

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12 Březen 2014, 0:51, Peter Geoghegan wrote: On Tue, Mar 11, 2014 at 4:41 PM, Peter Geoghegan p...@heroku.com wrote: I think that in practice the general recommendation will be that when indexing at the top level, use jsonb_hash_ops. When indexing nested items, use the more flexible

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Oleg Bartunov
Also, GiST index is faster for create/update operations. I really hope we will improve jsonb indexing in the next one-two releases. For now I'd suggest people index expressional indexes to index just interesting keys or use GiST. On Wed, Mar 12, 2014 at 5:15 PM, Tomas Vondra t...@fuzzy.cz wrote:

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 6:20 AM, Tomas Vondra t...@fuzzy.cz wrote: I'm still not sure how would that look. Does that mean I'd have to create multiple GIN indexes - one for each possible key or something like that? Can you give an example? It could mean that you're obliged to create multiple

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 11:57 AM, Oleg Bartunov obartu...@gmail.com wrote: Also, GiST index is faster for create/update operations. I really hope we will improve jsonb indexing in the next one-two releases. For now I'd suggest people index expressional indexes to index just interesting keys

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Oleg Bartunov
On Thu, Mar 13, 2014 at 12:10 AM, Peter Geoghegan p...@heroku.com wrote: On Wed, Mar 12, 2014 at 11:57 AM, Oleg Bartunov obartu...@gmail.com wrote: Also, GiST index is faster for create/update operations. I really hope we will improve jsonb indexing in the next one-two releases. For now I'd

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Andrew Dunstan
On 03/12/2014 04:10 PM, Peter Geoghegan wrote: On Wed, Mar 12, 2014 at 11:57 AM, Oleg Bartunov obartu...@gmail.com wrote: Also, GiST index is faster for create/update operations. I really hope we will improve jsonb indexing in the next one-two releases. For now I'd suggest people index

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Josh Berkus
Andrew, Peter: Just so I'm clear on the limits here, lemme make sure I understand this: a) GIN indexing is limited to ~~1500chars b) The value, which includes everything other than the top level set of keys, is one item as far as GIN is concerned. Therefore: we are limited to indexing JSON

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 1:37 PM, Andrew Dunstan and...@dunslane.net wrote: One major use case for using treeish data types in the first place is that you don't know when you're designing the database exactly what shape the data will be. If you don't know that, then how are you supposed to know

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Andrew Dunstan
On 03/12/2014 04:58 PM, Peter Geoghegan wrote: In any case, let's focus on what we have right now. I think that the indexing facilities proposed here are solid. In any case they do not preclude working on better indexing strategies as the need emerges. I quite agree, didn't mean to suggest

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 20:40, Peter Geoghegan wrote: On Wed, Mar 12, 2014 at 6:20 AM, Tomas Vondra t...@fuzzy.cz wrote: I'm still not sure how would that look. Does that mean I'd have to create multiple GIN indexes - one for each possible key or something like that? Can you give an example? It could

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 21:58, Peter Geoghegan wrote: The use case you describe here doesn't sound like something similar to full text search. It sounds like something identical. I think this very depends on the definition of full text search. In any case, let's focus on what we have right now. I

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Peter Geoghegan
On Wed, Mar 12, 2014 at 2:30 PM, Tomas Vondra t...@fuzzy.cz wrote: I think that's unfounded assumption. Many users actually have very little control over the documents or queries - a nice example may be the mail archive, with headers stored in a hstore/jsonb. I have absolutely no control over

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 21:55, Josh Berkus wrote: Andrew, Peter: Just so I'm clear on the limits here, lemme make sure I understand this: a) GIN indexing is limited to ~~1500chars The exact message I get is this: ERROR: index row size 1944 exceeds maximum 1352 for index tmp_idx so it's 1352B. But

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Stephen Frost
* Tomas Vondra (t...@fuzzy.cz) wrote: So I think it's quite difficult to give simple and exact explanation in the docs, other than there are limits, but it's difficult to say when you hit them. Arrays have more-or-less the same issue... Thanks, Stephen signature.asc

Re: [HACKERS] jsonb and nested hstore

2014-03-12 Thread Tomas Vondra
On 12.3.2014 22:43, Peter Geoghegan wrote: On Wed, Mar 12, 2014 at 2:30 PM, Tomas Vondra t...@fuzzy.cz wrote: I think that's unfounded assumption. Many users actually have very little control over the documents or queries - a nice example may be the mail archive, with headers stored in a

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Alexander Korotkov
On Tue, Mar 11, 2014 at 5:19 AM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 10, 2014 at 4:19 AM, Alexander Korotkov aekorot...@gmail.com wrote: Here it is. So it looks like what you have here is analogous to the other problems that I fixed with both GiST and GIN. That isn't

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Tomas Vondra
Hi, I've spent a few hours stress-testing this a bit - loading a mail archive with ~1M of messages (with headers stored in a jsonb column) and then doing queries on that. Good news - no crashes or any such issues so far. The queries that I ran manually seem to return sane results. The only

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Peter Geoghegan
On Tue, Mar 11, 2014 at 3:58 PM, Tomas Vondra t...@fuzzy.cz wrote: ERROR: index row size 1416 exceeds maximum 1352 for index gin_idx All index AMs have similar restrictions. A good example of such header is dkim-signature which basically contains the whole message digitally signed with

Re: [HACKERS] jsonb and nested hstore

2014-03-11 Thread Peter Geoghegan
On Tue, Mar 11, 2014 at 4:41 PM, Peter Geoghegan p...@heroku.com wrote: I think that in practice the general recommendation will be that when indexing at the top level, use jsonb_hash_ops. When indexing nested items, use the more flexible default GIN opclass. That seems like a pretty smart

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 1:18 PM, Peter Geoghegan p...@heroku.com wrote: * The jsonb_hash_ops non-default GIN opclass is broken. It has its own idiosyncratic notion of what constitutes containment, that sees it only return, say, jsonb arrays that have a matching string as their leftmost

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:00 AM, Alexander Korotkov aekorot...@gmail.com wrote: I din't get comment about leftmost element. There is absolutely no distinguish between array elements. All elements are extracted into same keys independent of their indexes. It seems to have no change since I wrote

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:19 AM, Peter Geoghegan p...@heroku.com wrote: I don't have time to post that at the moment, but offhand I *think* your confusion may be due to the fact that the json_hash_ops opclass (as I call it) was previously consistent with the behavior of the other GIN opclass

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:21 AM, Peter Geoghegan p...@heroku.com wrote: Sorry, I realize now that that must be incorrect. Still, please take a look at the commit linked to. To be clear, I mean that my explanation of why this was missed before was incorrect, not my contention that it's a problem

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 2:19 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 10, 2014 at 3:00 AM, Alexander Korotkov aekorot...@gmail.com wrote: I din't get comment about leftmost element. There is absolutely no distinguish between array elements. All elements are extracted into same

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 3:47 AM, Alexander Korotkov aekorot...@gmail.com wrote: Fix is attached. Could you post a patch with regression tests, please? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Alexander Korotkov
On Mon, Mar 10, 2014 at 3:04 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 10, 2014 at 3:47 AM, Alexander Korotkov aekorot...@gmail.com wrote: Fix is attached. Could you post a patch with regression tests, please? Here it is. -- With best regards, Alexander Korotkov.

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Andrew Dunstan
On 03/10/2014 05:18 AM, Peter Geoghegan wrote: On Fri, Mar 7, 2014 at 9:00 AM, Bruce Momjian br...@momjian.us wrote: OK, it sounds like the adjustments are minimal, like not using the high-order bit. Attached patch is a refinement of the work of Oleg, Teodor and Andrew. Revisions are mostly

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Andrew Dunstan
On 03/10/2014 10:50 AM, Andrew Dunstan wrote: Thanks for your work on this. It's just occurred to me that we'll need to add hstore_to_jsonb functions and a cast to match the hstore_to_json functions and cast. That should be fairly simple - I'll work on that. It need not hold up progress

Re: [HACKERS] jsonb and nested hstore

2014-03-10 Thread Peter Geoghegan
On Mon, Mar 10, 2014 at 4:19 AM, Alexander Korotkov aekorot...@gmail.com wrote: Here it is. So it looks like what you have here is analogous to the other problems that I fixed with both GiST and GIN. That isn't surprising, and this does fix my test-case. I'm not terribly happy about the lack of

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Merlin Moncure
On Thu, Mar 6, 2014 at 10:33 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Mar 6, 2014 at 09:50:56PM +0400, Oleg Bartunov wrote: Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Andrew Dunstan
On 03/06/2014 11:33 PM, Bruce Momjian wrote: On Thu, Mar 6, 2014 at 09:50:56PM +0400, Oleg Bartunov wrote: Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to the flagship feature

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Bruce Momjian
On Fri, Mar 7, 2014 at 11:35:41AM -0500, Andrew Dunstan wrote: IIRC The sacrifice was one bit in the header (i.e. in the first int after the varlena header). We could now repurpose that (for example if we ever decided to use a new format). Oleg and Teodor made most of the adjustments on the

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Andrew Dunstan
On 03/07/2014 11:45 AM, Bruce Momjian wrote: On Fri, Mar 7, 2014 at 11:35:41AM -0500, Andrew Dunstan wrote: IIRC The sacrifice was one bit in the header (i.e. in the first int after the varlena header). We could now repurpose that (for example if we ever decided to use a new format). Oleg

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread Bruce Momjian
On Fri, Mar 7, 2014 at 11:57:48AM -0500, Andrew Dunstan wrote: If they can be done for 9.4, great, if not, we have to decide if these suboptimal cases are enough for us to delay the data type until 9.5. I don't know the answer, but I have to ask the question. AFAIK, there is no sacrifice

Re: [HACKERS] jsonb and nested hstore

2014-03-07 Thread David E. Wheeler
On Mar 6, 2014, at 1:51 AM, Peter Geoghegan p...@heroku.com wrote: It's true for perl. Syntax of hstore is close to hash/array syntax and it's easy serialize/deserialize hstore to/from perl. Syntax of hstore was inspired by perl. I understand that. There is a module on CPAN called

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Teodor Sigaev
Thank you for checking that. Teodor's goal was that new-hstore be 100% backwards-compatible with old-hstore. If we're breaking APIs, then it That's true. Binary format is fully compatible unless old hstore value has more than 2^28 key-value pairs (256 mln which is far from reachable by

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Peter Geoghegan
On Thu, Mar 6, 2014 at 12:23 AM, Teodor Sigaev teo...@sigaev.ru wrote: That's possible to introduce GUC variable for i/o functions which will control old bug-to-bug behavior. IMHO, this is much better option that stopping hstore development or split hstore to two branches. A GUC that controls

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Teodor Sigaev
In what sense are we really stopping hstore development if hstore2 lives as jsonb? I have a hard time imagining someone dealing with the incompatibility that a user-facing hstore2 would introduce, while still preferring hstore syntax over json syntax given the choice. There are very rich

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Peter Geoghegan
On Thu, Mar 6, 2014 at 1:32 AM, Teodor Sigaev teo...@sigaev.ru wrote: It's true for perl. Syntax of hstore is close to hash/array syntax and it's easy serialize/deserialize hstore to/from perl. Syntax of hstore was inspired by perl. I understand that. There is a module on CPAN called

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
On Thu, Mar 6, 2014 at 12:43 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Mar 6, 2014 at 12:23 AM, Teodor Sigaev teo...@sigaev.ru wrote: That's possible to introduce GUC variable for i/o functions which will control old bug-to-bug behavior. IMHO, this is much better option that stopping

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Andrew Dunstan
On 03/06/2014 08:16 AM, Oleg Bartunov wrote: On Thu, Mar 6, 2014 at 12:43 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Mar 6, 2014 at 12:23 AM, Teodor Sigaev teo...@sigaev.ru wrote: That's possible to introduce GUC variable for i/o functions which will control old bug-to-bug behavior.

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Bruce Momjian
On Thu, Mar 6, 2014 at 09:33:18AM -0500, Andrew Dunstan wrote: I hear you, and largely agree, as long as the compatibility issue is solved. If it's not, I think inventing a new hstore2 type is probably a lousy way to go. For good or ill, the world has pretty much settled on wanting to use

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: OK, just to summarize: JSONB and everything it shares with hstore will be in core hstore-specific code stays in contrib hstore contrib will create an hstore type to call contrib and core code 9.4 hstore has some differences from

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Ronan Dunklau
Le jeudi 6 mars 2014 09:33:18 Andrew Dunstan a écrit : On 03/06/2014 08:16 AM, Oleg Bartunov wrote: On Thu, Mar 6, 2014 at 12:43 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Mar 6, 2014 at 12:23 AM, Teodor Sigaev teo...@sigaev.ru wrote: That's possible to introduce GUC variable for

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Magnus Hagander
On Thu, Mar 6, 2014 at 4:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: OK, just to summarize: JSONB and everything it shares with hstore will be in core hstore-specific code stays in contrib hstore contrib will create an hstore type

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: However, if the new hstore type (compatible with the old one) is the wrapper around jsonb, rather than the other way around, I don't see any problem with it at all. Most future users are almost certainly going to use the json interfaces, but we don't

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Andrew Dunstan
On 03/06/2014 10:46 AM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: However, if the new hstore type (compatible with the old one) is the wrapper around jsonb, rather than the other way around, I don't see any problem with it at all. Most future users are almost certainly going

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Heikki Linnakangas
On 03/06/2014 05:46 PM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: However, if the new hstore type (compatible with the old one) is the wrapper around jsonb, rather than the other way around, I don't see any problem with it at all. Most future users are almost certainly going

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Merlin Moncure
On Thu, Mar 6, 2014 at 11:28 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: So here my opinion on what we should do: 1. Forget about hstore2 2. Add GIN and GIST operator classes to jsonb, if they're ready for commit pretty darn soon. If not, punt them to next release. For #2, would we

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to the flagship feature (jsonb), particularly, binary storage for nested structures and indexing. Their work was sponsored by Engine Yard. As

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Andrew Dunstan
On 03/06/2014 12:50 PM, Oleg Bartunov wrote: Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to the flagship feature (jsonb), particularly, binary storage for nested structures and

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
I meant in Release Notes for 9.4 On Thu, Mar 6, 2014 at 10:26 PM, Andrew Dunstan and...@dunslane.net wrote: On 03/06/2014 12:50 PM, Oleg Bartunov wrote: Hi there, Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Josh Berkus
On 03/06/2014 07:00 AM, Bruce Momjian wrote: What can we do to help people migrate to an hstore type that supports data types? Is there a function we can give them to flag possible problem data, or give them some function to format things the old way for migrations, etc. If they are going to

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Oleg Bartunov
On Thu, Mar 6, 2014 at 10:54 PM, Josh Berkus j...@agliodbs.com wrote: g? On 03/06/2014 09:50 AM, Oleg Bartunov wrote: Looks like consensus is done. I and Teodor are not happy with it, but what we can do :) One thing I want to do is to reserve our contribution to the flagship feature

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Daniele Varrazzo
On Thu, Mar 6, 2014 at 6:54 PM, Josh Berkus j...@agliodbs.com wrote: The actual storage upgrade of hstore--hstore2 is fairly painless from the user perspective; they don't have to do anything. The problem is that the input/output strings are different, something which I didn't think to check

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Peter Geoghegan
On Thu, Mar 6, 2014 at 12:58 PM, Daniele Varrazzo daniele.varra...@gmail.com wrote: Hello, psycopg developer here. Not following the entire thread as it's quite articulated and not of my direct interest (nor comprehension). But if you throw at me a few test cases I can make sure psycopg can

Re: [HACKERS] jsonb and nested hstore

2014-03-06 Thread Daniele Varrazzo
On Thu, Mar 6, 2014 at 9:10 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Mar 6, 2014 at 12:58 PM, Daniele Varrazzo daniele.varra...@gmail.com wrote: Hello, psycopg developer here. Not following the entire thread as it's quite articulated and not of my direct interest (nor comprehension).

  1   2   3   4   5   >