Bruce Momjian br...@momjian.us writes:
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and then values, independently of the
other issues under discussion. This means we
* Tom Lane (t...@sss.pgh.pa.us) wrote:
Bruce Momjian br...@momjian.us writes:
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and then values, independently of the
other
* Tom Lane (t...@sss.pgh.pa.us) wrote:
Stephen Frost sfr...@snowman.net writes:
* Tom Lane (t...@sss.pgh.pa.us) wrote:
Done --- 201409291 is the cutover point.
Just to clarify- the commit bumped the catversion to 201409292, so
version = 201409291 has the old format while version
On 09/29/2014 11:49 AM, Arthur Silva wrote:
What's the call on the stride length? Are we going to keep it hardcoded?
Please, yes. The complications caused by a variable stride length would
be horrible.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers
Arthur Silva arthur...@gmail.com writes:
What's the call on the stride length? Are we going to keep it hardcoded?
At the moment it's 32, but we could change it without forcing a new
initdb. I ran a simple test that seemed to show 32 was a good choice,
but if anyone else wants to try other
On Mon, Sep 29, 2014 at 12:30:40PM -0400, Tom Lane wrote:
Bruce Momjian br...@momjian.us writes:
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and then values,
On 09/26/2014 06:20 PM, Josh Berkus wrote:
Overall, I'm satisfied with the performance of the length-and-offset
patch.
Oh, also ... no bugs found.
So, can we get Beta3 out now?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list
Josh Berkus j...@agliodbs.com writes:
So, can we get Beta3 out now?
If nobody else steps up and says they want to do some performance
testing, I'll push the latest lengths+offsets patch tomorrow.
Are any of the other open items listed at
All,
So these results have become a bit complex. So spreadsheet time.
https://docs.google.com/spreadsheets/d/1Mokpx3EqlbWlFDIkF9qzpM7NneN9z-QOXWSzws3E-R4
Some details:
The Length-and-Offset test was performed using a more recent 9.4
checkout than the other two tests. This was regrettable,
On 2014-09-19 15:40:14 +0300, Heikki Linnakangas wrote:
On 09/18/2014 09:27 PM, Heikki Linnakangas wrote:
I'll try to write a more polished patch tomorrow. We'll then see what it
looks like, and can decide if we want it.
Ok, here are two patches. One is a refined version of my earlier patch,
On 09/25/2014 09:01 AM, Andres Freund wrote:
But independent of which version is chosen, we *REALLY* need to make the
decision soon. This issue has held up the next beta (like jsonb has
blocked previous beta) for *weeks*.
Yes, please!
--
Josh Berkus
PostgreSQL Experts Inc.
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
But independent of which version is chosen, we *REALLY* need to make the
decision soon. This issue has held up the next beta (like jsonb has
blocked previous beta) for *weeks*.
Personally it doesn't make me very happy that Heikki
On 09/25/2014 10:14 AM, Bruce Momjian wrote:
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
But independent of which version is chosen, we *REALLY* need to make the
decision soon. This issue has held up the next beta (like jsonb has
blocked previous beta) for *weeks*.
On 2014-09-25 10:18:24 -0700, Josh Berkus wrote:
On 09/25/2014 10:14 AM, Bruce Momjian wrote:
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
But independent of which version is chosen, we *REALLY* need to make the
decision soon. This issue has held up the next beta (like
On 09/25/2014 10:20 AM, Andres Freund wrote:
On 2014-09-25 10:18:24 -0700, Josh Berkus wrote:
On 09/25/2014 10:14 AM, Bruce Momjian wrote:
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
But independent of which version is chosen, we *REALLY* need to make the
decision soon.
On 09/25/2014 10:26 AM, Andres Freund wrote:
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
If Heikki says it's ready, I'll test. So far he's said that it wasn't
done yet.
http://www.postgresql.org/message-id/541c242e.3030...@vmware.com
Yeah, and that didn't include some of Tom's bug
On 2014-09-25 10:29:51 -0700, Josh Berkus wrote:
On 09/25/2014 10:26 AM, Andres Freund wrote:
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
If Heikki says it's ready, I'll test. So far he's said that it wasn't
done yet.
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
If Heikki says it's ready, I'll test. So far he's said that it wasn't
done yet.
http://www.postgresql.org/message-id/541c242e.3030...@vmware.com
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
Josh Berkus j...@agliodbs.com writes:
On 09/25/2014 10:26 AM, Andres Freund wrote:
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
If Heikki says it's ready, I'll test. So far he's said that it wasn't
done yet.
http://www.postgresql.org/message-id/541c242e.3030...@vmware.com
Yeah, and
On 09/25/2014 11:22 AM, Tom Lane wrote:
In the interests of pushing this forward, I will work today on
trying to finish and review Heikki's offsets-and-lengths patch
so that we have something we can do performance testing on.
I doubt that the performance testing will tell us anything we
don't
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and then values, independently of the
other issues under discussion. This means we *will* be breaking on-disk
compatibility with 9.4beta2, which means pg_upgrade will need to be taught
to
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and then values, independently of the
other issues under discussion. This means we *will* be breaking on-disk
compatibility
On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote:
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and then values, independently of the
other issues under discussion.
On Thu, Sep 25, 2014 at 09:00:07PM +0200, Andres Freund wrote:
On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote:
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and
Bruce Momjian wrote:
3. 9.3 multi-xact bugs spooked us into being more careful
Uh. Multixact changes in 9.3 were infinitely more invasive than the
jsonb changes will ever be. a) they touched basic visibility design and
routines,
which are complex, understood by very few people, and
I wrote:
The offsets-and-lengths patch seems like the approach we ought to
compare to my patch, but it looks pretty unfinished to me: AFAICS it
includes logic to understand offsets sprinkled into a mostly-lengths
array, but no logic that would actually *store* any such offsets,
which means
On 09/25/2014 08:10 PM, Tom Lane wrote:
I wrote:
The offsets-and-lengths patch seems like the approach we ought to
compare to my patch, but it looks pretty unfinished to me: AFAICS it
includes logic to understand offsets sprinkled into a mostly-lengths
array, but no logic that would actually
On 09/24/2014 08:16 AM, Tom Lane wrote:
Jan Wieck j...@wi3ck.info writes:
On 09/15/2014 09:46 PM, Craig Ringer wrote:
Anyway - this is looking like the change will go in, and with it a
catversion bump. Introduction of a jsonb version/flags byte might be
worthwhile at the same time. It seems
Heikki Linnakangas hlinnakan...@vmware.com writes:
On 09/24/2014 08:16 AM, Tom Lane wrote:
Heikki's patch would eat up the high-order JEntry bits, but the other
points remain.
If we don't need to be backwards-compatible with the 9.4beta on-disk
format, we don't necessarily need to eat the
On Fri, Sep 19, 2014 at 5:40 AM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
I think we should bite the bullet and break compatibility with 9.4beta2
format, even if we go with my patch. In a jsonb object, it makes sense to
store all the keys first, like Tom did, because of cache benefits,
Peter Geoghegan p...@heroku.com writes:
On Fri, Sep 19, 2014 at 5:40 AM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
I think we should bite the bullet and break compatibility with 9.4beta2
format, even if we go with my patch. In a jsonb object, it makes sense to
store all the keys
On 09/15/2014 09:46 PM, Craig Ringer wrote:
On 09/16/2014 07:44 AM, Peter Geoghegan wrote:
FWIW, I am slightly concerned about weighing use cases around very
large JSON documents too heavily. Having enormous jsonb documents just
isn't going to work out that well, but neither will equivalent
On Tue, Sep 23, 2014 at 10:02 PM, Jan Wieck j...@wi3ck.info wrote:
Is it worth paying a byte per value to save on possible upgrade pain?
This comment seems to have drowned in the discussion.
If there indeed has to be a catversion bump in the process of this, then I
agree with Craig.
-1. We
Jan Wieck j...@wi3ck.info writes:
On 09/15/2014 09:46 PM, Craig Ringer wrote:
Anyway - this is looking like the change will go in, and with it a
catversion bump. Introduction of a jsonb version/flags byte might be
worthwhile at the same time. It seems likely that there'll be more room
for
On 09/19/2014 07:07 AM, Tom Lane wrote:
Heikki Linnakangas hlinnakan...@vmware.com writes:
Tom: You mentioned earlier that your patch fixes some existing bugs.
What were they?
What I remember at the moment (sans caffeine) is that the routines for
assembling jsonb values out of field data
On 09/18/2014 09:27 PM, Heikki Linnakangas wrote:
On 09/18/2014 07:53 PM, Josh Berkus wrote:
On 09/16/2014 08:45 PM, Tom Lane wrote:
We're somewhat comparing apples and oranges here, in that I pushed my
approach to something that I think is of committable quality (and which,
not incidentally,
Heikki Linnakangas hlinnakan...@vmware.com writes:
Tom: You mentioned earlier that your patch fixes some existing bugs.
What were they?
What I remember at the moment (sans caffeine) is that the routines for
assembling jsonb values out of field data were lacking some necessary
tests for
On 09/16/2014 08:45 PM, Tom Lane wrote:
We're somewhat comparing apples and oranges here, in that I pushed my
approach to something that I think is of committable quality (and which,
not incidentally, fixes some existing bugs that we'd need to fix in any
case); while Heikki's patch was just
On 09/18/2014 07:53 PM, Josh Berkus wrote:
On 09/16/2014 08:45 PM, Tom Lane wrote:
We're somewhat comparing apples and oranges here, in that I pushed my
approach to something that I think is of committable quality (and which,
not incidentally, fixes some existing bugs that we'd need to fix in
On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan p...@heroku.com wrote:
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus j...@agliodbs.com wrote:
Actually, having the keys all at the same level *is* relevant for the
issue we're discussing. If those 270 keys are organized in a tree, it's
not the
On 09/16/2014 06:31 AM, Robert Haas wrote:
On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan p...@heroku.com wrote:
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus j...@agliodbs.com wrote:
Actually, having the keys all at the same level *is* relevant for the
issue we're discussing. If those 270
On Tue, Sep 16, 2014 at 12:47 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/16/2014 06:31 AM, Robert Haas wrote:
On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan p...@heroku.com wrote:
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus j...@agliodbs.com wrote:
Actually, having the keys all at the
On 09/16/2014 09:54 AM, Robert Haas wrote:
On Tue, Sep 16, 2014 at 12:47 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/16/2014 06:31 AM, Robert Haas wrote:
On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan p...@heroku.com wrote:
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus j...@agliodbs.com
On 09/16/2014 07:47 PM, Josh Berkus wrote:
On 09/16/2014 06:31 AM, Robert Haas wrote:
On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan p...@heroku.com wrote:
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus j...@agliodbs.com wrote:
Actually, having the keys all at the same level *is* relevant for
On Tue, Sep 16, 2014 at 3:12 PM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
I'll leave it up to the jury to decide if we care or not. It seems like a
fairly unusual use case, where you push around large enough arrays or
objects to notice. Then again, I'm sure *someone* will do it. People
On Tue, Sep 16, 2014 at 1:11 PM, Josh Berkus j...@agliodbs.com wrote:
Well, I can only judge from the use cases I personally have, none of
which involve more than 100 keys at any level for most rows. So far
I've seen some people argue hypotetical use cases involving hundreds of
keys per
Heikki, Robert:
On 09/16/2014 11:12 AM, Heikki Linnakangas wrote:
Are you looking for someone with a real life scenario, or just synthetic
test case? The latter is easy to do.
See attached test program. It's basically the same I posted earlier.
Here are the results from my laptop with Tom's
On Tue, Sep 16, 2014 at 3:24 PM, Josh Berkus j...@agliodbs.com wrote:
Do you feel that way *as a code maintainer*? That is, if you ended up
maintaining the JSONB code, would you still feel that it's worth the
extra complexity? Because that will be the main cost here.
I feel that Heikki
On 16/09/14 21:20, Robert Haas wrote:
In practice, I'm not very surprised that the impact doesn't seem too
bad when you're running SQL queries from the client. There's so much
other overhead, for de-TOASTing and client communication and even just
planner and executor costs, that this gets lost
On 09/16/2014 10:37 PM, Robert Haas wrote:
On Tue, Sep 16, 2014 at 3:24 PM, Josh Berkus j...@agliodbs.com wrote:
Do you feel that way *as a code maintainer*? That is, if you ended up
maintaining the JSONB code, would you still feel that it's worth the
extra complexity? Because that will be
On Tue, Sep 16, 2014 at 4:20 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Sep 16, 2014 at 1:11 PM, Josh Berkus j...@agliodbs.com wrote:
Well, I can only judge from the use cases I personally have, none of
which involve more than 100 keys at any level for most rows. So far
I've
- Цитат от Robert Haas (robertmh...@gmail.com), на 16.09.2014 в 22:20 -
In practice, I'm not very surprised that the impact doesn't seem too
bad when you're running SQL queries from the client. There's so much
other overhead, for de-TOASTing and client communication and even just
Heikki Linnakangas hlinnakan...@vmware.com writes:
On 09/16/2014 10:37 PM, Robert Haas wrote:
On Tue, Sep 16, 2014 at 3:24 PM, Josh Berkus j...@agliodbs.com wrote:
Do you feel that way *as a code maintainer*? That is, if you ended up
maintaining the JSONB code, would you still feel that it's
On 09/12/2014 01:30 PM, Heikki Linnakangas wrote:
Performance was one argument for sure. It's not hard to come up with a
case where the all-lengths approach is much slower: take a huge array
with, say, million elements, and fetch the last element in a tight loop.
And do that in a PL/pgSQL
On Mon, Sep 15, 2014 at 2:12 PM, Josh Berkus j...@agliodbs.com wrote:
If not, I think the corner case is so obscure as to be not worth
optimizing for. I can't imagine that more than a tiny minority of our
users are going to have thousands of keys per datum.
Worst case is linear cost scaling
On 09/15/2014 10:23 AM, Claudio Freire wrote:
Now, large small keys could be 200 or 2000, or even 20k. I'd guess
several should be tested to find the shape of the curve.
Well, we know that it's not noticeable with 200, and that it is
noticeable with 100K. It's only worth testing further if we
On Mon, Sep 15, 2014 at 4:09 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/15/2014 10:23 AM, Claudio Freire wrote:
Now, large small keys could be 200 or 2000, or even 20k. I'd guess
several should be tested to find the shape of the curve.
Well, we know that it's not noticeable with 200, and
On 09/15/2014 12:15 PM, Claudio Freire wrote:
So while you're right that it's perhaps above what would be a common
use case, the range somewhere between 200 and 100K for the tipping
point seems overly imprecise to me.
Well, then, you know how to solve that.
--
Josh Berkus
PostgreSQL Experts
On Mon, Sep 15, 2014 at 4:17 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/15/2014 12:15 PM, Claudio Freire wrote:
So while you're right that it's perhaps above what would be a common
use case, the range somewhere between 200 and 100K for the tipping
point seems overly imprecise to me.
On 09/15/2014 12:25 PM, Claudio Freire wrote:
On Mon, Sep 15, 2014 at 4:17 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/15/2014 12:15 PM, Claudio Freire wrote:
So while you're right that it's perhaps above what would be a common
use case, the range somewhere between 200 and 100K for the
On Mon, Sep 15, 2014 at 3:09 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/15/2014 10:23 AM, Claudio Freire wrote:
Now, large small keys could be 200 or 2000, or even 20k. I'd guess
several should be tested to find the shape of the curve.
Well, we know that it's not noticeable with 200, and
On 09/15/2014 02:16 PM, Robert Haas wrote:
On Mon, Sep 15, 2014 at 3:09 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/15/2014 10:23 AM, Claudio Freire wrote:
Now, large small keys could be 200 or 2000, or even 20k. I'd guess
several should be tested to find the shape of the curve.
Well, we
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus j...@agliodbs.com wrote:
Actually, having the keys all at the same level *is* relevant for the
issue we're discussing. If those 270 keys are organized in a tree, it's
not the same as having them all on one level (and not as problematic).
I believe
I couldn't get my hands on the twitter data but I'm generating my own. The
json template is http://paste2.org/wJ1dfcjw and data was generated with
http://www.json-generator.com/. It has 35 top level keys, just in case
someone is wondering.
I generated 1 random objects and I'm inserting them
On 09/16/2014 07:44 AM, Peter Geoghegan wrote:
FWIW, I am slightly concerned about weighing use cases around very
large JSON documents too heavily. Having enormous jsonb documents just
isn't going to work out that well, but neither will equivalent designs
in popular document database systems
On 09/11/2014 06:56 PM, Arthur Silva wrote:
In my testings with the github archive data the savings -
performance-penalty was fine, but I'm not confident in those results
since there were only 8 top level keys.
Well, we did want to see that the patch doesn't create a regression with
data
On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus j...@agliodbs.com wrote:
So, I finally got time to test Tom's latest patch on this.
TLDR: we want to go with Tom's latest patch and release beta3.
Figures:
So I tested HEAD against the latest lengths patch. Per Arthur Silva, I
checked
On Fri, Sep 12, 2014 at 1:00 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus j...@agliodbs.com wrote:
So, I finally got time to test Tom's latest patch on this.
TLDR: we want to go with Tom's latest patch and release beta3.
Figures:
So I tested
On 09/12/2014 10:00 AM, Robert Haas wrote:
On Fri, Sep 12, 2014 at 1:00 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus j...@agliodbs.com wrote:
So, I finally got time to test Tom's latest patch on this.
TLDR: we want to go with Tom's latest patch and
On Fri, Sep 12, 2014 at 1:11 PM, Josh Berkus j...@agliodbs.com wrote:
On 09/12/2014 10:00 AM, Robert Haas wrote:
On Fri, Sep 12, 2014 at 1:00 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus j...@agliodbs.com wrote:
So, I finally got time to test Tom's
Robert Haas robertmh...@gmail.com writes:
On Fri, Sep 12, 2014 at 1:11 PM, Josh Berkus j...@agliodbs.com wrote:
It's certainly possible that there is a test case for which Heikki's
approach is superior, but if so we haven't seen it. And since it's
approach is also more complicated, sticking
On 09/12/2014 08:52 PM, Tom Lane wrote:
Robert Haas robertmh...@gmail.com writes:
On Fri, Sep 12, 2014 at 1:11 PM, Josh Berkus j...@agliodbs.com wrote:
It's certainly possible that there is a test case for which Heikki's
approach is superior, but if so we haven't seen it. And since it's
So, I finally got time to test Tom's latest patch on this.
TLDR: we want to go with Tom's latest patch and release beta3.
Figures:
So I tested HEAD against the latest lengths patch. Per Arthur Silva, I
checked uncompressed times for JSONB against compressed times. This
changed the picture
* Josh Berkus (j...@agliodbs.com) wrote:
TLDR: we want to go with Tom's latest patch and release beta3.
Having not even read the rest- yes please. We really need to get beta3
out and figure out when we're going to actually release 9.4...
Admittedly, the last month has been good and we've been
On Thu, Sep 11, 2014 at 10:01 PM, Josh Berkus j...@agliodbs.com wrote:
So, I finally got time to test Tom's latest patch on this.
TLDR: we want to go with Tom's latest patch and release beta3.
Figures:
So I tested HEAD against the latest lengths patch. Per Arthur Silva, I
checked
On Sep 4, 2014, at 7:26 PM, Jan Wieck j...@wi3ck.info wrote:
This is only because the input data was exact copies of the same strings over
and over again. PGLZ can very well compress slightly less identical strings
of varying lengths too. Not as well, but well enough. But I suspect such
On 08/12/2014 10:58 AM, Robert Haas wrote:
What would really be ideal here is if the JSON code could inform the
toast compression code this many initial bytes are likely
incompressible, just pass them through without trying, and then start
compressing at byte N, where N is the byte following the
On 08/08/2014 10:21 AM, Andrew Dunstan wrote:
On 08/07/2014 11:17 PM, Tom Lane wrote:
I looked into the issue reported in bug #11109. The problem appears to be
that jsonb's on-disk format is designed in such a way that the leading
portion of any JSON array or object will be fairly
On 08/08/2014 11:18 AM, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
On 08/07/2014 11:17 PM, Tom Lane wrote:
I looked into the issue reported in bug #11109. The problem appears to be
that jsonb's on-disk format is designed in such a way that the leading
portion of any JSON array
On Wed, Aug 27, 2014 at 1:09 AM, Arthur Silva arthur...@gmail.com wrote:
It won't be faster by any means, but it should definitely be incorporated
if any format changes are made (like Tom already suggested).
I think it's important we gather at least 2 more things before making any
calls:
*
On 08/16/2014 02:19 AM, Tom Lane wrote:
I think the realistic alternatives at this point are either to
switch to all-lengths as in my test patch, or to use the hybrid approach
of Heikki's test patch. IMO the major attraction of Heikki's patch
is that it'd be upward compatible with existing beta
Heikki Linnakangas hlinnakan...@vmware.com writes:
On 08/16/2014 02:19 AM, Tom Lane wrote:
I think the realistic alternatives at this point are either to
switch to all-lengths as in my test patch, or to use the hybrid approach
of Heikki's test patch. ...
Personally I'd prefer to go to the
On 08/26/2014 07:51 AM, Tom Lane wrote:
My feeling about it at this point is that the apparent speed gain from
using offsets is illusory: in practically all real-world cases where there
are enough keys or array elements for it to matter, costs associated with
compression (or rather failure to
Josh Berkus j...@agliodbs.com writes:
Anyway, I called for feedback on by blog, and have gotten some:
http://www.databasesoup.com/2014/08/the-great-jsonb-tradeoff.html
I was hoping you'd get some useful data from that, but so far it seems
like a rehash of points made in the on-list thread :-(
On 08/26/2014 11:40 AM, Tom Lane wrote:
Josh Berkus j...@agliodbs.com writes:
Anyway, I called for feedback on by blog, and have gotten some:
http://www.databasesoup.com/2014/08/the-great-jsonb-tradeoff.html
I was hoping you'd get some useful data from that, but so far it seems
like a
Josh Berkus j...@agliodbs.com writes:
On 08/26/2014 11:40 AM, Tom Lane wrote:
I was hoping you'd get some useful data from that, but so far it seems
like a rehash of points made in the on-list thread :-(
Unfortunately even the outside commentors don't seem to understand that
storage size
On 2014-08-26 15:01:27 -0400, Tom Lane wrote:
Josh Berkus j...@agliodbs.com writes:
On 08/26/2014 11:40 AM, Tom Lane wrote:
I was hoping you'd get some useful data from that, but so far it seems
like a rehash of points made in the on-list thread :-(
Unfortunately even the outside
On 26 August 2014 11:34, Josh Berkus j...@agliodbs.com wrote:
On 08/26/2014 07:51 AM, Tom Lane wrote:
My feeling about it at this point is that the apparent speed gain from
using offsets is illusory: in practically all real-world cases where
there
are enough keys or array elements for it
Andres Freund and...@2ndquadrant.com writes:
On 2014-08-26 15:01:27 -0400, Tom Lane wrote:
Yeah, exactly. Given current hardware trends, data compression is
becoming more of a win not less as time goes on: CPU cycles are cheap
even compared to main memory access, let alone mass storage. So
On Tue, Aug 26, 2014 at 4:01 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Josh Berkus j...@agliodbs.com writes:
On 08/26/2014 11:40 AM, Tom Lane wrote:
I was hoping you'd get some useful data from that, but so far it seems
like a rehash of points made in the on-list thread :-(
Unfortunately even
On 2014-08-26 15:17:13 -0400, Tom Lane wrote:
Andres Freund and...@2ndquadrant.com writes:
On 2014-08-26 15:01:27 -0400, Tom Lane wrote:
Yeah, exactly. Given current hardware trends, data compression is
becoming more of a win not less as time goes on: CPU cycles are cheap
even compared
On Tue, Aug 26, 2014 at 12:27 PM, Andres Freund and...@2ndquadrant.com wrote:
Anyway, that's just to say that I don't really agree that CPU overhead
is a worthy price to pay for storage efficiency if the gains are small.
+1
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list
On 08/26/2014 12:27 PM, Andres Freund wrote:
Anyway, that's just to say that I don't really agree that CPU overhead
is a worthy price to pay for storage efficiency if the gains are small.
But in this case the gains aren't small; we're talking up to 60% smaller
storage.
Testing STORAGE EXTENDED
I wrote:
I wish it were cache-friendly too, per the upthread tangent about having
to fetch keys from all over the place within a large JSON object.
... and while I was typing that sentence, lightning struck. The existing
arrangement of object subfields with keys and values interleaved is
Tom, here's the results with github data (8 top level keys) only. Here's a
sample object https://gist.github.com/igrigorik/2017462
All-Lenghts + Cache-Aware EXTERNAL
Query 1: 516ms
Query 2: 350ms
The difference is small but I's definitely faster, which makes sense since
cache line misses are
On Tue, Aug 26, 2014 at 8:41 PM, Arthur Silva arthur...@gmail.com wrote:
The difference is small but I's definitely faster, which makes sense since
cache line misses are probably slightly reduced.
As in the previous runs, I ran the query a dozen times and took the average
after excluding runs
Peter Geoghegan p...@heroku.com writes:
I'm not surprised that it hasn't beaten HEAD. I haven't studied the
problem in detail, but I don't think that the cache awareness of the
new revision is necessarily a distinct advantage.
I doubt it's a significant advantage in the current state of the
It won't be faster by any means, but it should definitely be incorporated
if any format changes are made (like Tom already suggested).
I think it's important we gather at least 2 more things before making any
calls:
* Josh tests w/ cache aware patch, which should confirm cache aware is
indeed
On Thu, Aug 21, 2014 at 6:20 PM, Josh Berkus j...@agliodbs.com wrote:
On 08/20/2014 03:42 PM, Arthur Silva wrote:
What data are you using right now Josh?
The same data as upthread.
Can you test the three patches (9.4 head, 9.4 with Tom's cleanup of
Heikki's patch, and 9.4 with Tom's
On 08/20/2014 03:42 PM, Arthur Silva wrote:
What data are you using right now Josh?
The same data as upthread.
Can you test the three patches (9.4 head, 9.4 with Tom's cleanup of
Heikki's patch, and 9.4 with Tom's latest lengths-only) on your workload?
I'm concerned that my workload is unusual
1 - 100 of 187 matches
Mail list logo
