On Mon, Sep 29, 2014 at 12:30:40PM -0400, Tom Lane wrote:
> Bruce Momjian writes:
> > On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
> >> BTW, it seems like there is consensus that we ought to reorder the items
> >> in a jsonb object to have keys first and then values, independently of
Arthur Silva writes:
> What's the call on the stride length? Are we going to keep it hardcoded?
At the moment it's 32, but we could change it without forcing a new
initdb. I ran a simple test that seemed to show 32 was a good choice,
but if anyone else wants to try other cases, go for it.
On 09/29/2014 11:49 AM, Arthur Silva wrote:
> What's the call on the stride length? Are we going to keep it hardcoded?
Please, yes. The complications caused by a variable stride length would
be horrible.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers ma
On Mon, Sep 29, 2014 at 12:19 AM, Josh Berkus wrote:
> On 09/26/2014 06:20 PM, Josh Berkus wrote:
> > Overall, I'm satisfied with the performance of the length-and-offset
> > patch.
>
> Oh, also ... no bugs found.
>
> So, can we get Beta3 out now?
>
> --
> Josh Berkus
> PostgreSQL Experts Inc.
>
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > * Tom Lane (t...@sss.pgh.pa.us) wrote:
> >> Done --- 201409291 is the cutover point.
>
> > Just to clarify- the commit bumped the catversion to 201409292, so
> > version <= 201409291 has the old format while version > 201409291 ha
Stephen Frost writes:
> * Tom Lane (t...@sss.pgh.pa.us) wrote:
>> Done --- 201409291 is the cutover point.
> Just to clarify- the commit bumped the catversion to 201409292, so
> version <= 201409291 has the old format while version > 201409291 has
> the new format. There was no 201409291, so I s
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Bruce Momjian writes:
> > On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
> >> BTW, it seems like there is consensus that we ought to reorder the items
> >> in a jsonb object to have keys first and then values, independently of the
> >> other issu
Bruce Momjian writes:
> On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
>> BTW, it seems like there is consensus that we ought to reorder the items
>> in a jsonb object to have keys first and then values, independently of the
>> other issues under discussion. This means we *will* be bre
Josh Berkus writes:
> So, can we get Beta3 out now?
If nobody else steps up and says they want to do some performance
testing, I'll push the latest lengths+offsets patch tomorrow.
Are any of the other open items listed at
https://wiki.postgresql.org/wiki/PostgreSQL_9.4_Open_Items
things that we
On 09/26/2014 06:20 PM, Josh Berkus wrote:
> Overall, I'm satisfied with the performance of the length-and-offset
> patch.
Oh, also ... no bugs found.
So, can we get Beta3 out now?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hacke
All,
So these results have become a bit complex. So spreadsheet time.
https://docs.google.com/spreadsheets/d/1Mokpx3EqlbWlFDIkF9qzpM7NneN9z-QOXWSzws3E-R4
Some details:
The Length-and-Offset test was performed using a more recent 9.4
checkout than the other two tests. This was regrettable, and
On 09/25/2014 08:10 PM, Tom Lane wrote:
> I wrote:
>> The "offsets-and-lengths" patch seems like the approach we ought to
>> compare to my patch, but it looks pretty unfinished to me: AFAICS it
>> includes logic to understand offsets sprinkled into a mostly-lengths
>> array, but no logic that would
I wrote:
> The "offsets-and-lengths" patch seems like the approach we ought to
> compare to my patch, but it looks pretty unfinished to me: AFAICS it
> includes logic to understand offsets sprinkled into a mostly-lengths
> array, but no logic that would actually *store* any such offsets,
> which me
Bruce Momjian wrote:
> 3. 9.3 multi-xact bugs spooked us into being more careful
Uh. Multixact changes in 9.3 were infinitely more invasive than the
jsonb changes will ever be. a) they touched basic visibility design and
routines,
which are complex, understood by very few people, and ha
On Thu, Sep 25, 2014 at 09:00:07PM +0200, Andres Freund wrote:
> On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote:
> > On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
> > > BTW, it seems like there is consensus that we ought to reorder the items
> > > in a jsonb object to have keys first
On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote:
> On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
> > BTW, it seems like there is consensus that we ought to reorder the items
> > in a jsonb object to have keys first and then values, independently of the
> > other issues under discussio
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote:
> BTW, it seems like there is consensus that we ought to reorder the items
> in a jsonb object to have keys first and then values, independently of the
> other issues under discussion. This means we *will* be breaking on-disk
> compatibilit
BTW, it seems like there is consensus that we ought to reorder the items
in a jsonb object to have keys first and then values, independently of the
other issues under discussion. This means we *will* be breaking on-disk
compatibility with 9.4beta2, which means pg_upgrade will need to be taught
to
On 09/25/2014 11:22 AM, Tom Lane wrote:
> In the interests of pushing this forward, I will work today on
> trying to finish and review Heikki's offsets-and-lengths patch
> so that we have something we can do performance testing on.
> I doubt that the performance testing will tell us anything we
> d
Josh Berkus writes:
> On 09/25/2014 10:26 AM, Andres Freund wrote:
>> On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
>>> If Heikki says it's ready, I'll test. So far he's said that it wasn't
>>> done yet.
>> http://www.postgresql.org/message-id/541c242e.3030...@vmware.com
> Yeah, and that did
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
> If Heikki says it's ready, I'll test. So far he's said that it wasn't
> done yet.
http://www.postgresql.org/message-id/541c242e.3030...@vmware.com
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
Pos
On 2014-09-25 10:29:51 -0700, Josh Berkus wrote:
> On 09/25/2014 10:26 AM, Andres Freund wrote:
> > On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
> >> If Heikki says it's ready, I'll test. So far he's said that it wasn't
> >> done yet.
> >
> > http://www.postgresql.org/message-id/541c242e.3030
On 09/25/2014 10:26 AM, Andres Freund wrote:
> On 2014-09-25 10:25:24 -0700, Josh Berkus wrote:
>> If Heikki says it's ready, I'll test. So far he's said that it wasn't
>> done yet.
>
> http://www.postgresql.org/message-id/541c242e.3030...@vmware.com
Yeah, and that didn't include some of Tom's b
On 09/25/2014 10:20 AM, Andres Freund wrote:
> On 2014-09-25 10:18:24 -0700, Josh Berkus wrote:
>> On 09/25/2014 10:14 AM, Bruce Momjian wrote:
>>> On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
But independent of which version is chosen, we *REALLY* need to make the
decis
On 2014-09-25 10:18:24 -0700, Josh Berkus wrote:
> On 09/25/2014 10:14 AM, Bruce Momjian wrote:
> > On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
> >> But independent of which version is chosen, we *REALLY* need to make the
> >> decision soon. This issue has held up the next beta (
On 09/25/2014 10:14 AM, Bruce Momjian wrote:
> On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
>> But independent of which version is chosen, we *REALLY* need to make the
>> decision soon. This issue has held up the next beta (like jsonb has
>> blocked previous beta) for *weeks*.
>>
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote:
> But independent of which version is chosen, we *REALLY* need to make the
> decision soon. This issue has held up the next beta (like jsonb has
> blocked previous beta) for *weeks*.
>
> Personally it doesn't make me very happy that He
On 09/25/2014 09:01 AM, Andres Freund wrote:
> But independent of which version is chosen, we *REALLY* need to make the
> decision soon. This issue has held up the next beta (like jsonb has
> blocked previous beta) for *weeks*.
Yes, please!
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts
On 2014-09-19 15:40:14 +0300, Heikki Linnakangas wrote:
> On 09/18/2014 09:27 PM, Heikki Linnakangas wrote:
> >I'll try to write a more polished patch tomorrow. We'll then see what it
> >looks like, and can decide if we want it.
>
> Ok, here are two patches. One is a refined version of my earlier
Heikki Linnakangas writes:
> On 09/24/2014 08:16 AM, Tom Lane wrote:
>> Heikki's patch would eat up the high-order JEntry bits, but the other
>> points remain.
> If we don't need to be backwards-compatible with the 9.4beta on-disk
> format, we don't necessarily need to eat the high-order JEntry
On 09/24/2014 08:16 AM, Tom Lane wrote:
Jan Wieck writes:
On 09/15/2014 09:46 PM, Craig Ringer wrote:
Anyway - this is looking like the change will go in, and with it a
catversion bump. Introduction of a jsonb version/flags byte might be
worthwhile at the same time. It seems likely that there'
Jan Wieck writes:
> On 09/15/2014 09:46 PM, Craig Ringer wrote:
>> Anyway - this is looking like the change will go in, and with it a
>> catversion bump. Introduction of a jsonb version/flags byte might be
>> worthwhile at the same time. It seems likely that there'll be more room
>> for improvemen
On Tue, Sep 23, 2014 at 10:02 PM, Jan Wieck wrote:
>> Is it worth paying a byte per value to save on possible upgrade pain?
>>
>
> This comment seems to have drowned in the discussion.
>
> If there indeed has to be a catversion bump in the process of this, then I
> agree with Craig.
-1. We alread
On 09/15/2014 09:46 PM, Craig Ringer wrote:
On 09/16/2014 07:44 AM, Peter Geoghegan wrote:
FWIW, I am slightly concerned about weighing use cases around very
large JSON documents too heavily. Having enormous jsonb documents just
isn't going to work out that well, but neither will equivalent desi
Peter Geoghegan writes:
> On Fri, Sep 19, 2014 at 5:40 AM, Heikki Linnakangas
> wrote:
>> I think we should bite the bullet and break compatibility with 9.4beta2
>> format, even if we go with "my patch". In a jsonb object, it makes sense to
>> store all the keys first, like Tom did, because of ca
On Fri, Sep 19, 2014 at 5:40 AM, Heikki Linnakangas
wrote:
> I think we should bite the bullet and break compatibility with 9.4beta2
> format, even if we go with "my patch". In a jsonb object, it makes sense to
> store all the keys first, like Tom did, because of cache benefits, and the
> future p
On 09/19/2014 07:07 AM, Tom Lane wrote:
> Heikki Linnakangas writes:
>> Tom: You mentioned earlier that your patch fixes some existing bugs.
>> What were they?
>
> What I remember at the moment (sans caffeine) is that the routines for
> assembling jsonb values out of field data were lacking some
Heikki Linnakangas writes:
> Tom: You mentioned earlier that your patch fixes some existing bugs.
> What were they?
What I remember at the moment (sans caffeine) is that the routines for
assembling jsonb values out of field data were lacking some necessary
tests for overflow of the size/offset f
On 09/18/2014 09:27 PM, Heikki Linnakangas wrote:
On 09/18/2014 07:53 PM, Josh Berkus wrote:
On 09/16/2014 08:45 PM, Tom Lane wrote:
We're somewhat comparing apples and oranges here, in that I pushed my
approach to something that I think is of committable quality (and which,
not incidentally, f
On 09/18/2014 07:53 PM, Josh Berkus wrote:
On 09/16/2014 08:45 PM, Tom Lane wrote:
We're somewhat comparing apples and oranges here, in that I pushed my
approach to something that I think is of committable quality (and which,
not incidentally, fixes some existing bugs that we'd need to fix in an
On 09/16/2014 08:45 PM, Tom Lane wrote:
> We're somewhat comparing apples and oranges here, in that I pushed my
> approach to something that I think is of committable quality (and which,
> not incidentally, fixes some existing bugs that we'd need to fix in any
> case); while Heikki's patch was just
Heikki Linnakangas writes:
> On 09/16/2014 10:37 PM, Robert Haas wrote:
>> On Tue, Sep 16, 2014 at 3:24 PM, Josh Berkus wrote:
>>> Do you feel that way *as a code maintainer*? That is, if you ended up
>>> maintaining the JSONB code, would you still feel that it's worth the
>>> extra complexity?
- Цитат от Robert Haas (robertmh...@gmail.com), на 16.09.2014 в 22:20 -
>
> In practice, I'm not very surprised that the impact doesn't seem too
> bad when you're running SQL queries from the client. There's so much
> other overhead, for de-TOASTing and client communication and even just
On Tue, Sep 16, 2014 at 4:20 PM, Robert Haas wrote:
> On Tue, Sep 16, 2014 at 1:11 PM, Josh Berkus wrote:
> >>> Well, I can only judge from the use cases I personally have, none of
> >>> which involve more than 100 keys at any level for most rows. So far
> >>> I've seen some people argue hypote
On 09/16/2014 10:37 PM, Robert Haas wrote:
On Tue, Sep 16, 2014 at 3:24 PM, Josh Berkus wrote:
Do you feel that way *as a code maintainer*? That is, if you ended up
maintaining the JSONB code, would you still feel that it's worth the
extra complexity? Because that will be the main cost here.
On 16/09/14 21:20, Robert Haas wrote:
In practice, I'm not very surprised that the impact doesn't seem too
bad when you're running SQL queries from the client. There's so much
other overhead, for de-TOASTing and client communication and even just
planner and executor costs, that this gets lost i
On Tue, Sep 16, 2014 at 3:24 PM, Josh Berkus wrote:
> Do you feel that way *as a code maintainer*? That is, if you ended up
> maintaining the JSONB code, would you still feel that it's worth the
> extra complexity? Because that will be the main cost here.
I feel that Heikki doesn't have a reput
Heikki, Robert:
On 09/16/2014 11:12 AM, Heikki Linnakangas wrote:
> Are you looking for someone with a real life scenario, or just synthetic
> test case? The latter is easy to do.
>
> See attached test program. It's basically the same I posted earlier.
> Here are the results from my laptop with T
On Tue, Sep 16, 2014 at 1:11 PM, Josh Berkus wrote:
>>> Well, I can only judge from the use cases I personally have, none of
>>> which involve more than 100 keys at any level for most rows. So far
>>> I've seen some people argue hypotetical use cases involving hundreds of
>>> keys per level, but
On Tue, Sep 16, 2014 at 3:12 PM, Heikki Linnakangas
wrote:
> I'll leave it up to the jury to decide if we care or not. It seems like a
> fairly unusual use case, where you push around large enough arrays or
> objects to notice. Then again, I'm sure *someone* will do it. People do
> strange things,
On 09/16/2014 07:47 PM, Josh Berkus wrote:
On 09/16/2014 06:31 AM, Robert Haas wrote:
On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan wrote:
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus wrote:
Actually, having the keys all at the same level *is* relevant for the
issue we're discussing. If
On 09/16/2014 09:54 AM, Robert Haas wrote:
> On Tue, Sep 16, 2014 at 12:47 PM, Josh Berkus wrote:
>> On 09/16/2014 06:31 AM, Robert Haas wrote:
>>> On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan wrote:
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus wrote:
> Actually, having the keys all
On Tue, Sep 16, 2014 at 12:47 PM, Josh Berkus wrote:
> On 09/16/2014 06:31 AM, Robert Haas wrote:
>> On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan wrote:
>>> On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus wrote:
Actually, having the keys all at the same level *is* relevant for the
iss
On 09/16/2014 06:31 AM, Robert Haas wrote:
> On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan wrote:
>> On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus wrote:
>>> Actually, having the keys all at the same level *is* relevant for the
>>> issue we're discussing. If those 270 keys are organized in a t
On Mon, Sep 15, 2014 at 7:44 PM, Peter Geoghegan wrote:
> On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus wrote:
>> Actually, having the keys all at the same level *is* relevant for the
>> issue we're discussing. If those 270 keys are organized in a tree, it's
>> not the same as having them all on
On 09/16/2014 07:44 AM, Peter Geoghegan wrote:
> FWIW, I am slightly concerned about weighing use cases around very
> large JSON documents too heavily. Having enormous jsonb documents just
> isn't going to work out that well, but neither will equivalent designs
> in popular document database system
I couldn't get my hands on the twitter data but I'm generating my own. The
json template is http://paste2.org/wJ1dfcjw and data was generated with
http://www.json-generator.com/. It has 35 top level keys, just in case
someone is wondering.
I generated 1 random objects and I'm inserting them rep
On Mon, Sep 15, 2014 at 4:05 PM, Josh Berkus wrote:
> Actually, having the keys all at the same level *is* relevant for the
> issue we're discussing. If those 270 keys are organized in a tree, it's
> not the same as having them all on one level (and not as problematic).
I believe Robert meant th
On 09/15/2014 02:16 PM, Robert Haas wrote:
> On Mon, Sep 15, 2014 at 3:09 PM, Josh Berkus wrote:
>> On 09/15/2014 10:23 AM, Claudio Freire wrote:
>>> Now, large small keys could be 200 or 2000, or even 20k. I'd guess
>>> several should be tested to find the shape of the curve.
>>
>> Well, we know
On Mon, Sep 15, 2014 at 3:09 PM, Josh Berkus wrote:
> On 09/15/2014 10:23 AM, Claudio Freire wrote:
>> Now, large small keys could be 200 or 2000, or even 20k. I'd guess
>> several should be tested to find the shape of the curve.
>
> Well, we know that it's not noticeable with 200, and that it is
On 09/15/2014 12:25 PM, Claudio Freire wrote:
> On Mon, Sep 15, 2014 at 4:17 PM, Josh Berkus wrote:
>> On 09/15/2014 12:15 PM, Claudio Freire wrote:
>>> So while you're right that it's perhaps above what would be a common
>>> use case, the range "somewhere between 200 and 100K" for the tipping
>>>
On Mon, Sep 15, 2014 at 4:17 PM, Josh Berkus wrote:
> On 09/15/2014 12:15 PM, Claudio Freire wrote:
>> So while you're right that it's perhaps above what would be a common
>> use case, the range "somewhere between 200 and 100K" for the tipping
>> point seems overly imprecise to me.
>
> Well, then,
On 09/15/2014 12:15 PM, Claudio Freire wrote:
> So while you're right that it's perhaps above what would be a common
> use case, the range "somewhere between 200 and 100K" for the tipping
> point seems overly imprecise to me.
Well, then, you know how to solve that.
--
Josh Berkus
PostgreSQL Expe
On Mon, Sep 15, 2014 at 4:09 PM, Josh Berkus wrote:
> On 09/15/2014 10:23 AM, Claudio Freire wrote:
>> Now, large small keys could be 200 or 2000, or even 20k. I'd guess
>> several should be tested to find the shape of the curve.
>
> Well, we know that it's not noticeable with 200, and that it is
On 09/15/2014 10:23 AM, Claudio Freire wrote:
> Now, large small keys could be 200 or 2000, or even 20k. I'd guess
> several should be tested to find the shape of the curve.
Well, we know that it's not noticeable with 200, and that it is
noticeable with 100K. It's only worth testing further if we
On Mon, Sep 15, 2014 at 2:12 PM, Josh Berkus wrote:
> If not, I think the corner case is so obscure as to be not worth
> optimizing for. I can't imagine that more than a tiny minority of our
> users are going to have thousands of keys per datum.
Worst case is linear cost scaling vs number of key
On 09/12/2014 01:30 PM, Heikki Linnakangas wrote:
>
> Performance was one argument for sure. It's not hard to come up with a
> case where the all-lengths approach is much slower: take a huge array
> with, say, million elements, and fetch the last element in a tight loop.
> And do that in a PL/pgSQ
On 09/12/2014 08:52 PM, Tom Lane wrote:
Robert Haas writes:
On Fri, Sep 12, 2014 at 1:11 PM, Josh Berkus wrote:
It's certainly possible that there is a test case for which Heikki's
approach is superior, but if so we haven't seen it. And since it's
approach is also more complicated, sticking
Robert Haas writes:
> On Fri, Sep 12, 2014 at 1:11 PM, Josh Berkus wrote:
>> It's certainly possible that there is a test case for which Heikki's
>> approach is superior, but if so we haven't seen it. And since it's
>> approach is also more complicated, sticking with the simpler
>> lengths-only
On Fri, Sep 12, 2014 at 1:11 PM, Josh Berkus wrote:
> On 09/12/2014 10:00 AM, Robert Haas wrote:
>> On Fri, Sep 12, 2014 at 1:00 PM, Robert Haas wrote:
>>> On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus wrote:
So, I finally got time to test Tom's latest patch on this.
TLDR: we want
On 09/12/2014 10:00 AM, Robert Haas wrote:
> On Fri, Sep 12, 2014 at 1:00 PM, Robert Haas wrote:
>> On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus wrote:
>>> So, I finally got time to test Tom's latest patch on this.
>>>
>>> TLDR: we want to go with Tom's latest patch and release beta3.
>>>
>>> Fig
On Fri, Sep 12, 2014 at 1:00 PM, Robert Haas wrote:
> On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus wrote:
>> So, I finally got time to test Tom's latest patch on this.
>>
>> TLDR: we want to go with Tom's latest patch and release beta3.
>>
>> Figures:
>>
>> So I tested HEAD against the latest len
On Thu, Sep 11, 2014 at 9:01 PM, Josh Berkus wrote:
> So, I finally got time to test Tom's latest patch on this.
>
> TLDR: we want to go with Tom's latest patch and release beta3.
>
> Figures:
>
> So I tested HEAD against the latest lengths patch. Per Arthur Silva, I
> checked uncompressed times
On 09/11/2014 06:56 PM, Arthur Silva wrote:
>
> In my testings with the github archive data the savings <->
> performance-penalty was fine, but I'm not confident in those results
> since there were only 8 top level keys.
Well, we did want to see that the patch doesn't create a regression with
dat
On Thu, Sep 11, 2014 at 10:01 PM, Josh Berkus wrote:
> So, I finally got time to test Tom's latest patch on this.
>
> TLDR: we want to go with Tom's latest patch and release beta3.
>
> Figures:
>
> So I tested HEAD against the latest lengths patch. Per Arthur Silva, I
> checked uncompressed time
* Josh Berkus (j...@agliodbs.com) wrote:
> TLDR: we want to go with Tom's latest patch and release beta3.
Having not even read the rest- yes please. We really need to get beta3
out and figure out when we're going to actually release 9.4...
Admittedly, the last month has been good and we've been f
So, I finally got time to test Tom's latest patch on this.
TLDR: we want to go with Tom's latest patch and release beta3.
Figures:
So I tested HEAD against the latest lengths patch. Per Arthur Silva, I
checked uncompressed times for JSONB against compressed times. This
changed the picture cons
On Sep 4, 2014, at 7:26 PM, Jan Wieck wrote:
> This is only because the input data was exact copies of the same strings over
> and over again. PGLZ can very well compress slightly less identical strings
> of varying lengths too. Not as well, but well enough. But I suspect such
> input data wou
On 08/08/2014 11:18 AM, Tom Lane wrote:
Andrew Dunstan writes:
On 08/07/2014 11:17 PM, Tom Lane wrote:
I looked into the issue reported in bug #11109. The problem appears to be
that jsonb's on-disk format is designed in such a way that the leading
portion of any JSON array or object will be f
On 08/08/2014 10:21 AM, Andrew Dunstan wrote:
On 08/07/2014 11:17 PM, Tom Lane wrote:
I looked into the issue reported in bug #11109. The problem appears to be
that jsonb's on-disk format is designed in such a way that the leading
portion of any JSON array or object will be fairly incompressib
On 08/12/2014 10:58 AM, Robert Haas wrote:
What would really be ideal here is if the JSON code could inform the
toast compression code "this many initial bytes are likely
incompressible, just pass them through without trying, and then start
compressing at byte N", where N is the byte following th
On Wed, Aug 27, 2014 at 1:09 AM, Arthur Silva wrote:
> It won't be faster by any means, but it should definitely be incorporated
> if any format changes are made (like Tom already suggested).
>
> I think it's important we gather at least 2 more things before making any
> calls:
> * Josh tests w/
It won't be faster by any means, but it should definitely be incorporated
if any format changes are made (like Tom already suggested).
I think it's important we gather at least 2 more things before making any
calls:
* Josh tests w/ cache aware patch, which should confirm cache aware is
indeed pref
Peter Geoghegan writes:
> I'm not surprised that it hasn't beaten HEAD. I haven't studied the
> problem in detail, but I don't think that the "cache awareness" of the
> new revision is necessarily a distinct advantage.
I doubt it's a significant advantage in the current state of the code;
I'm hap
On Tue, Aug 26, 2014 at 8:41 PM, Arthur Silva wrote:
> The difference is small but I's definitely faster, which makes sense since
> cache line misses are probably slightly reduced.
> As in the previous runs, I ran the query a dozen times and took the average
> after excluding runs with a high devi
Tom, here's the results with github data (8 top level keys) only. Here's a
sample object https://gist.github.com/igrigorik/2017462
All-Lenghts + Cache-Aware EXTERNAL
Query 1: 516ms
Query 2: 350ms
The difference is small but I's definitely faster, which makes sense since
cache line misses are prob
I wrote:
> I wish it were cache-friendly too, per the upthread tangent about having
> to fetch keys from all over the place within a large JSON object.
> ... and while I was typing that sentence, lightning struck. The existing
> arrangement of object subfields with keys and values interleaved is
On 08/26/2014 12:27 PM, Andres Freund wrote:
> Anyway, that's just to say that I don't really agree that CPU overhead
> is a worthy price to pay for storage efficiency if the gains are small.
But in this case the gains aren't small; we're talking up to 60% smaller
storage.
Testing STORAGE EXTENDE
On Tue, Aug 26, 2014 at 12:27 PM, Andres Freund wrote:
> Anyway, that's just to say that I don't really agree that CPU overhead
> is a worthy price to pay for storage efficiency if the gains are small.
+1
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org
On 2014-08-26 15:17:13 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2014-08-26 15:01:27 -0400, Tom Lane wrote:
> >> Yeah, exactly. Given current hardware trends, data compression is
> >> becoming more of a win not less as time goes on: CPU cycles are cheap
> >> even compared to main memo
On Tue, Aug 26, 2014 at 4:01 PM, Tom Lane wrote:
> Josh Berkus writes:
>> On 08/26/2014 11:40 AM, Tom Lane wrote:
>>> I was hoping you'd get some useful data from that, but so far it seems
>>> like a rehash of points made in the on-list thread :-(
>
>> Unfortunately even the outside commentors do
Andres Freund writes:
> On 2014-08-26 15:01:27 -0400, Tom Lane wrote:
>> Yeah, exactly. Given current hardware trends, data compression is
>> becoming more of a win not less as time goes on: CPU cycles are cheap
>> even compared to main memory access, let alone mass storage. So I'm
>> thinking w
On 26 August 2014 11:34, Josh Berkus wrote:
> On 08/26/2014 07:51 AM, Tom Lane wrote:
> > My feeling about it at this point is that the apparent speed gain from
> > using offsets is illusory: in practically all real-world cases where
> there
> > are enough keys or array elements for it to matter,
On 2014-08-26 15:01:27 -0400, Tom Lane wrote:
> Josh Berkus writes:
> > On 08/26/2014 11:40 AM, Tom Lane wrote:
> >> I was hoping you'd get some useful data from that, but so far it seems
> >> like a rehash of points made in the on-list thread :-(
>
> > Unfortunately even the outside commentors d
Josh Berkus writes:
> On 08/26/2014 11:40 AM, Tom Lane wrote:
>> I was hoping you'd get some useful data from that, but so far it seems
>> like a rehash of points made in the on-list thread :-(
> Unfortunately even the outside commentors don't seem to understand that
> storage size *is* related t
On 08/26/2014 11:40 AM, Tom Lane wrote:
> Josh Berkus writes:
>> Anyway, I called for feedback on by blog, and have gotten some:
>> http://www.databasesoup.com/2014/08/the-great-jsonb-tradeoff.html
>
> I was hoping you'd get some useful data from that, but so far it seems
> like a rehash of poin
Josh Berkus writes:
> Anyway, I called for feedback on by blog, and have gotten some:
> http://www.databasesoup.com/2014/08/the-great-jsonb-tradeoff.html
I was hoping you'd get some useful data from that, but so far it seems
like a rehash of points made in the on-list thread :-(
On 08/26/2014 07:51 AM, Tom Lane wrote:
> My feeling about it at this point is that the apparent speed gain from
> using offsets is illusory: in practically all real-world cases where there
> are enough keys or array elements for it to matter, costs associated with
> compression (or rather failure
Heikki Linnakangas writes:
> On 08/16/2014 02:19 AM, Tom Lane wrote:
>> I think the realistic alternatives at this point are either to
>> switch to all-lengths as in my test patch, or to use the hybrid approach
>> of Heikki's test patch. ...
>> Personally I'd prefer to go to the all-lengths approa
On 08/16/2014 02:19 AM, Tom Lane wrote:
I think the realistic alternatives at this point are either to
switch to all-lengths as in my test patch, or to use the hybrid approach
of Heikki's test patch. IMO the major attraction of Heikki's patch
is that it'd be upward compatible with existing beta
1 - 100 of 189 matches
Mail list logo
