On Nov 20, 2010, at 9:31 PM, Terry Laurenzo wrote:
Assuming that the JSON datatype (at a minimum) normalizes text for storage,
then the text storage option accounts for about the most expensive path but
with none of the benefits of an internal binary form (smaller size, ability
to cheaply
On Sun, Nov 21, 2010 at 12:31 AM, Terry Laurenzo t...@laurenzo.org wrote:
What we see from this is that parsing JSON text and generating a binary
representation is cheap, representing approximately 10% of the base case
time. Conversely, anything that involves generating JSON text is expensive,
On 11/21/2010 12:31 AM, Terry Laurenzo wrote:
I copied the 5 sample documents from json.org http://json.org's
example section for these tests. These are loaded into a table with a
varchar column 1000 times each (so the test table has 5000 rows in
it). In all situations, the binary
I've got a new stripped down version of the binary json plugin on github:
https://github.com/tlaurenzo/pgjson
With all due warning of contrived benchmarks, I wrote some tests to see
where things stand. The test script is here:
On Mon, Nov 8, 2010 at 9:52 PM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
Are there any activities in JSON data types for the next commitfest?
I'm leaning toward the view that we shouldn't commit a JSON
implementation to core (or contrib) for 9.1. We have at least three
competing
Robert,
I think I agree. At a minimum, I would like to see the chosen of the
competing priorities live on as an outside module for use by previous
versions. Even having proposed one, and soon to be two of the competing
implementations, it makes me nervous to commit to one at this juncture.
I'm
Are there any activities in JSON data types for the next commitfest?
I'd like to help you if you are working on the topic,
or I can make up previous works and discussions by myself.
On Mon, Oct 25, 2010 at 1:36 AM, Terry Laurenzo t...@laurenzo.org wrote:
Yeah, my concern is not whether the
It doesn't do particularly well on my previous example of [1,2,3]. It
comes out slightly shorter on [a,b,c] and better if the strings
need any escaping. I don't think the float4 and float8 formats are
very useful; how could you be sure that the output was going to look
the same as the
On Sun, Oct 24, 2010 at 2:21 AM, Terry Laurenzo t...@laurenzo.org wrote:
It doesn't do particularly well on my previous example of [1,2,3]. It
comes out slightly shorter on [a,b,c] and better if the strings
need any escaping. I don't think the float4 and float8 formats are
very useful; how
Yeah, my concern is not whether the overhead will be zero; it's
whether it will be small, yet allow large gains on other operations.
Like, how much slower will it be to pull out a moderately complex 1MB
JSON blob (not just a big string) out of a single-row, single-column
table? If it's 5%
I'm still going to write up a proposed grammar that takes these items into
account - just ran out of time tonight.
The binary format I was thinking of is here:
http://github.com/tlaurenzo/pgjson/blob/master/pgjson/shared/include/json/jsonbinary.h
This was just a quick brain dump and I
On Sat, Oct 23, 2010 at 8:33 PM, Terry Laurenzo t...@laurenzo.org wrote:
I'm still going to write up a proposed grammar that takes these items into
account - just ran out of time tonight.
The binary format I was thinking of is here:
On Wed, Oct 20, 2010 at 6:39 AM, Terry Laurenzo t...@laurenzo.org wrote:
The answer may be to have both a jsontext and jsonbinary type as each will
be optimized for a different case.
I want to choose one format for JSON rather than having two types.
It should be more efficient than other format
Good points. In addition, any binary format needs to support object
property traversal without having to do a deep scan of all descendants.
BSON handles this with explicit lengths for document types (objects and
arrays) so that entire parts of the tree can be skipped during sibling
traversal.
It
* Terry Laurenzo:
Agreed. BSON was born out of implementations that either lacked
arbitrary precision numbers or had a strong affinity to an
int/floating point way of thinking about numbers. I believe that if
BSON had an arbitrary precision number type, it would be a proper
superset of
On 10/20/2010 01:15 PM, Florian Weimer wrote:
* Terry Laurenzo:
Agreed. BSON was born out of implementations that either lacked
arbitrary precision numbers or had a strong affinity to an
int/floating point way of thinking about numbers. I believe that if
BSON had an arbitrary precision
On Sat, Oct 16, 2010 at 12:59 PM, Terry Laurenzo t...@laurenzo.org wrote:
- It is directly iterable without parsing and/or constructing an AST
- It is its own representation. If iterating and you want to tear-off a
value to be returned or used elsewhere, its a simple buffer copy plus
On 10/19/2010 10:44 AM, Robert Haas wrote:
On Sat, Oct 16, 2010 at 12:59 PM, Terry Laurenzot...@laurenzo.org wrote:
- It is directly iterable without parsing and/or constructing an AST
- It is its own representation. If iterating and you want to tear-off a
value to be returned or
Agreed. BSON was born out of implementations that either lacked arbitrary
precision numbers or had a strong affinity to an int/floating point way of
thinking about numbers. I believe that if BSON had an arbitrary precision
number type, it would be a proper superset of JSON.
As an aside, the max
On Tue, Oct 19, 2010 at 11:22 AM, Terry Laurenzo t...@laurenzo.org wrote:
Agreed. BSON was born out of implementations that either lacked arbitrary
precision numbers or had a strong affinity to an int/floating point way of
thinking about numbers. I believe that if BSON had an arbitrary
On Oct 19, 2010, at 12:17 PM, Robert Haas wrote:
I think we should take a few steps back and ask why we think that
binary encoding is the way to go. We store XML as text, for example,
and I can't remember any complaints about that on -bugs or
-performance, so why do we think JSON will be
On Tue, Oct 19, 2010 at 11:22 AM, Terry Laurenzo t...@laurenzo.org wrote:
Perhaps we should enumerate the attributes of what would make a good binary
encoding?
Not sure if we're discussing the internal storage format or the binary
send/recv format, but in my humble opinion, some attributes of a
On Tue, Oct 19, 2010 at 3:40 PM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
On Tue, Oct 19, 2010 at 3:17 PM, Robert Haas robertmh...@gmail.com wrote:
I think we should take a few steps back and ask why we think that
binary encoding is the way to go. We store XML as text, for example,
and
On Tue, Oct 19, 2010 at 2:46 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Oct 19, 2010 at 3:40 PM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
On Tue, Oct 19, 2010 at 3:17 PM, Robert Haas robertmh...@gmail.com
wrote:
I think we should take a few steps back and ask why we think
On Tue, Oct 19, 2010 at 12:17 PM, Robert Haas robertmh...@gmail.com wrote:
I think we should take a few steps back and ask why we think that
binary encoding is the way to go. We store XML as text, for example,
and I can't remember any complaints about that on -bugs or
-performance, so why do
After spending a week in the morass of this, I have to say that I am less
certain than I was on any front regarding the text/binary distinction. I'll
take some time and benchmark different cases. My hypothesis is that a well
implemented binary structure and conversions will add minimal
2010/10/19 Greg Stark gsst...@mit.edu:
On Tue, Oct 19, 2010 at 12:17 PM, Robert Haas robertmh...@gmail.com wrote:
I think we should take a few steps back and ask why we think that
binary encoding is the way to go. We store XML as text, for example,
and I can't remember any complaints about
Terry Laurenzo t...@laurenzo.org writes:
After spending a week in the morass of this, I have to say that I am less
certain than I was on any front regarding the text/binary distinction. I'll
take some time and benchmark different cases. My hypothesis is that a well
implemented binary
Greg Stark gsst...@mit.edu writes:
The elephant in the room is if the binary encoded form is smaller then
it occupies less ram and disk bandwidth to copy it around.
It seems equally likely that a binary-encoded form could be larger
than the text form (that's often true for our other datatypes).
On Tue, Oct 19, 2010 at 4:51 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Terry Laurenzo t...@laurenzo.org writes:
After spending a week in the morass of this, I have to say that I am less
certain than I was on any front regarding the text/binary distinction.
I'll
take some time and benchmark
Excerpts from David E. Wheeler's message of mar oct 19 16:36:20 -0300 2010:
On Oct 19, 2010, at 12:17 PM, Robert Haas wrote:
I think we should take a few steps back and ask why we think that
binary encoding is the way to go. We store XML as text, for example,
and I can't remember any
On Tue, Oct 19, 2010 at 6:56 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Greg Stark gsst...@mit.edu writes:
The elephant in the room is if the binary encoded form is smaller then
it occupies less ram and disk bandwidth to copy it around.
It seems equally likely that a binary-encoded form could be
I hear ya. It might be a premature optimization but I still think there may be
benefit for the case of large scale extraction and in- database transformation
of large JSON datastructures. We have terabytes of this stuff and I'd like
something between the hip nosql options and a fully
On Tue, Oct 19, 2010 at 9:15 PM, Terri Laurenzo t...@laurenzo.org wrote:
I hear ya. It might be a premature optimization but I still think there may
be benefit for the case of large scale extraction and in- database
transformation of large JSON datastructures. We have terabytes of this
On Sun, Oct 17, 2010 at 5:18 AM, Hitoshi Harada umi.tan...@gmail.com wrote:
Reading your proposal, I'm now +1 for BSON-like style. Especially JS
engine's capabilities to map external data to the language
representation are good news.
Hmm, we could store postgres' data types as-is with their
I like as simple design as we can accept. ISTM format, I/O interface,
simple get/set, mapping tuple from/to object, and indexing are minimum
requirement.
+1 to small start, but simple get/set are already debatable...
For example, text/json conversion:
A. SELECT 'json'::json;
B. SELECT
Hi all -
I independently started some work on a similar capability as was contributed
back in August by Joey Adams for a json datatype. Before starting, I did a
quick search but for some reason didn't turn this existing thread up.
What I've been working on is out on github for now:
2010/10/17 Terry Laurenzo t...@laurenzo.org:
Hi all -
I independently started some work on a similar capability as was contributed
back in August by Joey Adams for a json datatype. Before starting, I did a
quick search but for some reason didn't turn this existing thread up.
What I've been
On 08/12/2010 06:27 AM, David Fetter wrote:
+1 for putting it in core in 9.1 :)
There are times I really wish I could get object graphs, or at least
hierarchically nested object trees, of objects matching various
criteria. JSON might be a reasonable representation, and one that's
On Fri, Sep 17, 2010 at 11:12 PM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
On Sat, Sep 18, 2010 at 11:46 AM, Robert Haas robertmh...@gmail.com wrote:
itagaki.takah...@gmail.com wrote:
One of my proposal is we don't have to keep the original input text.
We store JSON data in effective
On Sat, Sep 18, 2010 at 4:03 PM, Robert Haas robertmh...@gmail.com wrote:
Hmm, yeah. I'd be tempted to try to keep the user's original
whitespace as far as possible, but disregard it as far as equality
comparison goes. However, I'm not quite sure what the right thing to
do about 0 vs 0.0 is.
On Sat, Sep 18, 2010 at 11:48 PM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
As for whitespace preservation, I don't think we should go out of our
way to keep it intact. Sure, preserving formatting for input and
output makes some sense because we'd have to go out of our way to
normalize
On Fri, Aug 13, 2010 at 7:33 PM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
Updated patch: the JSON code has all been moved into core, so this
patch is now for a built-in data type.
I have a question about the design of the JSON type. Why do we need to
store the value in UTF8 encoding?
On Fri, Sep 17, 2010 at 8:32 AM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
On Fri, Aug 13, 2010 at 7:33 PM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
Updated patch: the JSON code has all been moved into core, so this
patch is now for a built-in data type.
I have a question
On Sat, Sep 18, 2010 at 6:45 AM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
Why do we need to store the value in UTF8 encoding?
because the JSON code needs to handle Unicode escapes like
\u266B, but there is no simple and efficient way (that I know of) to
convert single characters
Itagaki Takahiro wrote:
On Sat, Sep 18, 2010 at 6:45 AM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
Why do we need to store the value in UTF8 encoding?
because the JSON code needs to handle Unicode escapes like
\u266B, but there is no simple and efficient way (that I know of) to
On Fri, Sep 17, 2010 at 10:28 PM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
One of my proposal is we don't have to keep the original input text.
We store JSON data in effective internal formats. If users want to get
human-readable output, they can use stringify() with indentation
On Sat, Sep 18, 2010 at 11:46 AM, Robert Haas robertmh...@gmail.com wrote:
itagaki.takah...@gmail.com wrote:
One of my proposal is we don't have to keep the original input text.
We store JSON data in effective internal formats. If users want to get
human-readable output, they can use
On Tue, Sep 14, 2010 at 04:06, Andrew Dunstan and...@dunslane.net wrote:
On 09/13/2010 09:30 PM, Itagaki Takahiro wrote:
Hi,
Anyone working on JSON datatype?
If no, I'm going to submit simplified version of JSON datatype patch.
What's the state of the GSOC project?
Well, GSoC itself is
Hi,
Anyone working on JSON datatype?
If no, I'm going to submit simplified version of JSON datatype patch.
On Wed, Aug 25, 2010 at 2:34 PM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
On Fri, Aug 13, 2010 at 7:33 PM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
Updated patch: the
On 09/13/2010 09:30 PM, Itagaki Takahiro wrote:
Hi,
Anyone working on JSON datatype?
If no, I'm going to submit simplified version of JSON datatype patch.
What's the state of the GSOC project?
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make
2010/8/25 Robert Haas robertmh...@gmail.com:
On Wed, Aug 25, 2010 at 1:34 AM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
* Should we accept a scalar value as a valid JSON?
According to RFC, the root element of JSON text must be an object
or array. But to_json() and from_json() accept
Hitoshi Harada umi.tan...@gmail.com writes:
It depends on use cases, but in my mind plain text will do for us. If
we have JavaScript engine in PostgreSQL like pl/v8 and it handles
on-disk format as-is, then we should choose the kind of format, but in
either text or binary format way it is
On Wed, Aug 25, 2010 at 1:34 AM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
* Should we accept a scalar value as a valid JSON?
According to RFC, the root element of JSON text must be an object
or array. But to_json() and from_json() accept scalar values.
This seems a bit like the XML
Hi, I start to review JSON patch.
On Fri, Aug 13, 2010 at 7:33 PM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
Updated patch: the JSON code has all been moved into core, so this
patch is now for a built-in data type.
I think the patch can be split into two pieces:
1. Basic I/O support
On Sat, 2010-07-24 at 18:57 -0400, Joseph Adams wrote:
I've been developing it as a contrib module because:
* I'd imagine it's easier than developing it as a built-in datatype
right away (e.g. editing a .sql.in file versus editing pg_type.h ).
* As a module, it has PGXS support, so people can
On Sat, Jul 24, 2010 at 06:57:18PM -0400, Joseph Adams wrote:
Update: I'm in the middle of cleaning up the JSON code (
http://git.postgresql.org/gitweb?p=json-datatype.git;a=summary if you
want to see the very latest ), so I haven't addressed all of the major
problems with it yet.
On Fri,
On Wed, 2010-08-11 at 15:27 -0700, David Fetter wrote:
I've been developing it as a contrib module because:
* I'd imagine it's easier than developing it as a built-in datatype
right away (e.g. editing a .sql.in file versus editing pg_type.h ).
* As a module, it has PGXS support, so
On Wed, Aug 11, 2010 at 03:40:36PM -0700, Joshua D. Drake wrote:
On Wed, 2010-08-11 at 15:27 -0700, David Fetter wrote:
I've been developing it as a contrib module because:
* I'd imagine it's easier than developing it as a built-in
datatype right away (e.g. editing a .sql.in file
On 08/11/2010 07:33 PM, David Fetter wrote:
I would be curious to the benefit of putting it in core. I have no
problem with the type but in core?
If it's not in core, the vast majority of users will not have it
installed, and nothing, in core or otherwise, will be able to count on
it.
Andrew Dunstan wrote:
On 08/11/2010 07:33 PM, David Fetter wrote:
I would be curious to the benefit of putting it in core. I have no
problem with the type but in core?
If it's not in core, the vast majority of users will not have it
installed, and nothing, in core or otherwise, will
On Wed, Aug 11, 2010 at 07:39:37PM -0400, Andrew Dunstan wrote:
On 08/11/2010 07:33 PM, David Fetter wrote:
I would be curious to the benefit of putting it in core. I have no
problem with the type but in core?
If it's not in core, the vast majority of users will not have it
installed, and
Excerpts from Joseph Adams's message of mar ago 10 04:03:43 -0400 2010:
An overview, along with my thoughts, of the utility functions:
FN_EXTRA, FN_EXTRA_ALLOC, FN_MCXT macros
* Useful-ometer: ()o
TypeInfo structure and getTypeInfo function
* Useful-ometer:
Alvaro Herrera alvhe...@commandprompt.com writes:
Excerpts from Joseph Adams's message of mar ago 10 04:03:43 -0400 2010:
An overview, along with my thoughts, of the utility functions:
FN_EXTRA, FN_EXTRA_ALLOC, FN_MCXT macros
* Useful-ometer: ()o
TypeInfo structure and
I, for one, think it would be great if the JSON datatype were all in
core :-) However, if and how much JSON code should go into core is up for
discussion. Thoughts, anyone?
in my opinion: As soon as possible. Spinning PostgreSQL as the
Ajax-enabled-database has many great uses.
Harald
--
Update: I'm in the middle of cleaning up the JSON code (
http://git.postgresql.org/gitweb?p=json-datatype.git;a=summary if you
want to see the very latest ), so I haven't addressed all of the major
problems with it yet.
On Fri, Jul 23, 2010 at 2:34 PM, Robert Haas robertmh...@gmail.com wrote:
-
On Sat, Jul 24, 2010 at 06:57:18PM -0400, Joseph Adams wrote:
A particularly useful aspect of the JSON support is the ability to
convert PostgreSQL arrays to JSON arrays (using to_json ), as there
currently isn't any streamlined way to parse arrays in the PostgreSQL
format client-side (that I
On Fri, Jul 23, 2010 at 2:18 AM, Joseph Adams
joeyadams3.14...@gmail.com wrote:
This is a work-in-progress patch of my GSoC project: Add JSON datatype
to PostgreSQL. It provides the following:
* JSON datatype: A TEXT-like datatype for holding JSON-formatted
text. Although the JSON RFC
On Fri, Jul 23, 2010 at 2:34 PM, Robert Haas robertmh...@gmail.com wrote:
- elog() must be used except for can't happen situations. Compare
the way numeric_in() throws an error message versus json_in().
Er... that should have said elog() must NOT be used except for can't
happen situations.
69 matches
Mail list logo
