On Monday 03 August 2009 22:11:08 Tom Lane wrote:
I'm starting to look at this patch. I observe that it's setting the
default output format to HEX. If changing the default behavior was
agreed to, or even discussed, I do not remember where. Shouldn't the
default stay the same?
I did pose
Peter Eisentraut pete...@gmx.net writes:
On Monday 03 August 2009 22:11:08 Tom Lane wrote:
I'm starting to look at this patch. I observe that it's setting the
default output format to HEX. If changing the default behavior was
agreed to, or even discussed, I do not remember where. Shouldn't
On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote:
Peter Eisentraut pete...@gmx.net writes:
On Monday 03 August 2009 22:11:08 Tom Lane wrote:
I'm starting to look at this patch. I observe that it's setting the
default output format to HEX. If changing the default behavior was
On Tue, Aug 4, 2009 at 16:31, Robert Haasrobertmh...@gmail.com wrote:
On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote:
Peter Eisentraut pete...@gmx.net writes:
On Monday 03 August 2009 22:11:08 Tom Lane wrote:
I'm starting to look at this patch. I observe that it's setting
Robert Haas robertmh...@gmail.com writes:
On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote:
If this seems reasonable, I can make a note of the point in the commit
message, so that we won't forget when the time comes.
Or, what we could do is start an open items for 8.5 list
--On Dienstag, August 04, 2009 10:28:48 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
While I've been poking at the pg_dump issues, it's occurred to me that
changing the default would be a great forcing function for finding out
any lurking problems.
+1
--
Thanks
Bernd
--
Bernd Helmle maili...@oopsware.de writes:
--On Samstag, Juli 11, 2009 13:40:44 +0300 Peter Eisentraut
pete...@gmx.net wrote:
OK, here is an updated patch. It has the setting as enum, completed
documentation, and libpq support. I'll add it to the commit fest in the
hope that someone else
I wrote:
While I've been poking at the pg_dump issues, it's occurred to me that
changing the default would be a great forcing function for finding out
any lurking problems. What I'm inclined to do now is to commit it
*with* the change of default, and let it be that way at least for a
few
One other stylistic gripe: I don't much like inserting a GUC variable
definition into builtins.h --- that file has traditionally only
contained function extern declarations. The best alternative I can
think of is to move the bytea-related stuff into a new include file
include/utils/bytea.h. Has
On Tue, Aug 4, 2009 at 12:18 AM, Tom Lanet...@sss.pgh.pa.us wrote:
One other stylistic gripe: I don't much like inserting a GUC variable
definition into builtins.h --- that file has traditionally only
contained function extern declarations. The best alternative I can
think of is to move the
Greg Stark gsst...@mit.edu writes:
On Tue, Aug 4, 2009 at 12:18 AM, Tom Lanet...@sss.pgh.pa.us wrote:
One other stylistic gripe: I don't much like inserting a GUC variable
definition into builtins.h --- that file has traditionally only
contained function extern declarations. The best
Tom Lane wrote:
Greg Stark gsst...@mit.edu writes:
On Tue, Aug 4, 2009 at 12:18 AM, Tom Lanet...@sss.pgh.pa.us wrote:
One other stylistic gripe: I don't much like inserting a GUC variable
definition into builtins.h --- that file has traditionally only
contained function extern
Alvaro Herrera alvhe...@commandprompt.com writes:
I vote for a new bytea.h file that does not slurp in byteain/byteaout,
to avoid breaking 3rd party code. miscadmin.h seems the worst solution,
since it's already included in 210 other files.
Well, unless you want to leave *all* the bytea
Tom Lane wrote:
Alvaro Herrera alvhe...@commandprompt.com writes:
I vote for a new bytea.h file that does not slurp in byteain/byteaout,
to avoid breaking 3rd party code. miscadmin.h seems the worst solution,
since it's already included in 210 other files.
Well, unless you want to leave
Alvaro Herrera alvhe...@commandprompt.com writes:
Tom Lane wrote:
Well, unless you want to leave *all* the bytea functions in builtins.h
there will still be some risk there. I'd actually sooner break calls
of byteaout than other things, because in reality every caller of
byteaout is going to
Bernd Helmle maili...@oopsware.de writes:
--On Samstag, Juli 11, 2009 13:40:44 +0300 Peter Eisentraut
pete...@gmx.net wrote:
OK, here is an updated patch. It has the setting as enum, completed
documentation, and libpq support. I'll add it to the commit fest in the
hope that someone else
--On Montag, August 03, 2009 15:11:08 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
I'm starting to look at this patch. I observe that it's setting the
default output format to HEX. If changing the default behavior was
agreed to, or even discussed, I do not remember where. Shouldn't the
default
Bernd Helmle maili...@oopsware.de writes:
--On Montag, August 03, 2009 15:11:08 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
I'm starting to look at this patch. I observe that it's setting the
default output format to HEX. If changing the default behavior was
agreed to, or even discussed, I do
--On Freitag, Juli 24, 2009 20:50:16 +0200 Bernd Helmle
maili...@oopsware.de wrote:
I don't believe i can do very much this weekend...
I have to delay that until sunday, but will get my hands on some
performance and function tests again, since i have access on the customer
machine then.
Bernd Helmle maili...@oopsware.de writes:
I've attached a slightly edited patch which fixes a compiler warning in
encode.c, too.
Bernd, are you done reviewing this or did you intend to do more?
It's still marked as needs review on the commitfest page.
regards, tom lane
--On Freitag, Juli 24, 2009 11:38:06 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
Bernd, are you done reviewing this or did you intend to do more?
It's still marked as needs review on the commitfest page.
I hoped to get more profiling data like Andrew suggested, but haven't
enough time to do it
--On Samstag, Juli 11, 2009 13:40:44 +0300 Peter Eisentraut
pete...@gmx.net wrote:
OK, here is an updated patch. It has the setting as enum, completed
documentation, and libpq support. I'll add it to the commit fest in the
hope that someone else can look it over in detail.
I've started
Bernd Helmle wrote:
--On Samstag, Juli 11, 2009 13:40:44 +0300 Peter Eisentraut
pete...@gmx.net wrote:
OK, here is an updated patch. It has the setting as enum, completed
documentation, and libpq support. I'll add it to the commit fest in the
hope that someone else can look it over in
--On Dienstag, Juli 21, 2009 16:49:45 -0400 Andrew Dunstan
and...@dunslane.net wrote:
You just tested COPY, not pg_dump, right? Some pg_dump numbers would be
interesting, both for text and custom formats.
Plain COPY, yes. I planned testing pg_dump for this round of my review but
ran out of
Bernd Helmle wrote:
--On Dienstag, Juli 21, 2009 16:49:45 -0400 Andrew Dunstan
and...@dunslane.net wrote:
You just tested COPY, not pg_dump, right? Some pg_dump numbers would be
interesting, both for text and custom formats.
Plain COPY, yes. I planned testing pg_dump for this round of my
Peter Eisentraut pete...@gmx.net writes:
OK, here is an updated patch. It has the setting as enum, completed
documentation, and libpq support. I'll add it to the commit fest in the hope
that someone else can look it over in detail.
I found that there is another issue that should be
On Wednesday 08 July 2009 01:07:08 Tom Lane wrote:
Peter Eisentraut pete...@gmx.net writes:
Here is a first cut at a new hex bytea input and output format. Example:
...
SET bytea_output_hex = true;
Should the configuration parameter be a boolean or an enum, opening
possibilities for
Hello, Bernd.
You wrote:
BH --On Dienstag, Juli 07, 2009 18:07:08 -0400 Tom Lane t...@sss.pgh.pa.us
BH wrote:
Enum. If we do this then it seems entirely fair that someone might
want other settings someday. Also, it seems silly to pick a format
partly on the grounds that it's expansible,
--On Dienstag, Juli 07, 2009 18:07:08 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
Enum. If we do this then it seems entirely fair that someone might
want other settings someday. Also, it seems silly to pick a format
partly on the grounds that it's expansible, and then not make the
control GUC
2009/7/8 Bernd Helmle maili...@oopsware.de:
--On Dienstag, Juli 07, 2009 18:07:08 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
Enum. If we do this then it seems entirely fair that someone might
want other settings someday. Also, it seems silly to pick a format
partly on the grounds that it's
On Wednesday 06 May 2009 18:47:57 Tom Lane wrote:
So the ambiguous-input problem is solved if we define the new format(s)
to be started by backslash and something that the old code would reject.
I'd keep it short, like \x, but there's still room for multiple
formats if anyone really wants to
Peter Eisentraut pete...@gmx.net writes:
Here is a first cut at a new hex bytea input and output format. Example:
...
SET bytea_output_hex = true;
Should the configuration parameter be a boolean or an enum, opening
possibilities for other formats?
Enum. If we do this then it seems
--On Samstag, Mai 30, 2009 00:47:16 +0300 Hannu Krosing
ha...@2ndquadrant.com wrote:
And we can also escape the need to uncompress TOAST'ed fields - just
markup the compression as another \c at the beginning of data.
Hmm i thought about that, but that seems only to make sense if there is an
--On Freitag, Mai 29, 2009 11:06:28 +0300 Peter Eisentraut
pete...@gmx.net wrote:
Btw., I have started to write some code for that.
Cool. Let me know if i can help out somewhere.
--
Thanks
Bernd
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To
On Friday 29 May 2009 04:26:35 Bruce Momjian wrote:
Added to TODO:
|Improve bytea COPY format
* http://archives.postgresql.org/pgsql-hackers/2009-05/msg00192.php
Btw., I have started to write some code for that.
--
Sent via pgsql-hackers mailing list
On Wed, 2009-05-06 at 18:33 +0300, Peter Eisentraut wrote:
On Tuesday 05 May 2009 17:38:33 Tom Lane wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Bernd Helmle maili...@oopsware.de wrote:
Another approach would be to just dump bytea columns in binary
format only (not sure
On Fri, 2009-05-29 at 11:06 +0300, Peter Eisentraut wrote:
On Friday 29 May 2009 04:26:35 Bruce Momjian wrote:
Added to TODO:
|Improve bytea COPY format
* http://archives.postgresql.org/pgsql-hackers/2009-05/msg00192.php
Btw., I have started to write some code for that.
why
Added to TODO:
|Improve bytea COPY format
* http://archives.postgresql.org/pgsql-hackers/2009-05/msg00192.php
---
Merlin Moncure wrote:
On Sat, May 16, 2009 at 11:23 AM, Stefan Kaltenbrunner
Bernd Helmle wrote:
--On Mittwoch, Mai 06, 2009 19:04:21 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
So I'm now persuaded that a better textual representation for bytea
should indeed make things noticeably better here. It would be
useful though to cross-check this thought by profiling a case
On Sat, May 16, 2009 at 11:23 AM, Stefan Kaltenbrunner
ste...@kaltenbrunner.cc wrote:
Bernd Helmle wrote:
--On Mittwoch, Mai 06, 2009 19:04:21 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
So I'm now persuaded that a better textual representation for bytea
should indeed make things noticeably
--On Mittwoch, Mai 06, 2009 19:04:21 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
So I'm now persuaded that a better textual representation for bytea
should indeed make things noticeably better here. It would be
useful though to cross-check this thought by profiling a case that
dumps a comparable
--On Mittwoch, Mai 06, 2009 19:04:21 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
So I'm now persuaded that a better textual representation for bytea
should indeed make things noticeably better here. It would be
useful though to cross-check this thought by profiling a case that
dumps a comparable
--On Dienstag, Mai 05, 2009 16:57:50 -0400 Andrew Dunstan
and...@dunslane.net wrote:
Hex will already provide some space savings over our current encoding
method for most byteas anyway. It's not like we'd be making things less
efficient space-wise. And in compressed archives the space
Bernd Helmle wrote:
--On Dienstag, Mai 05, 2009 16:57:50 -0400 Andrew Dunstan
and...@dunslane.net wrote:
Hex will already provide some space savings over our current encoding
method for most byteas anyway. It's not like we'd be making things less
efficient space-wise. And in compressed
On Tue, May 5, 2009 at 4:14 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
Tom Lane wrote:
I'm thinking plain old pairs-of-hex-digits might be the best
tradeoff if conversion speed is the criterion.
That's a lot less space-efficient than
Merlin Moncure wrote:
On Tue, May 5, 2009 at 4:14 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
Tom Lane wrote:
I'm thinking plain old pairs-of-hex-digits might be the best
tradeoff if conversion speed is the criterion.
Andrew Dunstan and...@dunslane.net writes:
Bernd Helmle wrote:
I'm dumb: I don't understand why a hex conversion would be
significantly faster than what we have now?
Quite apart from anything else you would not need the current loop over
the bytea input to calculate the result length - in
Andrew Dunstan wrote:
another nit with base64 is that properly encoded data requires
newlines according to the standard.
er, no, not as I read rfc 3548 s 2.1.
cheers
andrew
Why does encode('my text', 'base64') include newlines in its output? I
think MIME requires text to be broken
On Wed, May 6, 2009 at 8:02 AM, Andrew Dunstan and...@dunslane.net wrote:
Merlin Moncure wrote:
On Tue, May 5, 2009 at 4:14 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
Tom Lane wrote:
I'm thinking plain old pairs-of-hex-digits
On Tuesday 05 May 2009 17:38:33 Tom Lane wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Bernd Helmle maili...@oopsware.de wrote:
Another approach would be to just dump bytea columns in binary
format only (not sure how doable that is, though).
If that's not doable, perhaps a
Peter Eisentraut pete...@gmx.net writes:
For distinguishing various input formats, we could use the backslash
to escape the format specification without breaking backward
compatibilty, e.g.,
Oh, you're right! I had been thinking that byteain treats \x as
just meaning x if x isn't an octal
Bernd Helmle maili...@oopsware.de writes:
--On Dienstag, Mai 05, 2009 10:00:37 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
Seems like the right response might be some micro-optimization effort on
byteaout.
Hmm looking into profiler statistics seems to second your suspicion:
Normal COPY
From time to time we had complains about slow dump of large tables with
bytea columns, people often complaining about a) size and b) duration of
the dump.
That latter occurred recently to me, a customer would like to dump large
tables (approx. 12G in size) with pg_dump, but he was annoyed
Bernd Helmle maili...@oopsware.de writes:
From time to time we had complains about slow dump of large tables with
bytea columns, people often complaining about a) size and b) duration of
the dump.
That latter occurred recently to me, a customer would like to dump large
tables (approx. 12G
Bernd Helmle maili...@oopsware.de wrote:
Another approach would be to just dump bytea columns in binary
format only (not sure how doable that is, though).
If that's not doable, perhaps a base64 option for bytea COPY?
-Kevin
--
Sent via pgsql-hackers mailing list
Tom Lane t...@sss.pgh.pa.us writes:
Bernd Helmle maili...@oopsware.de writes:
That latter occurred recently to me, a customer would like to dump large
tables (approx. 12G in size) with pg_dump, but he was annoyed about the
performance. Using COPY BINARY reduced the time (unsurprisingly) to
Dimitri Fontaine dfonta...@hi-media.com writes:
Still, apart from lack of interest from developpers and/or resources, is
there some reason we don't have a pg_dump --binary option?
It seems rather antithetical to one of the main goals of pg_dump,
which is to provide a dump that can reliably be
Tom Lane t...@sss.pgh.pa.us writes:
It seems rather antithetical to one of the main goals of pg_dump,
which is to provide a dump that can reliably be loaded onto other
machines or newer versions of Postgres.
You're calling for a pg_export/pg_import tool suite, or I have to learn
to read again
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Bernd Helmle maili...@oopsware.de wrote:
Another approach would be to just dump bytea columns in binary
format only (not sure how doable that is, though).
If that's not doable, perhaps a base64 option for bytea COPY?
I'm thinking plain old
Tom Lane wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Bernd Helmle maili...@oopsware.de wrote:
Another approach would be to just dump bytea columns in binary
format only (not sure how doable that is, though).
If that's not doable, perhaps a base64 option for bytea COPY?
--On Dienstag, Mai 05, 2009 10:00:37 -0400 Tom Lane t...@sss.pgh.pa.us
wrote:
Seems like the right response might be some micro-optimization effort on
byteaout.
Hmm looking into profiler statistics seems to second your suspicion:
Normal COPY shows:
% cumulative self self
Tom Lane wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Bernd Helmle maili...@oopsware.de wrote:
Another approach would be to just dump bytea columns in binary
format only (not sure how doable that is, though).
If that's not doable, perhaps a base64 option for
Andrew Dunstan and...@dunslane.net writes:
Tom Lane wrote:
I'm thinking plain old pairs-of-hex-digits might be the best
tradeoff if conversion speed is the criterion. The main problem
in any case would be to decide how to control the format option.
Yeah. Any ideas on how to do that? I
Sorry got top-posting -- stupid iphone mail client.
We could eliminate the problem with old dumps by doing something like
\x to indicate a new-style hex dump.
That doesn't make us 100% safe against arbitrary user input but should
be pretty low risk.
--
Greg
On 5 May 2009, at 18:51,
Tom Lane t...@sss.pgh.pa.us wrote:
Unless we can think of a more bulletproof format selection mechanism
Would it make any sense to have an option on the COPY command to tell
it to use base64 for bytea columns?
-Kevin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
Tom Lane wrote:
I'm thinking plain old pairs-of-hex-digits might be the best
tradeoff if conversion speed is the criterion.
That's a lot less space-efficient than base64, though.
Well, base64 could give a 33% savings, but it's
Tom Lane wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
Tom Lane wrote:
I'm thinking plain old pairs-of-hex-digits might be the best
tradeoff if conversion speed is the criterion.
That's a lot less space-efficient than base64, though.
Well,
Tom Lane wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Bernd Helmle maili...@oopsware.de wrote:
Another approach would be to just dump bytea columns in binary
format only (not sure how doable that is, though).
If that's not doable, perhaps a base64 option for bytea COPY?
I'm
68 matches
Mail list logo
