On Thu, Mar 18, 2004 at 07:48:40AM +0100, Karel Zak wrote:
On Wed, Mar 17, 2004 at 11:02:38AM -0500, Tom Lane wrote:
Karel Zak [EMAIL PROTECTED] writes:
The formatting function API can be pretty simple:
text *my_copy_format(text *attrdata, int direction,
int nattrs, int
To be honest this idea strikes me as overkill - over
engineering. While there is a clear need for proper CSV import
(i.e. just setting DELIMITER to ',' doesn't work due to ','s in
strings) I cannot see how this would prove useful, or who would use
it?
While i have done a lot of messing around
Dear Tom,
On Wed, 17 Mar 2004, Tom Lane wrote:
If you want a GUI, it could be a GUI,
I do not want a GUI, I'm not a GUI guy;-) I was just wondering how GUI
could be adapted to deal with the tool if it is outside.
though I'd be worried about the portability price paid to have one. Or
are
On Thu, Mar 18, 2004 at 09:29:03AM +, Lee Kindness wrote:
To be honest this idea strikes me as overkill - over
engineering.
It was suggestion, maybe you're right :-)
While i have done a lot of messing around reading/writing the binary
format (and been stung by changes in that format)
On Wednesday 17 March 2004 21:23, Tom Lane wrote:
That is evidently a 7.1 database, not a 7.2 database. I'm surprised
that you don't get the other version check message first --- we must
have gotten the order of testing a mite confused ... anyway you need a
7.1 server.
Thank you for
Lee Kindness wrote:
To be honest this idea strikes me as overkill - over
engineering. While there is a clear need for proper CSV import
(i.e. just setting DELIMITER to ',' doesn't work due to ','s in
strings) I cannot see how this would prove useful, or who would use
it?
I agree. My modest
Andrew Dunstan wrote:
Lee Kindness wrote:
To be honest this idea strikes me as overkill - over
engineering. While there is a clear need for proper CSV import
(i.e. just setting DELIMITER to ',' doesn't work due to ','s in
strings) I cannot see how this would prove useful, or who would use
Karel Zak [EMAIL PROTECTED] writes:
On Wed, Mar 17, 2004 at 11:02:38AM -0500, Tom Lane wrote:
Karel Zak [EMAIL PROTECTED] writes:
This seems like it could only reasonably be implemented as a C function.
Why? I said it's pseudo code. It should use standard fmgr API like
every other
Silvio Mazzaro [EMAIL PROTECTED] writes:
However... it's strange but /var/lib/pgsql/data/PG_VERSION answers:
7.2
!!!
Yeah? Well, that explains something I was wondering about, which is why
the PG_VERSION mismatch complaint didn't come out first.
Where exactly did you get the server code you
Andrew Dunstan [EMAIL PROTECTED] writes:
Lee Kindness wrote:
To be honest this idea strikes me as overkill - over
engineering.
I agree. My modest proposal for handling CSVs would be to extend the
DELIMITER parameter to allow up to 3 characters - separator, quote and
escape. Escape would
On Thursday 18 March 2004 10:18, Fabien COELHO wrote:
On Wed, 17 Mar 2004, Tom Lane wrote:
though I'd be worried about the portability price paid to have one. Or
are you concerned about whether a GUI could invoke it? I don't see why
not --- the GUIs don't reimplement pg_dump, do they?
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
Lee Kindness wrote:
To be honest this idea strikes me as overkill - over
engineering.
I agree. My modest proposal for handling CSVs would be to extend the
DELIMITER parameter to allow up to 3 characters - separator, quote and
On Thu, 18 Mar 2004, Richard Huxton wrote:
On Thursday 18 March 2004 10:18, Fabien COELHO wrote:
On Wed, 17 Mar 2004, Tom Lane wrote:
though I'd be worried about the portability price paid to have one. Or
are you concerned about whether a GUI could invoke it? I don't see why
not
Richard Huxton [EMAIL PROTECTED] writes:
How would you run pg_dump on a remote machine?
Trivially. It's a client.
regards, tom lane
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Bruce Momjian wrote:
I have been poking around with our fsync default options to see if I can
improve them. One issue is that we never default to O_SYNC, but default
to O_DSYNC if it exists, which seems strange.
What I did was to beef up my test program and get it into CVS for folks
to run.
On Thursday 18 March 2004 17:51, Tom Lane wrote:
Richard Huxton [EMAIL PROTECTED] writes:
How would you run pg_dump on a remote machine?
Trivially. It's a client.
Eh? I'm assuming we're talking at cross purposes here. *I* can run it
trivially - ssh in and run it over there, or run it on my
Karel, Andrew, Fernando:
On Wed, Mar 17, 2004 at 11:02:38AM -0500, Tom Lane wrote:
Karel Zak [EMAIL PROTECTED] writes:
The formatting function API can be pretty simple:
text *my_copy_format(text *attrdata, int direction,
int nattrs, int attr, oid attrtype, oid
Andrew Dunstan wrote:
Bruce Momjian wrote:
I have been poking around with our fsync default options to see if I can
improve them. One issue is that we never default to O_SYNC, but default
to O_DSYNC if it exists, which seems strange.
What I did was to beef up my test program and get
I get the following warning compiling CVS HEAD:
[neilc:/Users/neilc/pgsql]% make -C src/backend/utils/error all
[ ... ]
gcc -no-cpp-precomp -O0 -Winline -fno-strict-aliasing -g -Wall
-Wmissing-prototypes -Wmissing-declarations -I../../../../src/include
-I/sw/include -c -o elog.o elog.c -MMD
Neil Conway [EMAIL PROTECTED] writes:
I get the following warning compiling CVS HEAD:
[neilc:/Users/neilc/pgsql]% make -C src/backend/utils/error all
[ ... ]
gcc -no-cpp-precomp -O0 -Winline -fno-strict-aliasing -g -Wall
-Wmissing-prototypes -Wmissing-declarations -I../../../../src/include
Tom Lane wrote:
Neil Conway [EMAIL PROTECTED] writes:
I get the following warning compiling CVS HEAD:
[neilc:/Users/neilc/pgsql]% make -C src/backend/utils/error all
[ ... ]
gcc -no-cpp-precomp -O0 -Winline -fno-strict-aliasing -g -Wall
-Wmissing-prototypes -Wmissing-declarations
Andrew Dunstan [EMAIL PROTECTED] writes:
*sigh*
my local (linux) man for gettimeofday says this:
struct timeval {
time_t tv_sec;/* seconds */
suseconds_ttv_usec; /* microseconds */
};
Yeah, but mine (HPUX) says that tv_sec
I attempted(!) to compile up CVS Head, and if you --enable-thread-safety,
you need to include the THREADS stuff to cc:
gmake[4]: Leaving directory `/home/ler/pg-dev/pgsql/src/port'
cc -O -Kinline initdb.o -L../../../src/interfaces/libpq -lpq
-L../../../src/port -L/usr/local/lib
The if() statement at line 418 in pquery.c seems a bit bereft of
controlled statement; looks like a broken log_executor_stats patch.
if (portal-strategy != PORTAL_MULTI_QUERY)
{
ereport(DEBUG3,
(errmsg_internal(PortalRun)));
Folks,
Jeremy handed me an interesting feature proposal at last night's SFPUG
meeting.
PG authentication methods ought to have drop-downs to other authentication
methods, in the same manner as SSH and PAM.
The idea would be this, if you had the following in your pg_hba.conf:
somedb jeremy
And, BTW, I deal with CSV *all the time* for my insurance clients, and I can
tell you that that format hasn't changed in 20 years. We can hard-code it
if it's easier.
Well many of my clients consider CSV Character Separated Value not
Comma... Thus I get data like this:
Hello,Good Bye
Hello
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -I. -I../../../src/include -D_GNU_SOURCE -c -o
bootparse.o bootparse.c
bootparse.y:26:26: access/strat.h: No such file or directory
In file included from bootparse.y:340:
bootscanner.l:24:26: access/strat.h: No such
Dear Tom,
I thought about it... how to solve the contradiction:
- backend vs external tool?
- new interface? new command? unix? windows?
- compatibility with old/existing interfaces?
- plugins, any one can contribute?
- don't bother DBA's
- communicate about it?
My 2 pence idea of the
On Thu, 18 Mar 2004, Josh Berkus wrote:
Jeremy handed me an interesting feature proposal at last night's SFPUG
meeting.
PG authentication methods ought to have drop-downs to other authentication
methods, in the same manner as SSH and PAM.
The idea would be this, if you had the
Is anyone working on these two todo items?
# CLUSTER
* Automatically maintain clustering on a table
* Add way to remove cluster specification on a table
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL
Joseph Shraibman wrote:
Is anyone working on these two todo items?
# CLUSTER
* Automatically maintain clustering on a table
No, and we don't know how to do it.
* Add way to remove cluster specification on a table
This patch is done and will be applied soon.
--
Bruce
Thanks. Fixed. Not sure how it happened.
---
Tom Lane wrote:
The if() statement at line 418 in pquery.c seems a bit bereft of
controlled statement; looks like a broken log_executor_stats patch.
if
On Thu, 18 Mar 2004, Larry Rosenman wrote:
I attempted(!) to compile up CVS Head, and if you --enable-thread-safety,
you need to include the THREADS stuff to cc:
gmake[4]: Leaving directory `/home/ler/pg-dev/pgsql/src/port'
cc -O -Kinline initdb.o -L../../../src/interfaces/libpq -lpq
Jonathan Gardner [EMAIL PROTECTED] writes:
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -I. -I../../../src/include -D_GNU_SOURCE -c -o
bootparse.o bootparse.c
bootparse.y:26:26: access/strat.h: No such file or directory
In file included from
--On Thursday, March 18, 2004 19:39:56 -0500 Tom Lane [EMAIL PROTECTED]
wrote:
Larry Rosenman [EMAIL PROTECTED] writes:
What is the concensus of the community?
AFAICS, initdb should not need to depend on libpq in the first place;
it never makes a connection to a live postmaster. I think it
Tom Lane [EMAIL PROTECTED] writes:
I could go with that too. The question here is do we have any popular
use-cases that aren't solved by that extension, but could be solved by
simple user-level data formatting functions? I'm not real eager to add
such a feature as an if we build it they
Larry Rosenman [EMAIL PROTECTED] writes:
What is the concensus of the community?
AFAICS, initdb should not need to depend on libpq in the first place;
it never makes a connection to a live postmaster. I think it would be
cleaner to get rid of that dependency instead of propagating thread junk
On Thursday 18 March 2004 04:30 pm, Tom Lane wrote:
Jonathan Gardner [EMAIL PROTECTED] writes:
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -I. -I../../../src/include -D_GNU_SOURCE -c -o
bootparse.o bootparse.c
bootparse.y:26:26: access/strat.h: No such
though I'd be worried about the portability price paid to have one. Or
are you concerned about whether a GUI could invoke it? I don't see why
not --- the GUIs don't reimplement pg_dump, do they?
Actually Tom, I think they do (where they have an export facility). How would
you run pg_dump on a
# CLUSTER
* Automatically maintain clustering on a table
* Add way to remove cluster specification on a table
I've done the latter - it's been sent to -patches. However, I need
someone to look at the shift/reduce problem I'm getting...
Chris
---(end of
On Thu, Mar 18, 2004 at 22:58:46 +,
Jon Jensen [EMAIL PROTECTED] wrote:
Is there some other way to do what I'm looking for here without the
authentication method fallthrough Josh proposes?
Assuming people aren't sharing accounts, you could let any authorized
postgres user connect using
On Thu, 18 Mar 2004, Bruno Wolff III wrote:
On Thu, Mar 18, 2004 at 22:58:46 +, Jon Jensen [EMAIL PROTECTED] wrote:
Is there some other way to do what I'm looking for here without the
authentication method fallthrough Josh proposes?
Assuming people aren't sharing accounts, you
On Fri, Mar 19, 2004 at 02:01:40 +,
Jon Jensen [EMAIL PROTECTED] wrote:
That's true, but that doesn't satisfy the need. I want an automated
process running as OS user postgres to authenticate with ident, but I'd
also like to be able have, say, phpPgAdmin (running as user apache)
Greg Stark [EMAIL PROTECTED] writes:
Tom Lane [EMAIL PROTECTED] writes:
I could go with that too. The question here is do we have any popular
use-cases that aren't solved by that extension, but could be solved by
simple user-level data formatting functions?
(I can't believe I'm saying this,
Bruce Momjian wrote:
Joseph Shraibman wrote:
* Add way to remove cluster specification on a table
This patch is done and will be applied soon.
I'm a bit confused, why would you want to uncluster a table?
---(end of broadcast)---
TIP 6: Have
Larry Rosenman wrote:
On Thu, 18 Mar 2004, Larry Rosenman wrote:
I attempted(!) to compile up CVS Head, and if you --enable-thread-safety,
you need to include the THREADS stuff to cc:
gmake[4]: Leaving directory `/home/ler/pg-dev/pgsql/src/port'
cc -O -Kinline initdb.o
This patch is done and will be applied soon.
I'm a bit confused, why would you want to uncluster a table?
You would want to remove the marker that says 'cluster this column in
the future'. At the moment, there is no way of removing all markers
from a table.
Chris
--On Thursday, March 18, 2004 23:03:16 -0500 Bruce Momjian
[EMAIL PROTECTED] wrote:
Larry Rosenman wrote:
On Thu, 18 Mar 2004, Larry Rosenman wrote:
I attempted(!) to compile up CVS Head, and if you
--enable-thread-safety, you need to include the THREADS stuff to cc:
gmake[4]: Leaving
Bruce Momjian wrote:
Because a CLUSTER with no argument clusters all previously clustered
tables in the db. This turns it off for that table.
My bad, I should have read the docs more closely.
---(end of broadcast)---
TIP 6: Have you searched our
--On Thursday, March 18, 2004 22:47:39 -0600 Larry Rosenman
[EMAIL PROTECTED] wrote:
--On Thursday, March 18, 2004 23:03:16 -0500 Bruce Momjian
[EMAIL PROTECTED] wrote:
I see that initdb is just the first of many /bin programs to be
compiled, so if we have to add the thread lib, we will
Hi,
I have done a patch for turning off clustering on a table entirely.
Unforunately, of the three syntaxes I can think of, all cause
shift/reduce errors:
SET WITHOUT CLUSTER;
DROP CLUSTER
CLUSTER ON NONE;
This is the new grammar that I added:
/* ALTER TABLE name SET WITHOUT CLUSTER */
|
Christopher Kings-Lynne [EMAIL PROTECTED] writes:
This is the new grammar that I added:
| ALTER TABLE relation_expr SET WITHOUT CLUSTER
Now, I have to change that relation_expr to qualified_name. However,
this causes shift/reduce errors. (Due to ALTER TABLE relation_expr SET
WITHOUT
Josh Berkus [EMAIL PROTECTED] writes:
Any reason why this is a bad idea?
It breaks client compatibility --- I don't think any existing clients
are prepared to be challenged multiple times, and indeed the protocol
spec specifically advises clients to drop the connection if they can't
handle the
On Friday 19 March 2004 02:01, Jon Jensen wrote:
On Thu, 18 Mar 2004, Bruno Wolff III wrote:
On Thu, Mar 18, 2004 at 22:58:46 +, Jon Jensen [EMAIL PROTECTED]
wrote:
Is there some other way to do what I'm looking for here without the
authentication method fallthrough Josh proposes?
I have updated my program with your suggested changes and put in
src/tools/fsync. Please see how you like it.
---
Zeugswetter Andreas SB SD wrote:
Running the attached test program shows on BSD/OS 4.3:
write
I have been poking around with our fsync default options to see if I can
improve them. One issue is that we never default to O_SYNC, but default
to O_DSYNC if it exists, which seems strange.
What I did was to beef up my test program and get it into CVS for folks
to run. What I found was that
Bruce Momjian [EMAIL PROTECTED] writes:
I have been poking around with our fsync default options to see if I can
improve them. One issue is that we never default to O_SYNC, but default
to O_DSYNC if it exists, which seems strange.
As I recall, that was based on testing on some different
Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
I have been poking around with our fsync default options to see if I can
improve them. One issue is that we never default to O_SYNC, but default
to O_DSYNC if it exists, which seems strange.
As I recall, that was based on testing
On Thu, Mar 18, 2004 at 01:50:32PM -0500, Bruce Momjian wrote:
I'm not sure I believe these numbers at all... my experience is that
getting trustworthy disk I/O numbers is *not* easy.
These numbers were reproducable on all the platforms I tested.
It's not because they are reproducable that
Bruce Momjian [EMAIL PROTECTED] writes:
Tom Lane wrote:
As I recall, that was based on testing on some different platforms.
But why perfer O_DSYNC over fdatasync if you don't prefer O_SYNC over
fsync?
It's what tested out as the best bet. I think we were using pgbench
as the test platform,
Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Tom Lane wrote:
As I recall, that was based on testing on some different platforms.
But why perfer O_DSYNC over fdatasync if you don't prefer O_SYNC over
fsync?
It's what tested out as the best bet. I think we were using
On Thu, Mar 18, 2004 at 02:22:10PM -0500, Bruce Momjian wrote:
OK, what better test do you suggest? Right now, there has been no
testing of these.
I suggest you start by doing atleast preallocating a 16 MB file
and do the tests on that, to atleast be somewhat simular to what
WAL does.
I
Bruce Momjian [EMAIL PROTECTED] writes:
Tom Lane wrote:
It's what tested out as the best bet. I think we were using pgbench
as the test platform, which as you know I have doubts about, but at
least it is testing one actual write/sync pattern Postgres can generate.
I assume pgbench has so
Kurt Roeckx wrote:
On Thu, Mar 18, 2004 at 02:22:10PM -0500, Bruce Momjian wrote:
OK, what better test do you suggest? Right now, there has been no
testing of these.
I suggest you start by doing atleast preallocating a 16 MB file
and do the tests on that, to atleast be somewhat
Kurt Roeckx [EMAIL PROTECTED] writes:
I have no idea what the access pattern is for normal WAL
operations or how many times it gets synched. Does it only do
f(data)sync() at commit time, or for every block it writes?
If we are using fsync/fdatasync, we issue those at commit time or when
Here are my results on Linux 2.6.1 using cvs version 1.7.
Those times with 20 seconds, you really hear the disk go crazy.
And I have the feeling something must be wrong. Those results
are reproducible.
Kurt
Simple write timing:
write0.139558
Compare fsync times
Kurt Roeckx wrote:
Here are my results on Linux 2.6.1 using cvs version 1.7.
Those times with 20 seconds, you really hear the disk go crazy.
And I have the feeling something must be wrong. Those results
are reproducible.
Wow, your O_SYNC times are great. Where can I buy some? :-)
Tom, Bruce,
My previous point about checking different fsync spacings corresponds to
different assumptions about average transaction size. I think a useful
tool for determining wal_sync_method has got to be able to reflect that
range of possibilities.
Questions:
1) This is an OSS project.
Josh Berkus [EMAIL PROTECTED] writes:
1) This is an OSS project. Why not just recruit a bunch of people on
PERFORMANCE and GENERAL to test the 4 different synch methods using real
databases? No test like reality, I say
I agree --- that is likely to yield *far* more useful results
Bruce Momjian [EMAIL PROTECTED] writes:
Well, I wrote the program to allow testing. I don't see a complex test
as being that much better than simple one. We don't need accurate
numbers. We just need to know if fsync or O_SYNC is faster.
Faster than what? The thing everyone is trying to
On Thu, Mar 18, 2004 at 03:34:21PM -0500, Bruce Momjian wrote:
Kurt Roeckx wrote:
Here are my results on Linux 2.6.1 using cvs version 1.7.
Those times with 20 seconds, you really hear the disk go crazy.
And I have the feeling something must be wrong. Those results
are
Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Well, I wrote the program to allow testing. I don't see a complex test
as being that much better than simple one. We don't need accurate
numbers. We just need to know if fsync or O_SYNC is faster.
Faster than what? The thing
Tom Lane wrote:
It really just shows whether the fsync fater the close has similar
timing to the one before the close. That was the best way I could think
to test it.
Sure, but where's the separate process part? What this seems to test
is whether a single process can sync its own
73 matches
Mail list logo
