Re: [PATCHES] xlog directory at initdb time
Am Mittwoch, 27. Dezember 2006 02:56 schrieb Euler Taveira de Oliveira: This simple patch lets someone specifies the xlog directory at initdb time. It uses symlinks to do it, and create and/or set permissions at the directory as appropriate. On the name of the option, it's not actually a data directory, so I'd just call it --xlogdir, parallel to --datadir. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] Patch to log usage of temporary files
On Tue, 2007-01-02 at 18:20 -0500, Tom Lane wrote: Bill Moran [EMAIL PROTECTED] writes: In response to Alvaro Herrera [EMAIL PROTECTED]: Please change things to save the stat() syscall when the feature is not in use. Do you have a suggestion on how to do that and still have the PG_TRACE1() work? That was specifically requested by Simon Riggs. Well, we are NOT paying a stat() call on every single file close, whether Simon wants it or not. Simon doesn't/wouldn't want the stat() call on each file close. If you put the PG_TRACE macro outside of the if test, yet prior to the file close, you can pass the filename through like this PG_TRACE1(temp__file__cleanup, vfdP-fileName); That way DTrace can make its own call to find out filesize, if it would like to... and we don't need to stat() before each temp file close. That's much more flexible and useful, as well as better performance. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] Patch to log usage of temporary files
Bill Moran wrote: + if (trace_temp_files != -1) Might be more robust to say if (trace_temp_files = 0) cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] Patch to log usage of temporary files
In response to Simon Riggs [EMAIL PROTECTED]:
On Tue, 2007-01-02 at 18:20 -0500, Tom Lane wrote:
Bill Moran [EMAIL PROTECTED] writes:
In response to Alvaro Herrera [EMAIL PROTECTED]:
Please change things to save the stat() syscall when the feature is not
in use.
Do you have a suggestion on how to do that and still have the PG_TRACE1()
work? That was specifically requested by Simon Riggs.
Well, we are NOT paying a stat() call on every single file close,
whether Simon wants it or not.
Simon doesn't/wouldn't want the stat() call on each file close.
If you put the PG_TRACE macro outside of the if test, yet prior to the
file close, you can pass the filename through like this
PG_TRACE1(temp__file__cleanup, vfdP-fileName);
That way DTrace can make its own call to find out filesize, if it would
like to... and we don't need to stat() before each temp file close.
That's much more flexible and useful, as well as better performance.
OK, I think I've managed to adjust this patch so that everyone is happy ;)
and it's better as well.
* PG_TRACE will work whether the GUC var is enabled or not, it sends the
fileName, as suggested by Simon
* stat() call is not made if trace_temp_files is disabled
* trace_temp_files is now an int: -1 disables, 0 and up equate to log if
the file is this size or larger
* Cleaned things up a bit -- should be more in line with PostgreSQL
coding standards
* failed stat is reported as LOG instead of ERROR
Done a bit of testing here, and everything seems to be in order.
--
Bill Moran
Collaborative Fusion Inc.
diff -c -r src.orig/backend/storage/file/fd.c src/backend/storage/file/fd.c
*** src.orig/backend/storage/file/fd.c Thu Dec 7 15:44:42 2006
--- src/backend/storage/file/fd.c Wed Jan 3 15:05:54 2007
***
*** 50,55
--- 50,56
#include access/xact.h
#include storage/fd.h
#include storage/ipc.h
+ #include utils/guc.h
/*
***
*** 938,944
void
FileClose(File file)
{
! Vfd *vfdP;
Assert(FileIsValid(file));
--- 939,946
void
FileClose(File file)
{
! Vfd *vfdP;
! struct stat filestats;
Assert(FileIsValid(file));
***
*** 968,973
--- 970,992
{
/* reset flag so that die() interrupt won't cause problems */
vfdP-fdstate = ~FD_TEMPORARY;
+ PG_TRACE1(temp__file__cleanup, vfdP-fileName);
+ if (trace_temp_files != -1)
+ {
+ if (stat(vfdP-fileName, filestats) == 0)
+ {
+ if (filestats.st_size = trace_temp_files)
+ {
+ ereport(LOG,
+ (errmsg(temp file: size %lu path \%s\,
+ filestats.st_size, vfdP-fileName)));
+ }
+ }
+ else
+ {
+ elog(LOG, Could not stat \%s\: %m, vfdP-fileName);
+ }
+ }
if (unlink(vfdP-fileName))
elog(LOG, failed to unlink \%s\: %m,
vfdP-fileName);
diff -c -r src.orig/backend/utils/misc/guc.c src/backend/utils/misc/guc.c
*** src.orig/backend/utils/misc/guc.c Wed Nov 29 09:50:07 2006
--- src/backend/utils/misc/guc.c Wed Jan 3 13:51:14 2007
***
*** 180,186
int log_min_messages = NOTICE;
int client_min_messages = NOTICE;
int log_min_duration_statement = -1;
!
int num_temp_buffers = 1000;
char *ConfigFileName;
--- 180,187
int log_min_messages = NOTICE;
int client_min_messages = NOTICE;
int log_min_duration_statement = -1;
! int trace_temp_files = -1;
!
int num_temp_buffers = 1000;
char *ConfigFileName;
***
*** 1471,1477
log_min_duration_statement,
-1, -1, INT_MAX / 1000, NULL, NULL
},
!
{
{bgwriter_delay, PGC_SIGHUP, RESOURCES,
gettext_noop(Background writer sleep time between rounds in milliseconds),
--- 1472,1478
log_min_duration_statement,
-1, -1, INT_MAX / 1000, NULL, NULL
},
!
{
{bgwriter_delay, PGC_SIGHUP, RESOURCES,
gettext_noop(Background writer sleep time between rounds in milliseconds),
***
*** 1657,1662
--- 1658,1673
},
server_version_num,
PG_VERSION_NUM, PG_VERSION_NUM, PG_VERSION_NUM, NULL, NULL
+ },
+
+ {
+ {trace_temp_files, PGC_USERSET, LOGGING_WHAT,
+ gettext_noop(Log the use of temp files larger than this size.),
+ gettext_noop(Size and location of each temp file is reported.),
+ NULL
+ },
+ trace_temp_files,
+ -1, -1, INT_MAX, NULL, NULL
},
/* End-of-list marker */
diff -c -r src.orig/backend/utils/misc/postgresql.conf.sample src/backend/utils/misc/postgresql.conf.sample
*** src.orig/backend/utils/misc/postgresql.conf.sample Mon Nov 20 20:23:37 2006
--- src/backend/utils/misc/postgresql.conf.sample Wed Jan 3 11:05:48 2007
***
*** 333,338
--- 333,341
#log_statement = 'none' # none, ddl, mod, all
#log_hostname = off
+ #trace_temp_files = -1 # Log usage of temporary files larger than
+ # the specified size (in bytes). -1 disables.
+ # 0
Re: [HACKERS] [PATCHES] Patch to log usage of temporary files
Andrew Dunstan [EMAIL PROTECTED] wrote: Bill Moran wrote: + if (trace_temp_files != -1) Might be more robust to say if (trace_temp_files = 0) Because it would allow for the easy addition of more negative numbers with magic value? ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] xlog directory at initdb time
Peter Eisentraut wrote:
On the name of the option, it's not actually a data directory, so I'd just
call it --xlogdir, parallel to --datadir.
Seems reasonable. Patch modified is attached.
--
Euler Taveira de Oliveira
http://www.timbira.com/
*** ./doc/src/sgml/ref/initdb.sgml.orig 2006-09-15 21:30:18.0 -0300
--- ./doc/src/sgml/ref/initdb.sgml 2007-01-03 23:34:13.0 -0200
***
*** 177,182
--- 177,193
/varlistentry
varlistentry
+ termoption-X replaceable
class=parameterdirectory/replaceable/option/term
+ termoption--xlogdir=replaceable
class=parameterdirectory/replaceable/option/term
+ listitem
+para
+ This option specifies the directory where the transaction log
+ should be stored. This option is only supported on systems that
support symbolic links.
+/para
+ /listitem
+ /varlistentry
+
+ varlistentry
termoption-U replaceable
class=parameterusername/replaceable/option/term
termoption--username=replaceable
class=parameterusername/replaceable/option/term
listitem
*** ./src/bin/initdb/initdb.c.orig 2006-12-22 22:43:12.0 -0200
--- ./src/bin/initdb/initdb.c 2007-01-03 23:34:45.0 -0200
***
*** 95,100
--- 95,101
static bool debug = false;
static bool noclean = false;
static bool show_setting = false;
+ static char *xlog_dir = ;
/* internal vars */
***
*** 112,117
--- 113,120
static char *system_views_file;
static bool made_new_pgdata = false;
static bool found_existing_pgdata = false;
+ static bool made_new_xlogdir = false;
+ static bool found_existing_xlogdir = false;
static char infoversion[100];
static bool caught_signal = false;
static bool output_failed = false;
***
*** 163,169
static char *get_id(void);
static char *get_encoding_id(char *encoding_name);
static char *get_short_version(void);
! static intcheck_data_dir(void);
static bool mkdatadir(const char *subdir);
static void set_input(char **dest, char *filename);
static void check_input(char *path);
--- 166,172
static char *get_id(void);
static char *get_encoding_id(char *encoding_name);
static char *get_short_version(void);
! static intcheck_data_dir(char *dir);
static bool mkdatadir(const char *subdir);
static void set_input(char **dest, char *filename);
static void check_input(char *path);
***
*** 610,615
--- 613,636
fprintf(stderr, _(%s: failed to remove
contents of data directory\n),
progname);
}
+
+ if (made_new_xlogdir)
+ {
+ fprintf(stderr, _(%s: removing transaction log
directory \%s\\n),
+ progname, xlog_dir);
+ if (!rmtree(xlog_dir, true))
+ fprintf(stderr, _(%s: failed to remove
transaction log directory\n),
+ progname);
+ }
+ else if (found_existing_xlogdir)
+ {
+ fprintf(stderr,
+ _(%s: removing contents of transaction
log directory \%s\\n),
+ progname, xlog_dir);
+ if (!rmtree(xlog_dir, false))
+ fprintf(stderr, _(%s: failed to remove
contents of transaction log directory\n),
+ progname);
+ }
/* otherwise died during startup, do nothing! */
}
else
***
*** 618,623
--- 639,649
fprintf(stderr,
_(%s: data directory \%s\ not removed at user's
request\n),
progname, pg_data);
+
+ if (made_new_xlogdir || found_existing_xlogdir)
+ fprintf(stderr,
+ _(%s: transaction log directory \%s\ not removed
at user's request\n),
+ progname, xlog_dir);
}
exit(1);
***
*** 919,931
}
/*
! * make sure the data directory either doesn't exist or is empty
*
* Returns 0 if nonexistent, 1 if exists and empty, 2 if not empty,
* or -1 if trouble accessing directory
*/
static int
! check_data_dir(void)
{
DIR*chkdir;
struct dirent *file;
--- 945,957
}
/*
! * make sure the directory either doesn't exist or is empty
*
* Returns 0 if nonexistent, 1 if exists and empty, 2 if not empty,
* or -1 if trouble accessing directory
*/
static int
! check_data_dir(char *dir)
{
DIR*chkdir;
struct dirent *file;
***
*** 933,939
Re: [PATCHES] Tablespace for temporary objects and sort files
On 12/27/06, Albert Cervera Areny [EMAIL PROTECTED] wrote:
Hi,
here's a new version of the patch against HEAD with both, table and sort
files working correctly for me. Regression tests work too.
I'd like to ask again the question I made on the first post as no answer
was
given at that time:
The GetTempTablespace function correctly returns a different tablespace
each time is called, but I store the position of the last tablespace used
with an integer and iterate through the list of tablespaces each time. I
tried to keep the iterator from call to call but I got a segfault, I
imagine due to the memory context. Should I try to keep the iterator? How
can I do it?
Now I'm working on some regression tests that could be added to
tablespace.source using something like:
SET temp_tablespaces='testspace';
CREATE TEMP TABLE temp_foo(a VARCHAR);
SELECT COUNT(pg_ls_dir('pg_tblspc/' || (SELECT oid FROM
pg_catalog.pg_tablespace WHERE spcname='testspace' )
seems is working fine... i actually looked for the files in the
tablespace directory...
have you looked my past comments?
--
regards,
Jaime Casanova
Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs and the universe trying
to produce bigger and better idiots.
So far, the universe is winning.
Richard Cook
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
