Re: [HACKERS] WAL fsync scheduling
> Bruce Momjian <[EMAIL PROTECTED]> writes: > > Other backend will see they are not the lowest > > WAIT_ON_FSYNC and will wait for their byte to be set to NOT_IN_COMMIT > > so they can then continue, knowing their data was synced. > > How will they wait? Without a semaphore involved, your answer must > be either "timed sleep" or "busy-wait loop", neither of which is > attractive ... Yes, either timed sleep or busy-wait. One nifty trick would be for each backend that is not going to do the fsync to just sleep with signals enabled, and for the fsyncing backend to signal the other backends to exit their sleep. That way, only one backend does the checking. This sleep thing was going to be a problem anyway with the old system. At least this way, they sleep/check only in cases where it is valuable. Can we use a semaphore for this system? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: AW: AW: [HACKERS] Could turn on -O2 in AIX
>> My solution would be to use INT_MIN for all ports, which has the advantage >> that the above problematic comparison can be converted to !=, >> since no integer will be smaller than INT_MIN. > I agree. When I was looking at this code this morning, I was wondering > what INT_MIN was supposed to represent anyway, if NOSTART_ABSTIME is > INT_MIN + 1. I think someone messed this up between 4.2 and Postgres95. > Thomas, any objection to this plan? I went ahead and committed this change, since Thomas hasn't weighed in with an objection ... regards, tom lane
Re: [HACKERS] WAL fsync scheduling
Bruce Momjian <[EMAIL PROTECTED]> writes: > Other backend will see they are not the lowest > WAIT_ON_FSYNC and will wait for their byte to be set to NOT_IN_COMMIT > so they can then continue, knowing their data was synced. How will they wait? Without a semaphore involved, your answer must be either "timed sleep" or "busy-wait loop", neither of which is attractive ... regards, tom lane
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.cxlog.c)
> > sleep(3) should conform to POSIX specification, if anyone has the > > reference they can check it to see what the effect of sleep(0) > > should be. > > Yes, but Posix also specifies sched_yield() which rather explicitly > allows a process to yield its timeslice. No idea how well that is > supported. OK, I have a new idea. There are two parts to transaction commit. The first is writing all dirty buffers or log changes to the kernel, and second is fsync of the log file. I suggest having a per-backend shared memory byte that has the following values: START_LOG_WRITE WAIT_ON_FSYNC NOT_IN_COMMIT backend_number_doing_fsync I suggest that when each backend starts a commit, it sets its byte to START_LOG_WRITE. When it gets ready to fsync, it checks all backends. If all are NOT_IN_COMMIT, it does fsync and continues. If one or more are in START_LOG_WRITE, it waits until no one is in START_LOG_WRITE. It then checks all WAIT_ON_FSYNC, and if it is the lowest backend in WAIT_ON_FSYNC, marks all others with its backend number, and does fsync. It then clears all backends with its number to NOT_IN_COMMIT. Other backend will see they are not the lowest WAIT_ON_FSYNC and will wait for their byte to be set to NOT_IN_COMMIT so they can then continue, knowing their data was synced. This allows a single backend not to sleep, and allows multiple backends to bunch up only when they are all about to commit. The reason backend numbers are written is so other backends entering the commit code will not interfere with the backends performing fsync. Comments? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
[HACKERS] WAL fsync scheduling
> > sleep(3) should conform to POSIX specification, if anyone has the > > reference they can check it to see what the effect of sleep(0) > > should be. > > Yes, but Posix also specifies sched_yield() which rather explicitly > allows a process to yield its timeslice. No idea how well that is > supported. OK, I have a new idea. There are two parts to transaction commit. The first is writing all dirty buffers or log changes to the kernel, and second is fsync of the log file. I suggest having a per-backend shared memory byte that has the following values: START_LOG_WRITE WAIT_ON_FSYNC NOT_IN_COMMIT backend_number_doing_fsync I suggest that when each backend starts a commit, it sets its byte to START_LOG_WRITE. When it gets ready to fsync, it checks all backends. If all are NOT_IN_COMMIT, it does fsync and continues. If one or more are in START_LOG_WRITE, it waits until no one is in START_LOG_WRITE. It then checks all WAIT_ON_FSYNC, and if it is the lowest backend in WAIT_ON_FSYNC, marks all others with its backend number, and does fsync. It then clears all backends with its number to NOT_IN_COMMIT. Other backend will see they are not the lowest WAIT_ON_FSYNC and will wait for their byte to be set to NOT_IN_COMMIT so they can then continue, knowing their data was synced. This allows a single backend not to sleep, and allows multiple backends to bunch up only when they are all about to commit. The reason backend numbers are written is so other backends entering the commit code will not interfere with the backends performing fsync. Comments? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Failure to recognise new database
Grant Finnemore <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Are you guys running with WAL enabled? If so, this is probably the >> BufferSync issue that Hiroshi thought I broke a couple days ago. >> Let me know... > Yes, I am running WAL enabled. OK, I put back the BufferSync call. Sorry 'bout that, chief... regards, tom lane
Re: [HACKERS] location of Unix socket
Nathan Myers <[EMAIL PROTECTED]> writes: >> * Oliver Elphick <[EMAIL PROTECTED]> [001117 16:41] wrote: could the socket location be made a configurable parameter in 7.1? > /tmp is a bad place for this stuff anyway. There have been *very long* discussions of this issue in the past, see for example the threads "flock patch breaks things here" and "postmaster locking issues" in pghackers around 8/31/98 and 10/10/98. Could we have some review of the archives before people go off on a new thread? The bottom line is that the location of the socket file is a fundamental part of the client/server protocol. You can't just move it around on a whim, or your clients will be unable to find your server. We have just accepted a patch that allows explicit runtime specification of the socket-file path. (I've got severe doubts about it, because of this issue --- but at least it doesn't affect people who don't use it.) But if the socket-file path becomes a site-configuration item then we will see a lot of complaints. Look at the frequency with which we see people asking about "Undefined variable client_encoding" notices --- that proves that those folk are using clients and servers that weren't configured identically. That notice is at least pretty harmless ... but if the configuration determines whether or not you can even contact the server, it's not harmless. I agree that /tmp was a stupid place to put the files, but we've got to tread very lightly about moving them, or we'll create worse problems than we solve. BTW: a prediction, Oliver: you *will* live to regret making a distribution-specific change in the socket file location. Dunno how long it will take, but I foresee bug reports from this. regards, tom lane
Re: [HACKERS] location of Unix socket
The 7.1 code will the socket location configurable. > At present the Unix socket's location is hard-coded as /tmp. > > As a result of a bug report, I have moved it in the Debian package to > /var/run/postgresql/. (The bug was that tmpreaper was deleting it and > thus blocking new connections.) > > I suppose that we cannot assume that /var/run exists across all target > systems, so could the socket location be made a configurable parameter > in 7.1? > > -- > Oliver Elphick[EMAIL PROTECTED] > Isle of Wight http://www.lfix.co.uk/oliver > PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 > GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C > > "For by grace are ye saved through faith; and that not > of yourselves. It is the gift of God; not of works, > lest any man should boast."Ephesians 2:8,9 > > > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Fundamental change of locking behavior in 7.1
"Hiroshi Inoue" <[EMAIL PROTECTED]> writes: > In PostgreSQL,'lock table' acquires a AccessExclusiveLock by default. > IMHO ExclusiveLock is sufficient for ordinary purpose. People who want that kind of lock can get it with LOCK TABLE IN EXCLUSIVE MODE. I do not think it's a good idea to change the default kind of lock acquired by a plain LOCK TABLE, if that's what you're suggesting ... regards, tom lane
Re: [HACKERS] Varchar standard compliance
Peter Eisentraut <[EMAIL PROTECTED]> writes:
> Is there a reason why the conversion from CHAR to CHAR(1) is done in
> analyze.c:transformColumnType rather than right in the
> grammar?
Well, transformColumnType does database access, which is verboten during
the grammar phase. (The grammar has to execute correctly even if we're
in transaction-abort state, else we'll not be able to recognize the
COMMIT or ROLLBACK command...)
You could possibly do the equivalent work in the grammar based strictly
on recognizing the keywords CHAR, NUMERIC, etc, but I think that
approach will probably run into a dead end at some point. Really,
the grammar is NOT the place to be making semantic deductions. It
should give back an undecorated parse tree and let parse_analyze fill
in semantic deductions. (We've been pretty lax about that in the past,
but I've been trying to move semantics code out of gram.y recently.)
> peter=# select cast('voodoo' as char(1));
> ?column?
> --
> v
> (1 row)
> peter=# select cast('voodoo' as char);
> ?column?
> --
> voodoo
> (1 row)
Possibly transformColumnType() should be applied to datatype names
appearing in casts (and other places?) as well as those appearing in
table column declarations. However, I find your example unconvincing:
I'd expect the result of that cast to be of type char(6), not char(1).
In short, I don't believe the above-quoted behavior is wrong.
regards, tom lane
Re: [HACKERS] location of Unix socket
On Fri, Nov 17, 2000 at 04:49:43PM -0800, Alfred Perlstein wrote: > * Oliver Elphick <[EMAIL PROTECTED]> [001117 16:41] wrote: > > At present the Unix socket's location is hard-coded as /tmp. > > > > As a result of a bug report, I have moved it in the Debian package to > > /var/run/postgresql/. (The bug was that tmpreaper was deleting it and > > thus blocking new connections.) > > > > I suppose that we cannot assume that /var/run exists across all target > > systems, so could the socket location be made a configurable parameter > > in 7.1? > > What about X sockets and ssh-agent sockets, and so on? > Where's the source to this thing? :) > > It would make more sense to fix tempreaper to ignore non regular > files. X sockets are in subdirectories, e.g. /tmp/.X11-unix/X0. /tmp is a bad place for this stuff anyway. Nathan Myers [EMAIL PROTECTED]
Re: [HACKERS] location of Unix socket
* Oliver Elphick <[EMAIL PROTECTED]> [001117 16:41] wrote: > At present the Unix socket's location is hard-coded as /tmp. > > As a result of a bug report, I have moved it in the Debian package to > /var/run/postgresql/. (The bug was that tmpreaper was deleting it and > thus blocking new connections.) > > I suppose that we cannot assume that /var/run exists across all target > systems, so could the socket location be made a configurable parameter > in 7.1? What about X sockets and ssh-agent sockets, and so on? Where's the source to this thing? :) It would make more sense to fix tempreaper to ignore non regular files. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] "I have the heart of a child; I keep it in a jar on my desk."
[HACKERS] location of Unix socket
At present the Unix socket's location is hard-coded as /tmp. As a result of a bug report, I have moved it in the Debian package to /var/run/postgresql/. (The bug was that tmpreaper was deleting it and thus blocking new connections.) I suppose that we cannot assume that /var/run exists across all target systems, so could the socket location be made a configurable parameter in 7.1? -- Oliver Elphick[EMAIL PROTECTED] Isle of Wight http://www.lfix.co.uk/oliver PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C "For by grace are ye saved through faith; and that not of yourselves. It is the gift of God; not of works, lest any man should boast."Ephesians 2:8,9
[HACKERS] Courses
Hey Guys, do you know of any intermediate to advanced Postgres Courses in the UK prefereably in London and if not would anyone with advanced knowledge be interested in setting some up. Thanks, Abe
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
> Bruce Momjian wrote: > > > > > I didn't want to do this during development, but now that there are no > > > more old-style internal functions left, I suppose you could make a good > > > argument that this is worth doing for old-style dynamically loaded > > > functions. Will put it on the to-do list. > > > > > > Are people satisfied with the notion of requiring an info function > > > to go with each dynamically loaded new-style function? If so, I'll > > > start working on that too. > > > > I think we need to balance portability with inconvenence for new users. > > > > I think mixing new/old function types in the same object file is pretty > > rare, and the confusion for programmers of having to label every > > function seems much more error-prone. > > > > I would support a single symbol to mark the entire object file. In > > fact, I would require old-style functions to add a symbol, and have > > new-style functions left alone. > > > > There are not that many functions out there, are there? People are > > having to recompile their C files anyway for the upgrade, don't they? > > Can't we insert that magic variable automatically using some > #includ/#define tricks ? > > So that people need just to recompile, but the result has the variable > nonetheless ? We thought of that. The problem is some new-style functions do not need to call any macros. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
> > For that matter, I think past version updates haven't even forced > > recompiles of user-defined functions, at least not ones that didn't poke > > into system innards. We can't get away with requiring a source code > > change --- people will scream about it. > > imho that is *not* sufficient to keep from Doing The Right Thing. And > DTRT usually means designing correctly for the future. I would support a > design that is cleanest for the next release, and put backward > compatibility a (no so) distant second. Agreed. > > > The nice thing about the info-marker idea is that we'll be able to > > extend it later, so that more info about the function is stored right > > where the function text is, and you don't have such a problem with > > keeping an SQL script file in sync with the function's real definition. > > I've been wanting to think about the "package" or "module" idea (which > others have brought up recently). If this kind of hook gets us more > options to support that (e.g. running a routine when the module is first > loaded to install new options into "SET key=val") then even better. Lack > of hooks of this nature lead us to push datatypes down into the core > when if we had full-up module support we could make more things > loadable. One idea we had was to define a SET variable or psql command-line switch so that LANGUAGE "C" would mean either old or new style. That way, pg_dump loads from old versions could set the switch before load without changing their code, and we could call all functions C. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Bruce Momjian wrote: > > > I didn't want to do this during development, but now that there are no > > more old-style internal functions left, I suppose you could make a good > > argument that this is worth doing for old-style dynamically loaded > > functions. Will put it on the to-do list. > > > > Are people satisfied with the notion of requiring an info function > > to go with each dynamically loaded new-style function? If so, I'll > > start working on that too. > > I think we need to balance portability with inconvenence for new users. > > I think mixing new/old function types in the same object file is pretty > rare, and the confusion for programmers of having to label every > function seems much more error-prone. > > I would support a single symbol to mark the entire object file. In > fact, I would require old-style functions to add a symbol, and have > new-style functions left alone. > > There are not that many functions out there, are there? People are > having to recompile their C files anyway for the upgrade, don't they? Can't we insert that magic variable automatically using some #includ/#define tricks ? So that people need just to recompile, but the result has the variable nonetheless ? --- Hannu
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
> For that matter, I think past version updates haven't even forced > recompiles of user-defined functions, at least not ones that didn't poke > into system innards. We can't get away with requiring a source code > change --- people will scream about it. imho that is *not* sufficient to keep from Doing The Right Thing. And DTRT usually means designing correctly for the future. I would support a design that is cleanest for the next release, and put backward compatibility a (no so) distant second. > The nice thing about the info-marker idea is that we'll be able to > extend it later, so that more info about the function is stored right > where the function text is, and you don't have such a problem with > keeping an SQL script file in sync with the function's real definition. I've been wanting to think about the "package" or "module" idea (which others have brought up recently). If this kind of hook gets us more options to support that (e.g. running a routine when the module is first loaded to install new options into "SET key=val") then even better. Lack of hooks of this nature lead us to push datatypes down into the core when if we had full-up module support we could make more things loadable. - Thomas
RE: [HACKERS] Fundamental change of locking behavior in 7.1
> -Original Message- > From: Zeugswetter Andreas SB > > Since I see, that Tom has implemented the "keep a AccessShareLock > lock until > transaction end" philisophy I would like to state a protest. > > This is a fundamental change in behavior and I would like to see > a vote on this. > > The one example we already know is: > > session1 session2 > begin work; begin work; > select * from tenk1 limit 1; > select * from tenk1 limit 1; > lock table tenk1; --now waits (why should it ?) > lock table tenk1; -- > NOTICE: Deadlock detected --> ABORT > In PostgreSQL,'lock table' acquires a AccessExclusiveLock by default. IMHO ExclusiveLock is sufficient for ordinary purpose. It doesn't conflict with AccessShareLock. Oracle doesn't have AccessExclusive(Share)Lock and I've been suspicious why users could acquire the lock explicitly. Comments ? Regards. Hiroshi Inoue
Re: [HACKERS] [rfc] new CREATE FUNCTION (and more)
Nathan Myers <[EMAIL PROTECTED]> writes:
> Why declare a function instead of a static struct?
Well, two reasons. One is that a function provides wiggle room if we
later decide that the library needs to do some computation before
handing back the function info struct. For example, Philip suggested
that the info function might be just a stub that causes an additional
library to be loaded. I'm not sure we'll ever really *need* such
flexibility in the future, but when it costs hardly anything to leave
the option open, why not?
The second reason is that if it's a function call, we only have one
primitive lookup operation that we expect the dynamic loader to be able
to support: find a function in a shlib by name. We have that already in
order to be able to call the real function. If it's a global variable,
then we need a second primitive lookup operation: find a global variable
in a shlib by name. Given the way that dynamic-link shared libraries
work, this is *not* necessarily the same as the first operation (code
and data are handled much differently in a shared library!) and might
not even be available on all platforms. At the very least it'd imply a
round of per-platform development and portability testing that I doubt
we can afford if we want to shoehorn this feature into the 7.1 schedule.
In short, using a variable looks like more work for less functionality,
and so the choice seems pretty clear-cut to me: use a function.
> Users are allowed to have functions that start
> with "pg" already, and that's quite a reasonable prefix for
> functions meant to be called by Postgres. Therefore, I suggest
> a prefix "_pg" instead of "pg". Thus,
> const struct _pg_user_function _pg_user_function_foo = { 2, };
The exact details of the name prefix need to be settled regardless
of whether the name is attached to a variable or a function. I was
thinking of pg_finfo_foo for a function named foo. We want to keep
the prefix reasonably short, so as to reduce the risk of duplicate-
symbol conflicts if the platform's linker truncates names to some
fixed length (I'm sure there are still some that do :-(). Using
a leading underscore (_pg_info_foo) might help but I worry about
creating conflicts with system names if we do that.
regards, tom lane
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Bruce Momjian <[EMAIL PROTECTED]> writes: > I would support a single symbol to mark the entire object file. In > fact, I would require old-style functions to add a symbol, and have > new-style functions left alone. That won't fly. > There are not that many functions out there, are there? People are > having to recompile their C files anyway for the upgrade, don't they? There's a big difference between having to recompile and having to change your source code. For that matter, I think past version updates haven't even forced recompiles of user-defined functions, at least not ones that didn't poke into system innards. We can't get away with requiring a source code change --- people will scream about it. The nice thing about the info-marker idea is that we'll be able to extend it later, so that more info about the function is stored right where the function text is, and you don't have such a problem with keeping an SQL script file in sync with the function's real definition. regards, tom lane
Re: AW: [HACKERS] Coping with 'C' vs 'newC' function language namesh
On Fri, Nov 17, 2000 at 09:43:28PM +1100, Philip Warner wrote:
> At 10:39 16/11/00 +0100, Zeugswetter Andreas SB wrote:
> >
> >Has anybody had time to look at how this is done in DB/2, Oracle ? Philip ?
> >
>
> Don't know about Oracle or DB2, but Dec/RDB has:
>
Well, I don't know 'nuthing about Oracle, but I _did_ sign up for the
OTN web site some time ago, specifically to get at Oracle docs. ;-)
After clicking around there for a bit I came up with this, which is my
interpretation of a 'flowchart' style language diagram:
CREATE [OR REPLACE] FUNCTION [.]
[( [IN|OUT|IN OUT] [NOCOPY] [, <...>] )]
RETURN
[{AUTHID {CURRENT_USER|DEFINER} | DETERMINISTIC | PARALLEL_ENABLE} <...>]
{IS|AS}
{ |
LANGUAGE {JAVA NAME ''|
C [NAME ] LIBRARY
[WITH CONTEXT] [PARAMETERS (...)]
}
}
The actual filesystem path to the DLL or .so is defined with a CREATE
LIBRARY command.
The WITH CONTEXT bit is a pointer to an opaque structure that the
underlying function is supposed to pass on to service routines it might
call, for their use.
It seems the parameters can be defined either in place with the name,
or with the PARAMETERS keyword.
Philip's comments about this directly calling into a C shared library
seem to apply, as well.
Ross
--
Open source code is like a natural resource, it's the result of providing
food and sunshine to programmers, and then staying out of their way.
[...] [It] is not going away because it has utility for both the developers
and users independent of economic motivations. Jim Flynn, Sunnyvale, Calif.
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.c xlog.c)
* Bruce Momjian <[EMAIL PROTECTED]> [001117 11:39]: > > * Bruce Momjian <[EMAIL PROTECTED]> [001117 11:23]: > > > > > sleep(3) should conform to POSIX specification, if anyone has the > > > > > reference they can check it to see what the effect of sleep(0) > > > > > should be. > > > > > > > > Yes, but Posix also specifies sched_yield() which rather explicitly > > > > allows a process to yield its timeslice. No idea how well that is > > > > supported. > > > > > > I have it on BSDI. We could add a configure check, and use it if it is > > > there. Another idea is to add a shared memory flag when someone enters > > > the 'commit' section of the transaction code. That way, a backend could > > > check to see if another process is _about_ to commit, and wait. > > On UnixWare, it requires the -Kthread or -Kpthread command, which then > > links in the threads library... > > > > I'm not sure that this is a good thing or not > > I would hope it just calls the function, and does not bring in thread > startup stuff. I suspect it DOES bring in the thread startup and all that implies... Tread lightly. The good news is UnixWare Threads are LWP's and the kernel is multithreaded... LER > > -- > Bruce Momjian| http://candle.pha.pa.us > [EMAIL PROTECTED] | (610) 853-3000 > + If your life is a hard drive, | 830 Blythe Avenue > + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 (voice) Internet: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.cxlog.c)
> * Bruce Momjian <[EMAIL PROTECTED]> [001117 11:23]: > > > > sleep(3) should conform to POSIX specification, if anyone has the > > > > reference they can check it to see what the effect of sleep(0) > > > > should be. > > > > > > Yes, but Posix also specifies sched_yield() which rather explicitly > > > allows a process to yield its timeslice. No idea how well that is > > > supported. > > > > I have it on BSDI. We could add a configure check, and use it if it is > > there. Another idea is to add a shared memory flag when someone enters > > the 'commit' section of the transaction code. That way, a backend could > > check to see if another process is _about_ to commit, and wait. > On UnixWare, it requires the -Kthread or -Kpthread command, which then > links in the threads library... > > I'm not sure that this is a good thing or not I would hope it just calls the function, and does not bring in thread startup stuff. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.c xlog.c)
* Bruce Momjian <[EMAIL PROTECTED]> [001117 11:23]: > > > sleep(3) should conform to POSIX specification, if anyone has the > > > reference they can check it to see what the effect of sleep(0) > > > should be. > > > > Yes, but Posix also specifies sched_yield() which rather explicitly > > allows a process to yield its timeslice. No idea how well that is > > supported. > > I have it on BSDI. We could add a configure check, and use it if it is > there. Another idea is to add a shared memory flag when someone enters > the 'commit' section of the transaction code. That way, a backend could > check to see if another process is _about_ to commit, and wait. On UnixWare, it requires the -Kthread or -Kpthread command, which then links in the threads library... I'm not sure that this is a good thing or not LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 (voice) Internet: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
> I didn't want to do this during development, but now that there are no > more old-style internal functions left, I suppose you could make a good > argument that this is worth doing for old-style dynamically loaded > functions. Will put it on the to-do list. > > Are people satisfied with the notion of requiring an info function > to go with each dynamically loaded new-style function? If so, I'll > start working on that too. I think we need to balance portability with inconvenence for new users. I think mixing new/old function types in the same object file is pretty rare, and the confusion for programmers of having to label every function seems much more error-prone. I would support a single symbol to mark the entire object file. In fact, I would require old-style functions to add a symbol, and have new-style functions left alone. There are not that many functions out there, are there? People are having to recompile their C files anyway for the upgrade, don't they? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.cxlog.c)
> > sleep(3) should conform to POSIX specification, if anyone has the > > reference they can check it to see what the effect of sleep(0) > > should be. > > Yes, but Posix also specifies sched_yield() which rather explicitly > allows a process to yield its timeslice. No idea how well that is > supported. I have it on BSDI. We could add a configure check, and use it if it is there. Another idea is to add a shared memory flag when someone enters the 'commit' section of the transaction code. That way, a backend could check to see if another process is _about_ to commit, and wait. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
AW: [HACKERS] Coping with 'C' vs 'newC' function language names
> > Just successfully loading an old-style C function doesn't > > guarantee that it works anyway. I pointed out before that the > > changes due to TOAST require each function that takes > > arguments of varlen types to expect toasted values. > > Couldn't the function handler detoast the values before handing them to > the function? That would be slower, but it would allow people to continue > to use the "simpler" interface. Are there really that many things you can do with a toasted value ? Off hand I can only think of equality functions. All others should need a detoasted value anyways. Andreas
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Couldn't the function handler detoast the values before handing them to > the function? That would be slower, but it would allow people to continue > to use the "simpler" interface. This would be a moderately expensive thing to do --- not impossible, but relatively expensive. The old-style function handler would need to look up all the function-argument datatypes during fmgr_info() so that it could save a bool array telling it which argument positions need detoasting. (You can't just detoast every Datum --- you have to at least verify that it's of a varlena type first.) On a per-invocation basis, however, it probably wouldn't be very costly. I didn't want to do this during development, but now that there are no more old-style internal functions left, I suppose you could make a good argument that this is worth doing for old-style dynamically loaded functions. Will put it on the to-do list. Are people satisfied with the notion of requiring an info function to go with each dynamically loaded new-style function? If so, I'll start working on that too. regards, tom lane
Re: [HACKERS] Varchar standard compliance
Peter Eisentraut <[EMAIL PROTECTED]> writes: >> On what grounds do you claim that behavior is incorrect? > Because SQL says so: > ::= > CHARACTER [] > | CHAR [] > | CHARACTER VARYING > | CHAR VARYING > | VARCHAR > 4) If is omitted, then a of 1 is implicit. Well, what that actually says is that CHAR means CHAR(1). The syntax does not allow VARCHAR without (n), so the thing we are noncompliant on is not what we consider the default n to be, but whether there is a default length for varchar at all. The spec is not offering one. I don't particularly want to enforce the spec's position that leaving off (n) is illegal, and given the choice between defaulting to VARCHAR(1) or VARCHAR(large), I'll take the second. The second one at least has some usefulness... > If we want to keep this, then there would really be no difference between > VARCHAR and TEXT, right? There's no real difference between VARCHAR without a length limit and TEXT, no. > I'm not partial to either side, but I wanted to know what the bit types > should do. I'd be inclined to stick with our existing VARCHAR behavior just on grounds of backwards compatibility. If you want to make the bit types behave differently, I wouldn't say that's indefensible. However, one advantage of treating BIT VARYING without (n) as unlimited is that you'd have the equivalent functionality to TEXT without having to make a third bit type... regards, tom lane
[HACKERS] Re: Database startup info
Thomas Lockhart <[EMAIL PROTECTED]> writes: > Would it be reasonable to label these fields for their likely 7.2 > functionality, rather than labeling them as they are now? Since this is > the first time they are appearing, it would be nice to not have to > change the names later... I don't have a problem with renaming "datallowconn" to "datoffline" (and reversing its sense) if you feel like doing that --- but please note that these are only field names, they do not constrain whatever command-level API we might put on top of the thing later. In any case, I'm not sure it's a good idea to call the thing "datoffline" when changing it doesn't actively throw off current connections. Names that are intended to be suggestive should be accurately suggestive, IMHO. (Maybe I should've called it datallownewconn.) As for datistemplate, that is NOT the same as datreadonly, and when we get around to supporting read-only databases there should be a separate column for that, IMHO. datistemplate is actually a permissions bit (are people who are neither superuser nor the database owner allowed to clone a particular database?) and has nothing to do with whether the DB is read-only. When we have read-only functionality, I'd want to change CREATE DATABASE to require the source to be both datistemplate and datreadonly --- but there are also substantial uses for databases that are readonly but not templates. So we need two bits. (Perhaps readonly status should apply to schemas, not databases, anyway --- haven't studied that part of the spec yet...) In short: I think datistemplate is fine as is. If you want to tweak the name or behavior of datallowconn, go for it (though implementing a command to set it might be a better plan than just tweaking the field name...) regards, tom lane
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Jan Wieck writes: > Just successfully loading an old-style C function doesn't > guarantee that it works anyway. I pointed out before that the > changes due to TOAST require each function that takes > arguments of varlen types to expect toasted values. Couldn't the function handler detoast the values before handing them to the function? That would be slower, but it would allow people to continue to use the "simpler" interface. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
AW: [HACKERS] Coping with 'C' vs 'newC' function language names
> > More to the point, I think we have to assume old-style interface if we > > see ... LANGUAGE 'C' with no other decoration, because any other > > assumption is guaranteed to break all existing user-defined functions. > > Just successfully loading an old-style C function doesn't > guarantee that it works anyway. I pointed out before that the > changes due to TOAST require each function that takes > arguments of varlen types to expect toasted values. Worst > case a dump might reload and anything works fine, but a month > later the first toasted value appears and the old-style C > function corrupts the data without a single warning. > > We need to WARN, WARN and explicitly WARN users of selfmade C > functions about this in any possible place! Imho the better solution would be, to always detoast values before we pass them to old-style C-functions. And toast the return value when the function returns. Andreas
Re: [HACKERS] Unhappy thoughts about pg_dump and objects inherited fromtemplate1
Tom Lane wrote: > To bring this back from future nice solutions to the reality of what > to do today, do people like the "template0" solution for now (7.1)? > I can work on it if so. Go for it. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] #
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Tom Lane wrote: > > More to the point, I think we have to assume old-style interface if we > see ... LANGUAGE 'C' with no other decoration, because any other > assumption is guaranteed to break all existing user-defined functions. Just successfully loading an old-style C function doesn't guarantee that it works anyway. I pointed out before that the changes due to TOAST require each function that takes arguments of varlen types to expect toasted values. Worst case a dump might reload and anything works fine, but a month later the first toasted value appears and the old-style C function corrupts the data without a single warning. We need to WARN, WARN and explicitly WARN users of selfmade C functions about this in any possible place! Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] #
Re: [HACKERS] Failure to recognise new database
> Are you guys running with WAL enabled? If so, this is probably the > BufferSync issue that Hiroshi thought I broke a couple days ago. > Let me know... Yes, I too am running with WAL enabled. - Thomas
[HACKERS] Fundamental change of locking behavior in 7.1
Since I see, that Tom has implemented the "keep a AccessShareLock lock until transaction end" philisophy I would like to state a protest. This is a fundamental change in behavior and I would like to see a vote on this. The one example we already know is: session1session2 begin work; begin work; select * from tenk1 limit 1; select * from tenk1 limit 1; lock table tenk1; --now waits (why should it ?) lock table tenk1; -- NOTICE: Deadlock detected --> ABORT I think this is not acceptable in committed read isolation. The AccessShareLock needs to be released after each statement finishes. Thank you Andreas
Re: AW: [HACKERS] Coping with 'C' vs 'newC' function language namesh
At 10:39 16/11/00 +0100, Zeugswetter Andreas SB wrote:
>
>Has anybody had time to look at how this is done in DB/2, Oracle ? Philip ?
>
Don't know about Oracle or DB2, but Dec/RDB has:
Create Function [Stored Name Is ] (...) Returns ;
[ External Name ] [ Location ]
[Language {ADA|C|Fortran|Pascal|General]
General Parameter Style [Not] Variant
where is the function name, the 'Stored Name' relates to
cross-schema functions, the 'External Name' is the name of the entry point,
'location' is the file location and the 'Variant' part corresponds to our
'iscacheable' attribute.
Functions themselves require no special coding. This is pretty much what I
think 'Language C' does for us now - just calling a shared library. There
is definitely a case to be made for coninuing to support standard calls to
general libraries unmodified. In Tom's current proposal, I think this will
be the case if there is no 'function-info' function available.
Philip Warner| __---_
Albatross Consulting Pty. Ltd. |/ - \
(A.B.N. 75 008 659 498) | /(@) __---_
Tel: (+61) 0500 83 82 81 | _ \
Fax: (+61) 0500 83 82 82 | ___ |
Http://www.rhyme.com.au |/ \|
|----
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/
AW: AW: AW: AW: [HACKERS] Coping with 'C' vs 'newC' function language nam esh
> Also, the grammar clause "LANGUAGE C" is actually part of the standard, so > naming it "LANGUAGE stdC" will make it *less* standard. (Not that I buy > Informix as being a "standard".) I only quoted Informix, because it is the only one where I know how it works. It might even be, that the Oracle and DB/2 interface is also similar to our "oldC", I simply don't know. The fact, that part of this is already in the standard (like "language c") makes me even more firm in my opinion, that more research is needed before advertising "newC". Andreas
AW: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.c xlog.c)
> > >> Ewe, so we have this 1/200 second delay for every transaction. Seems > > >> bad to me. > > > > > >I think as long as it becomes a tunable this isn't a bad idea at > > >all. Fixing it at 1/200 isn't so great because people not wrapping > > >large amounts of inserts/updates with transaction blocks will > > >suffer. > > > > I think the default should probably be no delay, and the documentation > > on enabling this needs to be clear and obvious (i.e. hard to miss). > > I just talked to Tom Lane about this. I think a sleep(0) just before > the flush would be the best. It would reliquish the cpu slice if > another process is ready to run. If no other backend is running, it > probably just returns. If there is another one, it gives it > a chance to > complete. On return from sleep(0), it can check if it still needs to > flush. This would tend to bunch up flushers so they flush only once, > while not delaying cases where only one backend is running. I don't think anything that simply yields the processor works on multiprocessor machines. The point is, that fsync is so expensive, that a wait time in the milliseconds is needed, and not micro seconds, to really improve tx throughput for many clients. I support the default to not delay point, since only a very heavily loaded database will see a lot of fsyncs in the same millisecond timeslice. A dba coping with a very heavily loaded database will need to tune anyway, so for him one additional config is no problem. Andreas
Re: [HACKERS] Failure to recognise new database
Tom Lane wrote: > Thomas Lockhart <[EMAIL PROTECTED]> writes: > >> Is it just me? > > > I'm pretty sure I saw something similar on a newly initialized database. > > Are you guys running with WAL enabled? If so, this is probably the > BufferSync issue that Hiroshi thought I broke a couple days ago. > Let me know... Yes, I am running WAL enabled. > > > regards, tom lane Regards, Grant -- > Poorly planned software requires a genius to write it > and a hero to use it. Grant Finnemore BSc(Eng) (mailto:[EMAIL PROTECTED]) Software Engineer Universal Computer Services Tel (+27)(11)712-1366PO Box 31266 Braamfontein 2017, South Africa Cell (+27)(82)604-553620th Floor, 209 Smit St., Braamfontein Fax (+27)(11)339-3421Johannesburg, South Africa
