Re: [HACKERS] Failure to recognise new database
Tom Lane wrote: Thomas Lockhart [EMAIL PROTECTED] writes: Is it just me? I'm pretty sure I saw something similar on a newly initialized database. Are you guys running with WAL enabled? If so, this is probably the BufferSync issue that Hiroshi thought I broke a couple days ago. Let me know... Yes, I am running WAL enabled. regards, tom lane Regards, Grant -- Poorly planned software requires a genius to write it and a hero to use it. Grant Finnemore BSc(Eng) (mailto:[EMAIL PROTECTED]) Software Engineer Universal Computer Services Tel (+27)(11)712-1366PO Box 31266 Braamfontein 2017, South Africa Cell (+27)(82)604-553620th Floor, 209 Smit St., Braamfontein Fax (+27)(11)339-3421Johannesburg, South Africa
AW: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.c xlog.c)
Ewe, so we have this 1/200 second delay for every transaction. Seems bad to me. I think as long as it becomes a tunable this isn't a bad idea at all. Fixing it at 1/200 isn't so great because people not wrapping large amounts of inserts/updates with transaction blocks will suffer. I think the default should probably be no delay, and the documentation on enabling this needs to be clear and obvious (i.e. hard to miss). I just talked to Tom Lane about this. I think a sleep(0) just before the flush would be the best. It would reliquish the cpu slice if another process is ready to run. If no other backend is running, it probably just returns. If there is another one, it gives it a chance to complete. On return from sleep(0), it can check if it still needs to flush. This would tend to bunch up flushers so they flush only once, while not delaying cases where only one backend is running. I don't think anything that simply yields the processor works on multiprocessor machines. The point is, that fsync is so expensive, that a wait time in the milliseconds is needed, and not micro seconds, to really improve tx throughput for many clients. I support the default to not delay point, since only a very heavily loaded database will see a lot of fsyncs in the same millisecond timeslice. A dba coping with a very heavily loaded database will need to tune anyway, so for him one additional config is no problem. Andreas
[HACKERS] Fundamental change of locking behavior in 7.1
Since I see, that Tom has implemented the "keep a AccessShareLock lock until transaction end" philisophy I would like to state a protest. This is a fundamental change in behavior and I would like to see a vote on this. The one example we already know is: session1session2 begin work; begin work; select * from tenk1 limit 1; select * from tenk1 limit 1; lock table tenk1; --now waits (why should it ?) lock table tenk1; -- NOTICE: Deadlock detected -- ABORT I think this is not acceptable in committed read isolation. The AccessShareLock needs to be released after each statement finishes. Thank you Andreas
Re: [HACKERS] Failure to recognise new database
Are you guys running with WAL enabled? If so, this is probably the BufferSync issue that Hiroshi thought I broke a couple days ago. Let me know... Yes, I too am running with WAL enabled. - Thomas
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Tom Lane wrote: More to the point, I think we have to assume old-style interface if we see ... LANGUAGE 'C' with no other decoration, because any other assumption is guaranteed to break all existing user-defined functions. Just successfully loading an old-style C function doesn't guarantee that it works anyway. I pointed out before that the changes due to TOAST require each function that takes arguments of varlen types to expect toasted values. Worst case a dump might reload and anything works fine, but a month later the first toasted value appears and the old-style C function corrupts the data without a single warning. We need to WARN, WARN and explicitly WARN users of selfmade C functions about this in any possible place! Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] #
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.cxlog.c)
sleep(3) should conform to POSIX specification, if anyone has the reference they can check it to see what the effect of sleep(0) should be. Yes, but Posix also specifies sched_yield() which rather explicitly allows a process to yield its timeslice. No idea how well that is supported. I have it on BSDI. We could add a configure check, and use it if it is there. Another idea is to add a shared memory flag when someone enters the 'commit' section of the transaction code. That way, a backend could check to see if another process is _about_ to commit, and wait. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.cxlog.c)
* Bruce Momjian [EMAIL PROTECTED] [001117 11:23]: sleep(3) should conform to POSIX specification, if anyone has the reference they can check it to see what the effect of sleep(0) should be. Yes, but Posix also specifies sched_yield() which rather explicitly allows a process to yield its timeslice. No idea how well that is supported. I have it on BSDI. We could add a configure check, and use it if it is there. Another idea is to add a shared memory flag when someone enters the 'commit' section of the transaction code. That way, a backend could check to see if another process is _about_ to commit, and wait. On UnixWare, it requires the -Kthread or -Kpthread command, which then links in the threads library... I'm not sure that this is a good thing or not I would hope it just calls the function, and does not bring in thread startup stuff. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] RE: [COMMITTERS] pgsql/src/backend/access/transam ( xact.c xlog.c)
* Bruce Momjian [EMAIL PROTECTED] [001117 11:39]: * Bruce Momjian [EMAIL PROTECTED] [001117 11:23]: sleep(3) should conform to POSIX specification, if anyone has the reference they can check it to see what the effect of sleep(0) should be. Yes, but Posix also specifies sched_yield() which rather explicitly allows a process to yield its timeslice. No idea how well that is supported. I have it on BSDI. We could add a configure check, and use it if it is there. Another idea is to add a shared memory flag when someone enters the 'commit' section of the transaction code. That way, a backend could check to see if another process is _about_ to commit, and wait. On UnixWare, it requires the -Kthread or -Kpthread command, which then links in the threads library... I'm not sure that this is a good thing or not I would hope it just calls the function, and does not bring in thread startup stuff. I suspect it DOES bring in the thread startup and all that implies... Tread lightly. The good news is UnixWare Threads are LWP's and the kernel is multithreaded... LER -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 (voice) Internet: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Re: [HACKERS] [rfc] new CREATE FUNCTION (and more)
Nathan Myers [EMAIL PROTECTED] writes:
Why declare a function instead of a static struct?
Well, two reasons. One is that a function provides wiggle room if we
later decide that the library needs to do some computation before
handing back the function info struct. For example, Philip suggested
that the info function might be just a stub that causes an additional
library to be loaded. I'm not sure we'll ever really *need* such
flexibility in the future, but when it costs hardly anything to leave
the option open, why not?
The second reason is that if it's a function call, we only have one
primitive lookup operation that we expect the dynamic loader to be able
to support: find a function in a shlib by name. We have that already in
order to be able to call the real function. If it's a global variable,
then we need a second primitive lookup operation: find a global variable
in a shlib by name. Given the way that dynamic-link shared libraries
work, this is *not* necessarily the same as the first operation (code
and data are handled much differently in a shared library!) and might
not even be available on all platforms. At the very least it'd imply a
round of per-platform development and portability testing that I doubt
we can afford if we want to shoehorn this feature into the 7.1 schedule.
In short, using a variable looks like more work for less functionality,
and so the choice seems pretty clear-cut to me: use a function.
Users are allowed to have functions that start
with "pg" already, and that's quite a reasonable prefix for
functions meant to be called by Postgres. Therefore, I suggest
a prefix "_pg" instead of "pg". Thus,
const struct _pg_user_function _pg_user_function_foo = { 2, };
The exact details of the name prefix need to be settled regardless
of whether the name is attached to a variable or a function. I was
thinking of pg_finfo_foo for a function named foo. We want to keep
the prefix reasonably short, so as to reduce the risk of duplicate-
symbol conflicts if the platform's linker truncates names to some
fixed length (I'm sure there are still some that do :-(). Using
a leading underscore (_pg_info_foo) might help but I worry about
creating conflicts with system names if we do that.
regards, tom lane
RE: [HACKERS] Fundamental change of locking behavior in 7.1
-Original Message- From: Zeugswetter Andreas SB Since I see, that Tom has implemented the "keep a AccessShareLock lock until transaction end" philisophy I would like to state a protest. This is a fundamental change in behavior and I would like to see a vote on this. The one example we already know is: session1 session2 begin work; begin work; select * from tenk1 limit 1; select * from tenk1 limit 1; lock table tenk1; --now waits (why should it ?) lock table tenk1; -- NOTICE: Deadlock detected -- ABORT In PostgreSQL,'lock table' acquires a AccessExclusiveLock by default. IMHO ExclusiveLock is sufficient for ordinary purpose. It doesn't conflict with AccessShareLock. Oracle doesn't have AccessExclusive(Share)Lock and I've been suspicious why users could acquire the lock explicitly. Comments ? Regards. Hiroshi Inoue
[HACKERS] location of Unix socket
At present the Unix socket's location is hard-coded as /tmp. As a result of a bug report, I have moved it in the Debian package to /var/run/postgresql/. (The bug was that tmpreaper was deleting it and thus blocking new connections.) I suppose that we cannot assume that /var/run exists across all target systems, so could the socket location be made a configurable parameter in 7.1? -- Oliver Elphick[EMAIL PROTECTED] Isle of Wight http://www.lfix.co.uk/oliver PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C "For by grace are ye saved through faith; and that not of yourselves. It is the gift of God; not of works, lest any man should boast."Ephesians 2:8,9
Re: [HACKERS] location of Unix socket
* Oliver Elphick [EMAIL PROTECTED] [001117 16:41] wrote: At present the Unix socket's location is hard-coded as /tmp. As a result of a bug report, I have moved it in the Debian package to /var/run/postgresql/. (The bug was that tmpreaper was deleting it and thus blocking new connections.) I suppose that we cannot assume that /var/run exists across all target systems, so could the socket location be made a configurable parameter in 7.1? What about X sockets and ssh-agent sockets, and so on? Where's the source to this thing? :) It would make more sense to fix tempreaper to ignore non regular files. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] "I have the heart of a child; I keep it in a jar on my desk."
Re: [HACKERS] location of Unix socket
On Fri, Nov 17, 2000 at 04:49:43PM -0800, Alfred Perlstein wrote: * Oliver Elphick [EMAIL PROTECTED] [001117 16:41] wrote: At present the Unix socket's location is hard-coded as /tmp. As a result of a bug report, I have moved it in the Debian package to /var/run/postgresql/. (The bug was that tmpreaper was deleting it and thus blocking new connections.) I suppose that we cannot assume that /var/run exists across all target systems, so could the socket location be made a configurable parameter in 7.1? What about X sockets and ssh-agent sockets, and so on? Where's the source to this thing? :) It would make more sense to fix tempreaper to ignore non regular files. X sockets are in subdirectories, e.g. /tmp/.X11-unix/X0. /tmp is a bad place for this stuff anyway. Nathan Myers [EMAIL PROTECTED]
Re: [HACKERS] Fundamental change of locking behavior in 7.1
"Hiroshi Inoue" [EMAIL PROTECTED] writes: In PostgreSQL,'lock table' acquires a AccessExclusiveLock by default. IMHO ExclusiveLock is sufficient for ordinary purpose. People who want that kind of lock can get it with LOCK TABLE IN EXCLUSIVE MODE. I do not think it's a good idea to change the default kind of lock acquired by a plain LOCK TABLE, if that's what you're suggesting ... regards, tom lane
Re: [HACKERS] Unhappy thoughts about pg_dump and objects inherited fromtemplate1
Tom Lane wrote: To bring this back from future nice solutions to the reality of what to do today, do people like the "template0" solution for now (7.1)? I can work on it if so. Go for it. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] #
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Jan Wieck writes: Just successfully loading an old-style C function doesn't guarantee that it works anyway. I pointed out before that the changes due to TOAST require each function that takes arguments of varlen types to expect toasted values. Couldn't the function handler detoast the values before handing them to the function? That would be slower, but it would allow people to continue to use the "simpler" interface. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] Varchar standard compliance
Peter Eisentraut [EMAIL PROTECTED] writes: On what grounds do you claim that behavior is incorrect? Because SQL says so: character string type ::= CHARACTER [ left paren length right paren ] | CHAR [ left paren length right paren ] | CHARACTER VARYING left paren length right paren | CHAR VARYING left paren length right paren | VARCHAR left paren length right paren 4) If length is omitted, then a length of 1 is implicit. Well, what that actually says is that CHAR means CHAR(1). The syntax does not allow VARCHAR without (n), so the thing we are noncompliant on is not what we consider the default n to be, but whether there is a default length for varchar at all. The spec is not offering one. I don't particularly want to enforce the spec's position that leaving off (n) is illegal, and given the choice between defaulting to VARCHAR(1) or VARCHAR(large), I'll take the second. The second one at least has some usefulness... If we want to keep this, then there would really be no difference between VARCHAR and TEXT, right? There's no real difference between VARCHAR without a length limit and TEXT, no. I'm not partial to either side, but I wanted to know what the bit types should do. I'd be inclined to stick with our existing VARCHAR behavior just on grounds of backwards compatibility. If you want to make the bit types behave differently, I wouldn't say that's indefensible. However, one advantage of treating BIT VARYING without (n) as unlimited is that you'd have the equivalent functionality to TEXT without having to make a third bit type... regards, tom lane
Re: [HACKERS] Varchar standard compliance
Peter Eisentraut [EMAIL PROTECTED] writes:
Is there a reason why the conversion from CHAR to CHAR(1) is done in
analyze.c:transformColumnType rather than right in the
grammar?
Well, transformColumnType does database access, which is verboten during
the grammar phase. (The grammar has to execute correctly even if we're
in transaction-abort state, else we'll not be able to recognize the
COMMIT or ROLLBACK command...)
You could possibly do the equivalent work in the grammar based strictly
on recognizing the keywords CHAR, NUMERIC, etc, but I think that
approach will probably run into a dead end at some point. Really,
the grammar is NOT the place to be making semantic deductions. It
should give back an undecorated parse tree and let parse_analyze fill
in semantic deductions. (We've been pretty lax about that in the past,
but I've been trying to move semantics code out of gram.y recently.)
peter=# select cast('voodoo' as char(1));
?column?
--
v
(1 row)
peter=# select cast('voodoo' as char);
?column?
--
voodoo
(1 row)
Possibly transformColumnType() should be applied to datatype names
appearing in casts (and other places?) as well as those appearing in
table column declarations. However, I find your example unconvincing:
I'd expect the result of that cast to be of type char(6), not char(1).
In short, I don't believe the above-quoted behavior is wrong.
regards, tom lane
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
For that matter, I think past version updates haven't even forced recompiles of user-defined functions, at least not ones that didn't poke into system innards. We can't get away with requiring a source code change --- people will scream about it. imho that is *not* sufficient to keep from Doing The Right Thing. And DTRT usually means designing correctly for the future. I would support a design that is cleanest for the next release, and put backward compatibility a (no so) distant second. The nice thing about the info-marker idea is that we'll be able to extend it later, so that more info about the function is stored right where the function text is, and you don't have such a problem with keeping an SQL script file in sync with the function's real definition. I've been wanting to think about the "package" or "module" idea (which others have brought up recently). If this kind of hook gets us more options to support that (e.g. running a routine when the module is first loaded to install new options into "SET key=val") then even better. Lack of hooks of this nature lead us to push datatypes down into the core when if we had full-up module support we could make more things loadable. - Thomas
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Bruce Momjian wrote: I didn't want to do this during development, but now that there are no more old-style internal functions left, I suppose you could make a good argument that this is worth doing for old-style dynamically loaded functions. Will put it on the to-do list. Are people satisfied with the notion of requiring an info function to go with each dynamically loaded new-style function? If so, I'll start working on that too. I think we need to balance portability with inconvenence for new users. I think mixing new/old function types in the same object file is pretty rare, and the confusion for programmers of having to label every function seems much more error-prone. I would support a single symbol to mark the entire object file. In fact, I would require old-style functions to add a symbol, and have new-style functions left alone. There are not that many functions out there, are there? People are having to recompile their C files anyway for the upgrade, don't they? Can't we insert that magic variable automatically using some #includ/#define tricks ? So that people need just to recompile, but the result has the variable nonetheless ? --- Hannu
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
For that matter, I think past version updates haven't even forced recompiles of user-defined functions, at least not ones that didn't poke into system innards. We can't get away with requiring a source code change --- people will scream about it. imho that is *not* sufficient to keep from Doing The Right Thing. And DTRT usually means designing correctly for the future. I would support a design that is cleanest for the next release, and put backward compatibility a (no so) distant second. Agreed. The nice thing about the info-marker idea is that we'll be able to extend it later, so that more info about the function is stored right where the function text is, and you don't have such a problem with keeping an SQL script file in sync with the function's real definition. I've been wanting to think about the "package" or "module" idea (which others have brought up recently). If this kind of hook gets us more options to support that (e.g. running a routine when the module is first loaded to install new options into "SET key=val") then even better. Lack of hooks of this nature lead us to push datatypes down into the core when if we had full-up module support we could make more things loadable. One idea we had was to define a SET variable or psql command-line switch so that LANGUAGE "C" would mean either old or new style. That way, pg_dump loads from old versions could set the switch before load without changing their code, and we could call all functions C. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Coping with 'C' vs 'newC' function language names
Bruce Momjian [EMAIL PROTECTED] writes: I would support a single symbol to mark the entire object file. In fact, I would require old-style functions to add a symbol, and have new-style functions left alone. That won't fly. There are not that many functions out there, are there? People are having to recompile their C files anyway for the upgrade, don't they? There's a big difference between having to recompile and having to change your source code. For that matter, I think past version updates haven't even forced recompiles of user-defined functions, at least not ones that didn't poke into system innards. We can't get away with requiring a source code change --- people will scream about it. The nice thing about the info-marker idea is that we'll be able to extend it later, so that more info about the function is stored right where the function text is, and you don't have such a problem with keeping an SQL script file in sync with the function's real definition. regards, tom lane
