Re: [PATCHES] [HACKERS] logging statement levels
Patch applied. --- Bruce Momjian wrote: > Andrew Dunstan wrote: > > >>Here are some options: > > >> > > >>1. change the type of "log_statement" option from boolean to string, > > >>with allowed values of "all, mod, ddl, none" with default "none". > > >>2. same as 1. but make boolean true values synonyms for "all" and > > >>boolean false values synonyms for "none". > > >>3. keep "log_statement" option as now and add a new option > > >>"log_statement_level" with the same options as 1. but default to "all", > > >>which will have no effect unless "log_statement" is true. > > >> > > >> > > > > > >I like 1. > > OK, here is a patch that implements #1. Here is sample output: > > test=> set client_min_messages = 'log'; > SET > test=> set log_statement = 'mod'; > SET > test=> select 1; >?column? > -- > 1 > (1 row) > > test=> update test set x=1; > LOG: statement: update test set x=1; > ERROR: relation "test" does not exist > test=> update test set x=1; > LOG: statement: update test set x=1; > ERROR: relation "test" does not exist > test=> copy test from '/tmp/x'; > LOG: statement: copy test from '/tmp/x'; > ERROR: relation "test" does not exist > test=> copy test to '/tmp/x'; > ERROR: relation "test" does not exist > test=> prepare xx as select 1; > PREPARE > test=> prepare xx as update x set y=1; > LOG: statement: prepare xx as update x set y=1; > ERROR: relation "x" does not exist > test=> explain analyze select 1;; >QUERY PLAN > > >Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.006..0.007 rows=1 > loops=1) >Total runtime: 0.046 ms > (2 rows) > > test=> explain analyze update test set x=1; > LOG: statement: explain analyze update test set x=1; > ERROR: relation "test" does not exist > test=> explain update test set x=1; > ERROR: relation "test" does not exist > > It checks PREPARE and EXECUTE ANALYZE too. The log_statement values are > 'none', 'mod', 'ddl', and 'all'. For 'all', it prints before the query > is parsed, and for ddl/mod, it does it right after parsing using the > node tag (or command tag for CREATE/ALTER/DROP), so any non-parse errors > will print after the log line. > > -- > Bruce Momjian| http://candle.pha.pa.us > [EMAIL PROTECTED] | (610) 359-1001 > + If your life is a hard drive, | 13 Roberts Road > + Christ can be your backup.| Newtown Square, Pennsylvania 19073 > Index: doc/src/sgml/runtime.sgml > === > RCS file: /cvsroot/pgsql-server/doc/src/sgml/runtime.sgml,v > retrieving revision 1.257 > diff -c -c -r1.257 runtime.sgml > *** doc/src/sgml/runtime.sgml 5 Apr 2004 03:02:03 - 1.257 > --- doc/src/sgml/runtime.sgml 6 Apr 2004 03:56:08 - > *** > *** 2121,2132 > > > > ! log_statement (boolean) > > > ! Causes each SQL statement to be logged. The default is > ! off. Only superusers can disable this option if it has been > ! enabled by an administrator. > > > > --- 2121,2141 > > > > ! log_statement (string) > > > ! Controls which SQL statement are logged. Valid values are > ! all, ddl, mod, and > ! none. ddl logs all data definition > ! commands like CREATE, ALTER, and > ! DROP commands. mod logs all > ! ddl statements, plus INSERT, > ! UPDATE, DELETE, TRUNCATE, > ! and COPY FROM. PREPARE and > ! EXPLAIN ANALYZE statements are also considered for > ! appropriate commands. The default is none. Only > ! superusers can reduce the detail of this option if it has been > ! set by an administrator. > > > > Index: src/backend/tcop/postgres.c > === > RCS file: /cvsroot/pgsql-server/src/backend/tcop/postgres.c,v > retrieving revision 1.397 > diff -c -c -r1.397 postgres.c > *** src/backend/tcop/postgres.c 24 Mar 2004 22:40:29 - 1.397 > --- src/backend/tcop/postgres.c 6 Apr 2004 03:56:11 - > *** > *** 87,92 > --- 87,94 > /* flag for logging end of session */ > boolLog_disconnections = false; > > + LogStmtLevel log_statement = LOGSTMT_NONE; > + > /* >* Flags for expensive function optimization -- JMH 3/9/92 >*/ > *** > *** 471,479 > List * > pg_parse_query(const
[PATCHES] LIKE vs regex queries
I have a database with about 250,000 entries in a table, PG 7.5, One of the fields is text. LIKE queries on this field execute much faster than the equivalent regex queries. Is this what you would expect? Should I prefer LIKE to regex? (I'm a regex fan, but the performance hit seems steep.) I've pasted sample output in below. order=# explain select dnum from item where description LIKE '%Ushio%'; QUERY PLAN -- Seq Scan on item (cost=0.00..7330.40 rows=349 width=16) Filter: (description ~~ '%Ushio%'::text) (2 rows) order=# select dnum from item where description LIKE '%Ushio%'; dnum -- B521479 MB105921 MB109239 MB110491 MB111390 MB111983 MB112854 MB115020 MB115020 MB120247 MB121532 (11 rows) Time: 855.540 ms == order=# explain select dnum from item where description ~ 'Ushio'; QUERY PLAN -- Seq Scan on item (cost=0.00..7330.40 rows=349 width=16) Filter: (description ~ 'Ushio'::text) ( order=# select dnum from item where description ~ 'Ushio'; dnum -- B521479 MB105921 MB109239 MB110491 MB111390 MB111983 MB112854 MB115020 MB115020 MB120247 MB121532 (11 rows) Time: 2409.043 ms = Daniel Graham [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] [HACKERS] logging statement levels
Andrew Dunstan wrote: > >>I think I'd prefer that to having it tied to the log_min_error_statement > >>level. But I don't care that much. > >> > >> > > > >OK, at least we understand each other. Right now we don't have any > >special "syntax error" log processing. We have errors logged through > >log_min_error_statement, and mod/ddl through the new log_statement. > > > >I can see a use case for having mod/ddl control of logging, and error > >control of logging, but why would you want to see syntax error queries > >but not other error queries? That's why I think log_min_error_statement > >is sufficient. If we add syntax logging,Thinks wouldn't that conflict with > >log_min_error_statement logging, because those are errors too. Maybe we > >need to add a 'synax' mode to log_min_error_statement above error that > >logs only syntax errors but not others. > > > > > > > > Thinks experiments yes, OK, I agree. Please forgive any > denseness. Not sure if we need another level. No problem. It is good to think through these things to make sure we have everything covered. > Why do we have log_min_error_statement default to PANIC level? Wouldn't > ERROR be a better default? Panic basically means off, meaning we don't print queries that generate errors. Should we print them by default? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] client_encoding in dump file
Pavel Stehule wrote: > Hello > > I send my first patch for PostgreSQL - maybe ugly patch. This patch > generate on top of dump file line with setting of current encoding. Its > useful for languages like czech with more than one wide used encoding. > We need informations about used encoding. FYI, this was fixed in 7.4.2. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] logging statement levels
Bruce Momjian wrote: Andrew Dunstan wrote: Bruce Momjian wrote: Right now we have log_min_error_statement: #log_min_error_statement = panic # Values in order of increasing severity: # debug5, debug4, debug3, debug2, debug1, # info, notice, warning, error, panic(off) which does allow control of printing only statements generating errors, which includes syntax errors. I don't see why this functionality should be mixed in with log_statement. Did you want a 'syntax error' level to log_statement, that would print only statements with syntax errors but not other errors? That doesn't seem very useful to me. It wasn't my idea, but I thought it was a good one. But it would go along with the idea of these settings as a list instead of a hierarchy, e.g.: log_statement = "syntax-errors, ddl, mod" In fact, I liked it so much that I thought "syntax-errors" should be the default instead of "none". I think I'd prefer that to having it tied to the log_min_error_statement level. But I don't care that much. OK, at least we understand each other. Right now we don't have any special "syntax error" log processing. We have errors logged through log_min_error_statement, and mod/ddl through the new log_statement. I can see a use case for having mod/ddl control of logging, and error control of logging, but why would you want to see syntax error queries but not other error queries? That's why I think log_min_error_statement is sufficient. If we add syntax logging,Thinks wouldn't that conflict with log_min_error_statement logging, because those are errors too. Maybe we need to add a 'synax' mode to log_min_error_statement above error that logs only syntax errors but not others. Thinks experiments yes, OK, I agree. Please forgive any denseness. Not sure if we need another level. Why do we have log_min_error_statement default to PANIC level? Wouldn't ERROR be a better default? cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] logging statement levels
Andrew Dunstan wrote: > Bruce Momjian wrote: > > >Right now we have log_min_error_statement: > > > > #log_min_error_statement = panic # Values in order of increasing severity: > > # debug5, debug4, debug3, debug2, debug1, > > # info, notice, warning, error, panic(off) > > > >which does allow control of printing only statements generating errors, > >which includes syntax errors. I don't see why this functionality should > >be mixed in with log_statement. > > > >Did you want a 'syntax error' level to log_statement, that would print > >only statements with syntax errors but not other errors? That doesn't > >seem very useful to me. > > > > > > > > It wasn't my idea, but I thought it was a good one. But it would go > along with the idea of these settings as a list instead of a hierarchy, > e.g.: > > log_statement = "syntax-errors, ddl, mod" > > In fact, I liked it so much that I thought "syntax-errors" should be the > default instead of "none". > > I think I'd prefer that to having it tied to the log_min_error_statement > level. But I don't care that much. OK, at least we understand each other. Right now we don't have any special "syntax error" log processing. We have errors logged through log_min_error_statement, and mod/ddl through the new log_statement. I can see a use case for having mod/ddl control of logging, and error control of logging, but why would you want to see syntax error queries but not other error queries? That's why I think log_min_error_statement is sufficient. If we add syntax logging, wouldn't that conflict with log_min_error_statement logging, because those are errors too. Maybe we need to add a 'synax' mode to log_min_error_statement above error that logs only syntax errors but not others. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [HACKERS] logging statement levels
Bruce Momjian wrote: Right now we have log_min_error_statement: #log_min_error_statement = panic # Values in order of increasing severity: # debug5, debug4, debug3, debug2, debug1, # info, notice, warning, error, panic(off) which does allow control of printing only statements generating errors, which includes syntax errors. I don't see why this functionality should be mixed in with log_statement. Did you want a 'syntax error' level to log_statement, that would print only statements with syntax errors but not other errors? That doesn't seem very useful to me. It wasn't my idea, but I thought it was a good one. But it would go along with the idea of these settings as a list instead of a hierarchy, e.g.: log_statement = "syntax-errors, ddl, mod" In fact, I liked it so much that I thought "syntax-errors" should be the default instead of "none". I think I'd prefer that to having it tied to the log_min_error_statement level. But I don't care that much. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] hint infrastructure setup (v3)
Dear Tom,
> In particular you can't any longer tell the difference between BOOLEAN
> and "boolean" (with quotes), which are not the same thing --- a quoted
> string is never a keyword, per spec. [...]
Ok, so you mean that on -boolean- the lexer returns a BOOLEAN_P token, but
with -"boolean"- it returns an Ident and -boolean- as a lval. Indeed, in
such a case I cannot recognize that simply boolean vs "boolean" if they
are both idents that look the same.
As a matter of fact, this can also be fixed with some post-filtering. Say,
all quoted idents could be returned with a leading " to show it was
dquoted, and the IDENT rules in the parser could remove when it is not
needed anymore to distinguish the case.
Not beautiful, I agree, but my point is that the current number of tokens
and number of states and automaton size are not inherent to SQL but to the
way the lexing/parsing is performed in postgresql.
> The basic point here is that eliminating tokens as you propose will
> result in small changes in behavior, none of which are good or per spec.
> Making the parser automaton smaller would be nice, but not at that
> price.
Ok. I don't want to change the spec. I still stand that it can be done,
although some more twicking is required. It was just a "proof of concept",
not a patch submission. Well, a "proof of concept" must still be a proof;-)
I attach a small patch that solve the boolean vs "boolean" issue, still as
a proof of concept that it is 'doable' to preserve semantics with a
different lexer/parser balance. I don't claim that it should be applied, I
just claim that the automaton size could be smaller, especially by
shortening the "unreserved_keyword" list.
> You have not proven that you can have the same result.
Well, I passed the regression tests, but that does not indeed prove
anything, because these issues are not tested at all.
Maybe you could consider to add the "regression" part of the attached
patcht, which creates a user "boolean" type.
Anyway, my motivation is about "hints" and "advises", and that does not
help a lot to solve these issues.
--
Fabien.*** ./src/backend/parser/gram.y.origTue Apr 6 18:15:39 2004
--- ./src/backend/parser/gram.y Tue Apr 6 17:56:46 2004
***
*** 95,100
--- 95,102
static Node *doNegate(Node *n);
static void doNegateFloat(Value *v);
+ #define clean_dqname(n) (((*(n))!='"')? (n): pstrdup((n)+1))
+
%}
***
*** 336,343
AGGREGATE ALL ALSO ALTER ANALYSE ANALYZE AND ANY ARRAY AS ASC
ASSERTION ASSIGNMENT AT AUTHORIZATION
! BACKWARD BEFORE BEGIN_P BETWEEN BIGINT BINARY BIT
! BOOLEAN_P BOTH BY
CACHE CALLED CASCADE CASE CAST CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
--- 338,345
AGGREGATE ALL ALSO ALTER ANALYSE ANALYZE AND ANY ARRAY AS ASC
ASSERTION ASSIGNMENT AT AUTHORIZATION
! BACKWARD BEFORE BEGIN_P BETWEEN BINARY BIT
! BOTH BY
CACHE CALLED CASCADE CASE CAST CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
***
*** 362,368
ILIKE IMMEDIATE IMMUTABLE IMPLICIT_P IN_P INCLUDING INCREMENT
INDEX INHERITS INITIALLY INNER_P INOUT INPUT_P
! INSENSITIVE INSERT INSTEAD INT_P INTEGER INTERSECT
INTERVAL INTO INVOKER IS ISNULL ISOLATION
JOIN
--- 364,370
ILIKE IMMEDIATE IMMUTABLE IMPLICIT_P IN_P INCLUDING INCREMENT
INDEX INHERITS INITIALLY INNER_P INOUT INPUT_P
! INSENSITIVE INSERT INSTEAD INTERSECT
INTERVAL INTO INVOKER IS ISNULL ISOLATION
JOIN
***
*** 386,398
PRECISION PRESERVE PREPARE PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE
! READ REAL RECHECK REFERENCES REINDEX RELATIVE_P RENAME REPEATABLE REPLACE
RESET RESTART RESTRICT RETURNS REVOKE RIGHT ROLLBACK ROW ROWS
RULE
SCHEMA SCROLL SECOND_P SECURITY SELECT SEQUENCE
SERIALIZABLE SESSION SESSION_USER SET SETOF SHARE
! SHOW SIMILAR SIMPLE SMALLINT SOME STABLE START STATEMENT
STATISTICS STDIN STDOUT STORAGE STRICT_P SUBSTRING SYSID
TABLE TEMP TEMPLATE TEMPORARY THEN TIME TIMESTAMP
--- 388,400
PRECISION PRESERVE PREPARE PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE
! READ RECHECK REFERENCES REINDEX RELATIVE_P RENAME REPEATABLE REPLACE
RESET RESTART RESTRICT RETURNS REVOKE RIGHT ROLLBACK ROW ROWS
RULE
SCHEMA SCROLL SECOND_P SECURITY SELECT SEQUENCE
SERIALIZABLE SESSION SESSION_USER SET SETOF SHARE
! SHOW SIMILAR SIMPLE SOME STABLE START STATEMENT
STATISTICS STDIN STDOUT STORAGE STRICT_P SUBSTRING SYSID
TABLE TEMP TEMPLATE TEMPORARY THEN TIME TIMESTAMP
***
*** 959,965
}
| IDENT
{
! $$ = makeS
[PATCHES] Translation updates for 7.5: pg_resetxlog-ru;pg_controldata-ru;pg_dump-ru;initdb-ru
Hello, Please install attached translation updates. This is for 7.5. initdb to to about ~50% the rest is complete $ msgfmt -cv initdb-ru.po 37 translated messages, 35 untranslated messages. $ msgfmt -cv pg_controldata-ru.po 41 translated messages. $ msgfmt -cv pg_dump-ru.po 335 translated messages. $ msgfmt -cv pg_resetxlog-ru.po 58 translated messages. $ msgfmt -cv psql-ru.po 464 translated messages. Thanks, -s pg_resetxlog-ru.po.gz;pg_controldata-ru.po.gz;pg_dump-ru.po.gz;initdb-ru.po.gz;psql-ru.po.gz pg_resetxlog-ru.po.gz Description: GNU Zip compressed data pg_controldata-ru.po.gz Description: GNU Zip compressed data pg_dump-ru.po.gz Description: GNU Zip compressed data initdb-ru.po.gz Description: GNU Zip compressed data psql-ru.po.gz Description: GNU Zip compressed data ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] logging statement levels
Andrew Dunstan wrote: > >>However, I think with a little extra work it might be possible to have both. > >> > >> > > > >Right now, the way it is done, only a real syntax error skips logging. > >If you referenced an invalid table or something, it does print the log > >just before the invalid table name mention. > > > >How would we test the command type before hitting a syntax error? I > >can't think of a way, and I am not sure it would even be meaningful. > > > > > > > > I agree that you can't test the statement type on a parse error. But > that doesn't mean to me that "mod" should suppress logging statements > with syntax errors. In fact, after the discussion surrounding this I > thought the consensus was to have these things as additive rather than > just one level selected. It is additive in that 'mod' also includes 'ddl' queries. > How to do it in the order you prefer? I would trap the parse error and > log the statement before emitting the error log. > > If I find a simple way I'll submit a further patch. > > Certainly your patch contains the guts of what needs to be done in any case. Right now we have log_min_error_statement: #log_min_error_statement = panic # Values in order of increasing severity: # debug5, debug4, debug3, debug2, debug1, # info, notice, warning, error, panic(off) which does allow control of printing only statements generating errors, which includes syntax errors. I don't see why this functionality should be mixed in with log_statement. Did you want a 'syntax error' level to log_statement, that would print only statements with syntax errors but not other errors? That doesn't seem very useful to me. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] logging statement levels
Andrew Dunstan wrote: > Bruce Momjian wrote: > > >Andrew Dunstan wrote: > > > > > >>Unless I'm missing something, this patch has the effect that with values > >>of "ddl" or "mod" for log_statement, a statement with a parse error > >>will not be logged, which was what I hoped to avoid. > >> > >> > > > >Right. The query type can't be determined during a syntax error because > >the parser couldn't identify the supplied command. I think that is > >fine. > > > >What it does allow is to for 'all' to display the command before the > >syntax error. > > > > > > > > If I had to make a choice I'd go the other way. Uh, what other way? > However, I think with a little extra work it might be possible to have both. Right now, the way it is done, only a real syntax error skips logging. If you referenced an invalid table or something, it does print the log just before the invalid table name mention. How would we test the command type before hitting a syntax error? I can't think of a way, and I am not sure it would even be meaningful. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] [HACKERS] logging statement levels
Bruce Momjian wrote: Andrew Dunstan wrote: Bruce Momjian wrote: Andrew Dunstan wrote: Unless I'm missing something, this patch has the effect that with values of "ddl" or "mod" for log_statement, a statement with a parse error will not be logged, which was what I hoped to avoid. Right. The query type can't be determined during a syntax error because the parser couldn't identify the supplied command. I think that is fine. What it does allow is to for 'all' to display the command before the syntax error. If I had to make a choice I'd go the other way. Uh, what other way? reverse the order rather than suppress the message. However, I think with a little extra work it might be possible to have both. Right now, the way it is done, only a real syntax error skips logging. If you referenced an invalid table or something, it does print the log just before the invalid table name mention. How would we test the command type before hitting a syntax error? I can't think of a way, and I am not sure it would even be meaningful. I agree that you can't test the statement type on a parse error. But that doesn't mean to me that "mod" should suppress logging statements with syntax errors. In fact, after the discussion surrounding this I thought the consensus was to have these things as additive rather than just one level selected. How to do it in the order you prefer? I would trap the parse error and log the statement before emitting the error log. If I find a simple way I'll submit a further patch. Certainly your patch contains the guts of what needs to be done in any case. cheers andrew cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] logging statement levels
Bruce Momjian wrote: Andrew Dunstan wrote: Unless I'm missing something, this patch has the effect that with values of "ddl" or "mod" for log_statement, a statement with a parse error will not be logged, which was what I hoped to avoid. Right. The query type can't be determined during a syntax error because the parser couldn't identify the supplied command. I think that is fine. What it does allow is to for 'all' to display the command before the syntax error. If I had to make a choice I'd go the other way. However, I think with a little extra work it might be possible to have both. I will look into it at some stage. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] O(samplesize) tuple sampling, proof of concept
On Mon, 05 Apr 2004 18:30:29 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >> noise-contributing factors. > >I think it would have to be visibility-bit updates. Can you try it with >a pre-vacuumed relation, so that there is no slowdown for updates? I'd like to avoid VACUUM to keep the dead tuples. Otherwise we'd have nothing to judge the quality of the row count estimation. SELECT count(*) ... should do as well. And I'll also issue a CHECKPOINT to make sure that the following ANALYSE doesn't have to write out dirty pages. >Yeah, so I managed to read it anyway ;-). It's the ones with >intricately intermixed "-" and "+" that I find difficult to follow. Vim nicely marks "+" and "-" lines in different colours. That makes you read -u diffs almost like a newspaper, your eyes automatically ignore lines that have the wrong colour. I can't get myself used to reading -c diffs. Jumping up and down to find corresponding lines makes me nervous. Anyway, just a matter of taste ... >Duh, you're right --- I was thinking that the old code doesn't need a >qsort, but it does. This seems a tad annoying considering that we know >the tuples were inserted into the pool in increasing order. I wonder if >it's worth using a more complex data structure to keep track of both >orders at once? I suppose that could easily wind up costing more than >the qsort though ... The least complex structure I can think of is a doubly linked list combined with an array. This can be done later, if we find it's worth it (which I doubt). Servus Manfred ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] hint infrastructure setup (v3)
> >>(b) write a new "recursive descendant" parser, and drop "gram.y"
>
> er, that's "recursive descent" :-)
Sorry for my French version.
> Well, unless you are a serious masochist,
I'm not a masochist;-) I'm arguing about where hint should/could be put.
> In fact, considering this thread I'm not sure any of the suggested steps
> will achieve Fabien's goal. ISTM that a smarter "training wheels"
> frontend, perhaps with some modest backend improvements, is likely to
> have better results. ("Oh, you found an error near there - now what do I
> suggest belongs there?")
I cannot see what you're suggesting "practically" as a frontend.
I don't think having another parser next to the first one for better error
messages is a serious option? I would not like to put another parser that
need to be kept synchronized with the first one. So either it is
integrated or linked with the current parser, or it is not there?
Out of the parser, the only information is the offending token (embedded
in the error string) and the character number in the string, that is quite
small a clue to give a hint.
--
Fabien Coelho - [EMAIL PROTECTED]
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [PATCHES] hint infrastructure setup (v3)
Dear Tom, > > I join a small proof-of-concept patch to drop some tokens out of the > > parser. > > I believe these were treated this way *specifically* because of the > keyword-is-not-an-identifier issue. SQL99 calls out most of these > as being keywords: Well, I think that the "reserved keywords" are fine as tokens in a lexer/parser, but think that the "unreserved keywords" should be dropped of the token status if possible. > and if we don't treat them as keywords then we will have a couple of > problems. One is case-conversion issues in locales where the standard > downcasing is not an extension of ASCII (Turkish is the only one I know > of offhand). Do you mean it should use an ASCII-only strcasecmp, not a possibly "localised" version? I agree, but this is just a "proof of concept" patch to show that you don't need so many tokens in the parser. > Another is that depending on where you put the renaming that this patch > removes without replacing :-(, I do not understand your point. It seems to me that the renaming is performed when a type name is expected? The "boolean" keyword (not token) is translated to system "bool" type in the GenericType rule?? ??? > it would be possible for the renaming transformation to get applied to > user-defined types with similar names, or for user-defined types to > unexpectedly shadow system definitions. I don't think that the patch changes the result of the parsing. It drops *TOKENS* out of the lexer, but they are still *KEYWORDS*, although they are not explicitly in the lexer list. "keyword.c" deals with tokens, the file name was ill-chosen. If you think that keywords can only be lexical tokens, then you end-up with an automaton larger than necessary, IMVHO. Note that the removed tokens are still "keywords" as they are treated *especially* anyway. It is not a semantical transformation. Also, if you don't want these names as candidate function names, they could be filtered out at some other point in the parser. They really don't need to be special tokens. My point is that you can have the very same *semantical* result with a smaller automaton if you chose a different trade-off within the lexer/parser/post filtering. I don't want to change the language. > The former would be surprising and the latter would violate the spec. I'm really not sure this is the case with the patch I sent. > Check the archives; I'm sure this was discussed in the 7.3 development > cycle and we concluded that treating these names as keywords was the > only practical solution. Hmmm... I can check the archive, but I cannot see how different the language is with the patch. Maybe there is a missing filter out, or strcasecmp is not the right version, but no more. I think it is a small technical issue in the parser internals, and has nothing to do with great principles and whether this or that is a keyword. It's about what keywords need to be tokens. -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] logging statement levels
Andrew Dunstan wrote: > > Unless I'm missing something, this patch has the effect that with values > of "ddl" or "mod" for log_statement, a statement with a parse error > will not be logged, which was what I hoped to avoid. Right. The query type can't be determined during a syntax error because the parser couldn't identify the supplied command. I think that is fine. What it does allow is to for 'all' to display the command before the syntax error. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] WIN32 psql Ctrl+C support
Where are we on this patch? --- Magnus Hagander wrote: > You need to make all variable access (including libpq, I think) in the > handler threadsafe. The control handler will execute on a different > thread from the main one (see > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc > /base/handlerroutine.asp). > > One way to do this could be to have the handler just set an event or a > variable, and then poll this one in the main thread. For more > information about these issues, see recent threads on signal handling on > pgsql-hackers-win32 (which deals with the server, but much the same > issues). > > > //Magnus > > > > -Original Message- > > From: Ludek Finstrle [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, January 14, 2004 1:03 PM > > To: [EMAIL PROTECTED] > > Subject: [PATCHES] WIN32 psql Ctrl+C support > > > > > > Hello, > > > > I write Ctrl+C support for Win32 psql client. I create diff > > againist CVS snapshoted today. > > > > Comments are welcome > > > > Luf > > > > ---(end of broadcast)--- > TIP 6: Have you searched our list archives? > >http://archives.postgresql.org > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] hint infrastructure setup (v3)
Fabien COELHO <[EMAIL PROTECTED]> writes: >> Another is that depending on where you put the renaming that this patch >> removes without replacing :-(, > I do not understand your point. It seems to me that the renaming is > performed when a type name is expected? The "boolean" keyword (not token) > is translated to system "bool" type in the GenericType rule?? ??? I mean that you removed functionality without putting it back; the modified parser will fail to recognize BOOLEAN as a type name at all, because it doesn't match "bool" which is in the catalogs. (And changing the entry to "boolean" is not a solution, it just moves the problem.) I assume you intended to handle this by doing the substitutions in type name lookup elsewhere in the parser, but I don't think that is a valid solution, because there is no longer enough information. In particular you can't any longer tell the difference between BOOLEAN and "boolean" (with quotes), which are not the same thing --- a quoted string is never a keyword, per spec. Possibly a better example than boolean is the REAL => pg_catalog.float4 transformation. If a user has defined his own type named foo.real, he ought to be able to refer to it as "real" (with quotes) and not get messed up by the keyword transformation. I think our original motivation for converting all these things to keywords was the realization that pg_dump would in fact screw up and fail to dump such a type definition correctly if "real" wasn't recognized as conflicting with a keyword (which is what prompts pg_dump to stick quotes on). The basic point here is that eliminating tokens as you propose will result in small changes in behavior, none of which are good or per spec. Making the parser automaton smaller would be nice, but not at that price. > My point is that you can have the very same *semantical* result with a > smaller automaton if you chose a different trade-off within the > lexer/parser/post filtering. I don't want to change the language. You have not proven that you can have the same result. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [BUGS] COPY allows parameters which corrupt output
Kris Jurka wrote:
>
> COPY allows DELIMITER and NULL to be specified which don't allow the data
> to be copied back in properly. Using any delimiter that could be part of
> a backslash escape sequence (\n \0 \t) will break if any of the data has a
> character matching the delimiter because it will be escaped and then be
> read as a special character.
>
> It also allows DELIMITER and NULL to overlap. No character in the NULL
> specification should be the DELIMITER.
The attached applied patch throws an error if the delimiter appears in
the COPY NULL string:
test=> copy pg_language to '/tmp/x' with delimiter '|';
COPY
test=> copy pg_language to '/tmp/x' with delimiter '|' null '|x';
ERROR: COPY delimiter must not appear in the NULL specification
test=> copy pg_language from '/tmp/x' with delimiter '|' null '|x';
ERROR: COPY delimiter must not appear in the NULL specification
It also throws an error if it conflicts with the default NULL string:
test=> copy pg_language to '/tmp/x' with delimiter '\\';
ERROR: COPY delimiter must not appear in the NULL specification
test=> copy pg_language to '/tmp/x' with delimiter '\\' NULL 'x';
COPY
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: src/backend/commands/copy.c
===
RCS file: /cvsroot/pgsql-server/src/backend/commands/copy.c,v
retrieving revision 1.218
diff -c -c -r1.218 copy.c
*** src/backend/commands/copy.c 10 Feb 2004 01:55:24 - 1.218
--- src/backend/commands/copy.c 6 Apr 2004 13:16:35 -
***
*** 780,785
--- 780,793
errmsg("COPY delimiter must be a single character")));
/*
+* Don't allow the delimiter to appear in the null string.
+*/
+ if (strchr(null_print, delim[0]) != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+errmsg("COPY delimiter must not appear in the NULL
specification")));
+
+ /*
* Don't allow COPY w/ OIDs to or from a table without them
*/
if (oids && !rel->rd_rel->relhasoids)
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [PATCHES] [HACKERS] logging statement levels
Unless I'm missing something, this patch has the effect that with values of "ddl" or "mod" for log_statement, a statement with a parse error will not be logged, which was what I hoped to avoid. cheers andrew Bruce Momjian wrote: Andrew Dunstan wrote: Here are some options: 1. change the type of "log_statement" option from boolean to string, with allowed values of "all, mod, ddl, none" with default "none". 2. same as 1. but make boolean true values synonyms for "all" and boolean false values synonyms for "none". 3. keep "log_statement" option as now and add a new option "log_statement_level" with the same options as 1. but default to "all", which will have no effect unless "log_statement" is true. I like 1. OK, here is a patch that implements #1. Here is sample output: test=> set client_min_messages = 'log'; SET test=> set log_statement = 'mod'; SET test=> select 1; ?column? -- 1 (1 row) test=> update test set x=1; LOG: statement: update test set x=1; ERROR: relation "test" does not exist test=> update test set x=1; LOG: statement: update test set x=1; ERROR: relation "test" does not exist test=> copy test from '/tmp/x'; LOG: statement: copy test from '/tmp/x'; ERROR: relation "test" does not exist test=> copy test to '/tmp/x'; ERROR: relation "test" does not exist test=> prepare xx as select 1; PREPARE test=> prepare xx as update x set y=1; LOG: statement: prepare xx as update x set y=1; ERROR: relation "x" does not exist test=> explain analyze select 1;; QUERY PLAN Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.006..0.007 rows=1 loops=1) Total runtime: 0.046 ms (2 rows) test=> explain analyze update test set x=1; LOG: statement: explain analyze update test set x=1; ERROR: relation "test" does not exist test=> explain update test set x=1; ERROR: relation "test" does not exist It checks PREPARE and EXECUTE ANALYZE too. The log_statement values are 'none', 'mod', 'ddl', and 'all'. For 'all', it prints before the query is parsed, and for ddl/mod, it does it right after parsing using the node tag (or command tag for CREATE/ALTER/DROP), so any non-parse errors will print after the log line. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] logging statement levels
Andrew Dunstan wrote:
> >>Here are some options:
> >>
> >>1. change the type of "log_statement" option from boolean to string,
> >>with allowed values of "all, mod, ddl, none" with default "none".
> >>2. same as 1. but make boolean true values synonyms for "all" and
> >>boolean false values synonyms for "none".
> >>3. keep "log_statement" option as now and add a new option
> >>"log_statement_level" with the same options as 1. but default to "all",
> >>which will have no effect unless "log_statement" is true.
> >>
> >>
> >
> >I like 1.
OK, here is a patch that implements #1. Here is sample output:
test=> set client_min_messages = 'log';
SET
test=> set log_statement = 'mod';
SET
test=> select 1;
?column?
--
1
(1 row)
test=> update test set x=1;
LOG: statement: update test set x=1;
ERROR: relation "test" does not exist
test=> update test set x=1;
LOG: statement: update test set x=1;
ERROR: relation "test" does not exist
test=> copy test from '/tmp/x';
LOG: statement: copy test from '/tmp/x';
ERROR: relation "test" does not exist
test=> copy test to '/tmp/x';
ERROR: relation "test" does not exist
test=> prepare xx as select 1;
PREPARE
test=> prepare xx as update x set y=1;
LOG: statement: prepare xx as update x set y=1;
ERROR: relation "x" does not exist
test=> explain analyze select 1;;
QUERY PLAN
Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.006..0.007 rows=1
loops=1)
Total runtime: 0.046 ms
(2 rows)
test=> explain analyze update test set x=1;
LOG: statement: explain analyze update test set x=1;
ERROR: relation "test" does not exist
test=> explain update test set x=1;
ERROR: relation "test" does not exist
It checks PREPARE and EXECUTE ANALYZE too. The log_statement values are
'none', 'mod', 'ddl', and 'all'. For 'all', it prints before the query
is parsed, and for ddl/mod, it does it right after parsing using the
node tag (or command tag for CREATE/ALTER/DROP), so any non-parse errors
will print after the log line.
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: doc/src/sgml/runtime.sgml
===
RCS file: /cvsroot/pgsql-server/doc/src/sgml/runtime.sgml,v
retrieving revision 1.257
diff -c -c -r1.257 runtime.sgml
*** doc/src/sgml/runtime.sgml 5 Apr 2004 03:02:03 - 1.257
--- doc/src/sgml/runtime.sgml 6 Apr 2004 03:56:08 -
***
*** 2121,2132
! log_statement (boolean)
! Causes each SQL statement to be logged. The default is
! off. Only superusers can disable this option if it has been
! enabled by an administrator.
--- 2121,2141
! log_statement (string)
! Controls which SQL statement are logged. Valid values are
! all, ddl, mod, and
! none. ddl logs all data definition
! commands like CREATE, ALTER, and
! DROP commands. mod logs all
! ddl statements, plus INSERT,
! UPDATE, DELETE, TRUNCATE,
! and COPY FROM. PREPARE and
! EXPLAIN ANALYZE statements are also considered for
! appropriate commands. The default is none. Only
! superusers can reduce the detail of this option if it has been
! set by an administrator.
Index: src/backend/tcop/postgres.c
===
RCS file: /cvsroot/pgsql-server/src/backend/tcop/postgres.c,v
retrieving revision 1.397
diff -c -c -r1.397 postgres.c
*** src/backend/tcop/postgres.c 24 Mar 2004 22:40:29 - 1.397
--- src/backend/tcop/postgres.c 6 Apr 2004 03:56:11 -
***
*** 87,92
--- 87,94
/* flag for logging end of session */
boolLog_disconnections = false;
+ LogStmtLevel log_statement = LOGSTMT_NONE;
+
/*
* Flags for expensive function optimization -- JMH 3/9/92
*/
***
*** 471,479
List *
pg_parse_query(const char *query_string)
{
! List *raw_parsetree_list;
! if (log_statement)
ereport(LOG,
(errmsg("statement: %s", query_string)));
--- 473,482
List *
pg_parse_query(const char *query_string)
{
! List *raw_parsetree_list,
