Re: [PATCHES] Autovacuum cancellation
Tom Lane [EMAIL PROTECTED] writes: I think there's a window where the process waiting directly on autovacuum could have already fired its deadlock check before it was waiting directly on autovacuum. I think you don't understand what that code is doing. If there's an autovac anywhere in the dependency graph, it'll find it. That'll teach me to try to read code from a patch directly without trying to apply it or at least read the original source next to it. I thought I had seen this code recently enough to apply the patch from memory -- clearly not. I assume the right thing happens if multiple deadlock check signals fire for the same autovacuum? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[PATCHES] fix ZIC dependency on postgres.h
I attached patch which removes dependency on postgres.h during ZIC compilation. It was discussed there: http://archives.postgresql.org/pgsql-hackers/2007-10/msg01261.php Zdenek Index: src/include/pgtime.h === RCS file: /zfs_data/cvs_pgsql/cvsroot/pgsql/src/include/pgtime.h,v retrieving revision 1.17 diff -c -r1.17 pgtime.h *** src/include/pgtime.h 4 Aug 2007 19:29:25 - 1.17 --- src/include/pgtime.h 26 Oct 2007 10:14:09 - *** *** 13,18 --- 13,19 #ifndef _PGTIME_H #define _PGTIME_H + #include c.h /* * The API of this library is generally similar to the corresponding Index: src/timezone/ialloc.c === RCS file: /zfs_data/cvs_pgsql/cvsroot/pgsql/src/timezone/ialloc.c,v retrieving revision 1.8 diff -c -r1.8 ialloc.c *** src/timezone/ialloc.c 26 Jan 2007 17:45:42 - 1.8 --- src/timezone/ialloc.c 26 Oct 2007 10:12:51 - *** *** 6,13 * $PostgreSQL: pgsql/src/timezone/ialloc.c,v 1.7 2005/10/15 02:49:51 momjian Exp $ */ - #include postgres.h - #include private.h --- 6,11 Index: src/timezone/localtime.c === RCS file: /zfs_data/cvs_pgsql/cvsroot/pgsql/src/timezone/localtime.c,v retrieving revision 1.17 diff -c -r1.17 localtime.c *** src/timezone/localtime.c 4 Aug 2007 19:29:25 - 1.17 --- src/timezone/localtime.c 26 Oct 2007 10:12:51 - *** *** 12,19 * ([EMAIL PROTECTED]). */ - #include postgres.h - #include fcntl.h #include private.h --- 12,17 Index: src/timezone/scheck.c === RCS file: /zfs_data/cvs_pgsql/cvsroot/pgsql/src/timezone/scheck.c,v retrieving revision 1.7 diff -c -r1.7 scheck.c *** src/timezone/scheck.c 15 Oct 2005 02:49:51 - 1.7 --- src/timezone/scheck.c 26 Oct 2007 10:12:51 - *** *** 6,13 * $PostgreSQL: pgsql/src/timezone/scheck.c,v 1.6 2005/06/20 08:00:51 neilc Exp $ */ - #include postgres.h - #include private.h --- 6,11 Index: src/timezone/zic.c === RCS file: /zfs_data/cvs_pgsql/cvsroot/pgsql/src/timezone/zic.c,v retrieving revision 1.21 diff -c -r1.21 zic.c *** src/timezone/zic.c 1 Feb 2007 19:10:30 - 1.21 --- src/timezone/zic.c 26 Oct 2007 10:12:51 - *** *** 6,13 * $PostgreSQL: pgsql/src/timezone/zic.c,v 1.20 2007/01/26 17:45:42 neilc Exp $ */ - #include postgres.h - #ifdef HAVE_GETOPT_H #include getopt.h #endif --- 6,11 ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Autovacuum cancellation
Alvaro Herrera wrote: /* * Look for a blocking autovacuum. There will only ever * be one, since the autovacuum workers are careful * not to operate concurrently on the same table. */ I think that's a bit unaccurate. You could have multiple autovacuum workers operating on different tables participating in a deadlock. The reason that can't happen is that autovacuum never holds a lock while waiting for another. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] Fix ecpg dependency on postgres.h
I attached the patch which fix ecpg dependency on postgresql.h. This was
discussed on hackers:
http://archives.postgresql.org/pgsql-hackers/2007-10/msg01261.php
Directory pgsql/src/interfaces/ecpg/preproc/parser should be removed and
there is new file pgsql/src/interfaces/ecpg/preproc/parser.c which is
little bit modified version (only different headers are included) of
pgsql/src/backend/parser/parser.c.
Zdenek
diff -rc pgsql/src/interfaces/ecpg/preproc/Makefile pgsql_ecpg/src/interfaces/ecpg/preproc/Makefile
*** pgsql/src/interfaces/ecpg/preproc/Makefile Wed Aug 22 10:20:58 2007
--- pgsql_ecpg/src/interfaces/ecpg/preproc/Makefile Fri Oct 26 09:59:40 2007
***
*** 59,67
c_keywords.o keywords.o preproc.o parser.o: preproc.h
- parser.c: $(top_srcdir)/src/backend/parser/parser.c
- rm -f $@ $(LN_S) $ .
-
distprep: $(srcdir)/preproc.c $(srcdir)/preproc.h $(srcdir)/pgc.c
install: all installdirs
--- 59,64
***
*** 74,80
rm -f '$(DESTDIR)$(bindir)/ecpg$(X)'
clean distclean:
! rm -f *.o ecpg$(X) parser.c
# garbage from partial builds
@rm -f y.tab.c y.tab.h
# garbage from development
--- 71,77
rm -f '$(DESTDIR)$(bindir)/ecpg$(X)'
clean distclean:
! rm -f *.o ecpg$(X)
# garbage from partial builds
@rm -f y.tab.c y.tab.h
# garbage from development
Only in pgsql/src/interfaces/ecpg/preproc: parser
diff -rc pgsql/src/interfaces/ecpg/preproc/parser.c pgsql_ecpg/src/interfaces/ecpg/preproc/parser.c
*** pgsql/src/interfaces/ecpg/preproc/parser.c Fri Oct 26 09:54:55 2007
--- pgsql_ecpg/src/interfaces/ecpg/preproc/parser.c Fri Oct 26 10:01:28 2007
***
*** 0
--- 1,169
+ /*-
+ *
+ * parser.c
+ * Main entry point/driver for PostgreSQL grammar
+ *
+ * Note that the grammar is not allowed to perform any table access
+ * (since we need to be able to do basic parsing even while inside an
+ * aborted transaction). Therefore, the data structures returned by
+ * the grammar are raw parsetrees that still need to be analyzed by
+ * analyze.c and related files.
+ *
+ *
+ * Portions Copyright (c) 1996-2007, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * $PostgreSQL: pgsql/src/backend/parser/parser.c,v 1.70 2007/01/06 19:14:17 tgl Exp $
+ *
+ *-
+ */
+
+ #include extern.h
+ #include preproc.h
+ #include nodes/pg_list.h
+
+
+ List *parsetree; /* result of parsing is left here */
+
+ static bool have_lookahead; /* is lookahead info valid? */
+ static int lookahead_token; /* one-token lookahead */
+ static YYSTYPE lookahead_yylval; /* yylval for lookahead token */
+ static YYLTYPE lookahead_yylloc; /* yylloc for lookahead token */
+
+
+ /*
+ * raw_parser
+ * Given a query in string form, do lexical and grammatical analysis.
+ *
+ * Returns a list of raw (un-analyzed) parse trees.
+ */
+ List *
+ raw_parser(const char *str)
+ {
+ int yyresult;
+
+ parsetree = NIL; /* in case grammar forgets to set it */
+ have_lookahead = false;
+
+ scanner_init(str);
+ parser_init();
+
+ yyresult = base_yyparse();
+
+ scanner_finish();
+
+ if (yyresult)/* error */
+ return NIL;
+
+ return parsetree;
+ }
+
+
+ /*
+ * Intermediate filter between parser and base lexer (base_yylex in scan.l).
+ *
+ * The filter is needed because in some cases the standard SQL grammar
+ * requires more than one token lookahead. We reduce these cases to one-token
+ * lookahead by combining tokens here, in order to keep the grammar LALR(1).
+ *
+ * Using a filter is simpler than trying to recognize multiword tokens
+ * directly in scan.l, because we'd have to allow for comments between the
+ * words. Furthermore it's not clear how to do it without re-introducing
+ * scanner backtrack, which would cost more performance than this filter
+ * layer does.
+ */
+ int
+ filtered_base_yylex(void)
+ {
+ int cur_token;
+ int next_token;
+ YYSTYPE cur_yylval;
+ YYLTYPE cur_yylloc;
+
+ /* Get next token --- we might already have it */
+ if (have_lookahead)
+ {
+ cur_token = lookahead_token;
+ base_yylval = lookahead_yylval;
+ base_yylloc = lookahead_yylloc;
+ have_lookahead = false;
+ }
+ else
+ cur_token = base_yylex();
+
+ /* Do we need to look ahead for a possible multiword token? */
+ switch (cur_token)
+ {
+ case NULLS_P:
+ /*
+ * NULLS FIRST and NULLS LAST must be reduced to one token
+ */
+ cur_yylval = base_yylval;
+ cur_yylloc = base_yylloc;
+ next_token = base_yylex();
+ switch (next_token)
+ {
+ case FIRST_P:
+ cur_token = NULLS_FIRST;
+ break;
+ case LAST_P:
+ cur_token = NULLS_LAST;
+ break;
+ default:
+ /* save the lookahead token for
Re: [PATCHES] fix ZIC dependency on postgres.h
Zdenek Kotala [EMAIL PROTECTED] writes: I attached patch which removes dependency on postgres.h during ZIC compilation. I consider it poor style to have an include file (particularly one that's fairly widely used, like pgtime.h) including any of postgres.h postgres_fe.h or c.h. That's pre-empting a decision that needs to be made on a per-c-file basis. So I fixed it as attached, instead. regards, tom lane Index: ialloc.c === RCS file: /cvsroot/pgsql/src/timezone/ialloc.c,v retrieving revision 1.8 diff -c -r1.8 ialloc.c *** ialloc.c26 Jan 2007 17:45:42 - 1.8 --- ialloc.c26 Oct 2007 13:28:46 - *** *** 6,12 * $PostgreSQL: pgsql/src/timezone/ialloc.c,v 1.8 2007/01/26 17:45:42 neilc Exp $ */ ! #include postgres.h #include private.h --- 6,12 * $PostgreSQL: pgsql/src/timezone/ialloc.c,v 1.8 2007/01/26 17:45:42 neilc Exp $ */ ! #include postgres_fe.h #include private.h Index: localtime.c === RCS file: /cvsroot/pgsql/src/timezone/localtime.c,v retrieving revision 1.17 diff -c -r1.17 localtime.c *** localtime.c 4 Aug 2007 19:29:25 - 1.17 --- localtime.c 26 Oct 2007 13:28:46 - *** *** 12,18 * ([EMAIL PROTECTED]). */ ! #include postgres.h #include fcntl.h --- 12,19 * ([EMAIL PROTECTED]). */ ! /* this file needs to build in both frontend and backend contexts */ ! #include c.h #include fcntl.h Index: scheck.c === RCS file: /cvsroot/pgsql/src/timezone/scheck.c,v retrieving revision 1.7 diff -c -r1.7 scheck.c *** scheck.c15 Oct 2005 02:49:51 - 1.7 --- scheck.c26 Oct 2007 13:28:46 - *** *** 6,12 * $PostgreSQL: pgsql/src/timezone/scheck.c,v 1.7 2005/10/15 02:49:51 momjian Exp $ */ ! #include postgres.h #include private.h --- 6,12 * $PostgreSQL: pgsql/src/timezone/scheck.c,v 1.7 2005/10/15 02:49:51 momjian Exp $ */ ! #include postgres_fe.h #include private.h Index: zic.c === RCS file: /cvsroot/pgsql/src/timezone/zic.c,v retrieving revision 1.21 diff -c -r1.21 zic.c *** zic.c 1 Feb 2007 19:10:30 - 1.21 --- zic.c 26 Oct 2007 13:28:46 - *** *** 6,12 * $PostgreSQL: pgsql/src/timezone/zic.c,v 1.21 2007/02/01 19:10:30 momjian Exp $ */ ! #include postgres.h #ifdef HAVE_GETOPT_H #include getopt.h --- 6,12 * $PostgreSQL: pgsql/src/timezone/zic.c,v 1.21 2007/02/01 19:10:30 momjian Exp $ */ ! #include postgres_fe.h #ifdef HAVE_GETOPT_H #include getopt.h ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] Autovacuum cancellation
Heikki Linnakangas [EMAIL PROTECTED] writes: Alvaro Herrera wrote: /* * Look for a blocking autovacuum. There will only ever * be one, since the autovacuum workers are careful * not to operate concurrently on the same table. */ I think that's a bit unaccurate. You could have multiple autovacuum workers operating on different tables participating in a deadlock. The reason that can't happen is that autovacuum never holds a lock while waiting for another. And that's not true either. It may only want low-grade locks (eg AccessShareLock on a system catalog) but deadlock is nonetheless entirely possible in principle. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] Autovacuum cancellation
Gregory Stark [EMAIL PROTECTED] writes: I assume the right thing happens if multiple deadlock check signals fire for the same autovacuum? Multiple signals shouldn't be a problem, but late-arriving ones could be. It might be worth having autovac explicitly clear QueryCancelPending after it's finished a table, so that a SIGINT sent because of activity on one table couldn't force cancellation of vacuum on the next one. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] Fix ecpg dependency on postgres.h
Zdenek Kotala [EMAIL PROTECTED] writes: I attached the patch which fix ecpg dependency on postgresql.h. This was discussed on hackers: Applied with revisions (again, wrong place to include postgres_fe.h) regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Autovacuum cancellation
Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: I assume the right thing happens if multiple deadlock check signals fire for the same autovacuum? Multiple signals shouldn't be a problem, but late-arriving ones could be. It might be worth having autovac explicitly clear QueryCancelPending after it's finished a table, so that a SIGINT sent because of activity on one table couldn't force cancellation of vacuum on the next one. Ok, committed; I snuck that in as well, but I'm not sure how to test that it works. I adjusted the comments -- I think they're more correct now. I also added a puny paragraph to lmgr/README. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
