Re: [RRR] [HACKERS] Seeking Mentors for Funded Reviewers
Hi, I would like to be Mentor for Funded Reviewers. My mission will be: 1) We are empowered to create a better world together. 2) Together we co-create our existence. 3) Together we make Postgresql project a success. I am looking for long and fruitful association with Postgresql. I will require to get training in technical, functional and culture of postgresql. Pl let me know, if you decide positively. Regards, Vijay. - Original Message - From: Joshua D. Drake j...@commandprompt.com Date: Thursday, January 27, 2011 2:00 am Subject: Re: [RRR] [HACKERS] Seeking Mentors for Funded Reviewers To: Robert Haas robertmh...@gmail.com Cc: Richard Broersma richard.broer...@gmail.com, Simon Riggs si...@2ndquadrant.com, Josh Berkus j...@agliodbs.com, pgsql-rrreview...@postgresql.org, postgres hackers pgsql-hackers@postgresql.org On Wed, 2011-01-26 at 14:15 -0500, Robert Haas wrote: On Wed, Jan 26, 2011 at 1:32 PM, Richard Broersma richard.broer...@gmail.com wrote: On Wed, Jan 26, 2011 at 3:12 AM, Simon Riggs si...@2ndquadrant.com wrote: You're paying the reviewers; are you paying the mentors? The answer to this question is that we can fund mentor (teacher). However, the amount to fund a mentor would be significantly less that the amount to fund a reviewer (student). The mentors are part of the educational process. Usually, in an educational process, it's the teachers who get paid, and the students who have to pay to get educated. I realize this is somewhat different because we want to encourage people to get involved in the project, but it still seems weird. Not somewhat, completely. Most of the teachers we have are already getting paid to work on PostgreSQL. There are some exceptions of coursebut if you look at the list of people that are qualified to actuallyreview code, they are getting paid *for PostgreSQL*. Now, that isn't to say you don't bring up a good point, you do. I thinkit may be worthwhile to find a way to also compensate mentors but as you say the goal here is encourage people to get involved. However there is the underlying goal of educating future PostgreSQL contributors, and let's face it --- reviewing code sucks and money is a great motivator (especially in today's economy or if you are a starving student). And I actually kind of agree with David Fetter. Aside from the scenario he mentioned (people who don't get paid stop volunteering, a phenomenon that has been documented to occur in other contexts), You have people that are in it for the money. There is nothing wrong with that. Hopefully through this grant they will gain enough skill and public notice to pick up a job where they might be able to give back to the community on a paid basis (probably not, but maybe). If people stop volunteering cause there is no money, then we care why? They are likely not vested in the community anyway. Either way, the mission has been accomplished. They were paid to be educated and learn the review/commitfest process, they did so. If they wish to move on, that's up to them. Do we want them to stay? Of course! However, I fail how to see the concern has anything to do with the grant process. there's also the problem that people might sign up to get the money but then do a lousy job. Well that is the risk we all face and if the mentor feedback was that the person did a lousy job (let's assume they were just lazy, not that they tried really hard but weren't up to the task), then they would risk ever receiving future grants. People sometimes do a lousy job now too, but at least we can count on the fact that everyone who signs up to do it has some intrinsic motivation. I think anyone who is going to make it through a grant process specifically for this purpose is going to have some intrinsic motivationbeyond money. We aren't talking about shelling out 50k here. Sincerely, Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: In pg_test_fsync, use K(1024) rather than k(1000) for write size units.
2011/1/27 Bruce Momjian br...@momjian.us: Bruce Momjian wrote: Peter Eisentraut wrote: We use small k in postgresql.conf, so pg_test_fsync should use the same. Using kB would be more accurate in any case. OK, done with the attached applied patch. FYI, I had used 'k' because this page suggests that k is 1000 and K is 1024, at least by the JEDEC memory standards: http://en.wikipedia.org/wiki/Kilo I can't find any reference to that on this page? The following does indeed say: URL:http://en.wikipedia.org/wiki/JEDEC_memory_standards quote kilo (K): A multiplier equal to 1,024 [..] The specification notes that these prefixes are included in the document only to reflect common usage. It refers to the IEEE/ASTM SI 10-1997 standard as stating, that this practice frequently leads to confusion and is deprecated. /quote If you want to make the difference explicit, consider using KiB (1024, note the extra i) vs. kB (1000); although doing so is probably not consistent with any other uses in PostgreSQL. URL:http://en.wikipedia.org/wiki/Kibibyte quote The unit symbol for the kibibyte is KiB. The unit was established by the International Electrotechnical Commission (IEC) in 1999 and has been accepted for use by all major standards organizations. /quote Nicolas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Query Optimizer + Parallel Operators
On 26.01.2011 16:46, Felix Schmidt @ Oracle wrote: Everybody, I'm interested in the query optimizer of PostgreSQL DB. Where could I find useful documentation or could you send me a pointer in the source code? The relevant source code is in src/backend/optimizer directory. If you google around, you'll find introductory presentations, but I can't recommend any particular one. What kind of parallelism does PostgreSQL use for operators, like selection or join? The short answer is none. Each PostgreSQL backend is a one single-threaded process, one query will only utilize one CPU (http://wiki.postgresql.org/wiki/FAQ#How_does_PostgreSQL_use_CPU_resources.3F). If you search the archives, you'll find discussion on how it might one day be improved, but nothing concrete has been done. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] arrays as pl/perl input arguments [PATCH]
Hi, On Jan 27, 2011, at 9:31 AM, Alex Hunsaker wrote: Find attached v3 of the patch. changes include: - fix deep recursion due to accidental reversal of check in encode_array_literal - add proper support for stringifying composite/row types. I did not find a good way to quote these from the perl on the fly, so instead we compute it the same way we used to and store the string inside the new object along with the array :(. - misc whitespace and code touchups pg_to_perl_arrays_v3.patch.gz Nice improvement. It passes all the regression tests on my OS X system. I have only a minor suggestion, I think is_array is worth mentioning in the utility functions chapter of the pl/perl documentation, it would be also more clear to use it in regression tests as opposed to manually checking whether the ref is equal to 'PostgreSQL::InServer::ARRAY'. /A -- Alexey Klyukin The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] make -j2 error in ecpg/pgtypeslib in current GIT
Hi, I get reproducible make -j2 compile error in ecpg/pgtypeslib. The problem is that after this below is executed: rm -f pgstrcasecmp.c ln -s ../../../../src/port/pgstrcasecmp.c . the compilation of pgstrcasecmp.c doesn't happen and this fails: ar crs libpgtypes.a numeric.o datetime.o common.o dt_common.o timestamp.o interval.o pgstrcasecmp.o ar: pgstrcasecmp.o: No such file or directory The above happens if I run make -j2 from the toplevel directory after ./configure. The second run passes. With make clean ; make -j2 in src/interfaces/ecpg the same error occurs again. If I run make without -j2 then the compile failure doesn't happen. The system is an uptodate Fedora 14. Best regards, Zoltán Böszörményi -- -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt, Austria Web: http://www.postgresql-support.de http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [RRR] [HACKERS] Seeking Mentors for Funded Reviewers
On Thu, Jan 27, 2011 at 3:49 AM, ghatpa...@vsnl.net wrote: Hi, I would like to be Mentor for Funded Reviewers. My mission will be: 1) We are empowered to create a better world together. 2) Together we co-create our existence. 3) Together we make Postgresql project a success. I am looking for long and fruitful association with Postgresql. I will require to get training in technical, functional and culture of postgresql. Pl let me know, if you decide positively. Regards, Vijay. I guess it's not up to me anyway, but this seems a bit odd considering that I can't recall you ever reviewing a patch yourself. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
Hi, I have a talk with MinGW developer, because I am not so familiar with the UNIX configure and build system, can you help to resolve the problem please. jon_y: XiaoboGu: try actually reading config.log 20:18 jon_y: look for winsock2.h in it 20:18 jon_y: you should find it, along with the error nearby 20:19 tjtag: thats fixed it, will do that also ktietz 20:20 XiaoboGu: d:/amber/devtool/mingw64-1.0-20100913/lib/gcc/../../x86_64-w64-mingw32/include/winsock2.h:1035:37: note: previous declaration of 'accept' was here 20:21 jon_y: like I said, the accept() probe doesn't work due to difference in expectation 20:23 tjtag: brb 20:23 XiaoboGu: the path is OK 20:24 XiaoboGu: I really can't understand this 20:24 jon_y: its probing accept() for known implementations 20:24 XiaoboGu: do you mean the configure script has a problem 20:25 jon_y: none of them match the one in winsock2.h 20:25 jon_y: I don't know, it could be mingw-w64's fault too 20:25 jon_y: please contact psql and ask for a list of accept() implementations checked On Tue, Jan 25, 2011 at 10:54 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/25/2011 09:15 AM, Xiaobo Gu wrote: Hi Andrew, The config.log is as following So what is the declaration of accept at d:/amber/devtool/rtools/mingw64/lib/gcc/../../x86_64-w64-mingw32/include/winsock2.h:1035:37: ? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] COPYRIGHT vs psql \copyright
It seems that psql's \copyright haven't been kept up-to-date with the changes to COPYRIGHT file around 2001. psql \copyright says: PostgreSQL Data Base Management System Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group This software is based on Postgres95, formerly known as Postgres, which contains the following notice: Portions Copyright(c) 1994, Regents of the University of California While COPYRIGHT says: PostgreSQL Database Management System (formerly known as Postgres, then as Postgres95) Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group Portions Copyright (c) 1994, The Regents of the University of California The rest of the text is identical, except for line-wrapping and whitespace. I'll update psql's \copyright output to match COPYRIGHT. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions support for pg_dump, patch v27
On Thu, Jan 27, 2011 at 20:01, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Ok, done. Of course, it solves the whole problem Itagaki had with adminpack because we stop relying on dependencies to get it right now. I found pg_restore with -c option fails when an extension is created in pg_catalog. Since pg_catalog is an implicit search target, so we might need the catalog to search_path explicitly. Note that I can restore adminpack with not errors because it has explicit pg_catalog. prefix in the installer script. postgres=# CREATE EXTENSION cube WITH SCHEMA pg_catalog; $ pg_dump -Fc postgres postgres.dump $ pg_restore -d postgres -c postgres.dump pg_restore: [archiver (db)] Error while PROCESSING TOC: pg_restore: [archiver (db)] Error from TOC entry 1; 3996 16678 EXTENSION cube pg_restore: [archiver (db)] could not execute query: ERROR: no schema has been selected to create in BTW, I have a minor comments for the code. extern bool extension_relocatable_p(Oid ext_oid); What is _p ? Also, we could make the function static because it is used only in extension.c. -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] setlocale and gettext in Postgres
Hi all, I see now the following lines in libintl.h of version 0.18.1.1 which didn't exist in 0.17 version. /* Support for the locale chosen by the user. */ #if (defined __APPLE__ defined __MACH__) || defined _WIN32 || defined __WIN32__ || defined __CYGWIN__ #undef setlocale #define setlocale libintl_setlocale extern char *setlocale (int, const char *); . . The macro may cause a trouble especially on Windows. Though libintl_setlocale() calls setlocale() internally, what the Mingw version of libintl library calls is the one in msvcrt.dll not in the libraries which main programs link. Unfortunately locale-aware functions like printf() calls in main programs would see C locales not the locales set by libintl_setlocale(). Attached is a patch to disable the macro on Windows. regards, Hiroshi Inoue *** port.h.orig 2011-01-15 05:29:13.43600 +0900 --- port.h 2011-01-23 21:56:19.55400 +0900 *** *** 176,181 --- 176,188 #ifdef printf #undef printf #endif + /* + * Versions of libintl = 0.18? try to replace setlocale() with a macro + * to their own versions. Disable the macro, if it exists. + */ + #if defined(setlocale) defined(WIN32) + #undef setlocale + #endif extern intpg_vsnprintf(char *str, size_t count, const char *fmt, va_list args); extern int -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Allowing multiple concurrent base backups
On Tue, Jan 25, 2011 at 1:02 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Jan 25, 2011 at 6:02 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, perhaps the code would be more readable if instead of the forcePageWrites counter that counts exclusive and non-exclusive backups, and an exclusiveBackup boolean indicating if one of the in-progress backups is an exclusive one, we had a counter that only counts non-exclusive backups, plus a boolean indicating if an exclusive backup is in progress in addition to them. Attached is a patch for that (against master branch, including only xlog.c). I read this patch and previous-posted one. Those look good. Comments: + * do_pg_start_backup is the workhorse of the user-visible pg_stop_backup() + * function. Typo: s/do_pg_start_backup/do_pg_stop_backup It's helpful to explain about this behavior in pg_basebackup.sgml or elsewhere. When I read the patch, I found that pg_stop_backup removes the backup history file as soon as it creates the file, if archive_mode is not enabled. This looks like oversight. We should prevent pg_stop_backup from removing the fresh history file? Or we should prevent pg_stop_backup from creating the history file from the beginning since it's not used at all if archiving is disabled? (If archiving is enabled, the history file can be used to clean the archived files up). Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sepgsql contrib module
2011/1/27 KaiGai Kohei kai...@ak.jp.nec.com: - Error messages become obtaining %m, when the error was originated from the libselinux interfaces. It will provides DBA a hint why interactions with SELinux does not work well. No space before the : %m, please. Also this word looks like a typo: seuciryt -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
On Thu, Jan 27, 2011 at 9:14 PM, Xiaobo Gu guxiaobo1...@gmail.com wrote:
On Thu, Jan 27, 2011 at 8:56 PM, Xiaobo Gu guxiaobo1...@gmail.com wrote:
On Thu, Jan 27, 2011 at 8:51 PM, Andrew Dunstan and...@dunslane.net wrote:
On 01/27/2011 07:41 AM, Xiaobo Gu wrote:
On Thu, Jan 27, 2011 at 8:37 PM, Andrew Dunstanand...@dunslane.net
wrote:
On 01/27/2011 07:31 AM, Xiaobo Gu wrote:
Hi,
I have a talk with MinGW developer, because I am not so familiar with
the UNIX configure and build system, can you help to resolve the
problem please.
jon_y: XiaoboGu: try actually reading config.log
20:18jon_y: look for winsock2.h in it
20:18jon_y: you should find it, along with the error nearby
20:19tjtag: thats fixed it, will do that also ktietz
20:20XiaoboGu:
d:/amber/devtool/mingw64-1.0-20100913/lib/gcc/../../x86_64-w64-mingw32/include/winsock2.h:1035:37:
note: previous declaration of 'accept' was here
20:21jon_y: like I said, the accept() probe doesn't work due to
difference in expectation
20:23tjtag: brb
20:23XiaoboGu: the path is OK
20:24XiaoboGu: I really can't understand this
20:24jon_y: its probing accept() for known implementations
20:24XiaoboGu: do you mean the configure script has a problem
20:25jon_y: none of them match the one in winsock2.h
20:25jon_y: I don't know, it could be mingw-w64's fault too
20:25jon_y: please contact psql and ask for a list of accept()
implementations checked
I am going to ignore your requests until you stop top-answering.
I am sorry, do you mean there are problems when dealing with the socke
function accept ?
Clearly there is a problem, or configure would have worked. You need to
answer the question I asked previously, namely what is the declaration of
accept() in the mingw64 headers? You have that source and I don't so I can't
answer the question.
#ifndef __WINSOCK_WS1_SHARED
/* these 46 functions have the same prototypes as in winsock2 */
WINSOCK_API_LINKAGE SOCKET WSAAPI accept(SOCKET s,struct sockaddr
*addr,int *addrlen);
WINSOCK_API_LINKAGE int WSAAPI bind(SOCKET s,const struct sockaddr
*name,int namelen);
WINSOCK_API_LINKAGE int WSAAPI closesocket(SOCKET s);
WINSOCK_API_LINKAGE int WSAAPI connect(SOCKET s,const struct
sockaddr *name,int namelen);
And these are type defines for SOCKET and struct sockaddr
#ifndef ___WSA_SOCKET_TYPES_H
#define ___WSA_SOCKET_TYPES_H
#if 0
typedef UINT_PTR SOCKET;
#else
typedef INT_PTR SOCKET;
#endif
#define INVALID_SOCKET (SOCKET)(~0)
#define SOCKET_ERROR (-1)
#endif /* ___WSA_SOCKET_TYPES_H */
struct sockaddr {
u_short sa_family;
char sa_data[14];
};
And the definitions for u_short and UINT_PTR INT_PTR are in :
_bsd_types.h
typedef unsigned short u_short;
basetsd.h
#ifndef _W64
#define _W64
#endif
#ifdef _WIN64
__MINGW_EXTENSION typedef __int64 INT_PTR,*PINT_PTR;
__MINGW_EXTENSION typedef unsigned __int64 UINT_PTR,*PUINT_PTR;
__MINGW_EXTENSION typedef __int64 LONG_PTR,*PLONG_PTR;
__MINGW_EXTENSION typedef unsigned __int64 ULONG_PTR,*PULONG_PTR;
#define __int3264 __int64
#else
typedef int INT_PTR,*PINT_PTR;
typedef unsigned int UINT_PTR,*PUINT_PTR;
typedef long LONG_PTR,*PLONG_PTR;
typedef unsigned long ULONG_PTR,*PULONG_PTR;
#define __int3264 __int32
#endif
Xiaobo Gu
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions support for pg_dump, patch v27
Itagaki Takahiro itagaki.takah...@gmail.com writes:
I found pg_restore with -c option fails when an extension is created
in pg_catalog. Since pg_catalog is an implicit search target, so we
might need the catalog to search_path explicitly.
Note that I can restore adminpack with not errors because it has
explicit pg_catalog. prefix in the installer script.
Nice catch, thank you very much (again) for finding those :)
Please find inline the patch I've just applied that fixes the issue by
managing the current search path and creation namespace before executing
the script, the same way that schemacmds.c does.
http://git.postgresql.org/gitweb?p=postgresql-extension.git;a=commitdiff;h=5c14c989426c0811e5bf8b993ae9a966fbf903c4
BTW, I have a minor comments for the code.
extern bool extension_relocatable_p(Oid ext_oid);
What is _p ? Also, we could make the function static
because it is used only in extension.c.
predicate. Maybe I've done too much Emacs Lisp coding at the time I
added that function, but it looked natural (enough) to me :)
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
commit 5c14c989426c0811e5bf8b993ae9a966fbf903c4 (HEAD,
refs/remotes/origin/extension, refs/heads/extension)
Author: Dimitri Fontaine d...@tapoueh.org
Date: Thu Jan 27 14:40:10 2011 +0100
Fully set the creation namespace before to execute the extenions's script.
Modified src/backend/commands/extension.c
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 87310fb..4a8c757 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -415,6 +415,8 @@ execute_extension_script(ExtensionControlFile *control,
char *filename = get_extension_absolute_path(control-script);
char *old_cmsgs = NULL, *old_lmsgs = NULL; /* silence compiler */
boolreset_cmsgs = false, reset_lmsgs = false;
+ Oid target_schema;
+ OverrideSearchPath *overridePath;
/*
* Set create_extension and create_extension_with_user_data so that the
@@ -453,6 +455,21 @@ execute_extension_script(ExtensionControlFile *control,
SetConfigOption(log_min_messages, warning, PGC_SUSET,
PGC_S_SESSION);
}
+ if (control-relocatable)
+ target_schema = LookupCreationNamespace(schema);
+ else
+ target_schema = get_namespace_oid(schema, false);
+
+ /*
+* Temporarily make the new namespace be the front of the search path,
as
+* well as the default creation target namespace. This will be undone
at
+* the end of this routine, or upon error.
+*/
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath-schemas = lcons_oid(target_schema, overridePath-schemas);
+ /* XXX should we clear overridePath-useTemp? */
+ PushOverrideSearchPath(overridePath);
+
/*
* On failure, ensure that create_extension does not remain set.
*/
@@ -474,7 +491,8 @@ execute_extension_script(ExtensionControlFile *control,
if (control-relocatable)
{
List *search_path = fetch_search_path(false);
- Oid first_schema, target_schema;
+ Oid first_schema = linitial_oid(search_path);
+ list_free(search_path);
if (!execute_sql_string(sql))
ereport(ERROR,
@@ -495,10 +513,6 @@ execute_extension_script(ExtensionControlFile *control,
* We skip this step if the target schema happens to be
the
* first schema of the current search_path.
*/
- first_schema = linitial_oid(search_path);
- target_schema = LookupCreationNamespace(schema);
- list_free(search_path);
-
if (first_schema != target_schema)
AlterExtensionNamespace_oid(CreateExtensionAddress.objectId,
target_schema);
@@ -531,6 +545,9 @@ execute_extension_script(ExtensionControlFile *control,
}
PG_END_TRY();
+ /* Reset search path to normal state */
+ PopOverrideSearchPath();
+
if (reset_cmsgs)
SetConfigOption(client_min_messages,
old_cmsgs, PGC_SUSET,
PGC_S_SESSION);
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
On Thu, Jan 27, 2011 at 7:31 AM, Xiaobo Gu guxiaobo1...@gmail.com wrote: 20:25 jon_y: please contact psql and ask for a list of accept() implementations checked It looks like we check each argument and the return type for a couple of possibilities: argument 1 can be int or unsigned int argument 2 can be struct sockaddr * or const struct sockaddr * or void * argument 3 can be int * or size_t * or socklen_t * or unsigned int * or void * the return type can be int or unsigned int PASCAL Here's the code: AC_DEFUN([AC_FUNC_ACCEPT_ARGTYPES], [AC_MSG_CHECKING([types of arguments for accept()]) AC_CACHE_VAL(ac_cv_func_accept_return,dnl [AC_CACHE_VAL(ac_cv_func_accept_arg1,dnl [AC_CACHE_VAL(ac_cv_func_accept_arg2,dnl [AC_CACHE_VAL(ac_cv_func_accept_arg3,dnl[for ac_cv_func_accept_return in 'int' 'unsigned int PASCAL'; do for ac_cv_func_accept_arg1 in 'int' 'unsigned int'; do for ac_cv_func_accept_arg2 in 'struct sockaddr *' 'const struct sockaddr *' 'void *'; do for ac_cv_func_accept_arg3 in 'int' 'size_t' 'socklen_t' 'unsigned int' 'void'; do AC_TRY_COMPILE( [#ifdef HAVE_SYS_TYPES_H #include sys/types.h#endif #ifdef HAVE_SYS_SOCKET_H #include sys/socket.h #endif extern $ac_cv_func_accept_return accept ($ac_cv_func_accept_arg1, $ac_cv_func_ac cept_arg2, $ac_cv_func_accept_arg3 *);], [], [ac_not_found=no; break 4], [ac_not_found=yes]) done done done done if test $ac_not_found = yes; then AC_MSG_ERROR([could not determine argument types]) fi if test $ac_cv_func_accept_arg3 = void; then ac_cv_func_accept_arg3=int fi])dnl AC_CACHE_VAL ])dnl AC_CACHE_VAL ])dnl AC_CACHE_VAL ])dnl AC_CACHE_VAL -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
On 01/27/2011 08:51 AM, Robert Haas wrote: On Thu, Jan 27, 2011 at 7:31 AM, Xiaobo Guguxiaobo1...@gmail.com wrote: 20:25jon_y: please contact psql and ask for a list of accept() implementations checked It looks like we check each argument and the return type for a couple of possibilities: argument 1 can be int or unsigned int argument 2 can be struct sockaddr * or const struct sockaddr * or void * argument 3 can be int * or size_t * or socklen_t * or unsigned int * or void * the return type can be int or unsigned int PASCAL Yeah. it looks like the return type is the likely culprit. It looks like it wants either a signed or unsigned int64. Also, the mingw64 API uses the WSAAPI qualifier instead of PASCAL. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 3: add facility to identify further no-work cases
On Thu, Jan 27, 2011 at 1:14 AM, Noah Misch n...@leadboat.com wrote:
Based on downthread discussion, I figure this will all change a good deal.
I'll
still briefly explain the patch as written. Most of the patch is plumbing to
support the new syntax, catalog entries, and FuncExpr field. The important
changes are in parse_coerce.c. I modified find_coercion_pathway() and
find_typmod_coercion_function() to retrieve pg_cast.castexemptor alongside
pg_cast.castfunc. Their callers (coerce_type() and coerce_type_typmod(),
respectively) then call the new apply_exemptor_function(), which calls the
exemptor function, if any, returns the value to place in FuncExpr.funcexempt,
and possibly updates the CoercionPathType the caller is about to use.
build_coercion_expression(), unchanged except to populate FuncExpr.funcexempt,
remains responsible for creating the appropriate node (RelabelType, FuncExpr).
Finally, I change GetCoerceExemptions to use FuncExpr.funcexempt.
OK. I was thinking that instead moving this into
eval_const_expressions(), we just make the logic in
find_coercion_pathway() call the exemptor function (or whatever we
call it) right around here:
switch (castForm-castmethod)
{
case COERCION_METHOD_FUNCTION:
result = COERCION_PATH_FUNC;
*funcid = castForm-castfunc;
break;
case COERCION_METHOD_INOUT:
result = COERCION_PATH_COERCEVIAIO;
break;
case COERCION_METHOD_BINARY:
result = COERCION_PATH_RELABELTYPE;
break;
default:
elog(ERROR, unrecognized
castmethod: %d,
(int) castForm-castmethod);
break;
}
If it's COERCION_METHOD_FUNCTION, then instead of unconditionally
setting the result to COERCION_PATH_FUNC, we inquire - based on the
typemods - whether it's OK to downgrade to a
COERCION_PATH_RELABELTYPE. The only fly in the ointment is that
find_coercion_pathway() doesn't current get the typemods. Not sure
how ugly that would be to fix.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Mingw-users] What's the relationship between GCC and MinGW
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 1/27/2011 14:43, Xiaobo Gu wrote: On 1/27/2011 11:02, Xiaobo Gu wrote: MSYS does not provide GCC, it only provides some UNIX like tools on Windows to emulate the *NIX environment. Yes it does, you are using it. OK, my fault, I now adjust the environments of my two computers(one is 32bit Windows XP SP3, the other one is 64bit Windows 7 Home basic) to the same setup layout: 1.MinGW64 is installed into D:\Amber\Devtool\MinGW64-1.0-20100913; 2.MSYS in installed into D:\Amber\Devtool\msys 3.Add D:/Amber/Devtool/MinGW64-1.0-20100913 /mingw to D:\Amber\Devtool\msys\etc\fstab 4. PostgreSQL source code is in D:\Amber\devproj\postgresql-9.0.2 From the MSYS sh prompt(D:\Amber\Devtool\msys\msys.bat), I do the following: 1. gcc -v sh: gcc: command not found 2. configure --without-zlib --host=x86_64-w64-mingw32 --with-system-tzdata=/usr/share/zoneinfo The content of config.log file is as following: Next time, please post your giant file as an attachment instead of inline. Looks like accept() detection doesn't work. You'll have to compare accept() declaration to the list of prototypes configure attempts to test. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (MingW32) iEYEARECAAYFAk1BRrkACgkQp56AKe10wHcrQACfZiHJjF1qfccF3Va83wrW+gJ4 dPcAn2amjoamy2mcDSpoRzpJzjXS/na3 =n6Mt -END PGP SIGNATURE- 0xED74C077.asc Description: application/pgp-keys -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
On Thu, Jan 27, 2011 at 10:01 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/27/2011 08:51 AM, Robert Haas wrote: On Thu, Jan 27, 2011 at 7:31 AM, Xiaobo Guguxiaobo1...@gmail.com wrote: 20:25jon_y: please contact psql and ask for a list of accept() implementations checked It looks like we check each argument and the return type for a couple of possibilities: argument 1 can be int or unsigned int argument 2 can be struct sockaddr * or const struct sockaddr * or void * argument 3 can be int * or size_t * or socklen_t * or unsigned int * or void * the return type can be int or unsigned int PASCAL Yeah. it looks like the return type is the likely culprit. It looks like it wants either a signed or unsigned int64. Also, the mingw64 API uses the WSAAPI qualifier instead of PASCAL. I am still trying, but I think it may be the first argument, because MinGW64 define SOCKET as a pointer, but you accept int or unsigned int Xiaobo Gu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
On 01/27/2011 09:19 AM, Xiaobo Gu wrote: On Thu, Jan 27, 2011 at 10:01 PM, Andrew Dunstanand...@dunslane.net wrote: On 01/27/2011 08:51 AM, Robert Haas wrote: On Thu, Jan 27, 2011 at 7:31 AM, Xiaobo Guguxiaobo1...@gmail.comwrote: 20:25jon_y: please contact psql and ask for a list of accept() implementations checked It looks like we check each argument and the return type for a couple of possibilities: argument 1 can be int or unsigned int argument 2 can be struct sockaddr * or const struct sockaddr * or void * argument 3 can be int * or size_t * or socklen_t * or unsigned int * or void * the return type can be int or unsigned int PASCAL Yeah. it looks like the return type is the likely culprit. It looks like it wants either a signed or unsigned int64. Also, the mingw64 API uses the WSAAPI qualifier instead of PASCAL. I am still trying, but I think it may be the first argument, because MinGW64 define SOCKET as a pointer, but you accept int or unsigned int No it doesn't. It defines it as an INT_PTR or UINT_PTR which in turn is an __int64, I think. But you're right, we might need to look at the first argument as well as the return type. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 3: add facility to identify further no-work cases
Robert Haas robertmh...@gmail.com writes: OK. I was thinking that instead moving this into eval_const_expressions(), we just make the logic in find_coercion_pathway() call the exemptor function (or whatever we call it) right around here: No, no, no, no. Didn't you read yesterday's discussion? parse_coerce is entirely the wrong place for this. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
On Thu, Jan 27, 2011 at 10:44 PM, Xiaobo Gu guxiaobo1...@gmail.com wrote: On Thu, Jan 27, 2011 at 9:32 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/27/2011 07:56 AM, Xiaobo Gu wrote: Clearly there is a problem, or configure would have worked. You need to answer the question I asked previously, namely what is the declaration of accept() in the mingw64 headers? You have that source and I don't so I can't answer the question. #ifndef __WINSOCK_WS1_SHARED /* these 46 functions have the same prototypes as in winsock2 */ WINSOCK_API_LINKAGE SOCKET WSAAPI accept(SOCKET s,struct sockaddr *addr,int *addrlen); WINSOCK_API_LINKAGE int WSAAPI bind(SOCKET s,const struct sockaddr *name,int namelen); WINSOCK_API_LINKAGE int WSAAPI closesocket(SOCKET s); WINSOCK_API_LINKAGE int WSAAPI connect(SOCKET s,const struct sockaddr *name,int namelen); Ok, now in src/config/ac_func_accept_argtypes.m4 on line 48, try adding the following alternatives for the return type: unsigned int WSAAPI int WSAAPI unsigned INT_PTR WSAAPI INT_PTR WSAAPI None of them works, I'll try to add '__int64' as an option to first argument, and see if one of them lets configure pass. AC_DEFUN([AC_FUNC_ACCEPT_ARGTYPES], [AC_MSG_CHECKING([types of arguments for accept()]) AC_CACHE_VAL(ac_cv_func_accept_return,dnl [AC_CACHE_VAL(ac_cv_func_accept_arg1,dnl [AC_CACHE_VAL(ac_cv_func_accept_arg2,dnl [AC_CACHE_VAL(ac_cv_func_accept_arg3,dnl [for ac_cv_func_accept_return in 'INT_PTR WSAAPI' 'int' 'unsigned int PASCAL' ; do for ac_cv_func_accept_arg1 in '__int64' 'int' 'unsigned int' ; do for ac_cv_func_accept_arg2 in 'struct sockaddr *' 'const struct sockaddr *' 'void *'; do for ac_cv_func_accept_arg3 in 'int' 'size_t' 'socklen_t' 'unsigned int' 'void'; do AC_TRY_COMPILE( The above combinition does not pass, another question, Because I just want to build the client side of PostgreSQL, can I ignore this error and let the configure pass Xiaobo Gu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW
On 01/27/2011 09:55 AM, Xiaobo Gu wrote: The above combinition does not pass, another question, Because I just want to build the client side of PostgreSQL, can I ignore this error and let the configure pass You can try, but I at least am only interested in getting the whole package to build. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] make -j2 error in ecpg/pgtypeslib in current GIT
Boszormenyi Zoltan z...@cybertec.at writes: I get reproducible make -j2 compile error in ecpg/pgtypeslib. FWIW, I can't reproduce that at all, either at -j2 or -j4, on either Fedora 13 or Fedora 14. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 3: add facility to identify further no-work cases
On Thu, Jan 27, 2011 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: OK. I was thinking that instead moving this into eval_const_expressions(), we just make the logic in find_coercion_pathway() call the exemptor function (or whatever we call it) right around here: No, no, no, no. Didn't you read yesterday's discussion? parse_coerce is entirely the wrong place for this. I not only read it, I participated in it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SSI patch version 14
I wrote: While the overall code coverage for PostgreSQL source code is: Overall coverage rate: lines..: 64.8% (130296 of 201140 lines) functions..: 72.0% (7997 of 11109 functions) By the way, I discovered that the above is lower if you just run the check target. The dcheck target helps with overall PostgreSQL code coverage, even though it was targeted at the SSI code. The coverage for predicate.c, after running both check and dcheck, was (formatted to match above): lines..: 69.8% (925 of 1325 lines) functions..: 85.7% (48 of 56 functions) Some minor tweaks to the regression tests boosts that to: lines..: 73.1% (968 of 1325 lines) functions..: 89.3% (50 of 56 functions) Most of the code not covered in the regular build (above) is in the OldSerXidX functions, which are covered well in a build with TEST_OLDSERXID defined. The 2PC code is very well covered now, except for the recovery-time function. We don't have a way to test that in the `make check` process, do we? There is some code which is not covered just because it is so hard to hit -- for example, code which is only executed if vacuum cleans up an index page when we are right at the edge of running out of the memory used to track predicate locks. It would be hard to include a test for that in the normal regression tests. The regression test changes are here: http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git;a=commitdiff;h=d4c1005d731c81049cc2622e50b7a2ebb99bbcac -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: In pg_test_fsync, use K(1024) rather than k(1000) for write size units.
Nicolas Barbier wrote: 2011/1/27 Bruce Momjian br...@momjian.us: Bruce Momjian wrote: Peter Eisentraut wrote: We use small k in postgresql.conf, so pg_test_fsync should use the same. ?Using kB would be more accurate in any case. OK, done with the attached applied patch. FYI, I had used 'k' because this page suggests that k is 1000 and K is 1024, at least by the JEDEC memory standards: ? ? ? ?http://en.wikipedia.org/wiki/Kilo I can't find any reference to that on this page? The following does indeed say: Sorry, I posed the wrong URL; it should have been: http://en.wikipedia.org/wiki/Bytes#Unit_symbol You can see the chart on the right. However, I agree 'kB' is the best. --- URL:http://en.wikipedia.org/wiki/JEDEC_memory_standards quote kilo (K): A multiplier equal to 1,024 [..] The specification notes that these prefixes are included in the document only to reflect common usage. It refers to the IEEE/ASTM SI 10-1997 standard as stating, that this practice frequently leads to confusion and is deprecated. /quote If you want to make the difference explicit, consider using KiB (1024, note the extra i) vs. kB (1000); although doing so is probably not consistent with any other uses in PostgreSQL. URL:http://en.wikipedia.org/wiki/Kibibyte quote The unit symbol for the kibibyte is KiB. The unit was established by the International Electrotechnical Commission (IEC) in 1999 and has been accepted for use by all major standards organizations. /quote Nicolas -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Wed, Jan 26, 2011 at 17:37, Andrew Dunstan and...@dunslane.net wrote: On 01/26/2011 11:26 AM, Bruce Momjian wrote: For those of you using git, I wanted to point out that it is fairly easy to remove git branches. For example, I can easily remove a branch on my github repository using: $ git branch -d :branch_name I don't believe that is revertable. What is scarey is that this could be done on our 'origin' as well. The ability to remove branches is a feature. I strongly encourage you to create topic branches for development work, then merge them onto the main branch, and then delete them. I almost never work directly on, say, REL9_0_STABLE or master, except for quite trivial changes. I thought we had some hooks on gitmaster to help prevent accidents like inadvertent branch deletion. We have hooks to prevent a number of things, but not the removal of branches (or tags). We'll send an email to committers telling you it's been done, but we don't prevent it. It would probably be pretty easy to add a hook preventing it though - do we want that? (we could still delete branches of course - but it would require an admin to do it directly on the git server, which is highly unlikely to happen by mistake) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: In pg_test_fsync, use K(1024) rather than k(1000) for write size units.
Bruce Momjian br...@momjian.us wrote: http://en.wikipedia.org/wiki/Bytes#Unit_symbol You can see the chart on the right. According to which, the JEDEC standard requires KB and the IEC standard requires KiB. What standard led us to use kB instead? It seems to generally mean 1000 instead of 1024. However, I agree 'kB' is the best. Why? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 3: add facility to identify further no-work cases
On Thu, Jan 27, 2011 at 10:42 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 27, 2011 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: OK. I was thinking that instead moving this into eval_const_expressions(), we just make the logic in find_coercion_pathway() call the exemptor function (or whatever we call it) right around here: No, no, no, no. Didn't you read yesterday's discussion? parse_coerce is entirely the wrong place for this. I not only read it, I participated in it. To put that another way, there's a difference between reading something and agreeing with it. I did the former, but not the latter. It seems to me that this discussion is unnecessarily antagonistic. Is it not OK for me to have a different idea about which way to go with something than you do? My personal view is that we ought to try to be increasing the number of places where type modifiers behave like they're really part of the type, rather than being an afterthought that we deal with when convenient and otherwise ignore. Otherwise, I see the chances of any substantive improvements in our type system as being just about zero. However, the larger point here is simply this: I'm trying to make some progress on reviewing and, where appropriate, committing the patches that were submitted for this CommitFest. I'd like to try to avoid the phenomenon where we're tripping over each other's feet. We're not making out very well on that front at the moment. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 11:13 AM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jan 26, 2011 at 17:37, Andrew Dunstan and...@dunslane.net wrote: On 01/26/2011 11:26 AM, Bruce Momjian wrote: For those of you using git, I wanted to point out that it is fairly easy to remove git branches. For example, I can easily remove a branch on my github repository using: $ git branch -d :branch_name I don't believe that is revertable. What is scarey is that this could be done on our 'origin' as well. The ability to remove branches is a feature. I strongly encourage you to create topic branches for development work, then merge them onto the main branch, and then delete them. I almost never work directly on, say, REL9_0_STABLE or master, except for quite trivial changes. I thought we had some hooks on gitmaster to help prevent accidents like inadvertent branch deletion. We have hooks to prevent a number of things, but not the removal of branches (or tags). We'll send an email to committers telling you it's been done, but we don't prevent it. It would probably be pretty easy to add a hook preventing it though - do we want that? (we could still delete branches of course - but it would require an admin to do it directly on the git server, which is highly unlikely to happen by mistake) I think it's highly unlikely to happen by mistake as it is. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Magnus Hagander mag...@hagander.net writes: On Wed, Jan 26, 2011 at 17:37, Andrew Dunstan and...@dunslane.net wrote: I thought we had some hooks on gitmaster to help prevent accidents like inadvertent branch deletion. We have hooks to prevent a number of things, but not the removal of branches (or tags). We'll send an email to committers telling you it's been done, but we don't prevent it. It would probably be pretty easy to add a hook preventing it though - do we want that? (we could still delete branches of course - but it would require an admin to do it directly on the git server, which is highly unlikely to happen by mistake) Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. Now, our committers all seem to be pretty careful people, so I don't feel strongly about having extra security on this --- but if it's easy to do, it's probably a good idea. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: In pg_test_fsync, use K(1024) rather than k(1000) for write size units.
Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: http://en.wikipedia.org/wiki/Bytes#Unit_symbol You can see the chart on the right. According to which, the JEDEC standard requires KB and the IEC standard requires KiB. What standard led us to use kB instead? It seems to generally mean 1000 instead of 1024. I assume Peter did lots of research when he added 'kB' to postgresql.conf. However, I agree 'kB' is the best. Why? No idea. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 3: add facility to identify further no-work cases
Robert Haas robertmh...@gmail.com writes: Is it not OK for me to have a different idea about which way to go with something than you do? Well, in this case I firmly believe that you're mistaken. My personal view is that we ought to try to be increasing the number of places where type modifiers behave like they're really part of the type, rather than being an afterthought that we deal with when convenient and otherwise ignore. And this argument is 100% irrelevant to the problem. The problem is that you want to put an optimization into the wrong phase of processing. That is going to hurt us, tomorrow if not today, and it has got *no* redeeming social value in terms of beng more flexible about what typmods are or how well integrated they are. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On 01/27/2011 11:29 AM, Tom Lane wrote: Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. Now, our committers all seem to be pretty careful people, so I don't feel strongly about having extra security on this --- but if it's easy to do, it's probably a good idea. Pushing a local topic branch by mistake seems much more likely to me. Some protection against that mightn't be a bad idea. Maybe for example a check on the branch name? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 3: add facility to identify further no-work cases
On Thu, Jan 27, 2011 at 11:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: My personal view is that we ought to try to be increasing the number of places where type modifiers behave like they're really part of the type, rather than being an afterthought that we deal with when convenient and otherwise ignore. And this argument is 100% irrelevant to the problem. The problem is that you want to put an optimization into the wrong phase of processing. That is going to hurt us, tomorrow if not today, and it has got *no* redeeming social value in terms of beng more flexible about what typmods are or how well integrated they are. The only thing we're deciding here is whether or not a cast requires a function. That's a function of the type OIDs and the typemods. I don't see why it's wrong to do the portion that involves the types in the same place as the portion that involves the typemods. Perhaps you could explain. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 17:36, Andrew Dunstan and...@dunslane.net wrote: On 01/27/2011 11:29 AM, Tom Lane wrote: Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. Now, our committers all seem to be pretty careful people, so I don't feel strongly about having extra security on this --- but if it's easy to do, it's probably a good idea. Pushing a local topic branch by mistake seems much more likely to me. Some protection against that mightn't be a bad idea. Maybe for example a check on the branch name? Or for that we could just disable branch creation *completely*, and then turn off that restriction that one time / year that we actually create a branch? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Magnus Hagander wrote: On Thu, Jan 27, 2011 at 17:36, Andrew Dunstan and...@dunslane.net wrote: On 01/27/2011 11:29 AM, Tom Lane wrote: Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. ?Now, our committers all seem to be pretty careful people, so I don't feel strongly about having extra security on this --- but if it's easy to do, it's probably a good idea. Pushing a local topic branch by mistake seems much more likely to me. Some protection against that mightn't be a bad idea. Maybe for example a check on the branch name? Or for that we could just disable branch creation *completely*, and then turn off that restriction that one time / year that we actually create a branch? Well, branch creation can always be undone --- branch removal seems like the big problem because it can't. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On 27.01.2011 18:41, Bruce Momjian wrote: Well, branch creation can always be undone --- branch removal seems like the big problem because it can't. Actually, all you need to do is to push the branch back to resurrect it. As long as your local branch is up-to-date with what was removed (or you know the commitid and still have that in your local repository), no-one will notice that it was gone for a moment. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 11:41 AM, Bruce Momjian br...@momjian.us wrote: Or for that we could just disable branch creation *completely*, and then turn off that restriction that one time / year that we actually create a branch? Well, branch creation can always be undone --- branch removal seems like the big problem because it can't. As I've repeatedly said, branch removal CAN be undone. I don't see any evidence that we have an actual problem here that needs worrying about. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Robert Haas wrote: On Thu, Jan 27, 2011 at 11:41 AM, Bruce Momjian br...@momjian.us wrote: Or for that we could just disable branch creation *completely*, and then turn off that restriction that one time / year that we actually create a branch? Well, branch creation can always be undone --- branch removal seems like the big problem because it can't. As I've repeatedly said, branch removal CAN be undone. I don't see any evidence that we have an actual problem here that needs worrying about. OK, someone removes a branch. If it is still in his local tree, he can push it back. If not, he has to go around and find someone who does have it, and who has the most recent copy? Can master be removed too? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Andrew Dunstan and...@dunslane.net writes: On 01/27/2011 11:29 AM, Tom Lane wrote: Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. Pushing a local topic branch by mistake seems much more likely to me. Yeah, that's probably true. Some protection against that mightn't be a bad idea. Maybe for example a check on the branch name? If we *don't* install branch-removal defenses on the server, then it's easy enough to clean up an erroneous branch push. Only if we do that does this scenario become a problem. I find myself agreeing with Robert that we may be creating an issue where none exists. At this point my vote is to leave it alone until and unless we see that people actually make this type of mistake regularly. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Bruce Momjian br...@momjian.us wrote: OK, someone removes a branch. As was explained earlier on this thread, it's not gone at that point; it's a dangling reference. I think that unless someone explicitly prunes the dangling references, they are left around for a week, and can easily be checked out again. If it is still in his local tree, he can push it back. If not, he has to go around and find someone who does have it, and who has the most recent copy? If it actually is gone from the server, you can fall back to this, yeah. Can master be removed too? I don't think so. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 01/27/2011 11:29 AM, Tom Lane wrote: Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. Pushing a local topic branch by mistake seems much more likely to me. Yeah, that's probably true. Some protection against that mightn't be a bad idea. Maybe for example a check on the branch name? If we *don't* install branch-removal defenses on the server, then it's easy enough to clean up an erroneous branch push. Only if we do that does this scenario become a problem. I find myself agreeing with Robert that we may be creating an issue where none exists. At this point my vote is to leave it alone until and unless we see that people actually make this type of mistake regularly. OK, I posted the information just so people would be aware of this issue --- I didn't expect it to be common or something we needed to protect against. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Upcoming back-branch updates
The Postgres core team will be wrapping update releases for all supported back branches this evening. These updates are prompted by a minor security issue (reported by Apple, curiously enough) as well as desire to push out some significant bug fixes that have been made in the last few weeks. My apologies for the short notice. I was supposed to report this decision to -hackers a week ago, and entirely forgot :-( regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SSI patch version 14
On Tue, 2011-01-25 at 05:57 -0500, Dan Ports wrote: This summary is right on. I would add one additional detail or clarification to the last point, which is that rather than checking for a cycle, we're checking for a transaction with both in and out conflicts, which every cycle must contain. To clarify, this means that it will get some false positives, right? For instance: T1: get snapshot T2: get snapshot insert R1 commit T1: read R1 write R2 T3: get snapshot read R2 T3: commit T1: commit -- throws error T1 has a conflict out to T2, and T1 has a conflict in from T3. T2 has a conflict in from T1. T3 has a conflict out to T1. T1 is canceled because it has both a conflict in and a conflict out. But the results are the same as a serial order of execution: T3, T1, T2. Is there a reason we can't check for a real cycle, which would let T1 succeed? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Spread checkpoint sync
Greg Smith wrote: I think a helpful next step here would be to put Robert's fsync compaction patch into here and see if that helps. There are enough backend syncs showing up in the difficult workloads (scale=1000, clients =32) that its impact should be obvious. Initial tests show everything expected from this change and more. This took me a while to isolate because of issues where the filesystem involved degraded over time, giving a heavy bias toward a faster first test run, before anything was fragmented. I just had to do a whole new mkfs on the database/xlog disks when switching between test sets in order to eliminate that. At a scale of 500, I see the following average behavior: Clients TPS backend-fsync 16 557 155 32 587 572 64 628 843 128 621 1442 256 632 2504 On one run through with the fsync compaction patch applied this turned into: Clients TPS backend-fsync 16 637 0 32 621 0 64 721 0 128 716 0 256 841 0 So not only are all the backend fsyncs gone, there is a very clear TPS improvement too. The change in results at =64 clients are well above the usual noise threshold in these tests. The problem where individual fsync calls during checkpoints can take a long time is not appreciably better. But I think this will greatly reduce the odds of running into the truly dysfunctional breakdown, where checkpoint and backend fsync calls compete with one another, that caused the worst-case situation kicking off this whole line of research here. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us PostgreSQL 9.0 High Performance: http://www.2ndQuadrant.com/books -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 11:52 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Jan 27, 2011 at 11:41 AM, Bruce Momjian br...@momjian.us wrote: Or for that we could just disable branch creation *completely*, and then turn off that restriction that one time / year that we actually create a branch? Well, branch creation can always be undone --- branch removal seems like the big problem because it can't. As I've repeatedly said, branch removal CAN be undone. I don't see any evidence that we have an actual problem here that needs worrying about. OK, someone removes a branch. If it is still in his local tree, he can push it back. If not, he has to go around and find someone who does have it, and who has the most recent copy? Can master be removed too? So if someone does this (which does not look at all likely to me): git push origin :REL9_0_STABLE git branch -r -D origin/REL9_0_STABLE git branch -d REL9_0_STABLE ...then, yes, they will need to find someone who has run 'git pull' since the last change that was made to that branch. OR they could get it back from the anonymous mirror of the canonical repository, which should always be up to date, OR I think there's an automatically updated mirror on github also. The master branch can be removed the same as any other one - just substitute master in place of REL9_0_STABLE in the above commands. But why would you do such a nutty thing? Worst case scenario looks to me like you type the first of those commands and then go oh crud. And if any of our 19 committers were unaware of the hazards of inserting random colons into their git commands, hopefully this discussion has awakened them to the error of their ways. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Robert Haas wrote: The master branch can be removed the same as any other one - just substitute master in place of REL9_0_STABLE in the above commands. But why would you do such a nutty thing? Worst case scenario looks to me like you type the first of those commands and then go oh crud. And if any of our 19 committers were unaware of the hazards of inserting random colons into their git commands, hopefully this --- discussion has awakened them to the error of their ways. Yes, that was really my goal --- to point out that some git operations are not reverable. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 18:19, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 27, 2011 at 11:52 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Jan 27, 2011 at 11:41 AM, Bruce Momjian br...@momjian.us wrote: Or for that we could just disable branch creation *completely*, and then turn off that restriction that one time / year that we actually create a branch? Well, branch creation can always be undone --- branch removal seems like the big problem because it can't. As I've repeatedly said, branch removal CAN be undone. I don't see any evidence that we have an actual problem here that needs worrying about. OK, someone removes a branch. If it is still in his local tree, he can push it back. If not, he has to go around and find someone who does have it, and who has the most recent copy? Can master be removed too? So if someone does this (which does not look at all likely to me): git push origin :REL9_0_STABLE git branch -r -D origin/REL9_0_STABLE git branch -d REL9_0_STABLE ...then, yes, they will need to find someone who has run 'git pull' since the last change that was made to that branch. OR they could get it back from the anonymous mirror of the canonical repository, which should always be up to date, OR I think there's an automatically updated mirror on github also. There is. Shouldn't we also be able to construct it from the latest mail to pgsql-committers, since it has the sha1 hash in it.. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
Robert Haas robertmh...@gmail.com wrote: So if someone does this (which does not look at all likely to me): git push origin :REL9_0_STABLE git branch -r -D origin/REL9_0_STABLE git branch -d REL9_0_STABLE ...then, yes, they will need to find someone who has run 'git pull' since the last change that was made to that branch. OR they could get it back from the anonymous mirror of the canonical repository, which should always be up to date, OR I think there's an automatically updated mirror on github also. I thought that git fsck by an administrator on the server would still show the original as a dangling commit, which could be checked out by the SHA1 ID. No? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Spread checkpoint sync
On Thu, Jan 27, 2011 at 12:18 PM, Greg Smith g...@2ndquadrant.com wrote: Greg Smith wrote: I think a helpful next step here would be to put Robert's fsync compaction patch into here and see if that helps. There are enough backend syncs showing up in the difficult workloads (scale=1000, clients =32) that its impact should be obvious. Initial tests show everything expected from this change and more. This took me a while to isolate because of issues where the filesystem involved degraded over time, giving a heavy bias toward a faster first test run, before anything was fragmented. I just had to do a whole new mkfs on the database/xlog disks when switching between test sets in order to eliminate that. At a scale of 500, I see the following average behavior: Clients TPS backend-fsync 16 557 155 32 587 572 64 628 843 128 621 1442 256 632 2504 On one run through with the fsync compaction patch applied this turned into: Clients TPS backend-fsync 16 637 0 32 621 0 64 721 0 128 716 0 256 841 0 So not only are all the backend fsyncs gone, there is a very clear TPS improvement too. The change in results at =64 clients are well above the usual noise threshold in these tests. The problem where individual fsync calls during checkpoints can take a long time is not appreciably better. But I think this will greatly reduce the odds of running into the truly dysfunctional breakdown, where checkpoint and backend fsync calls compete with one another, that caused the worst-case situation kicking off this whole line of research here. Dude! That's pretty cool. Thanks for doing that measurement work - that's really awesome. Barring objections, I'll go ahead and commit my patch. Based on what I saw looking at this, I'm thinking that the backend fsyncs probably happen in clusters - IOW, it's not 2504 backend fsyncs spread uniformly throughout the test, but clusters of 100 or more that happen in very quick succession, followed by relief when the background writer gets around to emptying the queue. During each cluster, the system probably slows way down, and then recovers when the queue is emptied. So the TPS improvement isn't at all a uniform speedup, but simply relief from the stall that would otherwise result from a full queue. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 12:24 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: So if someone does this (which does not look at all likely to me): git push origin :REL9_0_STABLE git branch -r -D origin/REL9_0_STABLE git branch -d REL9_0_STABLE ...then, yes, they will need to find someone who has run 'git pull' since the last change that was made to that branch. OR they could get it back from the anonymous mirror of the canonical repository, which should always be up to date, OR I think there's an automatically updated mirror on github also. I thought that git fsck by an administrator on the server would still show the original as a dangling commit, which could be checked out by the SHA1 ID. No? That's yet another way of undoing it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 3: add facility to identify further no-work cases
On Thu, Jan 27, 2011 at 09:02:26AM -0500, Robert Haas wrote:
OK. I was thinking that instead moving this into
eval_const_expressions(), we just make the logic in
find_coercion_pathway() call the exemptor function (or whatever we
call it) right around here:
switch (castForm-castmethod)
{
case COERCION_METHOD_FUNCTION:
result = COERCION_PATH_FUNC;
*funcid = castForm-castfunc;
break;
case COERCION_METHOD_INOUT:
result = COERCION_PATH_COERCEVIAIO;
break;
case COERCION_METHOD_BINARY:
result = COERCION_PATH_RELABELTYPE;
break;
default:
elog(ERROR, unrecognized
castmethod: %d,
(int) castForm-castmethod);
break;
}
If it's COERCION_METHOD_FUNCTION, then instead of unconditionally
setting the result to COERCION_PATH_FUNC, we inquire - based on the
typemods - whether it's OK to downgrade to a
COERCION_PATH_RELABELTYPE. The only fly in the ointment is that
find_coercion_pathway() doesn't current get the typemods. Not sure
how ugly that would be to fix.
Basically, this is a stylistic variation of the approach from my original at3
patch. I believe I looked at that particular positioning and decided that the
extra arguments and effects on non-parse_coerce.c callers were undesirable.
Debatable as any style issue, though. Note that you need to do the same thing
in find_typmod_coercion_function().
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 12:22 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: The master branch can be removed the same as any other one - just substitute master in place of REL9_0_STABLE in the above commands. But why would you do such a nutty thing? Worst case scenario looks to me like you type the first of those commands and then go oh crud. And if any of our 19 committers were unaware of the hazards of inserting random colons into their git commands, hopefully this --- discussion has awakened them to the error of their ways. Yes, that was really my goal --- to point out that some git operations are not reverable. That's true, but hopefully at this point it's clear that actually getting rid of a branch permanently would require rather a LOT of work, and that even if someone deliberately did their absolute best to get rid of a branch, an experienced git user could put it back in less than 15 minutes. I actually think a more likely scenario would be: we deliberately remove a branch - and then someone accidentally re-pushes it. But since we have no plans to actually do any such thing, that risk also seems more theoretical than actual. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upcoming back-branch updates
On Jan 27, 2011, at 9:17 AM, Tom Lane wrote: The Postgres core team will be wrapping update releases for all supported back branches this evening. These updates are prompted by a minor security issue (reported by Apple, curiously enough) as well as desire to push out some significant bug fixes that have been made in the last few weeks. Maybe we'll start to see more security-related reports from Apple. http://www.9to5mac.com/48841/apple-hires-ex-nsa-expert-to-lead-security-teams Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Caution when removing git branches
On Thu, Jan 27, 2011 at 17:52, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Jan 27, 2011 at 11:41 AM, Bruce Momjian br...@momjian.us wrote: Or for that we could just disable branch creation *completely*, and then turn off that restriction that one time / year that we actually create a branch? Well, branch creation can always be undone --- branch removal seems like the big problem because it can't. As I've repeatedly said, branch removal CAN be undone. I don't see any evidence that we have an actual problem here that needs worrying about. OK, someone removes a branch. If it is still in his local tree, he can push it back. If not, he has to go around and find someone who does have it, and who has the most recent copy? Can master be removed too? Correct. And *somebody* made the last commit on it, and that somebody hopefully still has the branch around - and you can find out who that is by looking at the committers email. No that's not a streamlined procedure, but it's hopefully ont something that will happen *often* at least :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SSI patch version 14
Jeff Davis pg...@j-davis.com wrote: To clarify, this means that it will get some false positives, right? Yes. But the example you're about to get into isn't one of them. For instance: T1: get snapshot T2: get snapshot insert R1 commit T1: read R1 write R2 T3: get snapshot read R2 T3: commit T1: commit -- throws error T1 has a conflict out to T2, and T1 has a conflict in from T3. T2 has a conflict in from T1. T3 has a conflict out to T1. T1 is canceled because it has both a conflict in and a conflict out. But the results are the same as a serial order of execution: T3, T1, T2. Is there a reason we can't check for a real cycle, which would let T1 succeed? Yes. Because T2 committed before T3 started, it's entirely possible that there is knowledge outside the database server that the work of T2 was done and committed before the start of T3, which makes the order of execution: T2, T3, T1, T2. So you can have anomalies. Let me give you a practical example. Pretend there are receipting terminals in public places for the organization. In most financial systems, those receipts are assigned to batches of some type. Let's say that's done by an insert for the new batch ID, which closes the old batch. Receipts are always added with the maximum batch ID, reflecting the open batch. Your above example could be: -- setup test=# create table ctl (batch_id int not null primary key); NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index ctl_pkey for table ctl CREATE TABLE test=# create table receipt (batch_id int not null, amt numeric(13,2) not null); CREATE TABLE test=# insert into ctl values (1),(2),(3); INSERT 0 3 test=# insert into receipt values ((select max(batch_id) from ctl), 50),((select max(batch_id) from ctl), 100); INSERT 0 2 -- receipting workstation -- T1 starts working on receipt insert transaction test=# begin transaction isolation level repeatable read; BEGIN test=# select 1; -- to grab snapshot, per above ?column? -- 1 (1 row) -- accounting workstation -- T2 closes old receipt batch; opens new test=# begin transaction isolation level repeatable read; BEGIN test=# insert into ctl values (4); INSERT 0 1 test=# commit; COMMIT -- receipting workstation -- T1 continues work on receipt test=# select max(batch_id) from ctl; max - 3 (1 row) test=# insert into receipt values (3, 1000); INSERT 0 1 -- accounting workstation -- T3 lists receipts from the closed batch -- (Hey, we inserted a new batch_id and successfully -- committed, right? The old batch is closed.) test=# begin transaction isolation level repeatable read; BEGIN test=# select * from receipt where batch_id = 3; batch_id | amt --+ 3 | 50.00 3 | 100.00 (2 rows) test=# commit; COMMIT -- receipting workstation -- T1 receipt insert transaction commits test=# commit; COMMIT Now, with serializable transactions, as you saw, T1 will be rolled back. With a decent software framework, it will be automatically retried, without any user interaction. It will select max(batch_id) of 4 this time, and the insert will succeed and be committed. Accounting's list is accurate. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SSI patch version 14
Kevin Grittner kevin.gritt...@wicourts.gov wrote: Now, with serializable transactions, as you saw, T1 will be rolled back. I should probably have mentioned, that if all the transactions were SERIALIZABLE and the report of transactions from the batch was run as SERIALIZABLE READ ONLY DEFERRABLE, the start of the report would block until it was certain that it had a snapshot which could not lead to an anomaly, so the BEGIN for T3 would wait until the COMMIT of T1, get a new snapshot which it would determine to be safe, and proceed. This would allow that last receipt to land in batch 3 and show up on accounting's receipt list with no rollbacks *or* anomalies. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Spread checkpoint sync
Robert Haas wrote: Based on what I saw looking at this, I'm thinking that the backend fsyncs probably happen in clusters - IOW, it's not 2504 backend fsyncs spread uniformly throughout the test, but clusters of 100 or more that happen in very quick succession, followed by relief when the background writer gets around to emptying the queue. That's exactly the case. You'll be running along fine, the queue will fill, and then hundreds of them can pile up in seconds. Since the worst of that seemed to be during the sync phase of the checkpoint, adding additional queue management logic to there is where we started at. I thought this compaction idea would be more difficult to implement than your patch proved to be though, so doing this first is working out quite well instead. This is what all the log messages from the patch look like here, at scale=500 and shared_buffers=256MB: DEBUG: compacted fsync request queue from 32768 entries to 11 entries That's an 8GB database, and from looking at the relative sizes I'm guessing 7 entries refer to the 1GB segments of the accounts table, 2 to its main index, and the other 2 are likely branches/tellers data. Since I know the production system I ran into this on has about 400 file segments on it regularly dirtied a higher shared_buffers than that, I expect this will demolish this class of problem on it, too. I'll have all the TPS over time graphs available to publish by the end of my day here, including tests at a scale of 1000 as well. Those should give a little more insight into how the patch is actually impacting high-level performance. I don't dare disturb the ongoing tests by copying all that data out of there until they're finished, will be a few hours yet. My only potential concern over committing this is that I haven't done a sanity check over whether it impacts the fsync mechanics in a way that might cause an issue. Your assumptions there are documented and look reasonable on quick review; I just haven't had much time yet to look for flaws in them. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us PostgreSQL 9.0 High Performance: http://www.2ndQuadrant.com/books -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TYPE 2: skip already-provable no-work rewrites
On Wed, Jan 26, 2011 at 07:31:40AM -0500, Robert Haas wrote: I'd also suggest that this big if-block you changed to a case statement could just as well stay as an if-block. There are only three cases, and we want to avoid rearranging things more than necessary. It complicates both review and back-patching to no good end. Okay. I've also left out the large reindent in ATRewriteTable for now. Easy to re-add it later if desired. I think you should collect up what's left of ALTER TABLE 0 and the stuff on this thread, rebase it, and submit it as a single patch on this thread that applies directly against the master branch. We may decide to split it back up again in some other way, but I think the current division isn't actually buying us much. Done as attached. This preserves compatibility with our current handling of composite type dependencies. The rest you've seen before. Thanks, nm diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 1d52be6..4efb02e 100644 *** a/doc/src/sgml/ref/alter_table.sgml --- b/doc/src/sgml/ref/alter_table.sgml *** *** 403,411 ALTER TABLE replaceable class=PARAMETERname/replaceable for details on the available parameters. Note that the table contents will not be modified immediately by this command; depending on the parameter you might need to rewrite the table to get the desired effects. ! That can be done with xref linkend=SQL-CLUSTER ! or one of the forms of commandALTER ! TABLE/ that forces a table rewrite. /para note --- 403,411 for details on the available parameters. Note that the table contents will not be modified immediately by this command; depending on the parameter you might need to rewrite the table to get the desired effects. ! That can be done with link linkend=SQL-VACUUMVACUUM ! FULL/, xref linkend=SQL-CLUSTER or one of the forms ! of commandALTER TABLE/ that forces a table rewrite. /para note *** *** 746,759 ALTER TABLE replaceable class=PARAMETERname/replaceable /para para ! Adding a column with a non-null default or changing the type of an ! existing column will require the entire table and indexes to be rewritten. ! This might take a significant amount of time for a large table; and it will ! temporarily require double the disk space. Adding or removing a system literaloid/ column likewise requires rewriting the entire table. /para para Adding a literalCHECK/ or literalNOT NULL/ constraint requires scanning the table to verify that existing rows meet the constraint. /para --- 746,770 /para para ! Adding a column with a non-null default will rewrite the entire table and ! all indexes. Changing the type of an existing column will do the same ! unless a binary-coercible cast implements the type conversion. Refer to ! xref linkend=SQL-CREATECAST for further information. A rewrite might ! take a significant amount of time for a large table, and it will temporarily ! require double the disk space. Adding or removing a system literaloid/ column likewise requires rewriting the entire table. /para para + Similar to the behavior of link linkend=SQL-VACUUMVACUUM FULL/, the + rewriting process eliminates any dead space in the table. Prior + to productnamePostgreSQL/ 9.0, literalSET DATA TYPE/ + outpaced literalVACUUM FULL/ at this task, so it was useful even absent + the need for a column type change. This speed advantage no longer holds, + and literalSET DATA TYPE/ may not even rewrite the table. +/para + +para Adding a literalCHECK/ or literalNOT NULL/ constraint requires scanning the table to verify that existing rows meet the constraint. /para *** *** 777,797 ALTER TABLE replaceable class=PARAMETERname/replaceable /para para - The fact that literalSET DATA TYPE/ requires rewriting the whole table - is sometimes an advantage, because the rewriting process eliminates - any dead space in the table. For example, to reclaim the space occupied - by a dropped column immediately, the fastest way is: - programlisting - ALTER TABLE table ALTER COLUMN anycol TYPE anytype; - /programlisting - where literalanycol/ is any remaining table column and - literalanytype/ is the same type that column already has. - This results in no semantically-visible change in the table, - but the command forces rewriting, which gets rid of no-longer-useful - data. -/para - -para The literalUSING/literal option of literalSET DATA TYPE/ can actually specify any expression involving the old values of the row; that is, it can refer to other columns as well as the one being converted. This allows --- 788,793
Re: [HACKERS] Caution when removing git branches
and...@dunslane.net (Andrew Dunstan) writes: On 01/27/2011 11:29 AM, Tom Lane wrote: Given that nobody is supposed to push temporary branches to the master repo anyway, an intended branch removal should be a pretty darn rare event. Now, our committers all seem to be pretty careful people, so I don't feel strongly about having extra security on this --- but if it's easy to do, it's probably a good idea. Pushing a local topic branch by mistake seems much more likely to me. Some protection against that mightn't be a bad idea. Maybe for example a check on the branch name? There seems to be a non-zero amount of value to this; I accidentally pushed some private branches into the Slony repo this afternoon, briefly, by accident. It wasn't troublesome to clean it up, so I'm not sure there's *huge* value in pushing a bunch of infrastructure into place to prevent such. If a problem: a) Is readily fixed, b) Is readily noticed, c) Gets you smacked down if you leave it unfixed, then I'm not sure it warrants going to extreme measures to prevent such a problem. -- select 'cbbrowne' || '@' || 'linuxdatabases.info'; http://linuxdatabases.info/info/slony.html If all those psychics know the winning lottery numbers, why are they all still working? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: In pg_test_fsync, use K(1024) rather than k(1000) for write size units.
Excerpts from Kevin Grittner's message of jue ene 27 13:22:12 -0300 2011: Bruce Momjian br...@momjian.us wrote: http://en.wikipedia.org/wiki/Bytes#Unit_symbol You can see the chart on the right. According to which, the JEDEC standard requires KB and the IEC standard requires KiB. What standard led us to use kB instead? It seems to generally mean 1000 instead of 1024. http://en.wikipedia.org/wiki/International_System_of_Units#Writing_unit_symbols_and_the_values_of_quantities -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pl/python SPI in subtransactions
On 26/01/11 04:51, Steve Singer wrote:
On 10-12-23 08:45 AM, Jan Urbański wrote:
I see you've merged the changes from the refactoring branch down but
haven't yet posted an updated patch. This review is based on
2f2b4a33bf344058620a5c684d1f2459e505c727
Thanks for the review, I'm attaching an updated patch against master.
The patch updates the excepted results of the regression tests so they
no longer expect the 'unrecognized error' warnings. No new unit test
are added to verify that behavior changes will continue to function as
intended (though they could be)
It's in fact just a correctness change, so I didn't include any new unit
tests.
No documentation updates are included. The current documentation is
silent on the behaviour of plpython when SPI calls generate errors so
this change doesn't invalidate any documentation but it would be nice if
we described what effect SQL invoked through SPI from the functions have
on the transaction. Maybe a Trapping Errors section?
Good point, the fact that you can now actually catch SPI errors should
be documented, I'll try to add a paragraph about that.
A concern I have is that some users might find this surprising. For
plpgsql the exception handling will rollback SQL from before the
exception and I suspect the other PL's are the same. It would be good
if a few more people chimed in on if they see this as a good idea.
It's not true for other PLs, but see below.
Another concern is that we are probably breaking some peoples code.
Consider the following:
BEGIN;
create table foo(a int4 primary key);
DO $$
r=plpy.execute(insert into foo values ('1'))
try :
r=plpy.execute(insert into foo values ('1'))
except:
plpy.log(something went wrong)
$$ language plpython2u;
select * FROM foo;
a
---
1
(1 row)
This code is going to behave different with the patch.
Right, without the patch you can never catch errors originating from
plpy.execute, so any error terminates the whole function, and so rolls
back the statement. FWIW PL/Perl works the same:
begin;
create table foo(i int primary key);
DO $$
spi_exec_query(insert into foo values ('1'));
eval { spi_exec_query(insert into foo values ('1')); };
elog(LOG, $@) if $@;
$$ language plperl;
select * from foo;
You will see a row in foo. AFAICS PL/Tcl also does it, but I don't have
it complied it to try. It does consitute a behaviour change, but we
didn't get any complains when the same thing happened for Perl.
I am finding the treatment of savepoints very strange.
If as a function author I'm able to recover from errors then I'd expect
(or maybe want) to be able to manage them through savepoints
Ooops, you found a bug there. In the attached patch you get the same
error (SPI_ERROR_TRANSACTION) as in master. Also added a unit test for that.
I've only glanced at your transaction manager patch, from what I can
tell it will give me another way of managing the inner transaction but
I'm not sure it will make the savepoint calls work(will it?). I also
wonder how useful in practice this patch will be if the other patch
doesn't also get applied (function others will be stuck with an all or
nothing as their options for error handling)
As long as you don't catch SPIError, nothing changes. And error in a SPI
call will propagate to the top of the Python stack and your function
will be terminated, its effects being rolled back. The function will
only work differently if you have bare except: clauses (that are
deprecated in 2.x and forbidden in 3.x IIRC) or catch Exception.
For example:
begin;
create table foo(i int primary key);
DO $$
plpy.execute(insert into foo values ('1'))
plpy.execute(insert into foo values ('1'))
$$ language plpython2u;
No row will be inserted and the whole transaction will be rolled back.
As for explicit subxact management, that's something that would be
unique to PL/Python, and my other patch implements it like this:
begin;
create table foo(i int primary key);
DO $$
try:
with plpy.subxact():
plpy.execute(insert into foo values ('1'))
plpy.execute(insert into foo values ('1'))
except plpy.SPIError:
plpy.log(no rows inserted)
$$ language plpython2u;
Thanks again for the review and for spotting that bug!
Cheers,
Jan
diff --git a/src/pl/plpython/expected/plpython_error.out b/src/pl/plpython/expected/plpython_error.out
index ea4a54c..4eeda6f 100644
*** a/src/pl/plpython/expected/plpython_error.out
--- b/src/pl/plpython/expected/plpython_error.out
*** CREATE FUNCTION sql_syntax_error() RETUR
*** 8,15
'plpy.execute(syntax error)'
LANGUAGE plpythonu;
SELECT sql_syntax_error();
- WARNING: plpy.SPIError: unrecognized error in PLy_spi_execute_query
- CONTEXT: PL/Python function sql_syntax_error
ERROR: plpy.SPIError: syntax error at or near syntax
LINE 1: syntax error
^
--- 8,13
*** CREATE FUNCTION exception_index_invalid_
*** 32,39
return rv[0]'
LANGUAGE
Re: [HACKERS] Re: In pg_test_fsync, use K(1024) rather than k(1000) for write size units.
Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Kevin Grittner's message of jue ene 27 13:22:12 -0300 2011: Bruce Momjian br...@momjian.us wrote: http://en.wikipedia.org/wiki/Bytes#Unit_symbol You can see the chart on the right. According to which, the JEDEC standard requires KB and the IEC standard requires KiB. What standard led us to use kB instead? It seems to generally mean 1000 instead of 1024. http://en.wikipedia.org/wiki/International_System_of_Units#Writing_unit_symbols_and_the_values_of_quantities That seems to agree with the other page that k means 10^3, not 2^10 -- or am I missing something? We are treating it as 2^10 in our GUCs, aren't we? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] REVIEW: PL/Python table functions
On 27/01/11 00:41, Jan Urbański wrote:
I'm also attaching an updated version that should apply on top of my
github refactor branch (or incrementally over the new set of refactor
patches that I will post shortly to the refactor thread).
Attached is a patch for master, as the refactorings have already been
merged.
Jan
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 16d78ae..167393e 100644
*** a/src/pl/plpython/Makefile
--- b/src/pl/plpython/Makefile
*** REGRESS = \
*** 79,84
--- 79,85
plpython_types \
plpython_error \
plpython_unicode \
+ plpython_composite \
plpython_drop
# where to find psql for running the tests
PSQLDIR = $(bindir)
diff --git a/src/pl/plpython/expected/plpython_composite.out b/src/pl/plpython/expected/plpython_composite.out
index ...1576588 .
*** a/src/pl/plpython/expected/plpython_composite.out
--- b/src/pl/plpython/expected/plpython_composite.out
***
*** 0
--- 1,309
+ CREATE FUNCTION multiout_simple(OUT i integer, OUT j integer) AS $$
+ return (1, 2)
+ $$ LANGUAGE plpythonu;
+ SELECT multiout_simple();
+ multiout_simple
+ -
+ (1,2)
+ (1 row)
+
+ SELECT * FROM multiout_simple();
+ i | j
+ ---+---
+ 1 | 2
+ (1 row)
+
+ SELECT i, j + 2 FROM multiout_simple();
+ i | ?column?
+ ---+--
+ 1 |4
+ (1 row)
+
+ SELECT (multiout_simple()).j + 3;
+ ?column?
+ --
+ 5
+ (1 row)
+
+ CREATE FUNCTION multiout_simple_setof(n integer = 1, OUT integer, OUT integer) RETURNS SETOF record AS $$
+ return [(1, 2)] * n
+ $$ LANGUAGE plpythonu;
+ SELECT multiout_simple_setof();
+ multiout_simple_setof
+ ---
+ (1,2)
+ (1 row)
+
+ SELECT * FROM multiout_simple_setof();
+ column1 | column2
+ -+-
+1 | 2
+ (1 row)
+
+ SELECT * FROM multiout_simple_setof(3);
+ column1 | column2
+ -+-
+1 | 2
+1 | 2
+1 | 2
+ (3 rows)
+
+ CREATE FUNCTION multiout_record_as(typ text,
+first text, OUT first text,
+second integer, OUT second integer,
+retnull boolean) RETURNS record AS $$
+ if retnull:
+ return None
+ if typ == 'dict':
+ return { 'first': first, 'second': second, 'additionalfield': 'must not cause trouble' }
+ elif typ == 'tuple':
+ return ( first, second )
+ elif typ == 'list':
+ return [ first, second ]
+ elif typ == 'obj':
+ class type_record: pass
+ type_record.first = first
+ type_record.second = second
+ return type_record
+ $$ LANGUAGE plpythonu;
+ SELECT * from multiout_record_as('dict', 'foo', 1, 'f');
+ first | second
+ ---+
+ foo | 1
+ (1 row)
+
+ SELECT multiout_record_as('dict', 'foo', 1, 'f');
+ multiout_record_as
+
+ (foo,1)
+ (1 row)
+
+ SELECT *, s IS NULL as snull from multiout_record_as('tuple', 'xxx', NULL, 'f') AS f(f, s);
+ f | s | snull
+ -+---+---
+ xxx | | t
+ (1 row)
+
+ SELECT *, f IS NULL as fnull, s IS NULL as snull from multiout_record_as('tuple', 'xxx', 1, 't') AS f(f, s);
+ f | s | fnull | snull
+ ---+---+---+---
+| | t | t
+ (1 row)
+
+ SELECT * from multiout_record_as('obj', NULL, 10, 'f');
+ first | second
+ ---+
+| 10
+ (1 row)
+
+ CREATE FUNCTION multiout_setof(n integer,
+OUT power_of_2 integer,
+OUT length integer) RETURNS SETOF record AS $$
+ for i in range(n):
+ power = 2 ** i
+ length = plpy.execute(select length('%d') % power)[0]['length']
+ yield power, length
+ $$ LANGUAGE plpythonu;
+ SELECT * FROM multiout_setof(3);
+ power_of_2 | length
+ +
+ 1 | 1
+ 2 | 1
+ 4 | 1
+ (3 rows)
+
+ SELECT multiout_setof(5);
+ multiout_setof
+
+ (1,1)
+ (2,1)
+ (4,1)
+ (8,1)
+ (16,2)
+ (5 rows)
+
+ CREATE FUNCTION multiout_return_table() RETURNS TABLE (x integer, y text) AS $$
+ return [{'x': 4, 'y' :'four'},
+ {'x': 7, 'y' :'seven'},
+ {'x': 0, 'y' :'zero'}]
+ $$ LANGUAGE plpythonu;
+ SELECT * FROM multiout_return_table();
+ x | y
+ ---+---
+ 4 | four
+ 7 | seven
+ 0 | zero
+ (3 rows)
+
+ CREATE FUNCTION multiout_array(OUT integer[], OUT text) RETURNS SETOF record AS $$
+ yield [[1], 'a']
+ yield [[1,2], 'b']
+ yield [[1,2,3], None]
+ $$ LANGUAGE plpythonu;
+ SELECT * FROM multiout_array();
+ column1 | column2
+ -+-
+ {1} | a
+ {1,2} | b
+ {1,2,3} |
+ (3 rows)
+
+ CREATE FUNCTION singleout_composite(OUT type_record) AS $$
+ return {'first': 1, 'second': 2}
+ $$ LANGUAGE plpythonu;
+ CREATE FUNCTION multiout_composite(OUT type_record) RETURNS SETOF type_record AS $$
+ return [{'first': 1, 'second': 2},
+{'first': 3, 'second': 4 }]
+ $$ LANGUAGE plpythonu;
Re: [HACKERS] REVIEW: PL/Python validator function
On 19/01/11 02:16, Hitoshi Harada wrote: Thanks. I tested the new version and looks ok. I'll mark it Ready for Commiter. Updated version against master. Jan diff --git a/src/include/catalog/pg_pltemplate.h b/src/include/catalog/pg_pltemplate.h index d987ddc..c0578f9 100644 *** a/src/include/catalog/pg_pltemplate.h --- b/src/include/catalog/pg_pltemplate.h *** DATA(insert ( pltcl t t pltcl_call_h *** 72,79 DATA(insert ( pltclu f f pltclu_call_handler _null_ _null_ $libdir/pltcl _null_ )); DATA(insert ( plperl t t plperl_call_handler plperl_inline_handler plperl_validator $libdir/plperl _null_ )); DATA(insert ( plperlu f f plperl_call_handler plperl_inline_handler plperl_validator $libdir/plperl _null_ )); ! DATA(insert ( plpythonu f f plpython_call_handler plpython_inline_handler _null_ $libdir/plpython _null_ )); ! DATA(insert ( plpython2u f f plpython_call_handler plpython_inline_handler _null_ $libdir/plpython2 _null_ )); ! DATA(insert ( plpython3u f f plpython3_call_handler plpython3_inline_handler _null_ $libdir/plpython3 _null_ )); #endif /* PG_PLTEMPLATE_H */ --- 72,79 DATA(insert ( pltclu f f pltclu_call_handler _null_ _null_ $libdir/pltcl _null_ )); DATA(insert ( plperl t t plperl_call_handler plperl_inline_handler plperl_validator $libdir/plperl _null_ )); DATA(insert ( plperlu f f plperl_call_handler plperl_inline_handler plperl_validator $libdir/plperl _null_ )); ! DATA(insert ( plpythonu f f plpython_call_handler plpython_inline_handler plpython_validator $libdir/plpython _null_ )); ! DATA(insert ( plpython2u f f plpython_call_handler plpython_inline_handler plpython_validator $libdir/plpython2 _null_ )); ! DATA(insert ( plpython3u f f plpython3_call_handler plpython3_inline_handler plpython3_validator $libdir/plpython3 _null_ )); #endif /* PG_PLTEMPLATE_H */ diff --git a/src/pl/plpython/expected/README b/src/pl/plpython/expected/README index f011528..27c995d 100644 *** a/src/pl/plpython/expected/README --- b/src/pl/plpython/expected/README *** *** 1,5 --- 1,7 Guide to alternative expected files: + plpython_error_0.out Python 2.4 and older + plpython_unicode.out any version, when server encoding != SQL_ASCII and client encoding = UTF8; else ... plpython_unicode_0.out any version, when server encoding != SQL_ASCII and client encoding != UTF8; else ... plpython_unicode_2.out Python 2.2 diff --git a/src/pl/plpython/expected/plpython_error.out b/src/pl/plpython/expected/plpython_error.out index ea4a54c..2b6141c 100644 *** a/src/pl/plpython/expected/plpython_error.out --- b/src/pl/plpython/expected/plpython_error.out *** *** 1,6 --- 1,30 -- test error handling, i forgot to restore Warn_restart in -- the trigger handler once. the errors and subsequent core dump were -- interesting. + /* Flat out Python syntax error + */ + CREATE FUNCTION python_syntax_error() RETURNS text + AS + '.syntaxerror' + LANGUAGE plpythonu; + ERROR: could not compile PL/Python function python_syntax_error + DETAIL: SyntaxError: invalid syntax (string, line 2) + /* With check_function_bodies = false the function should get defined + * and the error reported when called + */ + SET check_function_bodies = false; + CREATE FUNCTION python_syntax_error() RETURNS text + AS + '.syntaxerror' + LANGUAGE plpythonu; + SELECT python_syntax_error(); + ERROR: could not compile PL/Python function python_syntax_error + DETAIL: SyntaxError: invalid syntax (string, line 2) + /* Run the function twice to check if the hashtable entry gets cleaned up */ + SELECT python_syntax_error(); + ERROR: could not compile PL/Python function python_syntax_error + DETAIL: SyntaxError: invalid syntax (string, line 2) + RESET check_function_bodies; /* Flat out syntax error */ CREATE FUNCTION sql_syntax_error() RETURNS text diff --git a/src/pl/plpython/expected/plpython_error_0.out b/src/pl/plpython/expected/plpython_error_0.out index ...1fe3932 . *** a/src/pl/plpython/expected/plpython_error_0.out --- b/src/pl/plpython/expected/plpython_error_0.out *** *** 0 --- 1,145 + -- test error handling, i forgot to restore Warn_restart in + -- the trigger handler once. the errors and subsequent core dump were + -- interesting. + /* Flat out Python syntax error + */ + CREATE FUNCTION python_syntax_error() RETURNS text + AS + '.syntaxerror' + LANGUAGE plpythonu; + ERROR: could not compile PL/Python function python_syntax_error + DETAIL: SyntaxError: invalid syntax (line 2) + /* With check_function_bodies = false the function should get defined + * and the error reported when called + */ + SET check_function_bodies = false; + CREATE FUNCTION python_syntax_error() RETURNS text + AS + '.syntaxerror' + LANGUAGE plpythonu; + SELECT python_syntax_error(); + ERROR: could not compile PL/Python function python_syntax_error + DETAIL: SyntaxError:
Re: [HACKERS] pl/python invalidate functions with composite arguments
On 23/12/10 14:50, Jan Urbański wrote:
Here's a patch implementing properly invalidating functions that have
composite type arguments after the type changes, as mentioned in
http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's
an incremental patch on top of the plpython-refactor patch sent eariler.
Updated to master.
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index 982005b..cf7b7de 100644
*** a/src/pl/plpython/expected/plpython_types.out
--- b/src/pl/plpython/expected/plpython_types.out
*** SELECT * FROM test_type_conversion_array
*** 599,604
--- 599,656
ERROR: return value of function with array return type is not a Python sequence
CONTEXT: while creating return value
PL/Python function test_type_conversion_array_error
+ ---
+ --- Composite types
+ ---
+ CREATE TABLE employee (
+ name text,
+ basesalary integer,
+ bonus integer
+ );
+ INSERT INTO employee VALUES ('John', 100, 10), ('Mary', 200, 10);
+ CREATE OR REPLACE FUNCTION test_composite_table_input(e employee) RETURNS integer AS $$
+ return e['basesalary'] + e['bonus']
+ $$ LANGUAGE plpythonu;
+ SELECT name, test_composite_table_input(employee.*) FROM employee;
+ name | test_composite_table_input
+ --+
+ John |110
+ Mary |210
+ (2 rows)
+
+ ALTER TABLE employee DROP bonus;
+ SELECT name, test_composite_table_input(employee.*) FROM employee;
+ ERROR: KeyError: 'bonus'
+ CONTEXT: PL/Python function test_composite_table_input
+ ALTER TABLE employee ADD bonus integer;
+ UPDATE employee SET bonus = 10;
+ SELECT name, test_composite_table_input(employee.*) FROM employee;
+ name | test_composite_table_input
+ --+
+ John |110
+ Mary |210
+ (2 rows)
+
+ CREATE TYPE named_pair AS (
+ i integer,
+ j integer
+ );
+ CREATE OR REPLACE FUNCTION test_composite_type_input(p named_pair) RETURNS integer AS $$
+ return sum(p.values())
+ $$ LANGUAGE plpythonu;
+ SELECT test_composite_type_input(row(1, 2));
+ test_composite_type_input
+ ---
+ 3
+ (1 row)
+
+ ALTER TYPE named_pair RENAME TO named_pair_2;
+ SELECT test_composite_type_input(row(1, 2));
+ test_composite_type_input
+ ---
+ 3
+ (1 row)
+
--
-- Prepared statements
--
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index eeb23b7..e10b060 100644
*** a/src/pl/plpython/expected/plpython_types_3.out
--- b/src/pl/plpython/expected/plpython_types_3.out
*** SELECT * FROM test_type_conversion_array
*** 599,604
--- 599,656
ERROR: return value of function with array return type is not a Python sequence
CONTEXT: while creating return value
PL/Python function test_type_conversion_array_error
+ ---
+ --- Composite types
+ ---
+ CREATE TABLE employee (
+ name text,
+ basesalary integer,
+ bonus integer
+ );
+ INSERT INTO employee VALUES ('John', 100, 10), ('Mary', 200, 10);
+ CREATE OR REPLACE FUNCTION test_composite_table_input(e employee) RETURNS integer AS $$
+ return e['basesalary'] + e['bonus']
+ $$ LANGUAGE plpython3u;
+ SELECT name, test_composite_table_input(employee.*) FROM employee;
+ name | test_composite_table_input
+ --+
+ John |110
+ Mary |210
+ (2 rows)
+
+ ALTER TABLE employee DROP bonus;
+ SELECT name, test_composite_table_input(employee.*) FROM employee;
+ ERROR: KeyError: 'bonus'
+ CONTEXT: PL/Python function test_composite_table_input
+ ALTER TABLE employee ADD bonus integer;
+ UPDATE employee SET bonus = 10;
+ SELECT name, test_composite_table_input(employee.*) FROM employee;
+ name | test_composite_table_input
+ --+
+ John |110
+ Mary |210
+ (2 rows)
+
+ CREATE TYPE named_pair AS (
+ i integer,
+ j integer
+ );
+ CREATE OR REPLACE FUNCTION test_composite_type_input(p named_pair) RETURNS integer AS $$
+ return sum(p.values())
+ $$ LANGUAGE plpython3u;
+ SELECT test_composite_type_input(row(1, 2));
+ test_composite_type_input
+ ---
+ 3
+ (1 row)
+
+ ALTER TYPE named_pair RENAME TO named_pair_2;
+ SELECT test_composite_type_input(row(1, 2));
+ test_composite_type_input
+ ---
+ 3
+ (1 row)
+
--
-- Prepared statements
--
diff --git a/src/pl/plpython/plpython.c b/src/pl/plpython/plpython.c
index aafe556..54605fc 100644
*** a/src/pl/plpython/plpython.c
--- b/src/pl/plpython/plpython.c
*** typedef int Py_ssize_t;
*** 101,106
--- 101,107
#include nodes/makefuncs.h
#include parser/parse_type.h
Re: [HACKERS] pl/python tracebacks
On 23/12/10 14:56, Jan Urbański wrote:
Here's a patch implementing traceback support for PL/Python mentioned in
http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's
an incremental patch on top of the plpython-refactor patch sent eariler.
Updated to master.
diff --git a/src/pl/plpython/expected/plpython_do.out b/src/pl/plpython/expected/plpython_do.out
index a21b088..fb0f0e5 100644
*** a/src/pl/plpython/expected/plpython_do.out
--- b/src/pl/plpython/expected/plpython_do.out
*** NOTICE: This is plpythonu.
*** 3,6
--- 3,9
CONTEXT: PL/Python anonymous code block
DO $$ nonsense $$ LANGUAGE plpythonu;
ERROR: NameError: global name 'nonsense' is not defined
+ DETAIL: Traceback (most recent call last):
+ PL/Python anonymous code block, line 1, in module
+ nonsense
CONTEXT: PL/Python anonymous code block
diff --git a/src/pl/plpython/expected/plpython_error.out b/src/pl/plpython/expected/plpython_error.out
index ea4a54c..1e6295e 100644
*** a/src/pl/plpython/expected/plpython_error.out
--- b/src/pl/plpython/expected/plpython_error.out
*** CONTEXT: PL/Python function sql_syntax
*** 13,18
--- 13,21
ERROR: plpy.SPIError: syntax error at or near syntax
LINE 1: syntax error
^
+ DETAIL: Traceback (most recent call last):
+ PL/Python function sql_syntax_error, line 1, in module
+ plpy.execute(syntax error)
QUERY: syntax error
CONTEXT: PL/Python function sql_syntax_error
/* check the handling of uncaught python exceptions
*** CREATE FUNCTION exception_index_invalid(
*** 23,28
--- 26,34
LANGUAGE plpythonu;
SELECT exception_index_invalid('test');
ERROR: IndexError: list index out of range
+ DETAIL: Traceback (most recent call last):
+ PL/Python function exception_index_invalid, line 1, in module
+ return args[1]
CONTEXT: PL/Python function exception_index_invalid
/* check handling of nested exceptions
*/
*** CONTEXT: PL/Python function exception_
*** 37,42
--- 43,51
ERROR: plpy.SPIError: function test5(unknown) does not exist
LINE 1: SELECT test5('foo')
^
+ DETAIL: Traceback (most recent call last):
+ PL/Python function exception_index_invalid_nested, line 1, in module
+ rv = plpy.execute(SELECT test5('foo'))
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
QUERY: SELECT test5('foo')
CONTEXT: PL/Python function exception_index_invalid_nested
*** SELECT invalid_type_uncaught('rick');
*** 57,62
--- 66,74
WARNING: plpy.SPIError: unrecognized error in PLy_spi_prepare
CONTEXT: PL/Python function invalid_type_uncaught
ERROR: plpy.SPIError: type test does not exist
+ DETAIL: Traceback (most recent call last):
+ PL/Python function invalid_type_uncaught, line 3, in module
+ SD[plan] = plpy.prepare(q, [ test ])
CONTEXT: PL/Python function invalid_type_uncaught
/* for what it's worth catch the exception generated by
* the typo, and return None
*** SELECT invalid_type_reraised('rick');
*** 107,112
--- 119,127
WARNING: plpy.SPIError: unrecognized error in PLy_spi_prepare
CONTEXT: PL/Python function invalid_type_reraised
ERROR: plpy.Error: type test does not exist
+ DETAIL: Traceback (most recent call last):
+ PL/Python function invalid_type_reraised, line 6, in module
+ plpy.error(str(ex))
CONTEXT: PL/Python function invalid_type_reraised
/* no typo no messing about
*/
*** SELECT valid_type('rick');
*** 126,128
--- 141,238
(1 row)
+ /* error in nested functions to get a traceback
+ */
+ CREATE FUNCTION nested_error() RETURNS text
+ AS
+ 'def fun1():
+ plpy.error(boom)
+
+ def fun2():
+ fun1()
+
+ def fun3():
+ fun2()
+
+ fun3()
+ return not reached
+ '
+ LANGUAGE plpythonu;
+ SELECT nested_error();
+ ERROR: plpy.Error: boom
+ DETAIL: Traceback (most recent call last):
+ PL/Python function nested_error, line 10, in module
+ fun3()
+ PL/Python function nested_error, line 8, in fun3
+ fun2()
+ PL/Python function nested_error, line 5, in fun2
+ fun1()
+ PL/Python function nested_error, line 2, in fun1
+ plpy.error(boom)
+ CONTEXT: PL/Python function nested_error
+ /* raising plpy.Error is just like calling plpy.error
+ */
+ CREATE FUNCTION nested_error_raise() RETURNS text
+ AS
+ 'def fun1():
+ raise plpy.Error(boom)
+
+ def fun2():
+ fun1()
+
+ def fun3():
+ fun2()
+
+ fun3()
+ return not reached
+ '
+ LANGUAGE plpythonu;
+ SELECT nested_error_raise();
+ ERROR: plpy.Error: boom
+ DETAIL: Traceback (most recent call last):
+ PL/Python function nested_error_raise, line 10, in module
+ fun3()
+ PL/Python function nested_error_raise, line 8, in fun3
+ fun2()
+ PL/Python function nested_error_raise, line 5, in fun2
+ fun1()
+ PL/Python function nested_error_raise, line 2, in fun1
+
Re: [HACKERS] pl/python custom datatype parsers
On 23/12/10 15:15, Jan Urbański wrote:
Here's a patch implementing custom parsers for data types mentioned in
http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's
an incremental patch on top of the plpython-refactor patch sent eariler.
Updated to master.
diff --git a/contrib/hstore/Makefile b/contrib/hstore/Makefile
index 1d533fd..4e6ba7b 100644
*** a/contrib/hstore/Makefile
--- b/contrib/hstore/Makefile
***
*** 1,8
# contrib/hstore/Makefile
MODULE_big = hstore
OBJS = hstore_io.o hstore_op.o hstore_gist.o hstore_gin.o hstore_compat.o \
! crc32.o
DATA_built = hstore.sql
DATA = uninstall_hstore.sql
--- 1,17
# contrib/hstore/Makefile
MODULE_big = hstore
+
OBJS = hstore_io.o hstore_op.o hstore_gist.o hstore_gin.o hstore_compat.o \
! hstore_plpython.o crc32.o
!
! ifeq ($(with_python),yes)
!
! PG_CPPFLAGS := -I$(srcdir) -I$(top_builddir)/src/pl/plpython \
! $(python_includespec) -DHSTORE_PLPYTHON_SUPPORT
! SHLIB_LINK = $(python_libspec) $(python_additional_libs) \
! $(filter -lintl,$(LIBS)) $(CPPFLAGS)
! endif
DATA_built = hstore.sql
DATA = uninstall_hstore.sql
diff --git a/contrib/hstore/hstore.h b/contrib/hstore/hstore.h
index 8906397..6edfc70 100644
*** a/contrib/hstore/hstore.h
--- b/contrib/hstore/hstore.h
*** extern Pairs *hstoreArrayToPairs(ArrayTy
*** 174,179
--- 174,182
#define HStoreExistsAllStrategyNumber 11
#define HStoreOldContainsStrategyNumber 13 /* backwards compatibility */
+ /* PL/Python support */
+ extern void hstore_plpython_init(void);
+
/*
* defining HSTORE_POLLUTE_NAMESPACE=0 will prevent use of old function names;
* for now, we default to on for the benefit of people restoring old dumps
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index 0d6f0b6..92c8db9 100644
*** a/contrib/hstore/hstore_io.c
--- b/contrib/hstore/hstore_io.c
*** PG_MODULE_MAGIC;
*** 20,25
--- 20,26
/* old names for C functions */
HSTORE_POLLUTE(hstore_from_text, tconvert);
+ void _PG_init(void);
typedef struct
{
*** hstore_send(PG_FUNCTION_ARGS)
*** 1211,1213
--- 1212,1220
PG_RETURN_BYTEA_P(pq_endtypsend(buf));
}
+
+ void
+ _PG_init(void)
+ {
+ hstore_plpython_init();
+ }
diff --git a/contrib/hstore/hstore_plpython.c b/contrib/hstore/hstore_plpython.c
index ...4ba111a .
*** a/contrib/hstore/hstore_plpython.c
--- b/contrib/hstore/hstore_plpython.c
***
*** 0
--- 1,251
+ /*
+ * contrib/src/hstore_plpython.c
+ *
+ * bidirectional transformation between hstores and Python dictionary objects
+ */
+
+ /* Only build if PL/Python support is needed */
+ #if defined(HSTORE_PLPYTHON_SUPPORT)
+
+ #if defined(_MSC_VER) defined(_DEBUG)
+ /* Python uses #pragma to bring in a non-default libpython on VC++ if
+ * _DEBUG is defined */
+ #undef _DEBUG
+ /* Also hide away errcode, since we load Python.h before postgres.h */
+ #define errcode __msvc_errcode
+ #include Python.h
+ #undef errcode
+ #define _DEBUG
+ #elif defined (_MSC_VER)
+ #define errcode __msvc_errcode
+ #include Python.h
+ #undef errcode
+ #else
+ #include Python.h
+ #endif
+
+ #include postgres.h
+ #include utils/guc.h
+ #include utils/builtins.h
+ #include utils/syscache.h
+ #include catalog/namespace.h
+
+ #include plpython.h
+ #include hstore.h
+
+ static Oid get_hstore_oid(const char *name);
+ static void set_hstore_parsers(Oid);
+
+ static PyObject *hstore_to_dict(void *, Datum);
+ static Datum dict_to_hstore(void *, int32, PyObject *);
+
+ /* GUC variables */
+
+ static char *hstore_name;
+
+ /* Previous hstore OID */
+
+ static Oid previous;
+
+ PLyParsers parsers = {
+ .in = hstore_to_dict,
+ .out = dict_to_hstore
+ };
+
+ static PyObject *
+ hstore_to_dict(void *ignored, Datum d)
+ {
+ HStore *hstore = DatumGetHStoreP(d);
+ char*base;
+ HEntry *entries;
+ int count;
+ int i;
+ PyObject*ret;
+
+ base = STRPTR(hstore);
+ entries = ARRPTR(hstore);
+
+ ret = PyDict_New();
+
+ count = HS_COUNT(hstore);
+
+ for (i = 0; i count; i++)
+ {
+ PyObject *key, *val;
+
+ key = PyString_FromStringAndSize(HS_KEY(entries, base, i),
+ HS_KEYLEN(entries, i));
+ if (HS_VALISNULL(entries, i)) {
+ Py_INCREF(Py_None);
+ val = Py_None;
+ }
+ else {
+ val = PyString_FromStringAndSize(HS_VAL(entries, base, i),
+ HS_VALLEN(entries, i));
+ }
+
+ PyDict_SetItem(ret, key, val);
+ }
+
+ return ret;
+ }
+
+ static Datum
+ dict_to_hstore(void *ignored, int32 typmod, PyObject *dict)
+ {
+ HStore *hstore;
+ int pcount;
+ Pairs *pairs;
+ PyObject *key;
+ PyObject *value;
+ Py_ssize_t pos;
+ char
Re: [HACKERS] pl/python explicit subtransactions
On 23/12/10 15:32, Jan Urbański wrote:
Here's a patch implementing explicitly starting subtransactions mentioned in
http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's
an incremental patch on top of the spi-in-subxacts patch sent eariler.
Updated to the spi-in-subxacts version sent earlier.
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 16d78ae..33dddc6 100644
*** a/src/pl/plpython/Makefile
--- b/src/pl/plpython/Makefile
*** REGRESS = \
*** 79,84
--- 79,85
plpython_types \
plpython_error \
plpython_unicode \
+ plpython_subxact \
plpython_drop
# where to find psql for running the tests
PSQLDIR = $(bindir)
diff --git a/src/pl/plpython/expected/README b/src/pl/plpython/expected/README
index f011528..362cc0d 100644
*** a/src/pl/plpython/expected/README
--- b/src/pl/plpython/expected/README
*** plpython_unicode_2.out Python 2.2
*** 6,8
--- 6,11
plpython_unicode_3.out Python 2.3 through 3.1
plpython_types_3.out Python 3.1
+
+ plpython_subxact.out Python 2.6 through 3.1
+ plpython_subxact_0.out older Pythons that don't have the with statement
diff --git a/src/pl/plpython/expected/plpython_subxact.out b/src/pl/plpython/expected/plpython_subxact.out
index ...9888a20 .
*** a/src/pl/plpython/expected/plpython_subxact.out
--- b/src/pl/plpython/expected/plpython_subxact.out
***
*** 0
--- 1,330
+ -- test explicit subtransaction starting
+ /* Test table to see if transactions get properly rolled back
+ */
+ CREATE TABLE subxact_tbl (
+ i integer
+ );
+ /* Explicit case for Python 2.6
+ */
+ CREATE FUNCTION subxact_test(what_error text = NULL) RETURNS text
+ AS $$
+ import sys
+ subxact = plpy.subxact()
+ subxact.__enter__()
+ exc = True
+ try:
+ try:
+ plpy.execute(insert into subxact_tbl values(1))
+ plpy.execute(insert into subxact_tbl values(2))
+ if what_error == SPI:
+ plpy.execute(insert into subxact_tbl values('oops'))
+ elif what_error == Python:
+ plpy.attribute_error
+ except:
+ exc = False
+ subxact.__exit__(*sys.exc_info())
+ raise
+ finally:
+ if exc:
+ subxact.__exit__(None, None, None)
+ $$ LANGUAGE plpythonu;
+ SELECT subxact_test();
+ subxact_test
+ --
+
+ (1 row)
+
+ SELECT * FROM subxact_tbl;
+ i
+ ---
+ 1
+ 2
+ (2 rows)
+
+ TRUNCATE subxact_tbl;
+ SELECT subxact_test('SPI');
+ ERROR: plpy.SPIError: invalid input syntax for integer: oops
+ LINE 1: insert into subxact_tbl values('oops')
+^
+ QUERY: insert into subxact_tbl values('oops')
+ CONTEXT: PL/Python function subxact_test
+ SELECT * FROM subxact_tbl;
+ i
+ ---
+ (0 rows)
+
+ TRUNCATE subxact_tbl;
+ SELECT subxact_test('Python');
+ ERROR: AttributeError: 'module' object has no attribute 'attribute_error'
+ CONTEXT: PL/Python function subxact_test
+ SELECT * FROM subxact_tbl;
+ i
+ ---
+ (0 rows)
+
+ TRUNCATE subxact_tbl;
+ /* Context manager case for Python =2.6
+ */
+ CREATE FUNCTION subxact_ctx_test(what_error text = NULL) RETURNS text
+ AS $$
+ with plpy.subxact():
+ plpy.execute(insert into subxact_tbl values(1))
+ plpy.execute(insert into subxact_tbl values(2))
+ if what_error == SPI:
+ plpy.execute(insert into subxact_tbl values('oops'))
+ elif what_error == Python:
+ plpy.attribute_error
+ $$ LANGUAGE plpythonu;
+ SELECT subxact_ctx_test();
+ subxact_ctx_test
+ --
+
+ (1 row)
+
+ SELECT * FROM subxact_tbl;
+ i
+ ---
+ 1
+ 2
+ (2 rows)
+
+ TRUNCATE subxact_tbl;
+ SELECT subxact_ctx_test('SPI');
+ ERROR: plpy.SPIError: invalid input syntax for integer: oops
+ LINE 1: insert into subxact_tbl values('oops')
+^
+ QUERY: insert into subxact_tbl values('oops')
+ CONTEXT: PL/Python function subxact_ctx_test
+ SELECT * FROM subxact_tbl;
+ i
+ ---
+ (0 rows)
+
+ TRUNCATE subxact_tbl;
+ SELECT subxact_ctx_test('Python');
+ ERROR: AttributeError: 'module' object has no attribute 'attribute_error'
+ CONTEXT: PL/Python function subxact_ctx_test
+ SELECT * FROM subxact_tbl;
+ i
+ ---
+ (0 rows)
+
+ TRUNCATE subxact_tbl;
+ /* Nested subtransactions
+ */
+ CREATE FUNCTION subxact_nested_test(swallow boolean = 'f') RETURNS text
+ AS $$
+ plpy.execute(insert into subxact_tbl values(1))
+ with plpy.subxact():
+ plpy.execute(insert into subxact_tbl values(2))
+ try:
+ with plpy.subxact():
+ plpy.execute(insert into subxact_tbl values(3))
+ plpy.execute(error)
+ except plpy.SPIError, e:
+ if not swallow:
+ raise
+ plpy.notice(Swallowed %r % e)
+ return ok
+ $$ LANGUAGE plpythonu;
+ SELECT subxact_nested_test();
+ ERROR: plpy.SPIError: syntax error at or near error
+ LINE 1: error
+ ^
+ QUERY: error
+ CONTEXT: PL/Python function subxact_nested_test
+
Re: [HACKERS] pl/python custom exceptions for SPI
On 11/01/11 12:20, Jan Urbański wrote:
On 11/01/11 01:27, Tom Lane wrote:
Hannu Krosing ha...@2ndquadrant.com writes:
On 10.1.2011 17:20, Jan Urbański wrote:
I changed that patch to use Perl instead of sed to generate the
exceptions, which should be a more portable.
Updated as an incremental patch on to of the recently sent version of
explicit-subxacts.
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 33dddc6..1fe386b 100644
*** a/src/pl/plpython/Makefile
--- b/src/pl/plpython/Makefile
*** rpathdir = $(python_libdir)
*** 38,44
NAME = plpython$(python_majorversion)
OBJS = plpython.o
!
# Python on win32 ships with import libraries only for Microsoft Visual C++,
# which are not compatible with mingw gcc. Therefore we need to build a
--- 38,44
NAME = plpython$(python_majorversion)
OBJS = plpython.o
! SPIEXCEPTIONS = spiexceptions.h
# Python on win32 ships with import libraries only for Microsoft Visual C++,
# which are not compatible with mingw gcc. Therefore we need to build a
*** PSQLDIR = $(bindir)
*** 86,93
include $(top_srcdir)/src/Makefile.shlib
! all: all-lib
install: all installdirs install-lib
ifeq ($(python_majorversion),2)
--- 86,102
include $(top_srcdir)/src/Makefile.shlib
+ .PHONY: gen-spiexceptions
! # Generate spiexceptions.h from utils/errcodes.h
! spiexceptions.h: $(top_srcdir)/src/include/utils/errcodes.h
! $(PERL) $(srcdir)/generate-spiexceptions.pl $^ $(SPIEXCEPTIONS)
!
! gen-spiexceptions: $(SPIEXCEPTIONS)
!
! all: gen-spiexceptions all-lib
!
! distprep: gen-spiexceptions
install: all installdirs install-lib
ifeq ($(python_majorversion),2)
*** endif
*** 134,143
submake:
$(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X)
! clean distclean maintainer-clean: clean-lib
rm -f $(OBJS)
rm -rf results
rm -f regression.diffs regression.out
ifeq ($(PORTNAME), win32)
rm -f python${pytverstr}.def
endif
--- 143,157
submake:
$(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X)
! clean distclean: clean-lib
rm -f $(OBJS)
rm -rf results
rm -f regression.diffs regression.out
+
+ # since we distribute spiexceptions.h, only remove it in maintainer-clean
+ maintainer-clean: clean distclean
+ rm -f $(SPIEXCEPTIONS)
+
ifeq ($(PORTNAME), win32)
rm -f python${pytverstr}.def
endif
diff --git a/src/pl/plpython/expected/plpython_error.out b/src/pl/plpython/expected/plpython_error.out
index 4eeda6f..45ce136 100644
*** a/src/pl/plpython/expected/plpython_error.out
--- b/src/pl/plpython/expected/plpython_error.out
*** CREATE FUNCTION sql_syntax_error() RETUR
*** 8,14
'plpy.execute(syntax error)'
LANGUAGE plpythonu;
SELECT sql_syntax_error();
! ERROR: plpy.SPIError: syntax error at or near syntax
LINE 1: syntax error
^
QUERY: syntax error
--- 8,14
'plpy.execute(syntax error)'
LANGUAGE plpythonu;
SELECT sql_syntax_error();
! ERROR: spiexceptions.SyntaxError: syntax error at or near syntax
LINE 1: syntax error
^
QUERY: syntax error
*** CREATE FUNCTION exception_index_invalid_
*** 30,36
return rv[0]'
LANGUAGE plpythonu;
SELECT exception_index_invalid_nested();
! ERROR: plpy.SPIError: function test5(unknown) does not exist
LINE 1: SELECT test5('foo')
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--- 30,36
return rv[0]'
LANGUAGE plpythonu;
SELECT exception_index_invalid_nested();
! ERROR: spiexceptions.UndefinedFunction: function test5(unknown) does not exist
LINE 1: SELECT test5('foo')
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
*** return None
*** 50,56
'
LANGUAGE plpythonu;
SELECT invalid_type_uncaught('rick');
! ERROR: plpy.SPIError: type test does not exist
CONTEXT: PL/Python function invalid_type_uncaught
/* for what it's worth catch the exception generated by
* the typo, and return None
--- 50,56
'
LANGUAGE plpythonu;
SELECT invalid_type_uncaught('rick');
! ERROR: spiexceptions.UndefinedObject: type test does not exist
CONTEXT: PL/Python function invalid_type_uncaught
/* for what it's worth catch the exception generated by
* the typo, and return None
*** SELECT valid_type('rick');
*** 116,121
--- 116,159
(1 row)
+ /* Check catching specific types of exceptions
+ */
+ CREATE TABLE specific (
+ i integer PRIMARY KEY
+ );
+ NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index specific_pkey for table specific
+ CREATE FUNCTION specific_exception(i integer) RETURNS void AS
+ $$
+ from plpy import spiexceptions
+ try:
+ plpy.execute(insert into specific values (%s) % (i or NULL));
+ except
Re: [HACKERS] sepgsql contrib module
(2011/01/27 22:26), Robert Haas wrote: 2011/1/27 KaiGai Koheikai...@ak.jp.nec.com: - Error messages become obtaining %m, when the error was originated from the libselinux interfaces. It will provides DBA a hint why interactions with SELinux does not work well. No space before the : %m, please. Also this word looks like a typo: seuciryt The attached patch eliminated spaces before : %m, and fixed up the typo. Thanks, -- KaiGai Kohei kai...@ak.jp.nec.com sepgsql-v9.1-fixup.2.patch Description: application/octect-stream -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] mingw64
With the attached patch I have managed to get Postgres built and running
a clean set of regression tests using the Mingw64 toolset on my 64bit
Windows7Pro machine.
There's lots of work still to do (lots of warnings to make go away, for
example), but this is a pretty encouraging start.
It should also answer most of the questions XiaoboGu had about how to
build the client libraries, since this falls out along the way.
NB: on Windows 7 you really need to build and run as a non-admin user.
Windows blows away the PATH for your app otherwise. This took me a while
to get to the bottom of.
cheers
andrew
diff --git a/config/ac_func_accept_argtypes.m4 b/config/ac_func_accept_argtypes.m4
index 4ac5ce0..43e9595 100644
--- a/config/ac_func_accept_argtypes.m4
+++ b/config/ac_func_accept_argtypes.m4
@@ -45,8 +45,8 @@ AC_DEFUN([AC_FUNC_ACCEPT_ARGTYPES],
[AC_CACHE_VAL(ac_cv_func_accept_arg1,dnl
[AC_CACHE_VAL(ac_cv_func_accept_arg2,dnl
[AC_CACHE_VAL(ac_cv_func_accept_arg3,dnl
-[for ac_cv_func_accept_return in 'int' 'unsigned int PASCAL'; do
- for ac_cv_func_accept_arg1 in 'int' 'unsigned int'; do
+[for ac_cv_func_accept_return in 'int' 'unsigned int PASCAL' '__int64'; do
+ for ac_cv_func_accept_arg1 in 'int' 'unsigned int' '__int64'; do
for ac_cv_func_accept_arg2 in 'struct sockaddr *' 'const struct sockaddr *' 'void *'; do
for ac_cv_func_accept_arg3 in 'int' 'size_t' 'socklen_t' 'unsigned int' 'void'; do
AC_TRY_COMPILE(
diff --git a/configure b/configure
index f96c0ff..96a1037 100755
--- a/configure
+++ b/configure
@@ -18472,8 +18472,8 @@ else
if test ${ac_cv_func_accept_arg3+set} = set; then
$as_echo_n (cached) 6
else
- for ac_cv_func_accept_return in 'int' 'unsigned int PASCAL'; do
- for ac_cv_func_accept_arg1 in 'int' 'unsigned int'; do
+ for ac_cv_func_accept_return in 'int' 'unsigned int PASCAL' '__int64'; do
+ for ac_cv_func_accept_arg1 in 'int' 'unsigned int' '__int64'; do
for ac_cv_func_accept_arg2 in 'struct sockaddr *' 'const struct sockaddr *' 'void *'; do
for ac_cv_func_accept_arg3 in 'int' 'size_t' 'socklen_t' 'unsigned int' 'void'; do
cat conftest.$ac_ext _ACEOF
diff --git a/src/include/c.h b/src/include/c.h
index bec1a8c..1f2813c 100644
--- a/src/include/c.h
+++ b/src/include/c.h
@@ -58,7 +58,7 @@
#endif
#include postgres_ext.h
-#if _MSC_VER = 1400
+#if _MSC_VER = 1400 || defined(WIN64)
#define errcode __msvc_errcode
#include crtdefs.h
#undef errcode
diff --git a/src/include/port.h b/src/include/port.h
index 584a89e..c9f61b3 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -324,8 +324,12 @@ extern FILE *pgwin32_fopen(const char *, const char *);
#define fopen(a,b) pgwin32_fopen(a,b)
#endif
+#ifndef popen
#define popen(a,b) _popen(a,b)
+#endif
+#ifndef pclose
#define pclose(a) _pclose(a)
+#endif
/* New versions of MingW have gettimeofday, old mingw and msvc don't */
#ifndef HAVE_GETTIMEOFDAY
diff --git a/src/include/port/win32.h b/src/include/port/win32.h
index 1473d9e..9f4ad35 100644
--- a/src/include/port/win32.h
+++ b/src/include/port/win32.h
@@ -4,7 +4,9 @@
#define WIN32_ONLY_COMPILER
#endif
+#ifndef _WIN32_WINNT
#define _WIN32_WINNT 0x0501
+#endif
/*
* Always build with SSPI support. Keep it as a #define in case
* we want a switch to disable it sometime in the future.
@@ -18,8 +20,8 @@
#undef ERROR
#define _WINSOCKAPI_
-#include windows.h
#include winsock2.h
+#include windows.h
#include ws2tcpip.h
#undef small
#include process.h
diff --git a/src/include/port/win32/sys/socket.h b/src/include/port/win32/sys/socket.h
index 97a5041..e811919 100644
--- a/src/include/port/win32/sys/socket.h
+++ b/src/include/port/win32/sys/socket.h
@@ -13,6 +13,7 @@
*/
#include winsock2.h
#include ws2tcpip.h
+#include windows.h
#undef ERROR
#undef small
diff --git a/src/port/getaddrinfo.c b/src/port/getaddrinfo.c
index 4133aed..ba2706d 100644
--- a/src/port/getaddrinfo.c
+++ b/src/port/getaddrinfo.c
@@ -329,7 +329,7 @@ gai_strerror(int errcode)
return Not enough memory;
#endif
#ifdef EAI_NODATA
-#ifndef WIN32_ONLY_COMPILER /* MSVC complains because another case has the
+#ifndef WIN32/* MSVC complains because another case has the
* same value */
case EAI_NODATA:
return No host data of that type was found;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mingw64
On Fri, Jan 28, 2011 at 8:12 AM, Andrew Dunstan and...@dunslane.net wrote: With the attached patch I have managed to get Postgres built and running a clean set of regression tests using the Mingw64 toolset on my 64bit Windows7Pro machine. There's lots of work still to do (lots of warnings to make go away, for example), but this is a pretty encouraging start. It should also answer most of the questions XiaoboGu had about how to build the client libraries, since this falls out along the way. NB: on Windows 7 you really need to build and run as a non-admin user. Windows blows away the PATH for your app otherwise. This took me a while to get to the bottom of. Hi andrew, It's a great job you have done, but can you send me just the updated files, because I don't have SVN set up, and not fimiliar with the SVN commands. Thanks. Xiaobo Gu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mingw64
On 01/27/2011 08:23 PM, Xiaobo Gu wrote: On Fri, Jan 28, 2011 at 8:12 AM, Andrew Dunstanand...@dunslane.net wrote: With the attached patch I have managed to get Postgres built and running a clean set of regression tests using the Mingw64 toolset on my 64bit Windows7Pro machine. There's lots of work still to do (lots of warnings to make go away, for example), but this is a pretty encouraging start. It should also answer most of the questions XiaoboGu had about how to build the client libraries, since this falls out along the way. NB: on Windows 7 you really need to build and run as a non-admin user. Windows blows away the PATH for your app otherwise. This took me a while to get to the bottom of. Hi andrew, It's a great job you have done, but can you send me just the updated files, because I don't have SVN set up, and not fimiliar with the SVN commands. Thanks. We don't use SVN. You can apply the patch with something like patch -p 1 w64.patch cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mingw64
On Fri, Jan 28, 2011 at 8:12 AM, Andrew Dunstan and...@dunslane.net wrote: With the attached patch I have managed to get Postgres built and running a clean set of regression tests using the Mingw64 toolset on my 64bit Windows7Pro machine. There's lots of work still to do (lots of warnings to make go away, for example), but this is a pretty encouraging start. It should also answer most of the questions XiaoboGu had about how to build the client libraries, since this falls out along the way. NB: on Windows 7 you really need to build and run as a non-admin user. Windows blows away the PATH for your app otherwise. This took me a while to get to the bottom of. Hi I am on my 32bit Windows XP SP3 now, using a non-admin user named postgres, configure and make pass, but make install failed, configure --without-zlib --host=x86_64-w64-mingw32 --with-system-tzdata=/usr/share/zoneinfo --prefix=D:/psqlbin . $make .. $ make install D:/Amber/Devtool/MinGW64-1.0-20100913/bin/make -C src install make[1]: Entering directory `d:/amber/devproj/postgresql-9.0.2/src' /bin/mkdir -p 'D:/psqlbin/lib/postgresql/pgxs/src' process_begin: CreateProcess(NULL, /bin/mkdir -p D:/psqlbin/lib/postgresql/pgxs/ src, ...) failed. make (e=3): System can't find the specified path (translated from Chinese by me) make[1]: *** [installdirs-local] Error 3 make[1]: Leaving directory `d:/amber/devproj/postgresql-9.0.2/src' make: *** [install] Error 2 User postgres has full access previlige on D:/psqlbin directory I'll try on my 64bit Win7 home basic later. Xiaobo Gu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mingw64
On 01/27/2011 10:37 PM, Xiaobo Gu wrote: --with-system-tzdata=/usr/share/zoneinfo Why on earth are you doing this on Windows? That's crazy. Did you run make check? You should always do that after a build before you install. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mingw64
On Fri, Jan 28, 2011 at 11:44 AM, Andrew Dunstan and...@dunslane.net wrote: On 01/27/2011 10:37 PM, Xiaobo Gu wrote: --with-system-tzdata=/usr/share/zoneinfo Why on earth are you doing this on Windows? That's crazy. Did you run make check? You should always do that after a build before you install. configure does not pass if I omit --with-system-tzdata=/usr/share/zoneinfo Content of config.log This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by PostgreSQL configure 9.0.2, which was generated by GNU Autoconf 2.63. Invocation command line was $ ./configure --without-zlib --host=x86_64-w64-mingw32 --prefix=D:/psqlbin ## - ## ## Platform. ## ## - ## hostname = kzx-28F-tempdw uname -m = i686 uname -r = 1.0.16(0.48/3/2) uname -s = MINGW32_NT-5.1 uname -v = 2010-09-29 00:07 /usr/bin/uname -p = unknown /bin/uname -X = unknown /bin/arch = unknown /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown /usr/bin/hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown PATH: . PATH: /usr/local/bin PATH: /mingw/bin PATH: /bin PATH: /c/Program Files/Common Files/Microsoft Shared/Windows Live PATH: /c/WINDOWS/system32 PATH: /c/WINDOWS PATH: /c/WINDOWS/System32/Wbem PATH: /c/PROGRA~1/IBM/CLIENT~1 PATH: /c/PROGRA~1/IBM/CLIENT~1/Shared PATH: /c/PROGRA~1/IBM/CLIENT~1/Emulator PATH: /c/Program Files/TortoiseSVN/bin PATH: /c/Program Files/Common Files/Thunder Network/KanKan/Codecs PATH: /d/Amber/Program/MIT/Kerberos/bin PATH: . ## --- ## ## Core tests. ## ## --- ## configure:2069: checking build system type configure:2087: result: i686-pc-mingw32 configure:2109: checking host system type configure:2124: result: x86_64-w64-mingw32 configure:2148: checking which template to use configure:2247: result: win32 configure:2354: checking whether to build with 64-bit integer date/time support configure:2389: result: yes configure:2396: checking whether NLS is wanted configure:2430: result: no configure:2438: checking for default port number configure:2467: result: 5432 configure:2886: checking for block size configure:2926: result: 8kB configure:2938: checking for segment size configure:2971: result: 1GB configure:2983: checking for WAL block size configure:3024: result: 8kB configure:3036: checking for WAL segment size configure:3077: result: 16MB configure:3135: checking for x86_64-w64-mingw32-gcc configure:3151: found /mingw/bin/x86_64-w64-mingw32-gcc configure:3162: result: x86_64-w64-mingw32-gcc configure:3240: checking for C compiler version configure:3248: x86_64-w64-mingw32-gcc --version 5 x86_64-w64-mingw32-gcc.exe (GCC) 4.5.2 20100913 (prerelease) Copyright (C) 2010 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. configure:3252: $? = 0 configure:3259: x86_64-w64-mingw32-gcc -v 5 Using built-in specs. COLLECT_GCC=D:\Amber\Devtool\MinGW64-1.0-20100913\bin\x86_64-w64-mingw32-gcc.exe COLLECT_LTO_WRAPPER=d:/amber/devtool/mingw64-1.0-20100913/bin/../libexec/gcc/x86_64-w64-mingw32/4.5.2/lto-wrapper.exe Target: x86_64-w64-mingw32 Configured with: ../../../build/gcc/src/configure --target=x86_64-w64-mingw32 --prefix=/c/bb/vista64-mingw32/mingw-x86-x86_64/build/build/root --with-sysroot=/c/bb/vista64-mingw32/mingw-x86-x86_64/build/build/root --enable-languages=all,obj-c++ --enable-fully-dynamic-string --disable-multilib Thread model: win32 gcc version 4.5.2 20100913 (prerelease) (GCC) configure:3263: $? = 0 configure:3270: x86_64-w64-mingw32-gcc -V 5 x86_64-w64-mingw32-gcc.exe: '-V' option must have argument configure:3274: $? = 1 configure:3297: checking for C compiler default output file name configure:3319: x86_64-w64-mingw32-gccconftest.c 5 configure:3323: $? = 0 configure:3361: result: a.exe configure:3380: checking whether the C compiler works configure:3390: ./a.exe ./configure: line 3392: ./a.exe: Bad file number configure:3394: $? = 126 configure:3413: result: yes configure:3420: checking whether we are cross compiling configure:3422: result: yes configure:3425: checking for suffix of executables configure:3432: x86_64-w64-mingw32-gcc -o conftest.execonftest.c 5 configure:3436: $? = 0 configure:3462: result: .exe configure:3468: checking for suffix of object files configure:3494: x86_64-w64-mingw32-gcc -c conftest.c 5 configure:3498: $? = 0 configure:3523: result: o configure:3527: checking whether we are using the GNU C compiler configure:3556: x86_64-w64-mingw32-gcc -c conftest.c 5 configure:3563: $? = 0 configure:3580: result: yes configure:3589: checking whether x86_64-w64-mingw32-gcc accepts -g configure:3619: x86_64-w64-mingw32-gcc -c -g conftest.c 5 configure:3626: $? = 0 configure:3727: result: yes configure:3744: checking for
Re: [HACKERS] Extensions support for pg_dump, patch v27
On Thu, Jan 27, 2011 at 22:48, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Itagaki Takahiro itagaki.takah...@gmail.com writes: I found pg_restore with -c option fails when an extension is created in pg_catalog. Nice catch, thank you very much (again) for finding those :) Seems good. extern bool extension_relocatable_p(Oid ext_oid); predicate. Maybe I've done too much Emacs Lisp coding at the time I added that function, but it looked natural (enough) to me :) Hmmm, I like extension_is_relocatable() or so... Including the above, I wrote a patch on your patch for minor cleanup. Please merge reasonable parts in it. * access() is not portable. The pre-checking with access() doesn't seems needed because the same error will be raised in parse_extension_control_file(). * There are some dead code in the patch. For example, you exported ObjectAddresses to public, but it is not used in extension.c actually. I reverted some of changes. * Should we support absolute control file paths? Absolute paths are supported by get_extension_absolute_path(), but I'm not sure actual use-cases. * Each ereport(ERROR) should have a reasonable errcode unless they are an internal logic error, and whether the error message follows our guidline (starting with a lower case character, etc.) * Changed create_extension_with_user_data to a static variable. CreateExtensionAddress and create_extension are still exported. We could have better names for them -- CurrentExtensionAddress and in_create_extension? Or, in_create_extension might be replaced with CreateExtensionAddress.objectId != InvalidOid. * Added psql tab completion for CREATE/DROP/ALTER EXTENSION. * Use palloc0() instead of palloc() and memset(0). * Several code cleanup. -- Itagaki Takahiro extension-diff-on.v28a.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] FPI
I was thinking about full-page writes again tonight. I'm still
wondering about the feasibility of getting rid of full-page writes for
certain operations. We can do this, I think, in any case where we can
convince ourselves that if the original operation, or a redo of the
original operation, leaves behind a torn page, a subsequent redo will
still DTRT.
I think that both tuple freezing (XLOG_HEAP2_FREEZE) and heap
deletions (XLOG_HEAP_DELETE) are close to having this property. The
updates are pretty much unconditional - they certainly don't depend on
existing xmin/xmax/ctid/infomask bits being in any particular state.
But this (in the redo logic) is a problem:
if (XLByteLE(lsn, PageGetLSN(page)))
{
UnlockReleaseBuffer(buffer);
return;
}
A crash might leave the updated LSN on disk, while the rest of the
page isn't. If this is workable at all, we'd need to at least
suppress the above behavior (I believe it would still be OK after we
reach consistency, but not before).
Tonight I realized that I think we could also make this property hold
for updates, only with respect to the block containing the *old* tuple
(which is basically getting the equivalent of a delete). While
reducing WAL just for deletes and freezing might be judged not worth
the additional complexity, maybe being able to improve the update case
would put it over the top.
Thoughts? Any idea how much of our WAL volume comes from FPIs vs.
everything else? Index changes vs. heap changes?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Spread checkpoint sync
Robert Haas wrote: During each cluster, the system probably slows way down, and then recovers when the queue is emptied. So the TPS improvement isn't at all a uniform speedup, but simply relief from the stall that would otherwise result from a full queue. That does seem to be the case here. http://www.2ndquadrant.us/pgbench-results/index.htm now has results from my a long test series, at two database scales that caused many backend fsyncs during earlier tests. Set #5 is the existing server code, #6 is with the patch applied. There are zero backend fsync calls with the patch applied, which isn't surprising given how simple the schema is on this test case. An average of a 14% TPS gain appears at a scale of 500 and a 8% one at 1000; the attached CSV file summarizes the average figures for the archives. The gains do appear to be from smoothing out the dead period that normally occur during the sync phase of the checkpoint. For example, here are the fastest runs at scale=1000/clients=256 with and without the patch: http://www.2ndquadrant.us/pgbench-results/436/index.html (tps=361) http://www.2ndquadrant.us/pgbench-results/486/index.html (tps=380) Here the difference in how much less of a slowdown there is around the checkpoint end points is really obvious, and obviously an improvement. You can see the same thing to a lesser extent at the other end of the scale; here's the fastest runs at scale=500/clients=16: http://www.2ndquadrant.us/pgbench-results/402/index.html (tps=590) http://www.2ndquadrant.us/pgbench-results/462/index.html (tps=643) Where there are still very ugly maximum latency figures here in every case, these periods just aren't as wide with the patch in place. I'm moving onto some brief testing some of the newer kernel behavior here, then returning to testing the other checkpoint spreading ideas on top of this compation patch, presuming something like it will end up being committed first. I think it's safe to say I can throw away the changes to try and alter the fsync absorption code present in what I submitted before, as this scheme does a much better job of avoiding that problem than those earlier queue alteration ideas. I'm glad Robert grabbed the right one from the pile of ideas I threw out for what else might help here. P.S. Yes, I know I have other review work to do as well. Starting on the rest of that tomorrow. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us PostgreSQL 9.0 High Performance: http://www.2ndQuadrant.com/books ,,Unmodified,,Compacted Fsync,,, scale,clients,tps,max_latency,tps,max_latency,TPS Gain,% Gain 500,16,557,17963.41,631,17116.31,74,13.3% 500,32,587,25838.8,655,24311.54,68,11.6% 500,64,628,35198.39,727,38040.39,99,15.8% 500,128,621,41001.91,687,48195.77,66,10.6% 500,256,632,49610.39,747,46799.48,115,18.2% ,,, 1000,16,306,39298.95,321,40826.58,15,4.9% 1000,32,314,40120.35,345,27910.51,31,9.9% 1000,64,334,46244.86,358,45138.1,24,7.2% 1000,128,343,72501.57,372,47125.46,29,8.5% 1000,256,321,80588.63,350,83232.14,29,9.0% -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_ctl failover Re: [HACKERS] Latches, signals, and waiting
I did s/failover/promote. Here is the updated patch. I rebased the patch to current git master. I'm thinking about implementing a function which does a promotion for the standby. It will make pgpool lot easier to control the promotion since it allow to fire the promotion operation (either creating a trigger file or sending a signal) via SQL, not ssh etc. If there's enough interest, I will propose such a function for next CF. -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: RangeTypes
Updated patch. Changes: * Documentation for operators/functions * a comprehensive set of operators and functions * BTree opclass * Hash opclass * built-in range types: - PERIOD (timestamp) - PERIODTZ (timestamptz) - DATERANGE (date) - INTRANGE (int4) - NUMRANGE (numeric) * added subtype float function to the API, which will be useful for GiST * created canonical functions for intrange and daterange, so that: '[1,5]'::intrange = '[1,6)'::intrange * added length() function, written in SQL as: select upper($1) - lower($1) which uses polymorphic - operator to avoid the need to give the subtype subtract function and return type to the generic API Open items: * More documentation work * Settle any representation/alignment concerns * Should the new length() function be marked as immutable, stable, or volatile? It uses the polymorphic - operator, and I suppose someone could define a non-immutable version of that before calling length(). Then again, it is likely to be inlined anyway, right? * GiST - docs - catalog work - implementation * typmod support (optional) This is nearing completion. GiST is by far the most amount of effort remaining that I'm aware of. Comments about the API, naming, representation, interface, funcationality, grammar, etc. are welcome. Regards, Jeff Davis rangetypes-20110127.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_ctl failover Re: [HACKERS] Latches, signals, and waiting
On Fri, Jan 28, 2011 at 08:44, Tatsuo Ishii is...@postgresql.org wrote: I did s/failover/promote. Here is the updated patch. I rebased the patch to current git master. I'm thinking about implementing a function which does a promotion for the standby. It will make pgpool lot easier to control the promotion since it allow to fire the promotion operation (either creating a trigger file or sending a signal) via SQL, not ssh etc. I agree that having this available via SQL would be useful in a number of cases. pgpool or such being one, but also for example pgadmin. If there's enough interest, I will propose such a function for next CF. Just as a reminder, remember that next CF means 9.2. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
