Re: [PATCHES] Testing pg_terminate_backend()

2008-04-21 Thread Magnus Hagander
Bruce Momjian wrote:
 bruce wrote:
  Magnus Hagander wrote:
   Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
 Attached is my test script.   I ran it for 14 hours (asserts
 on), running 450 regression tests, with up to seven backends
 killed per regression test.

Hmm, there are something on the order of 1 SQL commands in
our regression tests, so even assuming perfect randomness
you've exercised SIGTERM on maybe 10% of them --- and of course
there's multiple places in a complex DDL command where SIGTERM
might conceivably be a problem.

Who was volunteering to run this 24x7 for awhile?
   
   That was me. As long as the script runs properly on linux, I can
   get that started as soon as I'm fed instructions on how to do
   it :-) Do I just fix the paths and set it running, or do I need
   to prepare something else?
  
  Nothing special to prepare.  Compile with asserts enabled, and run
  the script.  The comment at the top explains how to analyze the log
  for interesting error messages.
 
 Oh, you need to set a variable in the script indicating the average
 number of seconds it takes to run the regression test on your system.

Done that. Also, I needed to replace gmake with make, since I'm on
linux...

Anyway. It's been running for about 12 hours now, and I have *nothing*
in the output file. That tells me that the script doesn't appear to be
working - correct? It should output *something* there, right? (It's
obviously running, because I've got about 400,000 lines in nohup.out..)

Argh. So here I am looking at it now for details, and it seems the
script should be run from src/test/regress, and I ran it from the root
directory.. Oops. Also, I needed to place psql in the path - it failed
to find it, but hid the error message.

Just a hint if someone else is running it ;-)

//Magnus

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq Win32 Mutex performance patch

2008-04-21 Thread Magnus Hagander
Andrew Chernow wrote:
 Tom Lane wrote:
 Silently not locking is surely
  not very safe.
  
 
 Here is the dump code version of the patch.  If anyone wants the
 return value idea, let me know.

Here's a version of this patch that doesn't use malloc - instead, I had
to change the callers to it a bit.

This makes a difference only on Vista+ really, because prior to Vista
the function InitializeCriticalSection() *can* fail - it will throw an
exception and not return any error code. Which really isn't that
different from just crashing like this latest patch from Andrew would
have us do (on out of memory). 

I'm leaning towards going with the simpler one, which is the patch from
Andrew that crashes on out of memory.

Comments/preferences?

//Magnus
Index: fe-connect.c
===
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.357
diff -c -r1.357 fe-connect.c
*** fe-connect.c	31 Mar 2008 02:43:14 -	1.357
--- fe-connect.c	21 Apr 2008 11:35:33 -
***
*** 3827,3841 
  #ifndef WIN32
  	static pthread_mutex_t singlethread_lock = PTHREAD_MUTEX_INITIALIZER;
  #else
! 	static pthread_mutex_t singlethread_lock = NULL;
  	static long mutex_initlock = 0;
  
! 	if (singlethread_lock == NULL)
  	{
  		while (InterlockedExchange(mutex_initlock, 1) == 1)
! 			 /* loop, another thread own the lock */ ;
! 		if (singlethread_lock == NULL)
  			pthread_mutex_init(singlethread_lock, NULL);
  		InterlockedExchange(mutex_initlock, 0);
  	}
  #endif
--- 3827,3845 
  #ifndef WIN32
  	static pthread_mutex_t singlethread_lock = PTHREAD_MUTEX_INITIALIZER;
  #else
! 	static pthread_mutex_t singlethread_lock;
! 	static bool initialized = false;
  	static long mutex_initlock = 0;
  
! 	if (!initialized)
  	{
  		while (InterlockedExchange(mutex_initlock, 1) == 1)
! 			/* loop, another thread own the lock */ ;
! 		if (!initialized)
! 		{
  			pthread_mutex_init(singlethread_lock, NULL);
+ 			initialized = true;
+ 		}
  		InterlockedExchange(mutex_initlock, 0);
  	}
  #endif
Index: fe-secure.c
===
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-secure.c,v
retrieving revision 1.104
diff -c -r1.104 fe-secure.c
*** fe-secure.c	31 Mar 2008 02:43:14 -	1.104
--- fe-secure.c	21 Apr 2008 11:44:22 -
***
*** 809,823 
  #ifndef WIN32
  	static pthread_mutex_t init_mutex = PTHREAD_MUTEX_INITIALIZER;
  #else
! 	static pthread_mutex_t init_mutex = NULL;
  	static long mutex_initlock = 0;
  
! 	if (init_mutex == NULL)
  	{
  		while (InterlockedExchange(mutex_initlock, 1) == 1)
  			 /* loop, another thread own the lock */ ;
! 		if (init_mutex == NULL)
  			pthread_mutex_init(init_mutex, NULL);
  		InterlockedExchange(mutex_initlock, 0);
  	}
  #endif
--- 809,827 
  #ifndef WIN32
  	static pthread_mutex_t init_mutex = PTHREAD_MUTEX_INITIALIZER;
  #else
! 	static pthread_mutex_t init_mutex;
! 	static bool initialized = false;
  	static long mutex_initlock = 0;
  
! 	if (!initialized)
  	{
  		while (InterlockedExchange(mutex_initlock, 1) == 1)
  			 /* loop, another thread own the lock */ ;
! 		if (!initialized)
! 		{
  			pthread_mutex_init(init_mutex, NULL);
+ 			initialized = true;
+ 		}
  		InterlockedExchange(mutex_initlock, 0);
  	}
  #endif
Index: pthread-win32.c
===
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/pthread-win32.c,v
retrieving revision 1.15
diff -c -r1.15 pthread-win32.c
*** pthread-win32.c	1 Jan 2008 19:46:00 -	1.15
--- pthread-win32.c	21 Apr 2008 11:27:38 -
***
*** 35,51 
  void
  pthread_mutex_init(pthread_mutex_t *mp, void *attr)
  {
! 	*mp = CreateMutex(0, 0, 0);
  }
  
  void
  pthread_mutex_lock(pthread_mutex_t *mp)
  {
! 	WaitForSingleObject(*mp, INFINITE);
  }
  
  void
  pthread_mutex_unlock(pthread_mutex_t *mp)
  {
! 	ReleaseMutex(*mp);
  }
--- 35,51 
  void
  pthread_mutex_init(pthread_mutex_t *mp, void *attr)
  {
! 	InitializeCriticalSection(mp);
  }
  
  void
  pthread_mutex_lock(pthread_mutex_t *mp)
  {
! 	EnterCriticalSection(mp);
  }
  
  void
  pthread_mutex_unlock(pthread_mutex_t *mp)
  {
! 	LeaveCriticalSection(mp);
  }

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Proposed patch - psql wraps at window width

2008-04-21 Thread Bruce Momjian
Bryce Nesbitt wrote:
 The newline handling code was, by far, the most complex part of this 
 patch.  While I think it would be nicer to filter up past the newline, 
 I just don't see this as a common need.  Large text fields with newlines 
 are difficult to really view in psql anyway, and they are likely to be 
 the longest columns in any given query.   Bottom line: not worth messing 
 with.

OK, we can always return to it.

 I'm split on the new formatting style.  It looks a lot less grid-like, 
 which I might like once I get used to it (which will be a while, because 
 I can't run my own patch in daily use, as my servers are too old).  I 
 use a version of the wrapping patch that I backported to postgres 8.1, 
 which was prior to multibyte support, and much much simpler.

What new formatting style are you referring to above? Did I modify
what you sent as the original patch?

 I got this warning compiling:
 print.c:784: warning: pointer targets in passing argument 1 of 
 ?mb_strlen_max_width? differ in signedness
 And I did have trouble applying the patch -- I had to manually give it 
 the filename, and tell it to reverse the patch.

OK, fixed.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Proposed patch - psql wraps at window width

2008-04-21 Thread Bruce Momjian
Bryce Nesbitt wrote:
 1) \pset columns XX should make it clear that's for file output only.

OK, docs updated.

 2) There's an extra space, which breaks \pset border 2
 
 717c717
fputc(' ', fout);;
 ---
 fputc(' ', fout);
 842c842
fputs( | , fout);
 ---
 fputs( |, f

OK, got them fixed.

 2) With \pset border 2, the far left border, for symmetry, should work 
 like the middle borders.

OK, how does it look now with this patch?

 3) I'm getting bolder: how about having \pset format wrapped as the 
 default?  Any downsides?

I think we are going to want it as the default for many psql
informational commands, like \df.  Not sure about a more general
default.  We were just discussing using \x as a default for wide output
but it seems this wrap style is probably a better solution than
switching for \x for wide columns (less distracting for the user and
cleaner).  That will have to be a separate discussion once we are done.

Oh, I found a problem in my coding of the new wrap function I added. 
While I handled the case where a character might span multiple bytes, I
assumed all characters had a display width of one.  You can see from
pg_wcsformat()'s use of PQdsplen() that this isn't always the case. I
have modified the patch to properly use PQdsplen() but we are going to
need multi-byte users to test this once we are done.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/ref/psql-ref.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/psql-ref.sgml,v
retrieving revision 1.199
diff -c -c -r1.199 psql-ref.sgml
*** doc/src/sgml/ref/psql-ref.sgml	30 Mar 2008 18:10:20 -	1.199
--- doc/src/sgml/ref/psql-ref.sgml	21 Apr 2008 15:27:55 -
***
*** 1513,1519 
listitem
para
Sets the output format to one of literalunaligned/literal,
!   literalaligned/literal, literalhtml/literal,
literallatex/literal, or literaltroff-ms/literal.
Unique abbreviations are allowed.  (That would mean one letter
is enough.)
--- 1513,1520 
listitem
para
Sets the output format to one of literalunaligned/literal,
!   literalaligned/literal, literalwrapped/literal, 
!   literalhtml/literal,
literallatex/literal, or literaltroff-ms/literal.
Unique abbreviations are allowed.  (That would mean one letter
is enough.)
***
*** 1525,1532 
is intended to create output that might be intended to be read
in by other programs (tab-separated, comma-separated).
quoteAligned/quote mode is the standard, human-readable,
!   nicely formatted text output that is default. The
!   quoteacronymHTML/acronym/quote and
quoteLaTeX/quote modes put out tables that are intended to
be included in documents using the respective mark-up
language. They are not complete documents! (This might not be
--- 1526,1535 
is intended to create output that might be intended to be read
in by other programs (tab-separated, comma-separated).
quoteAligned/quote mode is the standard, human-readable,
!   nicely formatted text output that is default.
!   quoteWrapped/quote is like literalaligned/ but wraps
!   the output to fit the detected screen width or literal\pset
!   columns/.  The quoteacronymHTML/acronym/quote and
quoteLaTeX/quote modes put out tables that are intended to
be included in documents using the respective mark-up
language. They are not complete documents! (This might not be
***
*** 1537,1542 
--- 1540,1556 
/varlistentry
  
varlistentry
+   termliteralcolumns/literal/term
+   listitem
+   para
+   Controls the target width for the literalwrap/ format when
+   output is to a file, or the operating system does not support
+   screen width detection.
+   /para
+   /listitem
+   /varlistentry
+ 
+   varlistentry
termliteralborder/literal/term
listitem
para
Index: src/bin/psql/command.c
===
RCS file: /cvsroot/pgsql/src/bin/psql/command.c,v
retrieving revision 1.186
diff -c -c -r1.186 command.c
*** src/bin/psql/command.c	1 Jan 2008 19:45:55 -	1.186
--- src/bin/psql/command.c	21 Apr 2008 15:27:56 -
***
*** 1526,1531 
--- 1526,1534 
  		case PRINT_ALIGNED:
  	

Re: [PATCHES] Testing pg_terminate_backend()

2008-04-21 Thread Bruce Momjian
Magnus Hagander wrote:
 Done that. Also, I needed to replace gmake with make, since I'm on
 linux...
 
 Anyway. It's been running for about 12 hours now, and I have *nothing*
 in the output file. That tells me that the script doesn't appear to be
 working - correct? It should output *something* there, right? (It's
 obviously running, because I've got about 400,000 lines in nohup.out..)
 
 Argh. So here I am looking at it now for details, and it seems the
 script should be run from src/test/regress, and I ran it from the root
 directory.. Oops. Also, I needed to place psql in the path - it failed
 to find it, but hid the error message.
 
 Just a hint if someone else is running it ;-)

Yea, basically you need to rewrite my script.  :-(

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] float4/float8/int64 passed by value with tsearchfixup

2008-04-21 Thread Bruce Momjian
Tom Lane wrote:
 Gregory Stark [EMAIL PROTECTED] writes:
  So are you killing V0 for non-integral types? Because if not we should keep
  some sacrificial module to the regression tests to use to test for this
  problem.
 
 Well, we could potentially continue to have, say, the oldstyle
 geo_distance function used when not FLOAT8PASSBYVAL.  Not clear how
 useful that really is though.

So is this whole float4/float8/int64 issue closed?

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Proposed patch - psql wraps at window width

2008-04-21 Thread Alvaro Herrera
Bruce Momjian wrote:

 ! /* print a divider, middle columns only 
 */
 ! if ((j + 1) % col_count)
   {
 ! if (opt_border == 0)
 ! fputc(' ', fout);
 ! /* first line for values? */
 ! else if (line_count == 0  
 col_line_count == 0)
 ! fputs( | , fout);
 ! /* next value is beyond height? 
 */
 ! else if (line_count = 
 heights[j + 1])
 ! fputs(   , fout);
 ! /* start of another newline 
 string? */
 ! else if (col_line_count == 0)
 ! fputs( : , fout);
 ! else
 ! {
 ! /* Does the next column 
 wrap to this line? */
 ! struct lineptr 
 *this_line = col_lineptrs[j+1][line_count];
 ! boolstring_done = 
 *(this_line-ptr + bytes_output[j+1]) == 0;
 ! 
 ! fputs(string_done ?
  :  ; , fout);
 ! }
   }

I think it's a bad idea to use the same  :  separator in the two last
cases.  They are different and they should be displayed differently.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Proposed patch - psql wraps at window width

2008-04-21 Thread Bruce Momjian
Alvaro Herrera wrote:
 Bruce Momjian wrote:
 
  !   /* print a divider, middle columns only 
  */
  !   if ((j + 1) % col_count)
  {
  !   if (opt_border == 0)
  !   fputc(' ', fout);
  !   /* first line for values? */
  !   else if (line_count == 0  
  col_line_count == 0)
  !   fputs( | , fout);
  !   /* next value is beyond height? 
  */
  !   else if (line_count = 
  heights[j + 1])
  !   fputs(   , fout);
  !   /* start of another newline 
  string? */
  !   else if (col_line_count == 0)
  !   fputs( : , fout);
  !   else
  !   {
  !   /* Does the next column 
  wrap to this line? */
  !   struct lineptr 
  *this_line = col_lineptrs[j+1][line_count];
  !   boolstring_done = 
  *(this_line-ptr + bytes_output[j+1]) == 0;
  ! 
  !   fputs(string_done ?
   :  ; , fout);
  !   }
  }
 
 I think it's a bad idea to use the same  :  separator in the two last
 cases.  They are different and they should be displayed differently.

I confirmed with Alvaro that he didn't notice the first uses a colon and
the second a semicolon, so he is OK.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Proposed patch - psql wraps at window width

2008-04-21 Thread Alvaro Herrera
Alvaro Herrera wrote:
 Bruce Momjian wrote:
 

  !   fputs(string_done ?
   :  ; , fout);
  !   }
  }
 
 I think it's a bad idea to use the same  :  separator in the two last
 cases.  They are different and they should be displayed differently.

Bruce noted by IM that I misread the ? : expression :-)  Sorry for the
noise.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Proposed patch - psql wraps at window width

2008-04-21 Thread Bruce Momjian
Bruce Momjian wrote:
  I think it's a bad idea to use the same  :  separator in the two last
  cases.  They are different and they should be displayed differently.
 
 I confirmed with Alvaro that he didn't notice the first uses a colon and
 the second a semicolon, so he is OK.

FYI, I added a comment to the patch so others wouldn't confuse this. 
The ? : C syntax makes such confusion easy.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Testing pg_terminate_backend()

2008-04-21 Thread Magnus Hagander
Bruce Momjian wrote:
 Magnus Hagander wrote:
  Done that. Also, I needed to replace gmake with make, since I'm
  on linux...
  
  Anyway. It's been running for about 12 hours now, and I have
  *nothing* in the output file. That tells me that the script doesn't
  appear to be working - correct? It should output *something* there,
  right? (It's obviously running, because I've got about 400,000
  lines in nohup.out..)
  
  Argh. So here I am looking at it now for details, and it seems the
  script should be run from src/test/regress, and I ran it from the
  root directory.. Oops. Also, I needed to place psql in the path -
  it failed to find it, but hid the error message.
  
  Just a hint if someone else is running it ;-)
 
 Yea, basically you need to rewrite my script.  :-(

Not really, but it did need a couple of adjustments :-)

It's been running fine now for a number of hours, with output that
looks similar to the stuff you posted. I'll leave it running..

//Magnus

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Testing pg_terminate_backend()

2008-04-21 Thread Alvaro Herrera
Magnus Hagander wrote:

 It's been running fine now for a number of hours, with output that
 looks similar to the stuff you posted. I'll leave it running..

Perhaps it would be a good idea to leave it running on code with some
bugs on it, just to check if the problems show up.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Testing pg_terminate_backend()

2008-04-21 Thread Guillaume Smet
On Mon, Apr 21, 2008 at 7:25 PM, Magnus Hagander [EMAIL PROTECTED] wrote:
  Not really, but it did need a couple of adjustments :-)

  It's been running fine now for a number of hours, with output that
  looks similar to the stuff you posted. I'll leave it running..

If you can come up with an easily installable tarball, I can dedicate
1 or 2 boxes to run it 24/7.

-- 
Guillaume

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Testing pg_terminate_backend()

2008-04-21 Thread Bruce Momjian
Guillaume Smet wrote:
 On Mon, Apr 21, 2008 at 7:25 PM, Magnus Hagander [EMAIL PROTECTED] wrote:
   Not really, but it did need a couple of adjustments :-)
 
   It's been running fine now for a number of hours, with output that
   looks similar to the stuff you posted. I'll leave it running..
 
 If you can come up with an easily installable tarball, I can dedicate
 1 or 2 boxes to run it 24/7.

Sure.  I updated the script so it will be clearer what you have to
modify.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +
#!/bin/bash

REGRESSION_DURATION=# average duration of regression test, in seconds
OUTFILE=

[ $REGRESSION_DURATION -o ! $OUTFILE ] 
echo Must set REGRESSION_DURATION and OUTFILE in the script 12 
exit 1

[ ! -f parallel_schedule ] 
echo This must be from the Postgres src/test/regress directory 12 
exit 1

if gmake -h  /dev/null 21
thenMAKE=gmake
elseMAKE=make
fi

# To analyze output, use:
echo Running ...
echo To analyze the output log file, use:
echo grep '^\+ *[A-Z][A-Z]*:' /rtmp/regression.sigterm | sort | uniq | less


while :
do
(
SLEEP=`expr $RANDOM \* $REGRESSION_DURATION / 32767`
echo Sleeping $SLEEP seconds
sleep $SLEEP
echo Trying kill
# send up to 7 kill signals
for X in 1 2 3 4 5 6 7
do
psql -p 55432 -qt -c 
SELECT pg_terminate_backend(stat.procpid)
FROM (SELECT procpid FROM pg_stat_activity
ORDER BY random() LIMIT 1) AS stat
 template1 2 /dev/null
if [ $? -eq 0 ]
thenecho Kill sent
fi
sleep 5
done
) 
$MAKE check
wait
[ -s regression.diffs ]  cat regression.diffs  $OUTFILE
done

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] float4/float8/int64 passed by value with tsearch fixup

2008-04-21 Thread Guillaume Smet
On Mon, Apr 21, 2008 at 7:10 AM, Tom Lane [EMAIL PROTECTED] wrote:
  Note that we are already exercising both ends of the
  float8-byval option, so probably only --disable-float4-byval
  is very interesting to test explicitly.

I'll set up a new animal with --disable-float4-byval this week.

-- 
Guillaume

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


[PATCHES] column level privileges

2008-04-21 Thread Andrew Dunstan


(try 2)

Here is an updated patch that applies to HEAD.

I will update the wiki.

(What is the maximum attachment size that -patches will accept?)

cheers

andrew


I wrote:


This patch by Golden Lui was his work for the last Google SoC. I was 
his mentor for the project. I have just realised that he didn't send 
his final patch to the list.


I guess it's too late for the current commit-fest, but it really needs 
to go on a patch queue (my memory on this was jogged by Tom's recent 
mention of $Subject).


I'm going to see how much bitrot there is and see what changes are 
necessary to get it to apply.






-
Here is a README for the whole patch.

According to the SQL92 standard, there are four levels in the 
privilege hierarchy, i.e. database, tablespace, table, and column. 
Most commercial DBMSs support all the levels, but column-level 
privilege is hitherto unaddressed in the PostgreSQL, and this patch 
try to implement it.


What this patch have done:
1. The execution of GRANT/REVOKE for column privileges. Now only 
INSERT/UPDATE/REFERENCES privileges are supported, as SQL92 specified. 
SELECT privilege is now not supported. This part includes:
   1.1 Add a column named 'attrel' in pg_attribute catalog to store 
column privileges. Now all column privileges are stored, no matter 
whether they could be implied from table-level privilege.

   1.2 Parser for the new kind of GRANT/REVOKE commands.
   1.3 Execution of GRANT/REVOKE for column privileges. Corresponding 
column privileges will be added/removed automatically if no column is 
specified, as SQL standard specified.

2. Column-level privilege check.
   Now for UPDATE/INSERT/REFERENCES privilege, privilege check will be 
done ONLY on column level. Table-level privilege check was done in the 
function InitPlan. Now in this patch, these three kind of privilege 
are checked during the parse phase.
   2.1 For UPDATE/INSERT commands. Privilege check is done in the 
function transformUpdateStmt/transformInsertStmt.
   2.2 For REFERENCES, privilege check is done in the function 
ATAddForeignKeyConstraint. This function will be called whenever a 
foreign key constraint is added, like create table, alter table, etc.
   2.3 For COPY command, INSERT privilege is check in the function 
DoCopy. SELECT command is checked in DoCopy too.
3. While adding a new column to a table using ALTER TABLE command, set 
appropriate privilege for the new column according to privilege 
already granted on the table.

4. Allow pg_dump and pg_dumpall to dump in/out column privileges.
5. Add a column named objsubid in pg_shdepend catalog to record ACL 
dependencies between column and roles.

6. modify the grammar of ECPG to support column level privileges.
7. change psql's \z (\dp) command to support listing column privileges 
for tables and views. If \z(\dp) is run with a pattern, column 
privileges are listed after table level privileges.
8. Regression test for column-level privileges. I changed both 
privileges.sql and expected/privileges.out, so regression check is now 
all passed.


  







pg_colpriv_version_0.4-20080421.patch.gz
Description: GNU Zip compressed data

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY

2008-04-21 Thread Zoltan Boszormenyi

Zoltan Boszormenyi írta:

Zoltan Boszormenyi írta:

Decibel! írta:

On Apr 3, 2008, at 12:52 AM, Zoltan Boszormenyi wrote:

Where is the info in the sequence to provide restarting with
the _original_ start value?


There isn't any. If you want the sequence to start at some magic 
value, adjust the minimum value.


There's the START WITH option for IDENTITY columns and this below
is paragraph 8 under General rules of 14.10 truncate table statement
in 6WD2_02_Foundation_2007-12.pdf (page 902):

8) If RESTART IDENTITY is specified and the table descriptor of T 
includes a column descriptor IDCD of

  an identity column, then:
  a) Let CN be the column name included in IDCD and let SV be the 
start value included in IDCD.
  b) The following alter table statement is effectively executed 
without further Access Rule checking:

  ALTER TABLE TN ALTER COLUMN CN RESTART WITH SV

This says that the original start value is used, not the minimum value.
IDENTITY has the same options as CREATE SEQUENCE. In fact the
identity column specification links to 11.63 sequence generator 
definition

when it comes to IDENTITY sequence options. And surprise, surprise,
11.64 alter sequence generator statement now defines
ALTER SEQUENCE sn RESTART [WITH newvalue]
where omitting the WITH newval part also uses the original start 
value.


Best regards,
Zoltán Böszörményi


Attached patch implements the extension found in the current SQL200n 
draft,
implementing stored start value and supporting ALTER SEQUENCE seq 
RESTART;
Some error check are also added to prohibit CREATE SEQUENCE ... 
RESTART ...

and ALTER SEQUENCE ... START ...

Best regards,
Zoltán Böszörményi


Updated patch implements TRUNCATE ... RESTART IDENTITY
which restarts all owned sequences for the truncated table(s).
Regression tests updated, documentation added. pg_dump was
also extended to output original[1] START value for creating SEQUENCEs.

[1] For 8.3 and below I could only guesstimate it as MINVALUE for ascending
 and MAXVALUE for descending sequences.

Best regards,
Zoltán Böszörményi

--
--
Zoltán Böszörményi
Cybertec Schönig  Schönig GmbH
http://www.postgresql.at/



sql2008-compliant-seq-v2.patch.gz
Description: Unix tar archive

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Improve shutdown during online backup, take 2

2008-04-21 Thread Simon Riggs
On Wed, 2008-04-09 at 14:58 +0200, Albe Laurenz wrote:
 This patch replaces the previous version
 http://archives.postgresql.org/pgsql-patches/2008-04/msg4.php
 
 As suggested by Simon Riggs in
 http://archives.postgresql.org/pgsql-patches/2008-04/msg00013.php ,
 a smart shutdown will now wait for online backup mode to finish.
 The functions that touch backup_label have been moved to xlog.c.
 
 As suggested by Heikki Linnakangas in
 http://archives.postgresql.org/pgsql-patches/2008-04/msg00180.php ,
 I have introduced a new postmaster state PM_WAIT_BACKUP.
 In this state the postmaster will still accept connections.
 
 Thanks for the feedback!
 I'll try to add a link at the Wiki page.

Patch applies, and works as described. Looks good for final apply.

Few minor thoughts:

* Text in pg_ctl should be WARNING, not Warning.
* CancelBackup() API looks strange, not sure why
* Need to mention that CancelBackup() is not the right way to end a
backup, so that function and pg_stop_backup should reference each other

Other than those, I like it. Very useful patch.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches