Re: [HACKERS] Bug in new buffer freelist code
Jan Wieck <[EMAIL PROTECTED]> writes: > Yeah, looks like ... but I actually want to have a clean reproduction of > the error before I attempt to fix it. Well, I can tell you that just running "make check" over and over isn't a real efficient way to reproduce the problem. It might work to do something like stopping a FlushRelationBuffers call in its tracks with the debugger, meanwhile accessing the same rel with another backend. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Permissions and PGSQL
On Tue, Dec 16, 2003 at 13:41:32 +0500, Jean-Eric Cuendet <[EMAIL PROTECTED]> wrote: > Hi, > I use PgSql for a lot of our company's need and I lack some features. > I would like to know if there is plans to implement: > - User permissions based on columns? (Ex: User1 has Select on Column "CompayName" > but User2 has update on column "CompanyName" while User3 has create new row > on table). As the previous responder said that is no standard column security. However it should be possible to do something custom to do this using rules and/or triggers. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Bug in new buffer freelist code
Tom Lane wrote: Jan Wieck <[EMAIL PROTECTED]> writes: It seems to me that buffers that are thrown away via StrategyInvalidateBuffer() do not get their relnode and blocknum cleaned out. Mmmm. They definitely should be; if you look at the prior version of buf_table.c, BufTableDelete did this: /* * Clear the buffer's tag. This doesn't matter for the hash table, * since the buffer is already removed from it, but it ensures that * sequential searches through the buffer table won't think the buffer * is still valid for its old page. */ buf->tag.rnode.relNode = InvalidOid; buf->tag.rnode.tblNode = InvalidOid; I see you removed that from the current version, but the equivalent thing needs to be done someplace. Is StrategyInvalidateBuffer the right place? Yeah, looks like ... but I actually want to have a clean reproduction of the error before I attempt to fix it. Will look at it deeper. BTW, it kinda looks like the BM_DELETED flag bit is useless now? I think so, together with the BM_FREE which was redundant anyway. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] PostgreSQL port to pure Java?
Jean-Michel POURE wrote: Le Mardi 09 Décembre 2003 16:15, Ivelin Ivanov a écrit : I think that a co-bundle between an open source J2EE container like JBoss and a scalable database like PostgreSQL will be a blast. Why not cut all trees on earth and replace them with plastic? Before that, we need to port mankind DNA to Windows 3.1 in order to improve speed. That seems a bit harsh. Personally I think porting PostgreSQL to java is silly but there is some argument to the benefit from his idea. If you are a java programmer. Sincerely, Joshua Drake ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED]) -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PostgreSQL port to pure Java?
Jean-Michel POURE said: > Le Mardi 09 Décembre 2003 16:15, Ivelin Ivanov a écrit : >> I think that a co-bundle between an open source J2EE >> container like JBoss and a scalable database like >> PostgreSQL will be a blast. > > Why not cut all trees on earth and replace them with plastic? Before > that, we need to port mankind DNA to Windows 3.1 in order to improve > speed. > So you like the idea, I take it? cheers, and compliments of the season andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Bug in new buffer freelist code
Jan Wieck <[EMAIL PROTECTED]> writes: >> Let me know if there's any test I could run to confirm your theory, >> assuming I can make it happen again after I finish rebuilding. > You could add another assertion that checks that the CDB found is > actually pointing to the buffer that is being invalidated. Good thought. What would that look like exactly? > Looking at the bufmgr.c code for FlushRelationBuffers() ... it does > exactly what I described ... leaving the relnode and buffernum for an > invalidated buffer just where they are. Yeah. I don't think FlushRelationBuffers() is the place to fix it though; if you try to fix it at that level there will be several places to do it (DropBuffers etc). It's unclear to me what the division of labor is now between BufTableDelete and freelist.c, so I'm not sure where you *should* fix it ... but it should be as close as possible to where the buffer is removed from the hashtable, IMHO. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Bug in new buffer freelist code
Tom Lane wrote: BTW, I just managed to reproduce the hang, after a whole afternoon of trying ... only it was with a non-debug build. Sigh. Anyway, it seems my HP machine has a significantly higher probability of showing the problem than my Linux machine --- I have been unable to see the problem in thirty or forty "make checks" on that one. Let me know if there's any test I could run to confirm your theory, assuming I can make it happen again after I finish rebuilding. You could add another assertion that checks that the CDB found is actually pointing to the buffer that is being invalidated. Looking at the bufmgr.c code for FlushRelationBuffers() ... it does exactly what I described ... leaving the relnode and buffernum for an invalidated buffer just where they are. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Bug in new buffer freelist code
BTW, I just managed to reproduce the hang, after a whole afternoon of trying ... only it was with a non-debug build. Sigh. Anyway, it seems my HP machine has a significantly higher probability of showing the problem than my Linux machine --- I have been unable to see the problem in thirty or forty "make checks" on that one. Let me know if there's any test I could run to confirm your theory, assuming I can make it happen again after I finish rebuilding. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] PostgreSQL port to pure Java?
Le Mardi 09 Décembre 2003 16:15, Ivelin Ivanov a écrit : > I think that a co-bundle between an open source J2EE > container like JBoss and a scalable database like > PostgreSQL will be a blast. Why not cut all trees on earth and replace them with plastic? Before that, we need to port mankind DNA to Windows 3.1 in order to improve speed. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Bug in new buffer freelist code
Jan Wieck <[EMAIL PROTECTED]> writes: > It seems to me that buffers that are thrown away via > StrategyInvalidateBuffer() do not get their relnode and blocknum cleaned > out. Mmmm. They definitely should be; if you look at the prior version of buf_table.c, BufTableDelete did this: /* * Clear the buffer's tag. This doesn't matter for the hash table, * since the buffer is already removed from it, but it ensures that * sequential searches through the buffer table won't think the buffer * is still valid for its old page. */ buf->tag.rnode.relNode = InvalidOid; buf->tag.rnode.tblNode = InvalidOid; I see you removed that from the current version, but the equivalent thing needs to be done someplace. Is StrategyInvalidateBuffer the right place? BTW, it kinda looks like the BM_DELETED flag bit is useless now? regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Bug in new buffer freelist code
Tom Lane wrote:
I just had the parallel regression tests hang up due to what appears to
be a bug in the new ARC code. The CLUSTER test gets into an infinite
loop trying to do "CLUSTER clstr_1;". The loop is in
StrategyInvalidateBuffer's check that the buffer is already in the
freelist; it isn't, and the freelist is circular.
It seems to me that buffers that are thrown away via
StrategyInvalidateBuffer() do not get their relnode and blocknum cleaned
out. That causes FlushRelationBuffers() while doing a full scan of the
whole buffer pool to find buffers that once contained the block again.
If buffer 839 once contained that block, and it was given up that way,
and later on buffer 850 contains it, there is a CDB for it. If now
FlushRelationBuffers() scans the buffer pool, it will find buffer 839
first and call StrategyInvalidateBuffer() for it. That finds the CDB for
buffer 850, and add's buffer 839 to the list again. Later on FlushRB()
calls StrategyIB() for buffer 850 and we have the situation at hand.
Does that make sense?
Jan
(gdb) bt
#0 0x1fe8a8 in StrategyInvalidateBuffer (buf=0xc3a56f60) at freelist.c:733
#1 0x1fbf08 in FlushRelationBuffers (rel=0x400fa298, firstDelBlock=0)
at bufmgr.c:1596
#2 0x1479fc in swap_relfilenodes (r1=143786, r2=143915) at cluster.c:736
#3 0x147458 in rebuild_relation (OldHeap=0x2322b, indexOid=143788)
at cluster.c:455
#4 0x1473b0 in cluster_rel (rvtc=0x7b03bed8, recheck=0 '\000')
at cluster.c:395
#5 0x146ff4 in cluster (stmt=0x400b88a8) at cluster.c:232
#6 0x21c60c in ProcessUtility (parsetree=0x400b88a8, dest=0x400b88e8,
completionTag=0x7b03bbe8 "") at utility.c:1033
... etc ...
(gdb) p *buf
$5 = {bufNext = -1, data = 7211904, tag = {rnode = {tblNode = 17142,
relNode = 143906}, blockNum = 0}, buf_id = 850, flags = 14,
refcount = 0, io_in_progress_lock = 1721, cntx_lock = 1722,
cntxDirty = 0 '\000', wait_backend_id = 0}
(gdb) p *StrategyControl
$1 = {target_T1_size = 423, listUnusedCDB = 249, listHead = {464, 967, 1692,
1227}, listTail = {968, 645, 1528, 1694}, listSize = {364, 413, 584, 636},
listFreeBuffers = 839, num_lookup = 546939, num_hit = {1378, 246896, 282639,
3935}, stat_report = 0, cdb = {{prev = 386, next = 23, list = 3,
buf_tag = {rnode = {tblNode = 17142, relNode = 19080}, blockNum = 30},
buf_id = -1, t1_xid = 3402}}}
(gdb) p BufferDescriptors[839]
$2 = {bufNext = 839, data = 7121792, tag = {rnode = {tblNode = 17142,
relNode = 143906}, blockNum = 0}, buf_id = 839, flags = 14,
refcount = 0, io_in_progress_lock = 1699, cntx_lock = 1700,
cntxDirty = 0 '\000', wait_backend_id = 0}
So we've got a couple of problems here: buffers 839 and 850 both claim
to contain block 0 of rel 143906 (which is clstr_1), and the freelist
is circular.
This doesn't seem to be super reproducible, but there's definitely a
problem in there somewhere.
regards, tom lane
--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#== [EMAIL PROTECTED] #
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
[HACKERS] One regression failure with 7.4.1 on Debian 3.0r2
I have one regression failure on 7.4.1, which does not occur with 7.4 [EMAIL PROTECTED] more src/test/regress/regression.diffs *** ./expected/random.out Thu Feb 13 05:24:04 2003 --- ./results/random.outTue Dec 23 20:19:40 2003 *** *** 25,31 GROUP BY random HAVING count(random) > 1; random | count +--- ! (0 rows) SELECT random FROM RANDOM_TBL WHERE random NOT BETWEEN 80 AND 120; --- 25,32 GROUP BY random HAVING count(random) > 1; random | count +--- ! 103 | 2 ! (1 row) SELECT random FROM RANDOM_TBL WHERE random NOT BETWEEN 80 AND 120; [EMAIL PROTECTED] uname -a Linux bugsdb 2.4.23 #1 SMP Tue Dec 23 12:29:42 GMT 2003 i686 unknown -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Bug in new buffer freelist code
I just had the parallel regression tests hang up due to what appears to
be a bug in the new ARC code. The CLUSTER test gets into an infinite
loop trying to do "CLUSTER clstr_1;". The loop is in
StrategyInvalidateBuffer's check that the buffer is already in the
freelist; it isn't, and the freelist is circular.
(gdb) bt
#0 0x1fe8a8 in StrategyInvalidateBuffer (buf=0xc3a56f60) at freelist.c:733
#1 0x1fbf08 in FlushRelationBuffers (rel=0x400fa298, firstDelBlock=0)
at bufmgr.c:1596
#2 0x1479fc in swap_relfilenodes (r1=143786, r2=143915) at cluster.c:736
#3 0x147458 in rebuild_relation (OldHeap=0x2322b, indexOid=143788)
at cluster.c:455
#4 0x1473b0 in cluster_rel (rvtc=0x7b03bed8, recheck=0 '\000')
at cluster.c:395
#5 0x146ff4 in cluster (stmt=0x400b88a8) at cluster.c:232
#6 0x21c60c in ProcessUtility (parsetree=0x400b88a8, dest=0x400b88e8,
completionTag=0x7b03bbe8 "") at utility.c:1033
... etc ...
(gdb) p *buf
$5 = {bufNext = -1, data = 7211904, tag = {rnode = {tblNode = 17142,
relNode = 143906}, blockNum = 0}, buf_id = 850, flags = 14,
refcount = 0, io_in_progress_lock = 1721, cntx_lock = 1722,
cntxDirty = 0 '\000', wait_backend_id = 0}
(gdb) p *StrategyControl
$1 = {target_T1_size = 423, listUnusedCDB = 249, listHead = {464, 967, 1692,
1227}, listTail = {968, 645, 1528, 1694}, listSize = {364, 413, 584, 636},
listFreeBuffers = 839, num_lookup = 546939, num_hit = {1378, 246896, 282639,
3935}, stat_report = 0, cdb = {{prev = 386, next = 23, list = 3,
buf_tag = {rnode = {tblNode = 17142, relNode = 19080}, blockNum = 30},
buf_id = -1, t1_xid = 3402}}}
(gdb) p BufferDescriptors[839]
$2 = {bufNext = 839, data = 7121792, tag = {rnode = {tblNode = 17142,
relNode = 143906}, blockNum = 0}, buf_id = 839, flags = 14,
refcount = 0, io_in_progress_lock = 1699, cntx_lock = 1700,
cntxDirty = 0 '\000', wait_backend_id = 0}
So we've got a couple of problems here: buffers 839 and 850 both claim
to contain block 0 of rel 143906 (which is clstr_1), and the freelist
is circular.
This doesn't seem to be super reproducible, but there's definitely a
problem in there somewhere.
regards, tom lane
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] (Mis?)Behavior of \copy with -f and \i
Greetings... I've run into a problem with the way \copy behaves when psql is reading its input from a file using either the -f command line option or the \i command. (Not that it matters in this case, but this is PostgreSQL 7.4 on i686-pc-linux-gnu, compiled with gcc.) The following... ---8<--SNIP--- CREATE TABLE junk ( abbrev CHAR(1), name VARCHAR(80) ); \copy junk (abbrev, name) FROM STDIN WITH DELIMITER '|' NULL '' F|Foo B|Bar Z|Baz \. ---8<--SNIP--- ...works just fine when keyed directly into psql or is redirected from a file (i.e., psql < junk.sql). When using -f or \i, psql sits and waits for data and an EOF on the standard input and then proceeds to interpret the next four lines as commands. This was discussed several years ago in this thread: http://archives.postgresql.org/pgsql-hackers/2000-01/msg00361.php Peter Eisentraut declared that from that point on, stdin would be whatever stream the \copy command came from. I'd like to propose a variant on the "FROM" clause which makes good on Peter's declaration without breaking anything already using FROM STDIN and expecting it to really read from stdin. (I think this is for the better because there are lots of good uses for "psql -f foo.sql < foo.dat".) I'd be more than happy to write and test a patch if folks think this would be a good thing. I'm leaning toward "FROM -" as the syntax but am open to other ideas (i.e., "FROM HERE" or "FROM INPUT"). - Mark P.S.: I've been using Postgre(s(95)?|SQL) in its various forms for close to a decade, and what was a stable platform for lightweight storage has matured into something I'd pour a few million rows into without thinking twice about it. Thanks to everyone for all the great work! ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Permissions and PGSQL
I use PgSql for a lot of our company's need and I lack some features. I would like to know if there is plans to implement: - User permissions based on columns? (Ex: User1 has Select on Column "CompayName" but User2 has update on column "CompanyName" while User3 has create new row on table). These do not exist yet in pgsql. - Permissions on create table. This is the most lacking permission problem actually, since everyone that can log in PgSql can create tables... This does exist. The CREATE permission on schemas is what you want. Perhaps the CREATE permission on databases as well. You need to revoke these from the public schema. Or even just drop the public schema. Chris ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
