Re: [HACKERS] HOT patch, missing things

On 8/8/07, Simon Riggs <[EMAIL PROTECTED]> wrote: > > > > So pruning removes dead hot updated tuples, while defragging will remove > dead cold updated tuples and deletes, as well as rearranging space. No, pruning removes all dead tuples, irrespective of whether they are HOT or COLD updated and w

Re: [HACKERS] GUC for default heap fillfactor

On Thu, 2007-08-09 at 12:17 +0530, Pavan Deolasee wrote: > OK. I wish to have a config parameter so that DBA can set > the system level default and then override that (if required) for > each table. It will also be useful for experimenting with different > fillfactor. The only way today is either

Re: [HACKERS] HOT patch, missing things

On Thu, 2007-08-09 at 12:39 +0530, Pavan Deolasee wrote: > No, pruning removes all dead tuples, irrespective of whether they are > HOT or COLD updated and whether they are heap-only or not. It > handles line pointer redirection and marks > all dead tuples as ~LP_USED. Defragging just repairs the

Re: [HACKERS] Unexpected VACUUM FULL failure

On Wed, 2007-08-08 at 23:23 -0400, Tom Lane wrote: > I wrote: > > ... Since we've whacked the tqual.c logic around recently, > > the problem might actually lie there... > > In fact, I bet this is a result of the async-commit patch. The places > where vacuum.c bleats "HEAP_MOVED_OFF was expected"

[HACKERS] Compilation of pg 7.4.17 fails on HP-UX

Hello, I have just tried to compile postgresql 7.4.17 on a HP-UX 11.11 box (PA-RISC) and the compliation fails in gist. make[4]: Entering directory `/home/adrian/postgresql-7.4.17/src/backend/access/gist' gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes -Wmissing-declarations -I../../..

Re: [HACKERS] HOT patch, missing things

On 8/7/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > > There's three things clearly missing in the patch: > > 1. HOT updates on tables with expression or partial indexes. Hasn't been > done yet because it should be pretty straightforward and we've had more > important things to do. Though not

Re: [HACKERS] default_text_search_config and expression indexes

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Aug 09, 2007 at 02:36:41AM -0400, Bruce Momjian wrote: > Oleg Bartunov wrote: > > > Yea, seems more work than is necessary. If we require the configuration > > > to be always supplied, and document that mismatches are a problem, I > > > think

Re: [HACKERS] comunication protocol

Merlin Moncure wrote: On 8/8/07, Omar Bettin <[EMAIL PROTECTED]> wrote: Hi, I have installed postgresql to my client as a server for a progam (libpq.dll/VCL based) that I wrote for them. Every is working fine, but I noted some general slowness, compared with an older database system (iAny

Re: [HACKERS] comunication protocol

Gustavo Tonini wrote: Em Quarta 08 Agosto 2007 12:02, Omar Bettin escreveu: Hi, I have installed postgresql to my client as a server for a progam (libpq.dll/VCL based) that I wrote for them. Borland VCL? What component are you using? Gustavo. I use a strong modified PostgresD

Re: [HACKERS] default_text_search_config and expression indexes

On Thu, 9 Aug 2007, [EMAIL PROTECTED] wrote: Maybe I'm missing something, but it seems to me that the configuration is more attached to a column/index thatn to the whole database. If there's a default in an expression, I'd rather expect this default to be drawn from the index involved than from

Re: [HACKERS] HOT patch, missing things

On 8/9/07, Simon Riggs <[EMAIL PROTECTED]> wrote: > > > Whether I got the exact details of frugging & depruning correct or not: > if a tuple version is removed, then VACUUM doesn't need to remove it > later, so any non-VACUUM removal of rows must defer a VACUUM. > > ISTM that you are worried about

Re: [HACKERS] HOT patch, missing things

On Thu, 2007-08-09 at 15:46 +0530, Pavan Deolasee wrote: > > > On 8/9/07, Simon Riggs <[EMAIL PROTECTED]> wrote: > > Whether I got the exact details of frugging & depruning > correct or not: > if a tuple version is removed, then VACUUM doesn't need to > re

Re: [HACKERS] Function structure in formatting.c

On 8/9/07, Jaime Casanova <[EMAIL PROTECTED]> wrote: > > take your time, this seems like it will be for 8.4 anyway > I hear you, unfortunately "taking my time" usually means I forget about it for eight months and by the time I come back to it I've forgotten what I was doing =) I wasn't really exp

Re: [HACKERS] comunication protocol

On 8/9/07, Omar Bettin <[EMAIL PROTECTED]> wrote: > Merlin Moncure wrote: > > AFAIK, the fastest possible way to get data off the server, skipping > > all data and text processing is to write a SPI routine, and stream the > > data out locally to the server. I am doing exactly this in a > > partic

Re: [HACKERS] HOT patch, missing things

"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > HOT update is feasible iff > > - old and new tuples, both match the partiality condition OR > - old and new tuples, both don't match the condition ... > For functional index, we should apply the function to the old and new > tuple and compare the outc

[HACKERS] crypting prosrc in pg_proc

hello everybody, one of our customers wants to store the code of interpreted procedures (PL/pgSQL, PL/Perl) and so in an encrypted way. so the idea we had to add one more column to pg_proc telling us whether prosrc is encrypted or not. people could chose then whether to crypt codes there or

Re: [HACKERS] crypting prosrc in pg_proc

On Thu, Aug 09, 2007 at 03:39:06PM +0200, Hans-Juergen Schoenig wrote: >so the idea we had to add one more column to pg_proc telling us whether >prosrc is encrypted or not. people could chose then whether to crypt codes >there or not (speed of decryption can be an issue). >should no

Re: [HACKERS] crypting prosrc in pg_proc

Hans-Juergen Schoenig wrote: hello everybody, one of our customers wants to store the code of interpreted procedures (PL/pgSQL, PL/Perl) and so in an encrypted way. so the idea we had to add one more column to pg_proc telling us whether prosrc is encrypted or not. people could chose then whe

Re: [HACKERS] crypting prosrc in pg_proc

the idea is basically to hide codes - many companies want that and ask for it again and again. i would suggest keys to reside in $PGDATA. we do this for SSL and so already. initdb could create such keys so that they are unique to every database instance. decrypting could be avoided as muc

Re: [HACKERS] crypting prosrc in pg_proc

the idea is basically to hide codes - many companies want that and ask for it again and again. Hide code from who (or is that whom?)? The PL compiler(s) will need to decrypt the code. If a compiler can decrypt it, then anyone can decrypt it (because the compilers are open-source). And the

Re: [HACKERS] HOT patch, missing things

On Thu, Aug 09, 2007 at 01:25:14PM +0100, Gregory Stark wrote: > "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > > > HOT update is feasible iff > > > > - old and new tuples, both match the partiality condition OR > > - old and new tuples, both don't match the condition > ... > > For functional inde

Re: [HACKERS] GUC for default heap fillfactor

On Thu, Aug 09, 2007 at 09:57:48AM +0900, ITAGAKI Takahiro wrote: > > If HOT gets into 8.3, we might need a GUC to set database wide heap > > fillfactor to an appropriate value. > > I have no objection to do that, but we will need other default values soon, > something like default_[heap|btree|has

Re: [HACKERS] crypting prosrc in pg_proc

On Aug 9, 2007, at 4:47 PM, korry.douglas wrote: the idea is basically to hide codes - many companies want that and ask for it again and again. Hide code from who (or is that whom?)? the code should be hidden from the guy who is actually executing the function. so: some user i

Re: [HACKERS] crypting prosrc in pg_proc

On Aug 9, 2007, at 4:34 PM, Peter Eisentraut wrote: Am Donnerstag, 9. August 2007 16:09 schrieb Hans-Juergen Schoenig: the idea is basically to hide codes - many companies want that and ask for it again and again. If you want to design a security feature, you need to offer a threat and ris

Re: [HACKERS] crypting prosrc in pg_proc

Wouldn't it be better just to hide prosrc from the users you want to hide it from? (Note: my SOC student is working on column level privs, although unfortunately his work doesn't currently work with SELECT privs). In fact, in many cases, depending on the app, you can simply remove SELECT priv

Re: [HACKERS] crypting prosrc in pg_proc

Am Donnerstag, 9. August 2007 16:09 schrieb Hans-Juergen Schoenig: > the idea is basically to hide codes - many companies want that and   > ask for it again and again. If you want to design a security feature, you need to offer a threat and risk analysis, not just the whining of customers. -- P

Re: [HACKERS] crypting prosrc in pg_proc

Hans-Juergen Schoenig napsal(a): the idea is basically to hide codes - many companies want that and ask for it again and again. i would suggest keys to reside in $PGDATA. we do this for SSL and so already. initdb could create such keys so that they are unique to every database instance. dec

Re: [HACKERS] crypting prosrc in pg_proc

On Thu, 9 Aug 2007, Andrew Dunstan wrote: There are also some fairly impressive code obfuscators about, that your clients might find useful. All they really need is to find a sufficiently clever PL/Perl programmer. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

Re: [HACKERS] crypting prosrc in pg_proc

On Thu, Aug 09, 2007 at 04:34:48PM +0200, Peter Eisentraut wrote: > Am Donnerstag, 9. August 2007 16:09 schrieb Hans-Juergen Schoenig: > > the idea is basically to hide codes - many companies want that and ? > > ask for it again and again. > > If you want to design a security feature, you need to

Re: [HACKERS] crypting prosrc in pg_proc

On Aug 9, 2007, at 5:03 PM, Greg Smith wrote: On Thu, 9 Aug 2007, Andrew Dunstan wrote: There are also some fairly impressive code obfuscators about, that your clients might find useful. All they really need is to find a sufficiently clever PL/Perl programmer. we should make this a PL

Re: [HACKERS] comunication protocol

Merlin Moncure wrote: On 8/9/07, Omar Bettin <[EMAIL PROTECTED]> wrote: Merlin Moncure wrote: AFAIK, the fastest possible way to get data off the server, skipping all data and text processing is to write a SPI routine, and stream the data out locally to the server. I am doing exactl

Re: [HACKERS] crypting prosrc in pg_proc

My suggestion is to extend PL API and each PL language should offer wrap or encrypt function which generate encrypted code and this code will be store in the pg_proc. PL language will be responsible to detect if it raw or crypted code. PG_Dump will dump crypted procedure and author is respons

[HACKERS] change name of redirect_stderr?

Before I wrap up the CSVlog stuff, we need to decide whether or not to change the name of the redirect_stderr setting, and if so to what. The reason is that with CSVlogs it will no longer apply just to stderr (we will require it to be on for CSVlogs, in fact). I suggest "redirect_logs", alth

Re: [HACKERS] crypting prosrc in pg_proc

The basic problem is this: if you have to decrypt the code in order to give it to a compiler (PL/pgSQL, Java, Perl, ...) then there is a point in time where the source code is in plaintext form - it would be trivial to add an fprintf( stderr, "%s", plainTextForm ) to the PL handler to ste

Re: [HACKERS] crypting prosrc in pg_proc

On Thu, Aug 09, 2007 at 04:50:53PM +0200, Zdenek Kotala wrote: > Hans-Juergen Schoenig napsal(a): > >the idea is basically to hide codes - many companies want that and ask > >for it again and again. > > > >i would suggest keys to reside in $PGDATA. we do this for SSL and so > >already. > > > >ini

Re: [HACKERS] crypting prosrc in pg_proc

On Thu, Aug 09, 2007 at 04:42:19PM +0200, Hans-Juergen Schoenig wrote: > the problem here is that vendors of appliances don't want people to > spider their codes. this is a fact - it is not the idea of open > source to do so but bloody reality. in addition to that people are > not willing to

Re: [HACKERS] crypting prosrc in pg_proc

On Thu, Aug 09, 2007 at 05:16:43PM +0200, Hans-Juergen Schoenig wrote: > we are shipping appliances - users can only use SQL; no compilers and > no SSH logins allowed ... If you have shipped a physical thing to someone else, then they can do what they want with it irrespective of what "rules" yo

Re: [HACKERS] crypting prosrc in pg_proc

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew Sullivan wrote: > On Thu, Aug 09, 2007 at 04:42:19PM +0200, Hans-Juergen Schoenig wrote: > >> the problem here is that vendors of appliances don't want people to >> spider their codes. this is a fact - it is not the idea of open >> source t

Re: [HACKERS] crypting prosrc in pg_proc

Andrew Sullivan wrote: And they can always write it in C and deliver only compiled object code; although whether anyone would be foolish enough to run such object code without inspecting its safety is an excercise in speculation I don't care to pursue. The intended use for this is an app

Re: [HACKERS] crypting prosrc in pg_proc

Decibel! wrote: This is also related to the desire to be able to restrict access to the catalog tables. Doing so could potentially solve this problem; it solves other issues (such as being able to see all the databases that exist on a server, something that hosting environments care about).

Re: [HACKERS] Compilation of pg 7.4.17 fails on HP-UX

"Adrian Maier" <[EMAIL PROTECTED]> writes: > I have just tried to compile postgresql 7.4.17 on a HP-UX 11.11 box > (PA-RISC) and the compliation fails in gist. I do not think anyone cares about making 7.4.x run on platforms it did not support before. Use a newer PG release.

Re: [HACKERS] HOT patch, missing things

"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > I started with this. ISTM to support partial indexes, we must check > the old and new tuple against partiality match. > ... > For functional index, we should apply the function to the old and new > tuple and compare the outcome. If the results are same

Re: [HACKERS] crypting prosrc in pg_proc

On 8/9/07, Andrew Dunstan <[EMAIL PROTECTED]> wrote: > Decibel! wrote: > > This is also related to the desire to be able to restrict access to the > > catalog tables. Doing so could potentially solve this problem; it > > solves other issues (such as being able to see all the databases that > > exis

Re: [HACKERS] Problem with locks

"Gregory Stark" <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: > >> Gregory Stark <[EMAIL PROTECTED]> writes: >>> We're seeing a problem where occasionally a process appears to be granted a >>> lock but miss its semaphore signal. >> >> Kernel bug maybe? What's the platform?

[HACKERS] createlang/droplang -l outputs

Hi, The header in the list of already installed languages shown by createlang/droplang with the -l option is not printed from 8.2. PostgreSQL 8.2: $ createlang -l plpgsql | yes PostgreSQL 8.1: $ createlang -l Procedural Languages Name | Trusted? -+-- plpgsql

Re: [HACKERS] HOT patch, missing things

"Tom Lane" <[EMAIL PROTECTED]> writes: > We have so far managed to avoid any really strong dependencies on the > requirement of index-function immutability --- your queries may not work > very well if the function isn't immutable, but you are not at risk of > system-level data corruption. With thi

Re: [HACKERS] HOT patch, missing things

Gregory Stark <[EMAIL PROTECTED]> writes: > I agree about the costs for evaluating the expressions. But a COLD update is > certainly going to have to evaluate both expressions once. The only additional > cost here is that HOT is going to have to evaluate the *old* expression as > well. So it's at w

Re: [HACKERS] change name of redirect_stderr?

Andrew Dunstan <[EMAIL PROTECTED]> writes: > Before I wrap up the CSVlog stuff, we need to decide whether or not to > change the name of the redirect_stderr setting, and if so to what. The > reason is that with CSVlogs it will no longer apply just to stderr (we > will require it to be on for CSV

Re: [HACKERS] crypting prosrc in pg_proc

"Merlin Moncure" <[EMAIL PROTECTED]> writes: > ... IIRC the general > conclusion was that if you want to truly encrypt the sources for your > functions, the basic idea is to create a new stored procedure language > that wraps pl/pgsql and handles encryption there. > This would be relatively easy t

Re: [HACKERS] default_text_search_config and expression indexes

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Aug 09, 2007 at 02:03:13PM +0400, Oleg Bartunov wrote: > On Thu, 9 Aug 2007, [EMAIL PROTECTED] wrote: > > >Maybe I'm missing something [...] > I'm tired to repeat - index itself doesn't know about configuration ! > It just index tsvector data

Re: [HACKERS] crypting prosrc in pg_proc

Make a loadable PL "plpgsqlsec" or something like that that's just a thin wrapper around the plpgsql call handler, and all it does is decrypt the source text. perfect idea, simple perfect. i did not consider that yet. i was hoping for some enlightenment like that. would be a nice module for

Re: [HACKERS] RIP: Buildfarm member Baiji ??

The extinction of a dolphin ... sign of things to come? I've got a dolphin too (somebody's doing this on purpose ;): http://www.pgbuildfarm.org/cgi-bin/show_history.pl?nm=tucuxi&br=HEAD A few hours ago I realized the vmware instance it's running in had been suspended in a snapshot for the la

[HACKERS] 2D partitioning of VLDB - sane or not?

I am building up a schema for storing a bunch of data about proteins, which on a certain level can be modelled with quite simple tables. The problem is that the database I am building needs to house lots of it >10TB and growing, with one table in particular threatening to top 1TB. In the case of th

Re: [HACKERS] crypting prosrc in pg_proc

On Thursday 09 August 2007 11:30, Joshua D. Drake wrote: > Andrew Sullivan wrote: > > On Thu, Aug 09, 2007 at 04:42:19PM +0200, Hans-Juergen Schoenig wrote: > >> the problem here is that vendors of appliances don't want people to > >> spider their codes. this is a fact - it is not the idea of open

Re: [HACKERS] Compilation of pg 7.4.17 fails on HP-UX

On Thu, Aug 09, 2007 at 11:46:47AM -0400, Tom Lane wrote: > "Adrian Maier" <[EMAIL PROTECTED]> writes: > > I have just tried to compile postgresql 7.4.17 on a HP-UX 11.11 box > > (PA-RISC) and the compliation fails in gist. > > I do not think anyone cares about making 7.4.x run on platforms it

Re: [HACKERS] crypting prosrc in pg_proc

On Thu, Aug 09, 2007 at 11:41:02AM -0400, Andrew Dunstan wrote: > > > Decibel! wrote: > >This is also related to the desire to be able to restrict access to the > >catalog tables. Doing so could potentially solve this problem; it > >solves other issues (such as being able to see all the database

Re: [HACKERS] crypting prosrc in pg_proc

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hans-Juergen Schoenig wrote: > > On Aug 9, 2007, at 5:03 PM, Greg Smith wrote: > >> On Thu, 9 Aug 2007, Andrew Dunstan wrote: >> >>> There are also some fairly impressive code obfuscators about, that >>> your clients might find useful. >> >> All they

Re: [HACKERS] Wrote a connect-by feature

Bertram Scharpf wrote: Hi, Am Mittwoch, 08. Aug 2007, 14:04:23 +0200 schrieb Bertram Scharpf: Am Mittwoch, 08. Aug 2007, 00:52:41 +0200 schrieb Bertram Scharpf: Nobody's answering just a line. Is it such a bore?

Re: [HACKERS] Wrote a connect-by feature

Hi, Am Mittwoch, 08. Aug 2007, 14:04:23 +0200 schrieb Bertram Scharpf: > Am Mittwoch, 08. Aug 2007, 00:52:41 +0200 schrieb Bertram Scharpf: > > Nobody's answering just a line. Is it such a bore? Bertram -- Bertram Scharpf Stuttgart, Deuts

Re: [HACKERS] createlang/droplang -l outputs

Tomoaki Sato <[EMAIL PROTECTED]> writes: > The header in the list of already installed languages shown by > createlang/droplang with the -l option is not printed from 8.2. Ooops. I seem to have missed these uses of printQuery() when we put in the FETCH_COUNT patch last summer :-(. > +

Re: [HACKERS] Unexpected VACUUM FULL failure

"Simon Riggs" <[EMAIL PROTECTED]> writes: > Good hunch. I plugged this hole earlier, but on further inspection I can > see the plug wasn't wide enough. XLogAsyncCommitFlush() is good enough, > but HeapTupleSatisfiesVacuum() still allowed the inexact bookkeeping to > sometimes skip hint bit setting,

Re: [HACKERS] Compilation of pg 7.4.17 fails on HP-UX

Decibel! <[EMAIL PROTECTED]> writes: > On Thu, Aug 09, 2007 at 11:46:47AM -0400, Tom Lane wrote: >> "Adrian Maier" <[EMAIL PROTECTED]> writes: >>> I have just tried to compile postgresql 7.4.17 on a HP-UX 11.11 box >>> (PA-RISC) and the compliation fails in gist. >> >> I do not think anyone care

Re: [HACKERS] Wrote a connect-by feature

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew Dunstan wrote: > > > Bertram Scharpf wrote: >> Hi, >> >> Am Mittwoch, 08. Aug 2007, 14:04:23 +0200 schrieb Bertram Scharpf: >> >>> Am Mittwoch, 08. Aug 2007, 00:52:41 +0200 schrieb Bertram Scharpf: >>>

[HACKERS] pgcheck - data integrity check

Hi, I am working on a data integrity check tool (pgcheck). I would like to discuss the following issues: Right now I am working on a function, which should check the validity of pages in relation. The relation is passed to the function as its argument (its oid). For the integrity check of a page,

Re: [HACKERS] Unexpected VACUUM FULL failure

"Tom Lane" <[EMAIL PROTECTED]> writes: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > >> ISTM that if we call HeapTupleSatisfiesVacuum() with an additional >> boolean parameter, force, we can tell VF to always set the hint bits in >> every case, not just HEAP_MOVED_IN and HEAP_MOVED_OUT. > > Surel

Re: [HACKERS] Compilation of pg 7.4.17 fails on HP-UX

On 8/10/07, Tom Lane <[EMAIL PROTECTED]> wrote: > Decibel! <[EMAIL PROTECTED]> writes: > > On Thu, Aug 09, 2007 at 11:46:47AM -0400, Tom Lane wrote: > >> "Adrian Maier" <[EMAIL PROTECTED]> writes: > >>> I have just tried to compile postgresql 7.4.17 on a HP-UX 11.11 box > >>> (PA-RISC) and the c

Re: [HACKERS] Compilation of pg 7.4.17 fails on HP-UX

"Adrian Maier" <[EMAIL PROTECTED]> writes: > On 8/10/07, Tom Lane <[EMAIL PROTECTED]> wrote: >> [ digs around a bit... ] However, I might have been right for the >> wrong reasons. I'm thinking Adrian is trying to build for 64-bit >> HPPA (a beast the 7.4 supported-platforms matrix doesn't know ab