Re: [HACKERS] psql's describe command (for sequences) output improvement

2007-12-05 Thread Aftab Hussain
On Dec 4, 2007 7:47 PM, Peter Childs [EMAIL PROTECTED] wrote:



 On 04/12/2007, Aftab Hussain [EMAIL PROTECTED] wrote:
 
 
  Hi all,
 
  I have a patch which tries to improve the '\d some_sequence_name'
  command output in psql utility. Before sending the patch to pgsql-patches I
  just want to know your opinion whether the new output of the command is OK
  or it needs to be modified before sending it to pgsql-patches.
 
  For comparison purpose, here is the output of the '\d
  some_sequence_name' command:
 
  -- Say we have created a sequence named 'test_seq' already as CREATE
  SEQUENCE test_seq.
 
  --
  -- Output: before applying the patch.
  --
  testdb=# \d test_seq;
  Sequence public.test_seq
  Column |  Type
  ---+-
   sequence_name | name
   last_value| bigint
   increment_by  | bigint
   max_value | bigint
   min_value | bigint
   cache_value   | bigint
   log_cnt   | bigint
   is_cycled | boolean
   is_called | boolean
 
  --
  -- Output: after applying the patch.
  --
  testdb=# \d test_seq;
Sequence public.test_seq
   last_value | increment_by |  max_value  | min_value |
  cache_value | log_cnt | is_cycled | is_called
 
  +--+-+---+-+-+---+---
   1  | 1| 9223372036854775807 | 1 |
  1   | 1   | f | f
 
 
 
  Thanks in advance for your feedback(s).
 
  --
  Aftab.
 
 Why?

 is that not what

 select * from test_seq;

 does already.



 \d command return meta data not data.

 Peter Childs


In general, \d command is working perfectly for database objects.

For sequences, I think the current \d some_sequence command's output is
displaying information which does not help the end user very much. Also
isn't the newly display information (same as information provided by 'select
* from test_seq;' statement) is/can-be-considered the metadata information
about the sequences queried about (since for the returning sequences data we
have nextval('...'), currval('...') functions)?

Please correct me if I am wrong or have missed something.

Aftab Hussain.
EnterpriseDB.   www.enterprisedb.com


Re: [HACKERS] TOASTed size

2007-12-05 Thread Mark Kirkwood

Simon Riggs wrote:
  
That sounds more like what I was after.


So let me check my understanding: For TOASTed data pg_column_size()
tells you how many bytes the column value occupies when decompressed. So
there isn't any way of finding out how many bytes a column value
actually occupies when it is both compressed and external?

  


I dimly recall getting confused by this when writing this guy:

From what I can see: pg_column_size calls toast_datum_size for any 
variable length attribute - and then gets the external pointer and 
returns its va_extsize component (which looks to me like the 
*compressed* size.)


Cheers

Mark

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq


Re: [HACKERS] buildenv.pl/buildenv.bat

2007-12-05 Thread Magnus Hagander
On Tue, Dec 04, 2007 at 09:31:30AM -0500, Andrew Dunstan wrote:
 
 
 Magnus Hagander wrote:
 My recollection is that I changed the minimum amount necessary, because 
 I was expecting us to go into beta at anmy moment (silly me). That might 
 be why we still have both. There was an expectation that some cleanup 
 might be required during 8.4 development. I know I left other .bat files 
 as wrappers for the perl scripts, but that's obviously not appropriate 
 here. I'll see if I can adjust builddoc.bat so we can get rid of 
 buildenv.bat.
 
 As for documentation, you're probably right, I could easily have missed 
 it. I'll look into it.
 
 
 Great. Let me know if you need me to poke at anything.
 
 
   
 
 OK, I'm thinking that the best way might be to do away with buildenv.bat 
 altogether and replace the remaining references to it in .bat files with 
 something like this fragment:
 
 if not exist src\tools\msvc\buildenv.pl goto nobuildenv
 perl -e require 'src/tools/msvc/buildenv.pl'; while(($k,$v) = each 
 %ENV) { print [EMAIL PROTECTED] $k=$v\n]; }  tmp.bat
 call tmp.bat
 del tmp.bat
 :nobuildenv
 
 and adjust the docs accordingly.
 
 Thoughts?

Urgh.

I thought you specifically changed the earlier scripts to *avoid* having
them create a temporary file and then execute it. Which I agreed with :-)

I think I'd prefer we just get rid of the .bat's completely then. 

I wonder if we are daring enough to do it so late in the cycle, though..

//Magnus

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] [PATCHES] Better default_statistics_target

2007-12-05 Thread Greg Sabino Mullane

-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160


Simon spoke:
 The choice of 100 is because of the way the LIKE estimator is
 configured. Greg is not suggesting he measured it and found 100 to be
 best, he is saying that the LIKE operator is hard-coded at 100 and so
 the stats_target should reflect that.

Exactly.

 Setting it to 100 for all columns because of LIKE doesn't make much
 sense. I think we should set stats target differently depending upon the
 data type, but thats probably an 8.4 thing. Long text fields that might
 use LIKE should be set to 100. CHAR(1) and general fields should be set
 to 10.

Agreed, this would be a nice 8.4 thing. But what about 8.3 and 8.2? Is 
there a reason not to make this change? I know I've been lazy and not run 
any absolute figures, but rough tests show that raising it (from 10 to 
100) results in a very minor increase in analyze time, even for large 
databases. I think the burden of a slightly slower analyze time, which 
can be easily adjusted, both in postgresql.conf and right before running 
an analyze, is very small compared to the pain of some queries - which worked 
before - suddenly running much, much slower for no apparent reason at all.
Sure, 100 may have been chosen somewhat arbitrarily for the LIKE thing, 
but this is a current real-world performance regression (aka a bug, 
according to a nearby thread). Almost everyone agrees that 10 is too low, 
so why not make it 100, throw a big warning in the release notes, and 
then start some serious re-evaluation for 8.4?


- --
Greg Sabino Mullane [EMAIL PROTECTED]
End Point Corporation
PGP Key: 0x14964AC8 200712050920
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8


-BEGIN PGP SIGNATURE-

iD8DBQFHVrSivJuQZxSWSsgRAyDNAKCInH9SJRO8ly1L1MomJUPlBslBlgCeLQ1v
+w4ZumRcB5U5L3SGT0rk4AE=
=I8Ur
-END PGP SIGNATURE-



---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


[HACKERS] shared_buffers, wal_buffers, WAL files, data files

2007-12-05 Thread Guillaume Lelarge
Hi,

First, I'm not sure this mail should go to this mailing list. As it
refers to source code (mainly src/backend/postmaster/bgwriter.c and
src/backend/access/transam/xlog.c), I sent it here. I apologize if I'm
wrong.

I'm a bit puzzled by the different informations I can read on the
documentation, on the source files and other README files from the
source package. I'm actually using 8.2.5 source files.

I try to answer a simple question : what happens when I do a simple
INSERT on a just started PostgreSQL server.

# pg_ctl start
# psql foobase
[...]
foobase=# CREATE TABLE footable (i int4);
CREATE TABLE
foobase=# INSERT INTO footable (i) VALUES (1);
INSERT 0 1

From what I understand with the INSERT statement, here is what happens :
 * backend loads first (and only) block from footable file into a shared
   buffer
 * it modifies this block on the shared buffer, and sets it as dirty

After checkpoint_timeout seconds or after a manual CHECKPOINT (there's
also some other ways to have a checkpoint but in my simple example,
these two seem to be the most probable ones) :
 * bgwriter runs CreateCheckPoint
 * it will, beyond other things, flush this block from shared buffer to
   disk, fsync, sets the block as non-dirty...

... and here is my first question : flushes to the current XLOG file or
to footable data file ? I think it is to XLOG file but I don't find
where in the code the content of WAL files is flushed to data files. If
you know where I can look to get this information, I'm really interested.

From other parts of the documentation, it seems checkpoints are also in
charge of flushing WAL files to data files. I've read something like
this: a checkpoint writes dirty shared buffers, sync dirty  kernel
buffers, recycle WAL files. Also, because of the checkpoint_segments GUC.

And I don't find where wal_buffers are used in all this process.

One last thing (wrt this mail), I think I understand what
full_page_writes does but I wonder what this sentence means : Writes
full pages to WAL when first modified after a checkpoint. ? (it's the
last part that I don't understand)

Thanks for any pointers.

Regards.


-- 
Guillaume.
 http://www.postgresqlfr.org
 http://dalibo.com

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] buildenv.pl/buildenv.bat

2007-12-05 Thread Andrew Dunstan



Magnus Hagander wrote:

On Tue, Dec 04, 2007 at 09:31:30AM -0500, Andrew Dunstan wrote:
  

Magnus Hagander wrote:

My recollection is that I changed the minimum amount necessary, because 
I was expecting us to go into beta at anmy moment (silly me). That might 
be why we still have both. There was an expectation that some cleanup 
might be required during 8.4 development. I know I left other .bat files 
as wrappers for the perl scripts, but that's obviously not appropriate 
here. I'll see if I can adjust builddoc.bat so we can get rid of 
buildenv.bat.


As for documentation, you're probably right, I could easily have missed 
it. I'll look into it.
   


Great. Let me know if you need me to poke at anything.


 
  
OK, I'm thinking that the best way might be to do away with buildenv.bat 
altogether and replace the remaining references to it in .bat files with 
something like this fragment:


if not exist src\tools\msvc\buildenv.pl goto nobuildenv
perl -e require 'src/tools/msvc/buildenv.pl'; while(($k,$v) = each 
%ENV) { print [EMAIL PROTECTED] $k=$v\n]; }  tmp.bat

call tmp.bat
del tmp.bat
:nobuildenv

and adjust the docs accordingly.

Thoughts?



Urgh.

I thought you specifically changed the earlier scripts to *avoid* having
them create a temporary file and then execute it. Which I agreed with :-)

I think I'd prefer we just get rid of the .bat's completely then. 


I wonder if we are daring enough to do it so late in the cycle, though..


  


I did specifically do that, but I can't see a small workaround here that 
doesn't use this mechanism, since there's no eval function in the 
Windows shell.


I agree with getting rid of the remaining .bat files, or at least making 
them one line wrappers for perl scripts, but I think it's too late in 
the cycle for that now. As I explained, the reason I didn't make more 
changes before was because I thought it was too late then. I did just 
enough to make running the whole thing with the buildfarm client work 
reliably.


I think we could reasonably get away with this for one cycle and clean 
it up more thoroughly in 8.4.  There are exactly four places that use 
buildenv.bat: install.bat, builddoc.bat, pgbison,bat and pgflex.bat. Of 
these, the only one that really matters, I suspect, is builddoc.bat, but 
if we fix it we should do so consistently.


cheers

andrew



---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Open items for 8.3

2007-12-05 Thread Alvaro Herrera
Bruce Momjian wrote:
 In an attempt to move us toward 8.3 RC1 I have put all open item emails
 into the patches queue:
 
   http://momjian.postgresql.org/cgi-bin/pgpatches

FWIW it seems the only remaining issue is the ltree bug #3720:

http://archives.postgresql.org/pgsql-bugs/2007-11/msg00044.php

I am still seeing it with fresh HEAD:

alvherre=# SELECT '5.0.1.0'::ltree ~ '5.!0.!0.0'::lquery;
 ?column? 
--
 t
(1 ligne)

Is anyone working on this?

-- 
Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J
Hay que recordar que la existencia en el cosmos, y particularmente la
elaboración de civilizaciones dentro de él no son, por desgracia,
nada idílicas (Ijon Tichy)

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] autoconf

2007-12-05 Thread Alvaro Herrera
Patrick Welche wrote:
 I know that it doesn't matter as configure is in CVS, so there is no
 need for mere mortals to regenerate it, but why is
 
   RCS file: /projects/cvsroot/pgsql/configure.in,v
   revision 1.538
   date: 2007/11/26 12:31:07;  author: petere;  state: Exp;  lines: +2 -2
   Require a specific Autoconf version, instead of a lower bound only.
 
 a good idea?

To avoid the autoconf version used to regenerate configure from being
changed on a stable tree.

-- 
Alvaro Herrera   http://www.PlanetPostgreSQL.org/
Tulio: oh, para qué servirá este boton, Juan Carlos?
Policarpo: No, aléjense, no toquen la consola!
Juan Carlos: Lo apretaré una y otra vez.

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Open items for 8.3

2007-12-05 Thread Oleg Bartunov

On Wed, 5 Dec 2007, Alvaro Herrera wrote:


Bruce Momjian wrote:

In an attempt to move us toward 8.3 RC1 I have put all open item emails
into the patches queue:

http://momjian.postgresql.org/cgi-bin/pgpatches


FWIW it seems the only remaining issue is the ltree bug #3720:

http://archives.postgresql.org/pgsql-bugs/2007-11/msg00044.php

I am still seeing it with fresh HEAD:

alvherre=# SELECT '5.0.1.0'::ltree ~ '5.!0.!0.0'::lquery;
?column?
--
t
(1 ligne)

Is anyone working on this?



The problem is clear, Teodor probably will fix it.

Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq


Re: [HACKERS] [PATCHES] Better default_statistics_target

2007-12-05 Thread Guillaume Smet
On Dec 5, 2007 3:26 PM, Greg Sabino Mullane [EMAIL PROTECTED] wrote:
 Agreed, this would be a nice 8.4 thing. But what about 8.3 and 8.2? Is
 there a reason not to make this change? I know I've been lazy and not run
 any absolute figures, but rough tests show that raising it (from 10 to
 100) results in a very minor increase in analyze time, even for large
 databases. I think the burden of a slightly slower analyze time, which
 can be easily adjusted, both in postgresql.conf and right before running
 an analyze, is very small compared to the pain of some queries - which worked
 before - suddenly running much, much slower for no apparent reason at all.

As Tom stated it earlier, the ANALYZE slow down is far from being the
only consequence. The planner will also have more work to do and
that's the hard point IMHO.

Without studying the impacts of this change on a large set of queries
in different cases, it's quite hard to know for sure that it won't
have a negative impact in a lot of cases.

It's a bit too late in the cycle to change that IMHO, especially
without any numbers.

--
Guillaume

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] weird - invalid string enlargement request size

2007-12-05 Thread Walter Cruz
On Dec 4, 2007 7:19 PM, Tom Lane [EMAIL PROTECTED] wrote:
 Walter Cruz [EMAIL PROTECTED] writes:
  I posted on hackers cause I think that was a bug, os something.

 Yeah, I think so too.  Can you extract a reproducible test case?

I'm trying to extract that case!

[]'s
- Walter

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] weird - invalid string enlargement request size

2007-12-05 Thread Walter Cruz
Yes, I did.

What have I done.

My Linux version: Debian Etch.

My Postgres version:

PostgreSQL 8.2.5 on x86_64-unknown-linux-gnu, compiled by GCC gcc
(GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

It was compiled by myself.

My initdb was:
initdb -E LATIN1 --locale=pt_BR


By that initdb, the $LANG of the system was pt_BR.UTF-8 .

The pt_BR was not even enabled.

A simple query that shows the problem:

select true AS 
áaaa


(it's needed this exact number of chars, with one of them accented.)

The output of the SQL of Alvaro in this server is:

client_encoding;UNICODE
lc_collate;pt_BR.UTF-8
lc_ctype;pt_BR.UTF-8
lc_messages;pt_BR
lc_monetary;pt_BR
lc_numeric;pt_BR
lc_time;pt_BR
server_encoding;LATIN1


(I have enabled the pt_BR locale before the tests, but lc_collate and
lc_type are still pt_BR.UTF-8, due to initdb).

Just after that, I changed my $LANG to pt_BR, did a initdb.

The query

select true AS 
áaaa;

returns:

template1-# ;
NOTICE:  identifier
áaaa
will be truncated to
áaa
 áaa

On this second initdb:

template1=# select name, setting
template1-# from pg_settings
template1-# where name ~~ any ( array['lc_%', '%_encoding']);
  name   | setting
-+-
 client_encoding | LATIN1
 lc_collate  | pt_BR
 lc_ctype| pt_BR
 lc_messages | pt_BR
 lc_monetary | pt_BR
 lc_numeric  | pt_BR
 lc_time | pt_BR
 server_encoding | LATIN1


Is that information enough? I can give you more details, if needed.

[]'s
- Walter

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


[HACKERS] autoconf

2007-12-05 Thread Patrick Welche
I know that it doesn't matter as configure is in CVS, so there is no
need for mere mortals to regenerate it, but why is

  RCS file: /projects/cvsroot/pgsql/configure.in,v
  revision 1.538
  date: 2007/11/26 12:31:07;  author: petere;  state: Exp;  lines: +2 -2
  Require a specific Autoconf version, instead of a lower bound only.

a good idea?

Cheers,

Patrick

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Is postgres.gif missing in cvs?

2007-12-05 Thread Peter Eisentraut
Am Dienstag, 4. Dezember 2007 schrieb Tom Lane:
 The particular
 cases that were biting Devrim seemed to all be occurrences of 
 which perhaps is an allowed tag in his release.

 means whatever the last opening tag was (much like /).  So it should 
definitely be escaped.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Release Note Changes

2007-12-05 Thread Simon Riggs
On Fri, 2007-11-30 at 13:07 -0500, Tom Lane wrote:

 FWIW, I tend to agree with the folks who think Bruce trimmed too much
 this time.  But the release notes are, and always have been, intended to
 boil the CVS history down to something useful by eliminating irrelevant
 detail. 

OK, so given everything mentioned on this thread, there are three items
that are user noticeable and so don't fall into the category of
irrelevant detail:

- Merge Join performance has been substantially improved when low number
of duplicate join keys exist on the outer side of the join (Simon, Greg)

- Large I/O reduction during recovery when full_page_writes = on
(Heikki)

- WAL file switch performance has been improved. Recovery startup now
refers to the last checkpoint time, which may be anything up to the
checkpoint_timeout interval before a database crash. (Simon, Tom)

The last one seems important to me, but I can see that might be TMI for
some people, though I feel we should document it somewhere.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


[HACKERS] Needs advice on error detection

2007-12-05 Thread S. Horio

While using PostgreSQL 8.1.4 for Windows on Windows XP (SP2),
the following errors had occurred.

-
2007-10-11 12:38:21 LOG:  autovacuum: processing database hoge
2007-10-11 12:38:21 ERROR:  xlog flush request 0/2BEB2198 is not satisfied --- 
flushed only to 0/2BE32740
2007-10-11 12:38:21 CONTEXT:  writing block 205 of relation 1663/18096/27212
2007-10-11 12:38:22 ERROR:  xlog flush request 0/2BEB6A70 is not satisfied --- 
flushed only to 0/2BE32740
2007-10-11 12:38:22 CONTEXT:  writing block 206 of relation 1663/18096/27212
2007-10-11 12:38:23 ERROR:  xlog flush request 0/2BEBB1B8 is not satisfied --- 
flushed only to 0/2BE32740
2007-10-11 12:38:23 CONTEXT:  writing block 207 of relation 1663/18096/27212
...
2007-10-11 14:11:36 ERROR:  xlog flush request 0/2C27ABB8 is not satisfied --- 
flushed only to 0/2C25ED60
2007-10-11 14:11:36 CONTEXT:  writing block 258 of relation 1663/18096/27212
2007-10-11 14:11:36 WARNING:  could not write block 258 of 1663/18096/27212
2007-10-11 14:11:36 DETAIL:  Multiple failures --- write error may be permanent.
2007-10-11 14:11:36 ERROR:  xlog flush request 0/2C27ABB8 is not satisfied --- 
flushed only to 0/2C25ED60
2007-10-11 14:11:36 CONTEXT:  writing block 258 of relation1663/18096/27212
2007-10-11 14:11:36 WARNING:  could not write block 258 of 1663/18096/27212
2007-10-11 14:11:36 DETAIL:  Multiple failures --- write error may be permanent.
2007-10-11 14:11:36 LOG:  received fast shutdown request
2007-10-11 14:11:36 LOG:  shutting down
2007-10-11 14:11:36 PANIC:  xlog flush request 0/2C27ABB8 is not satisfied --- 
flushed only to 0/2C25ED60
2007-10-11 14:11:36 CONTEXT:  writing block 258 of relation 1663/18096/27212
-

We were using pg_ctl to start up PostgreSQL instead of using
Windows Services. And we weren't shutting down the database
before shutting down the OS. Because of that, we think that
we damaged the disk blocks which were used by the database. Since
our client application did not receive the error, we think
that the error was started during the auto vacuum. In order to
detect these errors, is it possible to force database to
be halted as a PANIC state or it is possible to make this kind
of unrecoverable errors to be sent to existing client connections?
Also, is it common to get these errors when database was not
properly shutdown before halting an OS?

-Best Regards,


---
Shigenori Horio360 N. Sepulveda Blvd., Suite 1040
Twin Sun Inc.El Segundo, CA 90245
[EMAIL PROTECTED] Tel: (310) 524-1800 ext 5, Fax: (310) 524-1818

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] TOASTed size

2007-12-05 Thread Gregory Stark
Simon Riggs [EMAIL PROTECTED] writes:

 On Wed, 2007-12-05 at 08:24 +, Gregory Stark wrote:
 Tom Lane [EMAIL PROTECTED] writes:
 
  Simon Riggs [EMAIL PROTECTED] writes:
  I'm thinking that there isn't any way currently of working out how big a
  compressed toast object is?
 
  pg_column_size() ?
 
 I was going to send the same thing but I think he's looking for the 
 compressed
 size of *external* data.
 
 In fact there isn't really any convenient way to find out something is stored
 external. pg_column_size reports the toast raw size of externally stored 
 data.
 
 There does seem to be a need for a more general pg_column_info which returns 
 a
 tuple (external bool, rawsize integer, storedsize integer).

 That sounds more like what I was after.

 So let me check my understanding: For TOASTed data pg_column_size()
 tells you how many bytes the column value occupies when decompressed.

Wait, no, it's supposed to be the actual size on disk. *checks* yeah, it's the
extsize which is the size of the datum in the toast table. So you could find
the compression ratio by calling length() and pg_column_size() at least for
text data.

I still think a single function returning those columns would be a nice thing
to have to make the api complete.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com
  Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL 
training!

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] TOASTed size

2007-12-05 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes:

 Simon Riggs [EMAIL PROTECTED] writes:
 I'm thinking that there isn't any way currently of working out how big a
 compressed toast object is?

 pg_column_size() ?

I was going to send the same thing but I think he's looking for the compressed
size of *external* data.

In fact there isn't really any convenient way to find out something is stored
external. pg_column_size reports the toast raw size of externally stored data.

There does seem to be a need for a more general pg_column_info which returns a
tuple (external bool, rawsize integer, storedsize integer).

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com
  Ask me about EnterpriseDB's PostGIS support!

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] TOASTed size

2007-12-05 Thread Gregory Stark

What it turns out is hard to determine is whether the column was stored
externally. To do that you have to rely on the trick of checking
pg_column_size(table.*) and that only works if it's the only column likely to
be stored externally.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com
  Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL 
training!

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] TOASTed size

2007-12-05 Thread Simon Riggs
On Wed, 2007-12-05 at 08:24 +, Gregory Stark wrote:
 Tom Lane [EMAIL PROTECTED] writes:
 
  Simon Riggs [EMAIL PROTECTED] writes:
  I'm thinking that there isn't any way currently of working out how big a
  compressed toast object is?
 
  pg_column_size() ?
 
 I was going to send the same thing but I think he's looking for the compressed
 size of *external* data.
 
 In fact there isn't really any convenient way to find out something is stored
 external. pg_column_size reports the toast raw size of externally stored data.
 
 There does seem to be a need for a more general pg_column_info which returns a
 tuple (external bool, rawsize integer, storedsize integer).

That sounds more like what I was after.

So let me check my understanding: For TOASTed data pg_column_size()
tells you how many bytes the column value occupies when decompressed. So
there isn't any way of finding out how many bytes a column value
actually occupies when it is both compressed and external?

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


[HACKERS] Uniform policy for author credits in contrib module documentation?

2007-12-05 Thread Tom Lane
As of CVS HEAD, some of the contrib module documentation pages have
extensive credit screeds, eg
http://developer.postgresql.org/pgdocs/postgres/cube.html
and some just have the author's name, with or without an email link,
and some don't have anything at all.

This bothers me; it seems like we should have a more uniform approach.

There are some arguments to be made for not having credits at all.
We don't make a habit of crediting individuals anywhere else in the SGML
docs; credits in the source code and/or CVS logs are supposed to be
enough.  And we do still have author credits in contrib/README, not to
mention the individual source code files.  And there's the whole issue
that files that have been there awhile have probably been tweaked by
a number of people besides the original author.  OTOH I dislike removing
credits that the authors might have expected to be there, and the
contrib modules mostly do have identifiable original authors.

If we do want to keep credits in the SGML pages, how extensive should
they be?  I already took it on myself to remove Gene Selkov's snailmail
address, but do we even want email addresses there?  A lot of them are
probably dead, and the ones that aren't are causing their owners to get
extra spam, because an email link is about the easiest thing to scrape
from a webpage that there could possibly be.

I don't have a strong opinion one way or the other, except that I think
we should have a uniform policy for all the contrib modules.

Comments?

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] [DOCS] Uniform policy for author credits in contrib module documentation?

2007-12-05 Thread Joe Conway

Tom Lane wrote:

As of CVS HEAD, some of the contrib module documentation pages have
extensive credit screeds, eg
http://developer.postgresql.org/pgdocs/postgres/cube.html
and some just have the author's name, with or without an email link,
and some don't have anything at all.

This bothers me; it seems like we should have a more uniform approach.


[snip]


I don't have a strong opinion one way or the other, except that I think
we should have a uniform policy for all the contrib modules.

Comments?


As far as I'm concerned, you can remove any credits for me from the 
contrib modules I've worked on (or I can do it if you'd prefer).


In any case +1 for a uniform policy, and +1 for removing credits from 
documentation.


Joe

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] weird - invalid string enlargement request size

2007-12-05 Thread Tom Lane
Walter Cruz [EMAIL PROTECTED] writes:
 My initdb was:
 initdb -E LATIN1 --locale=pt_BR
 By that initdb, the $LANG of the system was pt_BR.UTF-8 .
 A simple query that shows the problem:
 select true AS 
 áaaa

OK, I was able to reproduce this here:

postgres=# select true AS 
áaaa
;
ERROR:  invalid string enlargement request size 1073741823

What is happening is that vsnprintf() is failing because it doesn't like
its input data.  elog.c's EVALUATE_MESSAGE macro thinks that the only
possible reason for failure is that the output buffer isn't large
enough, so it enlarges and tries again, leading soon to the palloc error
displayed above.

Possibly we should put some effort into producing a more useful error
message here, but I'm reluctant to fool with it, because our historical
experience is that vsnprintf's behavior just isn't very consistent
across platforms.

In any case, the bottom-line reason why you're having a problem is that
the database encoding (LATIN1) is inconsistent with the encoding
specified by the server's locale (UTF-8), thus allowing vsnprintf to see
what it thinks is bad data.

PG 8.3 contains defenses to prevent that sort of inconsistency, and
I think that's probably all we need to do about this.

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Uniform policy for author credits in contrib module documentation?

2007-12-05 Thread Joshua D. Drake

Tom Lane wrote:

As of CVS HEAD, some of the contrib module documentation pages have
extensive credit screeds, eg
http://developer.postgresql.org/pgdocs/postgres/cube.html
and some just have the author's name, with or without an email link,
and some don't have anything at all.



I don't have a strong opinion one way or the other, except that I think
we should have a uniform policy for all the contrib modules.


Well once we push directly into the core documentation I agree that 
outside of release notes (although you just brought that up in another 
thread) we don't need to be mentioning contributions like that. Those 
who have contributed are in the logs.


Further those who have provided reasonable contribution really should be 
mentioned on the contributors page that is up for discussion which would 
make the rest of this moot yes?


Sincerely,

Joshua D. Drake



Comments?

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster




---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Uniform policy for author credits in contrib module documentation?

2007-12-05 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes:
 ...Further those who have provided reasonable contribution really should be 
 mentioned on the contributors page that is up for discussion which would 
 make the rest of this moot yes?

I don't have any objection to listing people on the contributors page
on the strength of their work on contrib modules.  But that seems
orthogonal to the question of what should be in the SGML docs ...

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Uniform policy for author credits in contrib module documentation?

2007-12-05 Thread Joshua D. Drake

Tom Lane wrote:

Joshua D. Drake [EMAIL PROTECTED] writes:
...Further those who have provided reasonable contribution really should be 
mentioned on the contributors page that is up for discussion which would 
make the rest of this moot yes?


I don't have any objection to listing people on the contributors page
on the strength of their work on contrib modules.  But that seems
orthogonal to the question of what should be in the SGML docs ...


All I was saying is I don't think we need the redundancy :). E.g; if 
they are worthy in the docs, they are worthy on the contributors page 
and thus not needed in the docs.


I am not of a strong opinion either way but it seems having names 
plastered everywhere just creates more management of information for no 
particular purpose.


Sincerely,

Joshua D. Drake



regards, tom lane




---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster