On Sun, Dec 09, 2007 at 02:40:37PM -0500, Andrew Dunstan wrote:
Magnus Hagander wrote:
You seem to have misunderstood what I am suggesting. Of course we should
document use of buildenv.pl in addition to the hacky fix to the .bat
files. The hack is the part that would be invisible. The
GetText is conversion po(EUC_JP) to SJIS.
Yes.
Are you sure about that? Why would gettext be converting to SJIS,
when
SJIS is nowhere in the environment it can see?
gettext is using GetACP () on Windows, wherever that gets it's info from
...
chcp did change the GetACP codepage in
Michael Akinde wrote:
Hi,
I am encountering problems when trying to run VACUUM FULL ANALYZE on a
particular table in my database; namely that the process crashes out
with the following problem:
INFO: vacuuming pg_catalog.pg_largeobject
ERROR: out of memory
DETAIL: Failed on request of
On Tue, 2007-12-11 at 10:59 +0100, Michael Akinde wrote:
I am encountering problems when trying to run VACUUM FULL ANALYZE on a
particular table in my database; namely that the process crashes out
with the following problem:
Probably just as well, since a VACUUM FULL on an 800GB table is
Hi,
I am encountering problems when trying to run VACUUM FULL ANALYZE on a
particular table in my database; namely that the process crashes out
with the following problem:
INFO: vacuuming pg_catalog.pg_largeobject
ERROR: out of memory
DETAIL: Failed on request of size 536870912.
INFO:
I'm starting work on next projects for 8.4.
Many applications have the need to store very large data volumes for
both archival and analysis. The analytic databases are commonly known as
Data Warehouses, though there isn't a common term for large archival
data stores. The use cases for those can
Many applications have the need to archive data after it has been
through the initial flurry of reads and updates that follows its
original insertion. Currently there is no specific feature support to
meet this requirement, so I propose to add this for 8.4.
Use Case: VLDB with tons of (now) read
Thanks for the rapid responses.
Stefan Kaltenbrunner wrote:
this seems simply a problem of setting maintenance_work_mem too high
(ie higher than what your OS can support - maybe an
ulimit/processlimit is in effect?) . Try reducing maintenance_work_mem
to say 128MB and retry.
If you promise
Michael Akinde wrote:
Thanks for the rapid responses.
Stefan Kaltenbrunner wrote:
this seems simply a problem of setting maintenance_work_mem too high
(ie higher than what your OS can support - maybe an
ulimit/processlimit is in effect?) . Try reducing maintenance_work_mem
to say 128MB and
Simon Riggs [EMAIL PROTECTED] writes:
So... VACUUM FREEZE table SET READ ONLY;
would be my first thought, but I'm guessing everybody will press me
towards supporting the more obvious
ALTER TABLE table SET READ ONLY;
This command will place a ShareLock (only) on the table, preventing
Michael Akinde wrote:
Thanks for the rapid responses.
Stefan Kaltenbrunner wrote:
this seems simply a problem of setting maintenance_work_mem too high (ie
higher than what your OS can support - maybe an ulimit/processlimit is in
effect?) . Try reducing maintenance_work_mem to say 128MB and
On Tue, Dec 11, 2007 at 12:30:43PM +0100, Michael Akinde wrote:
The way the process was running, it seems to have basically just
continually allocated memory until (presumably) it broke through the
slightly less than 1.2 GB shared memory allocation we had provided for
PostgreSQL (at least
[moved from -general]
Tom Lane [EMAIL PROTECTED] writes:
The bottom line here is that we have built a partitioning facility
out of spare parts, ie, a very generalized contradiction-proving
section of the planner.
That's true but I think the conclusions you reach are a stretch. The main
On Tue, 2007-12-11 at 11:12 +, Simon Riggs wrote:
Features
- Read Only Tables
- Compressed Tablespaces
I wonder if instead of read-only tables wouldn't it be better to have
some kind of automatic partitioning which permits to have different
chunks of the table data in different
On Tue, 2007-12-11 at 13:44 +0100, Csaba Nagy wrote:
Another advantage I guess would be that active data would more likely
stay in cache, as updated records would stay together and not spread
over the inactive.
And I forgot to mention that vacuum could mostly skip the archive part,
and only
Ühel kenal päeval, T, 2007-12-11 kell 13:44, kirjutas Csaba Nagy:
On Tue, 2007-12-11 at 11:12 +, Simon Riggs wrote:
Features
- Read Only Tables
- Compressed Tablespaces
I wonder if instead of read-only tables wouldn't it be better to have
some kind of automatic partitioning which
On Tue, 2007-12-11 at 14:58 +0200, Hannu Krosing wrote:
Ühel kenal päeval, T, 2007-12-11 kell 13:44, kirjutas Csaba Nagy:
Then put the active chunk on a high performance file system and the
archive tablespace on a compressed/slow/cheap file system and you're
done. Allow even the archive
Gevik Babakhani wrote:
After reading the thread of 2004 regarding user quotas, I understand
why the discussion moved towards having a tablespace quota as a
solution.
My reason to start this discussion was due the need of controlling
database size. Having tablespace quotas could allow one to
Magnus Hagander wrote:
On Mon, Dec 10, 2007 at 02:05:05PM +, Bruce Momjian wrote:
Log Message:
---
Document how to turn off disk write cache on popular operating systems.
Modified Files:
--
pgsql/doc/src/sgml:
wal.sgml (r1.46 - r1.47)
On 12/11/07, Simon Riggs [EMAIL PROTECTED] wrote:
Compressed Tablespaces
Using a streaming library like zlib, it will be easy to read/write data
files into a still-usable form but with much reduced size. Access to a
compressed table only makes sense as a SeqScan. That would be handled by
On 11/12/2007, Csaba Nagy [EMAIL PROTECTED] wrote:
On Tue, 2007-12-11 at 14:58 +0200, Hannu Krosing wrote:
Ühel kenal päeval, T, 2007-12-11 kell 13:44, kirjutas Csaba Nagy:
Then put the active chunk on a high performance file system and the
archive tablespace on a compressed/slow/cheap
Hmm, I'm now wondering if the log line number is correctly positioned.
Right now we have it just after the PID. So it suggests that following
PID and log line number is enough for tracking what a session does.
While this is not entirely incorrect, ISTM to be more logical to put it
closer to the
Stefan Kaltenbrunner wrote:
Michael Akinde wrote:
Incidentally, in the first error of the two I posted, the shared
memory setting was significantly lower (24 MB, I believe). I'll try
with 128 MB before I leave in the evening, though (assuming the other
tests I'm running complete by then).
On 11/12/2007, Simon Riggs [EMAIL PROTECTED] wrote:
Attach
--
Writing tables on one system and then moving that data to other systems
is fairly common. If we supported read-only tables then you might
consider how you would publish new versions to people.
For now, we just want to
Gregory Stark [EMAIL PROTECTED] writes:
I think this will be easiest to do for bitmap index scans. Since we gather up
all the pages we'll need before starting the heap scan we can easily skim
through them, issue posix_fadvises for at least a certain number ahead of the
actual read point and
On Tue, Dec 11, 2007 at 03:18:54PM +0100, Michael Akinde wrote:
The server has 4 GB RAM available, so even if it was trying to use 1.2
GB shared memory + 1 GB for maintenance_mem all at once, it still seems
odd that the process would fail. As far as I can tell (running ulimit -a
), the
On Mon, Dec 10, 2007 at 09:56:39AM +, Dave Page wrote:
Dave Page wrote:
Tom Lane wrote:
Dave Page [EMAIL PROTECTED] writes:
Gregory Stark wrote:
An alternative is leaving it in the project file but putting
something like
this in c.h:
Put it in win32.h, please. c.h shouldn't
On Tue, Dec 11, 2007 at 08:18:42AM -0500, Bruce Momjian wrote:
Magnus Hagander wrote:
On Mon, Dec 10, 2007 at 02:05:05PM +, Bruce Momjian wrote:
Log Message:
---
Document how to turn off disk write cache on popular operating systems.
Modified Files:
--
Another problem I just noticed is that it seems the bgwriter is
inheriting the session id from Postmaster; it doesn't have one of its
own.
--
Alvaro Herrera Developer, http://www.PostgreSQL.org/
People get annoyed when you try to debug them. (Larry Wall)
Alvaro Herrera wrote:
Another problem I just noticed is that it seems the bgwriter is
inheriting the session id from Postmaster; it doesn't have one of its
own.
Huh, sorry, I'm just blind, I neglected to look at the last digit ;-)
--
Alvaro Herrera Valdivia, Chile ICBM: S 39º 49'
Martijn van Oosterhout wrote:
IIRC you said you're on a 32-bit architecture? Which means any single
process only has 4GB address space. Take off 1GB for the kernel, 1GB
shared memory, 1 GB maintainence workmem and a collection of libraries,
stack space and general memory fragmentation and I can
Magnus Hagander [EMAIL PROTECTED] writes:
If you set it to fsync or fsync_writethrough it will write through the
cache.
Really? How much should we trust that?
regards, tom lane
---(end of broadcast)---
TIP 9: In versions
On Tue, Dec 11, 2007 at 10:09:51AM -0500, Tom Lane wrote:
Magnus Hagander [EMAIL PROTECTED] writes:
If you set it to fsync or fsync_writethrough it will write through the
cache.
Really? How much should we trust that?
I'd say as much as we shuold trust that checkbox in the Windows
On Tue, 2007-12-11 at 11:49 +, Gregory Stark wrote:
Simon Riggs [EMAIL PROTECTED] writes:
So... VACUUM FREEZE table SET READ ONLY;
would be my first thought, but I'm guessing everybody will press me
towards supporting the more obvious
ALTER TABLE table SET READ ONLY;
This
Simon Riggs [EMAIL PROTECTED] writes:
This command will place a ShareLock (only) on the table, preventing
anybody from writing to the table while we freeze it. The ShareLock is
incompatible with any transaction that has written to the table, so when
we acquire the lock all writers to the table
Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
Another change I did was to change a %.*s to %*s. The precision
marker seems useless AFAICT.
This is wrong, broken, will cause crashes on platforms where the PS
string is not null-terminated. (Hint: .* is a maximum width, * is a
For starters, if binary results is a feature you wish you could uninvent
then we are probably dead in the water for that reason. This goes to
the core of our concept. If there is no desire to synch client server
in regards to type handling, than this a waste of time. I think all of
this
On Tue, 2007-12-11 at 10:19 -0500, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
This command will place a ShareLock (only) on the table, preventing
anybody from writing to the table while we freeze it. The ShareLock is
incompatible with any transaction that has written to the
On Tue, 2007-12-11 at 13:44 +0100, Csaba Nagy wrote:
On Tue, 2007-12-11 at 11:12 +, Simon Riggs wrote:
Features
- Read Only Tables
- Compressed Tablespaces
I wonder if instead of read-only tables wouldn't it be better to have
some kind of automatic partitioning
That's definitely
Alvaro Herrera [EMAIL PROTECTED] writes:
I still propose that the log line number should be moved w.r.t. session
identifier.
No objection here.
I changed two more things: the VXID is not reported if not in a backend
(because AuxiliaryProcesses are said to never have one), and added
quotes
Am Montag, 10. Dezember 2007 schrieb Alvaro Herrera:
Peter Eisentraut wrote:
I figured it would make sense if pgarch.c used the same mechanism that
postmaster.c uses to report the various variants of regular and signal
exits.
Hmm. Getting rid of the (PID 0) is going to be a mess enough
Andrew Chernow [EMAIL PROTECTED] writes:
For starters, if binary results is a feature you wish you could uninvent
then we are probably dead in the water for that reason. This goes to
the core of our concept.
Not really: AFAICS you could implement exactly the API you have sketched
without
A very simple C function which I copied from the manual.
And I found that it's called twice.
Even in the function:
if (SRF_IS_FIRSTCALL()) {
ereport(INFO, (errcode(ERRCODE_IO_ERROR), errmsg(1)));
}
An example output. You will find two INFO: 1 there..
Why a function is called twice and how
On Tue, 2007-12-11 at 17:27 +0100, Peter Eisentraut wrote:
Am Montag, 10. Dezember 2007 schrieb Alvaro Herrera:
Peter Eisentraut wrote:
I figured it would make sense if pgarch.c used the same mechanism that
postmaster.c uses to report the various variants of regular and signal
exits.
Peter Eisentraut [EMAIL PROTECTED] writes:
Better patch.
Doesn't this patch break the behavior that is documented in the comment?
Specifically, the case where the restore_command dies on a signal and
this is reported to us by the controlling shell as exitcode 128.
We want the archiver to die,
Am Dienstag, 11. Dezember 2007 schrieb Tom Lane:
Peter Eisentraut [EMAIL PROTECTED] writes:
Better patch.
Doesn't this patch break the behavior that is documented in the comment?
Specifically, the case where the restore_command dies on a signal and
this is reported to us by the controlling
library users would have no idea whether
the values they provide are being sent as text or binary.
The putf interface currently abstracts how it actually sends it.
Although, you do put a C type rather than a string. There is a putstr
%pqtypstr, which puts the string representation of a type.
Hi all,
I'm not sure psql handles \dFp the right way. The query allows
translators to translate some columns' values but forgets to escape the
strings. So, here is a patch that escapes these translated strings.
Regards.
--
Guillaume.
http://www.postgresqlfr.org
http://dalibo.com
Index:
Simon.
VLDB Features I'm expecting to work on are
- Read Only Tables/WORM tables
- Advanced Partitioning
- Compression
plus related performance features
Just so you don't lose sight of it, one of the biggest VLDB features we're
missing is fault-tolerant bulk load. Unfortunately, I don't
your proposal completely glossed over the
issue of exactly what data structure would be exposed to clients for
anything more complex than an integer
Yeah. Forgot to include the below in the proposal and the last email.
Here is the lastest list for complex types. Most have been around since
On Tue, Dec 11, 2007 at 11:12:46AM +, Simon Riggs wrote:
Read-Only Tables
In the past when this topic came up, there was some discussion of doing this
at a level somewhere below the table horizon. There are a number of nasty
limitations for partitions currently (not the
Peter Eisentraut [EMAIL PROTECTED] writes:
Am Dienstag, 11. Dezember 2007 schrieb Tom Lane:
Doesn't this patch break the behavior that is documented in the comment?
Specifically, the case where the restore_command dies on a signal and
this is reported to us by the controlling shell as exitcode
Guillaume Lelarge [EMAIL PROTECTED] writes:
I'm not sure psql handles \dFp the right way. The query allows
translators to translate some columns' values but forgets to escape the
strings. So, here is a patch that escapes these translated strings.
This seems mighty ugly, and it's not the way we
Tom Lane a écrit :
Guillaume Lelarge [EMAIL PROTECTED] writes:
I'm not sure psql handles \dFp the right way. The query allows
translators to translate some columns' values but forgets to escape the
strings. So, here is a patch that escapes these translated strings.
This seems mighty ugly,
Guillaume Lelarge [EMAIL PROTECTED] writes:
Tom Lane a écrit :
This seems mighty ugly, and it's not the way we handle any other \d
command. Why is it needed for \dFp (and only that)?
The problem here is that Start parse is translated with Début de
l'analyse (which is a bad translation but
If I create a binary cursor on a recent version of PostgreSQL, how can I
tell if the timestamp data internally is an 8 byte double or an 8 byte
integer?
I see an #ifdef that changes the code path to compute timestamps as one
type or the other, but I do not know how to recognize the internal
Ühel kenal päeval, T, 2007-12-11 kell 10:53, kirjutas Josh Berkus:
Simon.
VLDB Features I'm expecting to work on are
- Read Only Tables/WORM tables
- Advanced Partitioning
- Compression
plus related performance features
Just so you don't lose sight of it, one of the biggest VLDB
-Original Message-
From: Alvaro Herrera [mailto:[EMAIL PROTECTED]
Sent: Tuesday, December 11, 2007 1:11 PM
To: Dann Corbit
Cc: pgsql-hackers@postgresql.org; [EMAIL PROTECTED]
Subject: Re: [GENERAL] Dumb question about binary cursors and
#ifdefHAVE_INT64_TIMESTAMP
Dann Corbit
Dann Corbit wrote:
If I create a binary cursor on a recent version of PostgreSQL, how can I
tell if the timestamp data internally is an 8 byte double or an 8 byte
integer?
I see an #ifdef that changes the code path to compute timestamps as one
type or the other, but I do not know how to
Dann Corbit [EMAIL PROTECTED] writes:
If I create a binary cursor on a recent version of PostgreSQL, how can I
tell if the timestamp data internally is an 8 byte double or an 8 byte
integer?
PQparameterStatus(conn, integer_datetimes)
regards, tom lane
On Tue, 2007-12-11 at 10:53 -0800, Josh Berkus wrote:
Simon.
VLDB Features I'm expecting to work on are
- Read Only Tables/WORM tables
- Advanced Partitioning
- Compression
plus related performance features
Just so you don't lose sight of it, one of the biggest VLDB features we're
Tom Lane a écrit :
Guillaume Lelarge [EMAIL PROTECTED] writes:
Tom Lane a écrit :
This seems mighty ugly, and it's not the way we handle any other \d
command. Why is it needed for \dFp (and only that)?
The problem here is that Start parse is translated with Début de
l'analyse (which is a
-Original Message-
From: Alvaro Herrera [mailto:[EMAIL PROTECTED]
Sent: Tuesday, December 11, 2007 1:11 PM
To: Dann Corbit
Cc: pgsql-hackers@postgresql.org; [EMAIL PROTECTED]
Subject: Re: [GENERAL] Dumb question about binary cursors and
#ifdefHAVE_INT64_TIMESTAMP
Dann Corbit
Guillaume Lelarge [EMAIL PROTECTED] writes:
Tom Lane a écrit :
We should be fixing it so that the translated strings never go to the
server and back at all. This doesn't seem amazingly hard for column
headings --- it'd take some API additions in print.c, I think.
I'll take a look at this.
Simon Riggs wrote:
I prefer archive_command = ... for the errdetail though. The commands
can be quite long so the extra words don't really add anything, plus
they require translation.
I did like this suggestion, but then I noticed, we don't actually report the
setting of the archive_command
I wrote:
describe.c's whole approach to this has always been pretty thoroughly
broken in my mind, because it makes untenable assumptions about the
client-side gettext() producing strings that are in the current
client_encoding. If they are not, the server will probably reject
the SQL query
Tom Lane wrote:
No, you are confusing the cases called shell was killed by a signal
and called command was killed by a signal, which the shell then turned
around and reported to us as exit 128.
Yes, I had missed that difference. Next try ...
--
Peter Eisentraut
On Tue, 2007-12-11 at 14:25 -0500, Andrew Sullivan wrote:
On Tue, Dec 11, 2007 at 11:12:46AM +, Simon Riggs wrote:
Read-Only Tables
In the past when this topic came up, there was some discussion of doing this
at a level somewhere below the table horizon. There are
On Tue, 2007-12-11 at 23:31 +0100, Peter Eisentraut wrote:
Simon Riggs wrote:
I prefer archive_command = ... for the errdetail though. The commands
can be quite long so the extra words don't really add anything, plus
they require translation.
I did like this suggestion, but then I
Simon Riggs wrote:
On Tue, 2007-12-11 at 23:31 +0100, Peter Eisentraut wrote:
Simon Riggs wrote:
I prefer archive_command = ... for the errdetail though. The commands
can be quite long so the extra words don't really add anything, plus
they require translation.
I did like this
Hannu,
COPY ... WITH ERRORS TO ...
Yeah, that's a start.
or something more advanced, like bulkload which can be continued after
crash ?
Well, we could also use a loader which automatically parallelized, but that
functionality can be done at the middleware level. WITH ERRORS is the
most
On Tue, 2007-12-11 at 10:53 -0800, Josh Berkus wrote:
Just so you don't lose sight of it, one of the biggest VLDB features we're
missing is fault-tolerant bulk load.
I actually had to cook up a version of this for Truviso recently. I'll
take a look at submitting a cleaned-up implementation for
On Tue, 2007-12-11 at 15:31 -0800, Josh Berkus wrote:
Here's the other VLDB features we're missing:
Parallel Query
Windowing Functions
Parallel Index Build (not sure how this works exactly, but it speeds Oracle
up considerably)
On-disk Bitmap Index (anyone game to finish GP patch?)
I
On Sun, Dec 9, 2007 at 6:37 PM, in message
[EMAIL PROTECTED], Alvaro Herrera
[EMAIL PROTECTED] wrote:
I have never understood what's the point of having an option to force a
password prompt. I wonder why don't we deprecate -W?
I occasionally find it useful for situations where I have a
On Tue, 11 Dec 2007, Josh Berkus wrote:
Just so you don't lose sight of it, one of the biggest VLDB features we're
missing is fault-tolerant bulk load. Unfortunately, I don't know anyone
who's working on it.
I'm curious what you feel is missing that pgloader doesn't fill that
requirement:
I'm still working on getting through our test cases and benchmarks,
but one thing I can say for sure -- a freshly loaded database under
8.3beta4 is over 13% smaller than the exact same database freshly
loaded into 8.2.5. It went from 216GB to 187GB.
Of course, the down side of this is that
Peter Eisentraut [EMAIL PROTECTED] writes:
Yes, I had missed that difference. Next try ...
Looks sane to me.
regards, tom lane
---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at
Alvaro Herrera [EMAIL PROTECTED] writes:
Simon Riggs wrote:
I think you should lose the The, if nothing else. Most Postgres
messages I recall say return code = xx not The return code ...
Right, that's because they are in errmessage() and not errdetail(). The
guidelines are different.
Right
Hi,
Here's another problem report on Windows. This time it is usage of SSL
connections and NOTIFY. I talked to Magnus on IRC and he directed me to
bug #2829:
http://archives.postgresql.org/pgsql-bugs/2006-12/msg00122.php
This report seems to be a little different, if only because the reported
Hi.
Yeah, As a part from which a problem happens, it is your suggestion.
This is only the check.
http://winpg.jp/~saito/pg83/message_check/gtext2.c
Therefore, a message needed is acquirable in the next operation.
gtext2 C UTF-8
http://winpg.jp/~saito/pg83/message_check/codeset_utf8_msg.txt
Alvaro Herrera [EMAIL PROTECTED] writes:
This guy is using 8.2.5. SSL seems to be able to fill his log files at
full speed.
Are you *sure* the server is 8.2.5? 8.2.5 shouldn't emit duplicate
messages, but 8.2.4 and before would:
2007-05-17 21:20 tgl
* src/backend/libpq/:
Billow Gao [EMAIL PROTECTED] writes:
A very simple C function which I copied from the manual.
And I found that it's called twice.
You do realize that it's *supposed* to be called twice? Once to
return the first row, and again to say it's done returning rows.
But the info messages you show are
Greg,
I'm curious what you feel is missing that pgloader doesn't fill that
requirement: http://pgfoundry.org/projects/pgloader/
Because pgloader is implemented in middleware, it carries a very high overhead
if you have bad rows. As little as 1% bad rows will slow down loading by 20%
due to
On 12/11/07, Tom Lane [EMAIL PROTECTED] wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
I dunno anything about how to fix the real problem (what's winsock error
10004?), but I don't think he'd be seeing full speed log filling in
8.2.5.
WSAEINTR, A blocking operation was interrupted by a call
Trevor Talbot [EMAIL PROTECTED] writes:
On 12/11/07, Tom Lane [EMAIL PROTECTED] wrote:
I dunno anything about how to fix the real problem (what's winsock error
10004?),
WSAEINTR, A blocking operation was interrupted by a call to
WSACancelBlockingCall.
Oh, then it's exactly the same thing as
Gregory Stark wrote:
I could swear this has been discussed in the past too. I seem to recall Luke
disparaging Postgres on the same basis but proposing an immensely complicated
solution. posix_fadvise or using libaio in a simplistic fashion as a kind of
fadvise would be fairly lightweight way
86 matches
Mail list logo
