> Joshua D. Drake wrote:
> ...
> ZFS is not an option; generally speaking.
Then in general, if the corruption occurred within the:
- read path, try again and hope it takes care of itself.
- write path, the best that can be hoped for is a single bit error
within the data itself which can be bot
I updated the following SE-PostgreSQL patches:
[1/5]
http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1076.patch
[2/5]
http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1076.patch
[3/5]
http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1
Paul Schlie wrote:
this can not be relied upon, a strategy potentially utilizing the suspect
data as if it were good likely needs to be adopted, accompanied somehow with
a persistent indication that the query results (or specific sub-results) are
themselves suspect, as it may often be a lesser ev
On Wed, Oct 1, 2008 at 9:39 AM, Tom Lane <[EMAIL PROTECTED]> wrote:
> "Gurjeet Singh" <[EMAIL PROTECTED]> writes:
> > On Wed, Oct 1, 2008 at 3:07 AM, Decibel! <[EMAIL PROTECTED]> wrote:
> >> ERROR: aggregates not allowed in WHERE clause
>
> > No, the real issue is that you are referencing the out
If you are concerned with data integrity (not caused by bugs in the code
itself), you may be interested in utilizing ZFS; however, be aware that I
found and reported a bug in their implementation of the Fletcher checksum
algorithm they use by default to attempt to verify the integrity of the data
s
"Gurjeet Singh" <[EMAIL PROTECTED]> writes:
> On Wed, Oct 1, 2008 at 3:07 AM, Decibel! <[EMAIL PROTECTED]> wrote:
>> ERROR: aggregates not allowed in WHERE clause
> No, the real issue is that you are referencing the outer table's column's
> max() in the inner query (correlated sub-query).
Yeah.
On Wed, Oct 1, 2008 at 3:07 AM, Decibel! <[EMAIL PROTECTED]> wrote:
> From -HEAD:
>
> ERROR: aggregates not allowed in WHERE clause
> STATEMENT: SELECT *
>FROM loans l
>WHERE id IN ( SELECT max(l.id)
>FROM loans
>
Joshua Drake wrote:
During a 41-month period we observed more than 400,000 instances of
checksum mistmatches, 8% of which were discovered during RAID
reconstruction, creating the possibility of real data loss.
They also have a wonderful term they mention, "Silent Data corruptions".
Exactely
Hi,
2008/10/1 Dimitri Fontaine <[EMAIL PROTECTED]>:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi,
>
> Le 30 sept. 08 à 20:03, Tom Lane a écrit :
>>
>>set_read_position(tupstore, &local_read_position);
>>tuple = tuplestore_gettuple(tupstore, ...);
>>get_read_posi
On Tue, Sep 30, 2008 at 5:51 PM, Tom Lane <[EMAIL PROTECTED]> wrote:
> "Gurjeet Singh" <[EMAIL PROTECTED]> writes:
> > I vote for contrib/pg_freespacemap functions to be included in the core
> > since FSM is in core.
>
> The old FSM was in core, too. That's not a helpful argument.
In the absenc
I just managed to make a backend dump core while fooling with the CTE
patch, and found out that the system failed to recover, because the
ensuing startup process *also* dumped core. Here's the backtrace:
Core was generated by `postgres: startup'.
Program terminated with signal 11, Segmentation fa
On 30 Sep 2008, at 10:17 PM, Decibel! <[EMAIL PROTECTED]> wrote:
On Sep 30, 2008, at 1:48 PM, Heikki Linnakangas wrote:
This has been suggested before, and the usual objection is
precisely that it only protects from errors in the storage layer,
giving a false sense of security.
If you ca
>
> I believe the idea was to make this as non-invasive as possible. And
> it would be really nice if this could be enabled without a dump/
> reload (maybe the upgrade stuff would make this possible?)
> --
It's all about the probability of a duplicate check being generated. If
you use a 32 bit ch
From -HEAD:
ERROR: aggregates not allowed in WHERE clause
STATEMENT: SELECT *
FROM loans l
WHERE id IN ( SELECT max(l.id)
FROM loans
JOIN customers c ON c.id =
l.customer_id
On Sep 30, 2008, at 1:48 PM, Heikki Linnakangas wrote:
This has been suggested before, and the usual objection is
precisely that it only protects from errors in the storage layer,
giving a false sense of security.
If you can come up with a mechanism for detecting non-storage errors
as well
On Tue, 30 Sep 2008 13:48:52 -0700
"Jeffrey Baker" <[EMAIL PROTECTED]> wrote:
>
> Practically all of them. Here is a good paper on various checksums,
> their failure rates, and practical applications.
>
> "Parity Lost and Parity Regained"
> http://www.usenix.org/event/fast08/tech/full_papers/k
On Sep 30, 2008, at 2:17 PM, [EMAIL PROTECTED] wrote:
A customer of ours has been having trouble with corrupted data for
some
time. Of course, we've almost always blamed hardware (and we've seen
RAID controllers have their firmware upgraded, among other
actions), but
the useful thing to kno
On Tue, Sep 30, 2008 at 1:41 PM, Bruce Momjian <[EMAIL PROTECTED]> wrote:
> Alvaro Herrera wrote:
> > A customer of ours has been having trouble with corrupted data for some
> > time. Of course, we've almost always blamed hardware (and we've seen
> > RAID controllers have their firmware upgraded,
Alvaro Herrera wrote:
> A customer of ours has been having trouble with corrupted data for some
> time. Of course, we've almost always blamed hardware (and we've seen
> RAID controllers have their firmware upgraded, among other actions), but
> the useful thing to know is when corruption has happen
> A customer of ours has been having trouble with corrupted data for some
> time. Of course, we've almost always blamed hardware (and we've seen
> RAID controllers have their firmware upgraded, among other actions), but
> the useful thing to know is when corruption has happened, and where.
That i
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
Le 30 sept. 08 à 20:03, Tom Lane a écrit :
set_read_position(tupstore, &local_read_position);
tuple = tuplestore_gettuple(tupstore, ...);
get_read_position(tupstore, &local_read_position);
rather than just tuplestore_get
On Tue, 30 Sep 2008, Heikki Linnakangas wrote:
Doesn't some filesystems include a per-block CRC, which would achieve the
same thing? ZFS?
Yes, there is a popular advoacy piece for ZFS with a high-level view of
why and how they implement that at
http://blogs.sun.com/bonwick/entry/zfs_end_to_e
Alvaro Herrera wrote:
Initially I'm aiming at a CRC32 sum
for each block. FlushBuffer would calculate the checksum and store it
in the CRC fork; ReadBuffer_common would read the page, calculate the
checksum, and compare it to the one stored in the CRC fork.
There's one fundamental problem with
Hello Alvaro,
some random thoughts while reading your proposal follow...
Alvaro Herrera wrote:
> So we've been tasked with adding CRCs to data files.
Disks get larger and relative reliability shrinks, it seems. So I agree
that this is a worthwhile thing to have. But shouldn't that be the job
of
On Tue, Sep 30, 2008 at 2:49 PM, Joshua Drake <[EMAIL PROTECTED]> wrote:
> On Tue, 30 Sep 2008 14:33:04 -0400
> "Jonah H. Harris" <[EMAIL PROTECTED]> wrote:
>
>> > I'd like to submit this for 8.4, but I want to ensure that -hackers
>> > at large approve of this feature before starting serious codin
On Tue, 30 Sep 2008 14:33:04 -0400
"Jonah H. Harris" <[EMAIL PROTECTED]> wrote:
> > I'd like to submit this for 8.4, but I want to ensure that -hackers
> > at large approve of this feature before starting serious coding.
>
> IMHO, this is a functionality that should be enabled by default (as it
>
Alvaro Herrera wrote:
A customer of ours has been having trouble with corrupted data for some
time. Of course, we've almost always blamed hardware (and we've seen
RAID controllers have their firmware upgraded, among other actions), but
the useful thing to know is when corruption has happened, an
Alvaro Herrera <[EMAIL PROTECTED]> writes:
> The implementation I'm envisioning requires the use of a new relation
> fork to store the per-block CRCs.
That seems bizarre, and expensive, and if you lose one block of the CRC
fork you lose confidence in a LOT of data. Why not keep the CRCs in the
pa
This patch changes the options field of pg_hba.conf to take name/value
pairs instead of a fixed string. This makes it a lot nicer to deal with
auth methods that need more than one parameter, such as LDAP.
While at it, it also adds map support to kerberos, gssapi and sspi and
not just ident - basic
On Tue, Sep 30, 2008 at 2:02 PM, Alvaro Herrera
<[EMAIL PROTECTED]> wrote:
> A customer of ours has been having trouble with corrupted data for some
> time. Of course, we've almost always blamed hardware (and we've seen
> RAID controllers have their firmware upgraded, among other actions), but
> t
Greg Stark <[EMAIL PROTECTED]> writes:
> On 24 Sep 2008, at 02:45, Tom Lane <[EMAIL PROTECTED]> wrote:
>> The next big
>> thing seems to be to figure out exactly how to do multiple references
>> to CTE outputs, so that we can de-bogotify the planner.
> I've looked and don't seem to still have the
A customer of ours has been having trouble with corrupted data for some
time. Of course, we've almost always blamed hardware (and we've seen
RAID controllers have their firmware upgraded, among other actions), but
the useful thing to know is when corruption has happened, and where.
So we've been
On Tue, Sep 30, 2008 at 2:19 AM, Robins Tharakan <[EMAIL PROTECTED]> wrote:
> Hi,
> While making a complex database back-end, I have at-hand about 200 odd
> functions and frankly 'management of functions' is already getting quite
> tedious. Since the count is certain to rise, I am looking for a goo
Gurjeet Singh wrote:
On Tue, Sep 30, 2008 at 4:49 PM, Heikki Linnakangas <
[EMAIL PROTECTED]> wrote:
Gurjeet Singh wrote:
On Tue, Sep 30, 2008 at 3:09 PM, Heikki Linnakangas <
[EMAIL PROTECTED]> wrote:
That's normal. VACUUM FULL creates new index pointers for the tuples it
moves, which can
On Tue, Sep 30, 2008 at 2:53 PM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
>> For convenience, would it be possible to see about having all this
>> provided by PostgreSQL?
>> a. pg_relation_size()
>> b. pg_relation_toast_size()
>> c. pg_relation_fsm_size()
>> d. pg_relation_indexes_size()
Dimitri Fontaine wrote:
Le mardi 30 septembre 2008, Heikki Linnakangas a écrit :
You forgot the toast size.
Yeah, pg_total_relation_size() - pg_relation_size() is not equal to the
total size of indexes because of that.
Oops. Thanks for pointing this to me...
But you can do SUM(pg_relation_s
I attach patch which cleans up code around PageGetTempPage. These changes were
discussed here:
http://archives.postgresql.org/pgsql-hackers/2008-08/msg00102.php
Zdenek
--
Zdenek Kotala Sun Microsystems
Prague, Czech Republic http://sun.com/postgresql
diff -cr
Le mardi 30 septembre 2008, Heikki Linnakangas a écrit :
> > You forgot the toast size.
>
> Yeah, pg_total_relation_size() - pg_relation_size() is not equal to the
> total size of indexes because of that.
Oops. Thanks for pointing this to me...
> But you can do SUM(pg_relation_size(index)) across
Dimitri Fontaine <[EMAIL PROTECTED]> writes:
> Question for the slow readers: this new FSM scheme being dynamic, it's no
> longer possible to have table bloat, right?
> (where table bloat is full of dead-for-any-transaction tuples, and you have
> to
> CLUSTER or VACUUM FULL to be able to reuse
KaiGai Kohei <[EMAIL PROTECTED]> writes:
> When I run "pg_ctl restart" of the vanilla pgsql-8.3.4 without any options,
> it does not work correctly.
Confirmed here, and fixed. HEAD was all right, the back branches not so
much.
regards, tom lane
--
Sent via pgsql-hackers
Dimitri Fontaine wrote:
Le mardi 30 septembre 2008, Heikki Linnakangas a écrit :
Dimitri Fontaine wrote:
Question for the slow readers: this new FSM scheme being dynamic, it's no
longer possible to have table bloat, right?
(where table bloat is full of dead-for-any-transaction tuples, and you
h
Gurjeet Singh wrote:
On Tue, Sep 30, 2008 at 6:09 PM, Dimitri Fontaine <[EMAIL PROTECTED]>wrote:
What's practical about pg_relation_size() and pg_total_relation_size() as
of
8.3 is that the diff is the cumulated indexes storage volume. Your proposal
makes it harder to get this information, but s
Le mardi 30 septembre 2008, Heikki Linnakangas a écrit :
> Dimitri Fontaine wrote:
> > Question for the slow readers: this new FSM scheme being dynamic, it's no
> > longer possible to have table bloat, right?
> > (where table bloat is full of dead-for-any-transaction tuples, and you
> > have to CLU
Philip Warner wrote:
Andrew Dunstan wrote:
Unfortunately, it quite possibly would. You would not be able to build
two indexes on the same table in parallel, even though they wouldn't
have conflicting locks.
I suppose so, but:
1. By the same logic it might speed things up; it might bu
Heikki Linnakangas napsal(a):
The FSM is not updated during WAL replay. That means that after crash
recovery, the FSM won't be completely up-to-date, but at roughly the
state it was at last checkpoint. In a warm stand-by, the FSM will
reflect the situation at last full backup. We need to thi
Dimitri Fontaine wrote:
Question for the slow readers: this new FSM scheme being dynamic, it's no
longer possible to have table bloat, right?
(where table bloat is full of dead-for-any-transaction tuples, and you have to
CLUSTER or VACUUM FULL to be able to reuse the space it takes)
No, it did
On Tue, Sep 30, 2008 at 6:09 PM, Dimitri Fontaine <[EMAIL PROTECTED]>wrote:
> Hi,
>
> Le mardi 30 septembre 2008, Heikki Linnakangas a écrit :
> > pg_relation_size() doesn't include the size of the FSM. Should it? I'm
> > thinking "no", but pg_total_relation_size() should.
>
> What's practical abo
Hi,
Le mardi 30 septembre 2008, Heikki Linnakangas a écrit :
> pg_relation_size() doesn't include the size of the FSM. Should it? I'm
> thinking "no", but pg_total_relation_size() should.
What's practical about pg_relation_size() and pg_total_relation_size() as of
8.3 is that the diff is the cum
"Gurjeet Singh" <[EMAIL PROTECTED]> writes:
> I vote for contrib/pg_freespacemap functions to be included in the core
> since FSM is in core.
The old FSM was in core, too. That's not a helpful argument.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hac
On Tue, Sep 30, 2008 at 4:32 PM, Heikki Linnakangas <
[EMAIL PROTECTED]> wrote:
>
> pg_relation_size() doesn't include the size of the FSM. Should it? I'm
> thinking "no",
No
> but pg_total_relation_size() should.
+1
> The FSM is not updated during WAL replay. That means that after crash
>
On Tue, Sep 30, 2008 at 4:49 PM, Heikki Linnakangas <
[EMAIL PROTECTED]> wrote:
> Gurjeet Singh wrote:
>
>> On Tue, Sep 30, 2008 at 3:09 PM, Heikki Linnakangas <
>> [EMAIL PROTECTED]> wrote:
>>
>>> That's normal. VACUUM FULL creates new index pointers for the tuples it
>>> moves, which can lead to
Gurjeet Singh wrote:
On Tue, Sep 30, 2008 at 3:09 PM, Heikki Linnakangas <
[EMAIL PROTECTED]> wrote:
That's normal. VACUUM FULL creates new index pointers for the tuples it
moves, which can lead to a bigger index. If it bothers, REINDEX will pack
the indexes tighter again.
That explains it...
I finally committed the new version of the FSM rewrite, after fixing the
latest round of Tom's comments. There's a few loose ends I'm going to
address as subsequent patches:
pg_relation_size() doesn't include the size of the FSM. Should it? I'm
thinking "no", but pg_total_relation_size() shoul
On Tue, Sep 30, 2008 at 3:09 PM, Heikki Linnakangas <
[EMAIL PROTECTED]> wrote:
> Gurjeet Singh wrote:
>
>>I noticed something strange today, and thought I should report it. I
>> vacuumed a database, and as expected, one of the table's size decreased
>> (other table were VACUUMed individually
Gurjeet Singh wrote:
I noticed something strange today, and thought I should report it. I
vacuumed a database, and as expected, one of the table's size decreased
(other table were VACUUMed individually earlier); but o my astonishment, the
size of the UNIQUE KEY index on one of the columns inc
Hi All,
I noticed something strange today, and thought I should report it. I
vacuumed a database, and as expected, one of the table's size decreased
(other table were VACUUMed individually earlier); but o my astonishment, the
size of the UNIQUE KEY index on one of the columns increased. Here's
When I run "pg_ctl restart" of the vanilla pgsql-8.3.4 without any options,
it does not work correctly.
cut here
[EMAIL PROTECTED] postgresql-8.3.4]$ ./root/bin/pg_ctl restart
LOG: received smart shutdown request
LOG: autovacuum launcher shutting down
waiting for server to shut
Tom Lane wrote:
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
Tom Lane wrote:
In fsm_rebuild_page, surely we needn't check "if (lchild < NodesPerPage)".
Yes, we do.
But the loop starting point is such that you must be visiting a parent
with at least one child, no?
Hmm, true, and that me
Abbas napsal(a):
On Mon, 2008-09-29 at 14:42 +0200, Zdenek Kotala wrote:
Do I have to perform performance tests too?
Yes, please. My colleague tested it and got 5% performance drop, but it was not
complete version and I tested full patch on Friday and It was surprise for me
... I got little
59 matches
Mail list logo
