Re: [HACKERS] pg_rawdump

2010-10-21 Thread Stephen R. van den Berg
Bruce Momjian wrote: >Greg Stark wrote: >> On Tue, Oct 19, 2010 at 1:12 PM, Stephen R. van den Berg >> wrote: >> > In order to simplify recovery at this point (enormously), it would >> > have been very helpful (at almost negligible cost), to have the name >> > of the table, the name of the column

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Stephen R. van den Berg
Greg Stark wrote: >On Thu, Oct 21, 2010 at 11:30 AM, Stephen R. van den Berg wrote: >> For the recovery information I'd like to reserve: >> identifier: 00: table OID >> ? ? ? ? ? ?01: table layout >So here's a proposal for something that could maybe be implemented. I >think I'm leaning against th

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Bruce Momjian
Greg Stark wrote: > On Tue, Oct 19, 2010 at 1:12 PM, Stephen R. van den Berg wrote: > > In order to simplify recovery at this point (enormously), it would > > have been very helpful (at almost negligible cost), to have the name > > of the table, the name of the columns, and the types of the > > co

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Stephen R. van den Berg
Kevin Grittner wrote: >Tom Lane wrote: >> Most of the data-loss reports I've seen appeared to come from >> people who wouldn't be capable of doing such recovery work even if >> better tools were available. >No doubt; but the recovery work often winds up in the hands of >people with more skills th

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Robert Haas
On Thu, Oct 21, 2010 at 5:21 PM, Kevin Grittner wrote: > Tom Lane wrote: > >> Most of the data-loss reports I've seen appeared to come from >> people who wouldn't be capable of doing such recovery work even if >> better tools were available. > > No doubt; but the recovery work often winds up in t

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Kevin Grittner
Tom Lane wrote: > Most of the data-loss reports I've seen appeared to come from > people who wouldn't be capable of doing such recovery work even if > better tools were available. No doubt; but the recovery work often winds up in the hands of people with more skills than those responsible for

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Tom Lane
"Stephen R. van den Berg" writes: > Tom Lane wrote: >> There are way too many scenarios where you'll have no hope of doing >> any such manual recovery anyway. > True. It's all a matter of statistics. Judging by the number of reports > I find by googling net-history, I'd have to conclude that th

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Greg Stark
On Thu, Oct 21, 2010 at 11:30 AM, Stephen R. van den Berg wrote: > For the recovery information I'd like to reserve: > identifier: 00: table OID >            01: table layout > So here's a proposal for something that could maybe be implemented. I think I'm leaning against this currently as there

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Stephen R. van den Berg
Tom Lane wrote: >"Stephen R. van den Berg" writes: >> If it's inserted in the "special" area, it will not break any >> compatibility. >I'll tell you what I really don't like about this proposal: we discuss >some scheme or other for taking over the "special space" in heap pages >at least once a ye

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Tom Lane
"Stephen R. van den Berg" writes: > Robert Haas wrote: >> and break on-disk compatibility just to make it easier to > If it's inserted in the "special" area, it will not break any > compatibility. I'll tell you what I really don't like about this proposal: we discuss some scheme or other for tak

Re: [HACKERS] pg_rawdump

2010-10-21 Thread Stephen R. van den Berg
Robert Haas wrote: >On Wed, Oct 20, 2010 at 5:30 PM, Stephen R. van den Berg wrote: >> Ideal would be: put the table-oid inside the header of each page >> (either in the official header, or in the special area). >> This way even lost blocks can be correlated to the same table. >> I'd still vote fo

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Robert Haas
On Wed, Oct 20, 2010 at 5:30 PM, Stephen R. van den Berg wrote: > Ideal would be: put the table-oid inside the header of each page > (either in the official header, or in the special area). > This way even lost blocks can be correlated to the same table. > I'd still vote for the latest known table

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Stephen R. van den Berg
Tom Lane wrote: >Aidan Van Dyk writes: >> If we're looking to have any sort of "out of catalog" documentation of >> table storage format, what about just having a new relation fork that >> just "appends" each and every change made to the table formats, >> including ones rolled back, etc. I presum

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Aidan Van Dyk
On Wed, Oct 20, 2010 at 1:09 PM, Tom Lane wrote: > [ thinks for a bit... ]  Perhaps we could stick some sort of unique > ID into tables, which could be correlated to the same unique ID > appearing in a metadata fork. It would be awfully nice if the table name was in the first few bytes of the se

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Tom Lane
Aidan Van Dyk writes: > On Wed, Oct 20, 2010 at 10:28 AM, Tom Lane wrote: >> I can see the potential usefulness of a self-documenting table storage >> format, but this proposal isn't that; it's just an unreliable kluge. > If we're looking to have any sort of "out of catalog" documentation of > t

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Aidan Van Dyk
On Wed, Oct 20, 2010 at 10:28 AM, Tom Lane wrote: > "Stephen R. van den Berg" writes: >> It's just that matching table and file, and subsequently figuring out >> some missing columns which may have been added/removed later, >> can be rather timeconsuming and could be made a lot easier (not necess

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Stephen R. van den Berg
Tom Lane wrote: >"Stephen R. van den Berg" writes: >> It's just that matching table and file, and subsequently figuring out >> some missing columns which may have been added/removed later, >> can be rather timeconsuming and could be made a lot easier (not necessarily >> perfect) if that informatio

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Tom Lane
"Stephen R. van den Berg" writes: > It's just that matching table and file, and subsequently figuring out > some missing columns which may have been added/removed later, > can be rather timeconsuming and could be made a lot easier (not necessarily > perfect) if that information would have been pre

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Stephen R. van den Berg
Roberto Mello wrote: >On Tue, Oct 19, 2010 at 6:13 PM, Stephen R. van den Berg wrote: >> Greg Stark wrote: >>>premise this on the idea that you've lost everything in the catalog >>>but not the data in other tables. Which seems like a narrow use case. >> It happens, more often than you'd think. ??

Re: [HACKERS] pg_rawdump

2010-10-20 Thread Stephen R. van den Berg
Tom Lane wrote: >"Stephen R. van den Berg" writes: >> In order to simplify recovery at this point (enormously), it would >> have been very helpful (at almost negligible cost), to have the name >> of the table, the name of the columns, and the types of the >> columns available. >> Why don't we ins

Re: [HACKERS] pg_rawdump

2010-10-19 Thread Roberto Mello
On Tue, Oct 19, 2010 at 6:13 PM, Stephen R. van den Berg wrote: > Greg Stark wrote: > >> But I'm not sure how useful. I mean, you can't really decipher >>everything properly without the data in the catalog -- and you have to >>premise this on the idea that you've lost everything in the catalog >>b

Re: [HACKERS] pg_rawdump

2010-10-19 Thread Tom Lane
"Stephen R. van den Berg" writes: > In order to simplify recovery at this point (enormously), it would > have been very helpful (at almost negligible cost), to have the name > of the table, the name of the columns, and the types of the > columns available. > Why don't we insert that data into the

Re: [HACKERS] pg_rawdump

2010-10-19 Thread Stephen R. van den Berg
Greg Stark wrote: >On Tue, Oct 19, 2010 at 1:12 PM, Stephen R. van den Berg wrote: >> In order to simplify recovery at this point (enormously), it would >> have been very helpful (at almost negligible cost), to have the name >> of the table, the name of the columns, and the types of the >> columns

Re: [HACKERS] pg_rawdump

2010-10-19 Thread Greg Stark
On Tue, Oct 19, 2010 at 1:12 PM, Stephen R. van den Berg wrote: > In order to simplify recovery at this point (enormously), it would > have been very helpful (at almost negligible cost), to have the name > of the table, the name of the columns, and the types of the > columns available. > > Why don