Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-06-26 Thread Fujii Masao
On Wed, Mar 28, 2012 at 10:08 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Wed, Mar 28, 2012 at 12:30 AM, Alvaro Herrera
 alvhe...@commandprompt.com wrote:

 Excerpts from Fujii Masao's message of mar mar 27 06:40:34 -0300 2012:

 Anyway, should I add this patch into the next CF? Or is anyone planning
 to commit the patch for 9.2?

 I think the correct thing to do here is add to next CF, and if some
 committer has enough interest in getting it quickly it can be grabbed
 from there and committed into 9.2.

 Yep! I've added it to next CF.

Attached is the revised version of the patch.

Regards,

-- 
Fujii Masao


pg_controldata_walfilename_v4.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-27 Thread Fujii Masao
On Mon, Mar 26, 2012 at 11:18 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 On Mon, Mar 26, 2012 at 2:50 AM, Magnus Hagander mag...@hagander.net wrote:
 s/segment/file/g?

 We're already using file to mean something different *internally*,
 don't we? And since pg_controldata shows fairly internal information,
 I'm not sure this is the best idea.

 Maybe compromise and call it segment file - that is both easier to
 understand than segment, and not actually using a term that means
 something else...

 It's also kind of wordy.  I think file is fine.

 +1 for file.  I think the internal usage of file to mean roughly
 4GB worth of WAL is going to go away soon anyway, as there won't be
 much reason to worry about the concept once LSN arithmetic is 64-bit.

Agreed. This would mean that the following lots of log messages need to
be changed after 64-bit LSN will have been committed.

errmsg(could not fdatasync log file %u, segment %u: %m,
   log, seg)));

Anyway, should I add this patch into the next CF? Or is anyone planning
to commit the patch for 9.2?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-27 Thread Alvaro Herrera

Excerpts from Fujii Masao's message of mar mar 27 06:40:34 -0300 2012:

 Anyway, should I add this patch into the next CF? Or is anyone planning
 to commit the patch for 9.2?

I think the correct thing to do here is add to next CF, and if some
committer has enough interest in getting it quickly it can be grabbed
from there and committed into 9.2.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-27 Thread Fujii Masao
On Wed, Mar 28, 2012 at 12:30 AM, Alvaro Herrera
alvhe...@commandprompt.com wrote:

 Excerpts from Fujii Masao's message of mar mar 27 06:40:34 -0300 2012:

 Anyway, should I add this patch into the next CF? Or is anyone planning
 to commit the patch for 9.2?

 I think the correct thing to do here is add to next CF, and if some
 committer has enough interest in getting it quickly it can be grabbed
 from there and committed into 9.2.

Yep! I've added it to next CF.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-26 Thread Magnus Hagander
On Fri, Mar 23, 2012 at 18:05, Fujii Masao masao.fu...@gmail.com wrote:
 On Sat, Mar 24, 2012 at 1:49 AM, Robert Haas robertmh...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 12:42 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 9:41 PM, Robert Haas robertmh...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 6:13 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 5:56 PM, Magnus Hagander mag...@hagander.net 
 wrote:
 Might it be a good idea to put it on it's own row instead of changing
 the format of an existing row, in order not to break scripts and
 programs that are parsing the previous output?

 Good idea! What row name should we use for the WAL file containing
 REDO record? Latest checkpoint's REDO file?

 Sounds good to me.  I like the idea, too.  The status quo is an
 unnecessary nuisance, so this will be a nice usability improvement.

 Attached patch adds new row Latest checkpoint's REDO WAL segment: into
 the result of pg_controldata. I used the term WAL segment for the row name
 instead of file because WAL segment is used in another row Bytes per 
 WAL
 segment:. But better name?

 s/segment/file/g?

 Yep, file might be more intuitive for a user than segment. Attached is the
 file version of the patch.

We're already using file to mean something different *internally*,
don't we? And since pg_controldata shows fairly internal information,
I'm not sure this is the best idea.

Maybe compromise and call it segment file - that is both easier to
understand than segment, and not actually using a term that means
something else...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-26 Thread Robert Haas
On Mon, Mar 26, 2012 at 2:50 AM, Magnus Hagander mag...@hagander.net wrote:
 s/segment/file/g?

 Yep, file might be more intuitive for a user than segment. Attached is 
 the
 file version of the patch.

 We're already using file to mean something different *internally*,
 don't we? And since pg_controldata shows fairly internal information,
 I'm not sure this is the best idea.

 Maybe compromise and call it segment file - that is both easier to
 understand than segment, and not actually using a term that means
 something else...

It's also kind of wordy.  I think file is fine.  There are a few
references to xlogid indicating a file number, but the actual field
name is just xlogid.  We also use the term file to mean the other
thing, as in XLOGfileslop, and I have a hard time believing anyone's
really going to get confused about what is meant here.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Mon, Mar 26, 2012 at 2:50 AM, Magnus Hagander mag...@hagander.net wrote:
 s/segment/file/g?

 We're already using file to mean something different *internally*,
 don't we? And since pg_controldata shows fairly internal information,
 I'm not sure this is the best idea.
 
 Maybe compromise and call it segment file - that is both easier to
 understand than segment, and not actually using a term that means
 something else...

 It's also kind of wordy.  I think file is fine.

+1 for file.  I think the internal usage of file to mean roughly
4GB worth of WAL is going to go away soon anyway, as there won't be
much reason to worry about the concept once LSN arithmetic is 64-bit.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Magnus Hagander
On Fri, Mar 23, 2012 at 05:06, Fujii Masao masao.fu...@gmail.com wrote:
 Hi,

 I'd like to propose to change pg_controldata so that it reports the name
 of WAL file containing the latest checkpoint's REDO record, as follows:

    $ pg_controldata $PGDATA
    ...
    Latest checkpoint's REDO location:    0/16D6ACC (file
 00010001)
    Latest checkpoint's TimeLineID:       1
    ...

 This simplifies very much the way to calculate the archive file cutoff point
 because the reported WAL file is just cutoff point itself. If the file name is
 not reported, we have to calculate the cutoff point from the reported
 location and timeline, which is complicated calculation. We can use
 pg_xlogfile_name function to calculate that, but it cannot be executed in
 the standby. Another problem is that pg_xlogfile_name always uses
 current timeline for the calculation, so if the reported timeline is not
 the same as current one, pg_xlogfile_name cannot return the correct WAL
 file name. Making pg_controldata report that WAL file name gets rid of
 such a complexity.

 You may think that archive_cleanup_command is usable for that purpose.
 That's true. But it's not usable simply for  the case where there are more
 than one standby servers. In this case, the archive file cutoff point needs
 to be calculated from each standby's REDO location and timeline.

 Attached patch changes pg_controldata as above. Thought?

Might it be a good idea to put it on it's own row instead of changing
the format of an existing row, in order not to break scripts and
programs that are parsing the previous output?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Fujii Masao
On Fri, Mar 23, 2012 at 2:06 PM, Jaime Casanova ja...@2ndquadrant.com wrote:
 On Thu, Mar 22, 2012 at 11:06 PM, Fujii Masao masao.fu...@gmail.com wrote:

 We can use
 pg_xlogfile_name function to calculate that, but it cannot be executed in
 the standby. Another problem is that pg_xlogfile_name always uses
 current timeline for the calculation, so if the reported timeline is not
 the same as current one, pg_xlogfile_name cannot return the correct WAL
 file name. Making pg_controldata report that WAL file name gets rid of
 such a complexity.


 i would think that pg_xlogfile_name() is not allowed in the standby
 because ThisTimelineId is not very well defined in recovery but if you
 extend pg_xlogfile_name() to also receive a timelineid as you
 suggested in [1] then that version of the function could be allowed in
 the standby.
 or there is something else i'm missing?

 is that enough for you to solve your problem?

Yes, we can more easily calculate the cutoff point by using that extended
pg_xlogfile_name(). But if pg_controldata reports the WAL file name, we
can calculate the cutoff point without starting the server. So I think that
it's worth changing pg_controldata that way even if we extend
pg_xlogfile_name().

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Magnus Hagander
On Fri, Mar 23, 2012 at 10:51, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 2:06 PM, Jaime Casanova ja...@2ndquadrant.com wrote:
 On Thu, Mar 22, 2012 at 11:06 PM, Fujii Masao masao.fu...@gmail.com wrote:

 We can use
 pg_xlogfile_name function to calculate that, but it cannot be executed in
 the standby. Another problem is that pg_xlogfile_name always uses
 current timeline for the calculation, so if the reported timeline is not
 the same as current one, pg_xlogfile_name cannot return the correct WAL
 file name. Making pg_controldata report that WAL file name gets rid of
 such a complexity.


 i would think that pg_xlogfile_name() is not allowed in the standby
 because ThisTimelineId is not very well defined in recovery but if you
 extend pg_xlogfile_name() to also receive a timelineid as you
 suggested in [1] then that version of the function could be allowed in
 the standby.
 or there is something else i'm missing?

 is that enough for you to solve your problem?

 Yes, we can more easily calculate the cutoff point by using that extended
 pg_xlogfile_name(). But if pg_controldata reports the WAL file name, we
 can calculate the cutoff point without starting the server. So I think that
 it's worth changing pg_controldata that way even if we extend
 pg_xlogfile_name().

+1 - I think they're both useful things, each on it's own.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Fujii Masao
On Fri, Mar 23, 2012 at 5:56 PM, Magnus Hagander mag...@hagander.net wrote:
 Might it be a good idea to put it on it's own row instead of changing
 the format of an existing row, in order not to break scripts and
 programs that are parsing the previous output?

Good idea! What row name should we use for the WAL file containing
REDO record? Latest checkpoint's REDO file?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Robert Haas
On Fri, Mar 23, 2012 at 6:13 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 5:56 PM, Magnus Hagander mag...@hagander.net wrote:
 Might it be a good idea to put it on it's own row instead of changing
 the format of an existing row, in order not to break scripts and
 programs that are parsing the previous output?

 Good idea! What row name should we use for the WAL file containing
 REDO record? Latest checkpoint's REDO file?

Sounds good to me.  I like the idea, too.  The status quo is an
unnecessary nuisance, so this will be a nice usability improvement.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Fujii Masao
On Fri, Mar 23, 2012 at 9:41 PM, Robert Haas robertmh...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 6:13 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 5:56 PM, Magnus Hagander mag...@hagander.net wrote:
 Might it be a good idea to put it on it's own row instead of changing
 the format of an existing row, in order not to break scripts and
 programs that are parsing the previous output?

 Good idea! What row name should we use for the WAL file containing
 REDO record? Latest checkpoint's REDO file?

 Sounds good to me.  I like the idea, too.  The status quo is an
 unnecessary nuisance, so this will be a nice usability improvement.

Attached patch adds new row Latest checkpoint's REDO WAL segment: into
the result of pg_controldata. I used the term WAL segment for the row name
instead of file because WAL segment is used in another row Bytes per WAL
segment:. But better name?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


pg_controldata_walfilename_v2.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Robert Haas
On Fri, Mar 23, 2012 at 12:42 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 9:41 PM, Robert Haas robertmh...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 6:13 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 5:56 PM, Magnus Hagander mag...@hagander.net 
 wrote:
 Might it be a good idea to put it on it's own row instead of changing
 the format of an existing row, in order not to break scripts and
 programs that are parsing the previous output?

 Good idea! What row name should we use for the WAL file containing
 REDO record? Latest checkpoint's REDO file?

 Sounds good to me.  I like the idea, too.  The status quo is an
 unnecessary nuisance, so this will be a nice usability improvement.

 Attached patch adds new row Latest checkpoint's REDO WAL segment: into
 the result of pg_controldata. I used the term WAL segment for the row name
 instead of file because WAL segment is used in another row Bytes per WAL
 segment:. But better name?

s/segment/file/g?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-23 Thread Fujii Masao
On Sat, Mar 24, 2012 at 1:49 AM, Robert Haas robertmh...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 12:42 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 9:41 PM, Robert Haas robertmh...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 6:13 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Fri, Mar 23, 2012 at 5:56 PM, Magnus Hagander mag...@hagander.net 
 wrote:
 Might it be a good idea to put it on it's own row instead of changing
 the format of an existing row, in order not to break scripts and
 programs that are parsing the previous output?

 Good idea! What row name should we use for the WAL file containing
 REDO record? Latest checkpoint's REDO file?

 Sounds good to me.  I like the idea, too.  The status quo is an
 unnecessary nuisance, so this will be a nice usability improvement.

 Attached patch adds new row Latest checkpoint's REDO WAL segment: into
 the result of pg_controldata. I used the term WAL segment for the row name
 instead of file because WAL segment is used in another row Bytes per WAL
 segment:. But better name?

 s/segment/file/g?

Yep, file might be more intuitive for a user than segment. Attached is the
file version of the patch.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


pg_controldata_walfilename_v3.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reporting WAL file containing checkpoint's REDO record in pg_controldata's result

2012-03-22 Thread Jaime Casanova
On Thu, Mar 22, 2012 at 11:06 PM, Fujii Masao masao.fu...@gmail.com wrote:

 We can use
 pg_xlogfile_name function to calculate that, but it cannot be executed in
 the standby. Another problem is that pg_xlogfile_name always uses
 current timeline for the calculation, so if the reported timeline is not
 the same as current one, pg_xlogfile_name cannot return the correct WAL
 file name. Making pg_controldata report that WAL file name gets rid of
 such a complexity.


i would think that pg_xlogfile_name() is not allowed in the standby
because ThisTimelineId is not very well defined in recovery but if you
extend pg_xlogfile_name() to also receive a timelineid as you
suggested in [1] then that version of the function could be allowed in
the standby.
or there is something else i'm missing?

is that enough for you to solve your problem?

[1] 
http://archives.postgresql.org/message-id/cahgqgwhwqjgeksmp2oteru8p0sj6x7ypqyh5qqeqcbqsxaa...@mail.gmail.com

-- 
Jaime Casanova         www.2ndQuadrant.com
Professional PostgreSQL: Soporte 24x7 y capacitación

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers