date:20161110

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-10 Thread Kyotaro HORIGUCHI

Thank you for the new patch.

At Fri, 11 Nov 2016 16:42:43 +0900, Michael Paquier  
wrote in 
> On Fri, Nov 11, 2016 at 12:28 AM, Stephen Frost  wrote:
> > We should probably include in here that we may skip a checkpoint if no
> > activity has happened, meaning that this is a safe setting to set for
> > environments which are idle for long periods.
> 
> OK, here is the interesting bit I just updated (I cut the diff a bit
> as the rest is just reformatting):
>  parameter is greater than zero, the server will switch to a new
>  segment file whenever this many seconds have elapsed since the last
>  segment file switch, and there has been any database activity,
> -including a single checkpoint.  (Increasing
> -checkpoint_timeout will reduce unnecessary
> -checkpoints on an idle system.)
> [...]
> +including a single checkpoint.  Checkpoints can however be skipped
> +if there is no database activity, making this parameter a safe
> +setting for environments which are idle for a long period of time.
> 
> > (I'm thinking embedded systems here).
> 
> (Those are most of my users :{) ).

Ok, (FWIW..,) it seems fine for me.

> On Fri, Nov 11, 2016 at 3:23 AM, David Steele  wrote:
> > On 11/10/16 1:03 PM, Stephen Frost wrote:
> >> Agreed. You certainly may wish to log checkpoints, even on an embedded
> >> or low I/o system, but logging that nothing is happening doesn't seem
> >> useful except perhaps for debugging.
> >
> > Sure, DEBUG1 or DEBUG2 makes sense.
> 
> OK. LOG was useful to avoid noise when debugging the thing, but DEBUG1
> is fine for me as well in the final version.

Agreed. DEBUG2 seems too deep for it.

Well, I think we had the final comment and it has been addressd
so I mark this as ready for committer soon.

Thank you all.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-10 Thread Michael Paquier

On Fri, Nov 11, 2016 at 12:28 AM, Stephen Frost  wrote:
> * Michael Paquier (michael.paqu...@gmail.com) wrote:
>> Thanks for the review! Waiting for a couple of days more is fine for
>> me. This won't change much. Attached is v15 with the fixes you
>> mentioned.
>
> I figured I'd go ahead and start looking into this (and it's pretty easy
> for me to discuss it with David, given he works in the same office ;).

Thanks!

> A couple initial comments:
>
>> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
>> index adab2f8..38c2385 100644
>> --- a/doc/src/sgml/config.sgml
>> +++ b/doc/src/sgml/config.sgml
>> @@ -2826,12 +2826,9 @@ include_dir 'conf.d'
>>  parameter is greater than zero, the server will switch to a new
>>  segment file whenever this many seconds have elapsed since the last
>>  segment file switch, and there has been any database activity,
>> -including a single checkpoint.  (Increasing
>> -checkpoint_timeout will reduce unnecessary
>> -checkpoints on an idle system.)
>> -Note that archived files that are closed early
>> -due to a forced switch are still the same length as completely full
>> -files.  Therefore, it is unwise to use a very short
>> +including a single checkpoint.  Note that archived files that are
>> +closed early due to a forced switch are still the same length as
>> +completely full files.  Therefore, it is unwise to use a very short
>>  archive_timeout  it will bloat your archive
>>  storage.  archive_timeout settings of a minute or so are
>>  usually reasonable.  You should consider using streaming 
>> replication,
>
> We should probably include in here that we may skip a checkpoint if no
> activity has happened, meaning that this is a safe setting to set for
> environments which are idle for long periods.

OK, here is the interesting bit I just updated (I cut the diff a bit
as the rest is just reformatting):
 parameter is greater than zero, the server will switch to a new
 segment file whenever this many seconds have elapsed since the last
 segment file switch, and there has been any database activity,
-including a single checkpoint.  (Increasing
-checkpoint_timeout will reduce unnecessary
-checkpoints on an idle system.)
[...]
+including a single checkpoint.  Checkpoints can however be skipped
+if there is no database activity, making this parameter a safe
+setting for environments which are idle for a long period of time.

> (I'm thinking embedded systems here).

(Those are most of my users :{) ).

On Fri, Nov 11, 2016 at 3:23 AM, David Steele  wrote:
> On 11/10/16 1:03 PM, Stephen Frost wrote:
>> Agreed. You certainly may wish to log checkpoints, even on an embedded
>> or low I/o system, but logging that nothing is happening doesn't seem
>> useful except perhaps for debugging.
>
> Sure, DEBUG1 or DEBUG2 makes sense.

OK. LOG was useful to avoid noise when debugging the thing, but DEBUG1
is fine for me as well in the final version.
-- 
Michael
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index adab2f8..d2a8ec2 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2826,17 +2826,16 @@ include_dir 'conf.d'
 parameter is greater than zero, the server will switch to a new
 segment file whenever this many seconds have elapsed since the last
 segment file switch, and there has been any database activity,
-including a single checkpoint.  (Increasing
-checkpoint_timeout will reduce unnecessary
-checkpoints on an idle system.)
-Note that archived files that are closed early
-due to a forced switch are still the same length as completely full
-files.  Therefore, it is unwise to use a very short
-archive_timeout  it will bloat your archive
-storage.  archive_timeout settings of a minute or so are
-usually reasonable.  You should consider using streaming replication,
-instead of archiving, if you want data to be copied off the master
-server more quickly than that.
+including a single checkpoint.  Checkpoints can however be skipped
+if there is no database activity, making this parameter a safe
+setting for environments which are idle for a long period of time.
+Note that archived files that are closed early due to a forced switch
+are still the same length as completely full files.  Therefore, it is
+unwise to use a very short archive_timeout  it will
+bloat your archive storage.  archive_timeout settings of
+a minute or so are usually reasonable.  You should consider using
+streaming replication, instead of archiving, if you want data to
+be copied off the master server more quickly than that.
 This

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2016-11-10 Thread Kyotaro HORIGUCHI

Hello,

> Aside from that, I'd like to comment this patch on other points
> later.

The start of this patch is that the fact that most of but not all
geometric operators use fuzzy comparson. But Tom pointed out that
the fixed fuzz factor is not reasonable but hard to find how to
modify it.

We can remove the fuzz factor altogether but I think we also
should provide a means usable to do similar things. At least "is
a point on a line" might be useless for most cases without any
fuzzing feature. (Nevertheless, it is a problem only when it is
being used to do that:) If we don't find reasonable policy on
fuzzing operations, it would be the proof that we shouldn't
change the behavior.

The 0001 patch adds many FP comparison functions individually
considering NaN. As the result the sort order logic involving NaN
is scattered around into the functions, then, you implement
generic comparison function using them. It seems inside-out to
me. Defining ordering at one place, then comparison using it
seems to be reasonable.

The 0002 patch repalces many native operators for floating point
numbers by functions having sanity check. This seems to be
addressing to Tom's comment. It is reasonable for comparison
operators but I don't think replacing all arithmetics is so. For
example, float8_div checks that

- If both of operands are not INF, result should not be INF.
- If the devident is not exactly zero, the result should not be zero.

The second assumption is wrong by itself. For an extreme case,
4.9e-324 / 1.7e+308 becomes exactly zero (by underflow). We
canont assert it to be wrong but the devedent is not zero. The
validity of the result varies according to its meaning. For the
case of box_cn,

> center->x = float8_div(float8_pl(box->high.x, box->low.x), 2.0);
> center->y = float8_div(float8_pl(box->high.y, box->low.y), 2.0);

If the center somehow goes extremely near to the origin, it could
result in a false error.

> =# select @@ box'(-8e-324, -8e-324), (4.9e-324, 4.9e-324)';
> ERROR: value out of range: underflow

I don't think this underflow is an error, and actually it is a
change of the current behavior without a reasonable reason. More
significant (and maybe unacceptable) side-effect is that it
changes the behavior of ordinary operators. I don't think this is
acceptable. More consideration is needed.

> =# select ('-8e-324'::float8 + '4.9e-324'::float8) / 2.0;
> ERROR: value out of range: underflow

In regard to fuzzy operations, libgeos seems to have several
types of this kind of feature. (I haven't looked closer into
them). Other than reducing precision seems overkill or
unappliable for PostgreSQL bulitins. As Jim said, can we replace
the fixed scale fuzz factor by precision reduction? Maybe, with a
GUC variable (I hear someone's roaring..) to specify the amount
defaults to fit the current assumption.

https://apt-browse.org/browse/ubuntu/trusty/universe/i386/libgeos++-dev/3.4.2-4ubuntu1/file/usr/include/geos/geom/BinaryOp.h

/*
* Define this to use PrecisionReduction policy
* in an attempt at by-passing binary operation
* robustness problems (handles TopologyExceptions)
*/
...
* Define this to use TopologyPreserving simplification policy
* in an attempt at by-passing binary operation
* robustness problems (handles TopologyExceptions)
(seems not activated)
...
* Use common bits removal policy.
* If enabled, this would be tried /before/
* Geometry snapping.
...
/*
* Use snapping policy
*/

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

75 matches

Mail list logo