Re: [HACKERS] record identical operator - Review

2013-10-06 Thread Bruce Momjian
On Thu, Oct 3, 2013 at 10:46:05AM -0700, Kevin Grittner wrote:    opcname - varchar_ops kd_point_ops cidr_ops text_pattern_ops varchar_pattern_ops bpchar_pattern_ops (6 rows) Do these all have operators defined too?

Re: [HACKERS] record identical operator - Review

2013-10-03 Thread Steve Singer
On 09/30/2013 09:08 AM, Kevin Grittner wrote: Steve Singer st...@ssinger.info wrote: How about To support matching of rows which include elements without a default B-tree operator class, the following operators are defined for composite type comparison: literal*=/,

Re: [HACKERS] record identical operator - Review

2013-10-03 Thread Bruce Momjian
On Thu, Oct 3, 2013 at 09:59:03AM -0400, Steve Singer wrote: Are there any outstanding issues on this patch preventing it from being committed? I have not received answers to my email of October 1: http://www.postgresql.org/message-id/20131001024620.gb13...@momjian.us -- Bruce

Re: [HACKERS] record identical operator

2013-10-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: I'm wary of inventing a completely new way of doing this. I don't think that there's any guarantee that the send/recv functions won't expose exactly the same implementation details as a direct check for binary equality. I don't follow this

Re: [HACKERS] record identical operator - Review

2013-10-03 Thread Stephen Frost
Steve, Thanks for following-up on this; I had meant to reply much sooner but other things got in the way. Thanks again, Stephen * Steve Singer (st...@ssinger.info) wrote: Are there any outstanding issues on this patch preventing it from being committed? I think we

Re: [HACKERS] record identical operator - Review

2013-10-03 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: On Fri, Sep 27, 2013 at 03:34:20PM -0700, Kevin Grittner wrote: We first need to document the existing record comparison operators. If they read the docs for comparing row_constructors and expect that to be the behavior they get when they compare records,

Re: [HACKERS] record identical operator - Review

2013-10-03 Thread Kevin Grittner
Steve Singer st...@ssinger.info wrote: You haven't adjusted the patch to reduce the duplication between the equality and comparison functions, if you disagree with me and feel that doing so would increase the code complexity and be inconsistent with how we do things elsewhere that is fine. I

Re: [HACKERS] record identical operator

2013-10-03 Thread Robert Haas
On Thu, Oct 3, 2013 at 11:12 AM, Stephen Frost sfr...@snowman.net wrote: Binary equality has existing precedent and is used in numerous places in the code for good reason. Users might be confused about the use of those semantics in those places also, but AFAICT nobody is. You've stated that

Re: [HACKERS] record identical operator

2013-10-03 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: If we were consistently copying and updating the value based on some external knowledge that the value has changed (similar to how slony works w/ triggers that dump change sets into a table- it doesn't consider has any value on this row changed?; the

Re: [HACKERS] record identical operator

2013-10-03 Thread Peter Geoghegan
On Thu, Oct 3, 2013 at 10:53 AM, Robert Haas robertmh...@gmail.com wrote: The point I want to make here is that we have an existing precedent to use bitwise equality when we want to make sure that values are equivalent for all purposes, regardless of what opclass or whatever is in use. There

Re: [HACKERS] record identical operator

2013-10-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: You could argue that HOT isn't user-visible, but we certainly advise people to think about structuring their indexing in a fashion that does not defeat HOT, so I think to some extent it is user-visible. I do think saying HOT is user-visible is

Re: [HACKERS] record identical operator

2013-10-03 Thread Hannu Krosing
On 10/04/2013 12:22 AM, Stephen Frost wrote: That said, I agree that PG, in general, is more 'open' to exposing implementation details than is perhaps ideal, Every *real* system is more open to exposing implementation details than is *ideal*. One very popular implementation detail which

Re: [HACKERS] record identical operator

2013-10-03 Thread Stephen Frost
* Hannu Krosing (ha...@krosing.net) wrote: The main exposed implementation detail of this operator is that it is very fast and can be recommended to be used at user level for speeding up equal query like this SELECT * FROM t WHERE guaranteed equal or equal where the plain equal will only

Re: [HACKERS] record identical operator - Review

2013-09-30 Thread Kevin Grittner
Steve Singer st...@ssinger.info wrote: How about    To support matching of rows which include elements without a default B-tree operator class, the following operators are defined for composite type comparison: literal*=/, literal*lt;gt;/, literal*lt;/,

Re: [HACKERS] record identical operator - Review

2013-09-30 Thread Bruce Momjian
On Fri, Sep 27, 2013 at 03:34:20PM -0700, Kevin Grittner wrote: The arguments for this patch are * We want the materialized view to return the same value as would be returned if the query were executed directly.  This not only means that the values should be the same according to a

Re: [HACKERS] record identical operator - Review

2013-09-29 Thread Steve Singer
On 09/28/2013 03:03 PM, Kevin Grittner wrote: para +To support matching of rows which include elements without a default +B-tree operator class, the following operators are defined for composite +type comparison: +literal*=/, +literal*lt;gt;/, +literal*lt;/, +

Re: [HACKERS] record identical operator - Review

2013-09-28 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Bruce Momjian br...@momjian.us wrote: I think we need to see a patch from Kevin that shows all the row comparisons documented and we can see the impact of the user-visibile part. First draft attached. I'm inclined to first submit a proposed

Re: [HACKERS] record identical operator - Review

2013-09-27 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: On Thu, Sep 19, 2013 at 06:31:38PM -0400, Steve Singer wrote: On 09/12/2013 06:27 PM, Kevin Grittner wrote: Attached is a patch for a bit of infrastructure I believe to be necessary for correct behavior of REFRESH MATERIALIZED VIEW CONCURRENTLY as well as

Re: [HACKERS] record identical operator

2013-09-26 Thread Robert Haas
On Tue, Sep 24, 2013 at 3:22 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: Now I admit that's an arguable point. We could certainly define an intermediate notion of equality that is more equal than whatever = does, but not as equal as exact binary

Re: [HACKERS] record identical operator - Review

2013-09-26 Thread Bruce Momjian
On Thu, Sep 19, 2013 at 06:31:38PM -0400, Steve Singer wrote: On 09/12/2013 06:27 PM, Kevin Grittner wrote: Attached is a patch for a bit of infrastructure I believe to be necessary for correct behavior of REFRESH MATERIALIZED VIEW CONCURRENTLY as well as incremental maintenance of matviews.

Re: [HACKERS] record identical operator - Review

2013-09-26 Thread Bruce Momjian
On Thu, Sep 26, 2013 at 05:50:08PM -0400, Bruce Momjian wrote: I think we need to see a patch from Kevin that shows all the row comparisons documented and we can see the impact of the user-visibile part. One interesting approach would be to only allow the operator to be called from SPI

Re: [HACKERS] record identical operator

2013-09-25 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Here is v2 of the patch which changes from the universally disliked operator names v1 used.  It also fixes bugs in the row comparisons for pass-by-reference types, fixes a couple nearby comments, and adds regression tests for a matview containing a box

Re: [HACKERS] record identical operator

2013-09-24 Thread Andres Freund
On 2013-09-23 21:21:53 -0400, Stephen Frost wrote: Here's an example to illustrate what I'm talking about when it comes down to you can't claim that you'll produce exactly what the query will always, with these types: =# table citext_table; id | name +--- 1 | one 3 | three

Re: [HACKERS] record identical operator

2013-09-24 Thread Hannu Krosing
On 09/23/2013 10:38 PM, Stephen Frost wrote: and heavily caveated. I'm not sure what caveats would be needed. It seems to me that a clear description of what it does would suffice. Like all the other non-default opclasses in core, it will be non-default because it is less frequently

Re: [HACKERS] record identical operator

2013-09-24 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Your example demonstrates that if a given query can generate two different outputs, A and B, based on the same input data, then the contents of the materialized view cannot be equal to be A and also equal to B. Well, no duh. Two different outputs

Re: [HACKERS] record identical operator

2013-09-24 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: Skipping out on much of the side-discussion to try and drive at the heart of this.. Robert Haas (robertmh...@gmail.com) wrote: I would suggest that you read the referenced papers for details as to how the algorithm works.  To make a long story short,

Re: [HACKERS] record identical operator

2013-09-24 Thread Robert Haas
On Tue, Sep 24, 2013 at 7:14 AM, Stephen Frost sfr...@snowman.net wrote: Of course, you don't need citext, or any other data type with a loose notion of equality, to generate that sort of problem: [...] rhaas=# set datestyle = 'german'; SET I'm talking about *planner differences* changing

Re: [HACKERS] record identical operator

2013-09-24 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: That's the point, and the whole point.  You have not shown that it doesn't.  You have not shown why adding a 12th non-default opclass is a particular problem here (although we have a consensus to use different operators, to reserve this operator

Re: [HACKERS] record identical operator

2013-09-24 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: We need justification to add operators, imv, especially ones that expose our internal binary representation of data. I believe I have done so. I worry that adding these will come back to bite us later How? and that we're making promises we won't be

Re: [HACKERS] record identical operator

2013-09-24 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: Stephen Frost sfr...@snowman.net wrote: I worry that adding these will come back to bite us later How? User misuse is certainly one consideration, but I wonder what's going to happen if we change our internal representation of data (eg: numerics

Re: [HACKERS] record identical operator

2013-09-24 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: Kevin Grittner (kgri...@ymail.com) wrote: Stephen Frost sfr...@snowman.net wrote: I worry that adding these will come back to bite us later How? User misuse is certainly one consideration, I think that one has been talked to death already, with the

Re: [HACKERS] record identical operator

2013-09-24 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: Stephen Frost sfr...@snowman.net wrote: The promise that we'll always return the binary representation of the data that we saw last.  When greatest(x,y) comes back 'false' for a MAX(), we then have to go check well, does the type consider them

Re: [HACKERS] record identical operator

2013-09-24 Thread Robert Haas
On Tue, Sep 24, 2013 at 12:00 PM, Stephen Frost sfr...@snowman.net wrote: It wouldn't address my concerns anyway, which are around these binary operators and the update-in-place approach being risky and setting us up for problems down the road. I think that if you want to hold up a bug fix to

Re: [HACKERS] record identical operator

2013-09-24 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Tue, Sep 24, 2013 at 12:00 PM, Stephen Frost sfr...@snowman.net wrote: It wouldn't address my concerns anyway, which are around these binary operators and the update-in-place approach being risky and setting us up for problems down the road.

Re: [HACKERS] record identical operator

2013-09-24 Thread Robert Haas
On Tue, Sep 24, 2013 at 1:04 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Tue, Sep 24, 2013 at 12:00 PM, Stephen Frost sfr...@snowman.net wrote: It wouldn't address my concerns anyway, which are around these binary operators and the

Re: [HACKERS] record identical operator

2013-09-24 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Now I admit that's an arguable point. We could certainly define an intermediate notion of equality that is more equal than whatever = does, but not as equal as exact binary equality. I suggested it up-thread and don't recall seeing a response, so

Re: [HACKERS] record identical operator

2013-09-24 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: I think the conservative (and therefore correct) approach is to decide that we're always going to update if we detect any difference at all. And there may be users who are surprised that a refresh changed parts of the table that have nothing to do

Re: [HACKERS] record identical operator

2013-09-24 Thread Merlin Moncure
On Tue, Sep 24, 2013 at 2:22 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: Now I admit that's an arguable point. We could certainly define an intermediate notion of equality that is more equal than whatever = does, but not as equal as exact binary

Re: [HACKERS] record identical operator

2013-09-23 Thread Robert Haas
On Fri, Sep 20, 2013 at 11:23 AM, Stephen Frost sfr...@snowman.net wrote: Perhaps I'm assuming things are farther along than they are.. I was assumed that 'incremental mat view' updates were actuallly 'partial'- iow, that it keeps track of the rows in the mat view which are impacted by the

Re: [HACKERS] record identical operator

2013-09-23 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: It seems odd to me that you have such strong opinions about what is and is not acceptable here given that you don't seem to familiar with the current state of this work. That'd be because the arguments that I've been trying to make around this

Re: [HACKERS] record identical operator

2013-09-23 Thread Robert Haas
On Mon, Sep 23, 2013 at 1:19 PM, Stephen Frost sfr...@snowman.net wrote: So I've gathered and I'm not terribly surprised that it's broken. It was unfortunate that we committed it as such, imv. I'm *not* convinced that means we should build on that in a direction that doesn't appear to be

Re: [HACKERS] record identical operator

2013-09-23 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: Robert Haas (robertmh...@gmail.com) wrote: Now, the next project Kevin's going to work on, and that he was working on when he discovered this problem, is incremental maintenance: that is, allowing us to update the view *without* needing to rerun the

Re: [HACKERS] record identical operator

2013-09-23 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: Unless we can tell whether there are any differences between two versions of a row, we can't accurately generate the delta to drive the incremental maintenance. This is predicated on the assumption that you simply generate the new view and then try

Re: [HACKERS] record identical operator

2013-09-23 Thread Robert Haas
On Mon, Sep 23, 2013 at 2:46 PM, Robert Haas robertmh...@gmail.com wrote: Anyway, that is exactly what Kevin is proposing to do here and, to be clear, he's NOT proposing to use the binary-identical semantics to identify the row to be updated. That will happen using the semantics of whatever

Re: [HACKERS] record identical operator

2013-09-23 Thread Stephen Frost
Robert, all, Skipping out on much of the side-discussion to try and drive at the heart of this.. * Robert Haas (robertmh...@gmail.com) wrote: I would suggest that you read the referenced papers for details as to how the algorithm works. To make a long story short, they do need to keep track

Re: [HACKERS] record identical operator

2013-09-23 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: I'm trying to explain that using that methodology is what landed us in this situation to begin with. I'm trying to figure out what situation you think we're in. Seriously, if you could apply the patch and show one example that demonstrates what you see

Re: [HACKERS] record identical operator

2013-09-23 Thread Robert Haas
On Mon, Sep 23, 2013 at 3:45 PM, Stephen Frost sfr...@snowman.net wrote: Robert, all, Skipping out on much of the side-discussion to try and drive at the heart of this.. * Robert Haas (robertmh...@gmail.com) wrote: I would suggest that you read the referenced papers for details as to how

Re: [HACKERS] record identical operator

2013-09-23 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: I don't know why there shouldn't be a question about that. Because anything else would be an internal optimization which must be proven to be correct, imv, also.. Suppose that the MAX() aggregate is in use. If 4 or 5 or 6 is updated so as to

Re: [HACKERS] record identical operator

2013-09-23 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: The only thing the paper says on the topic is that any incremental maintenance scheme is a heuristic.  There will always be cases when it would be faster and less resource-intensive to regenerate the data from the defining query.  There is at least

Re: [HACKERS] record identical operator

2013-09-23 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: In this case even something like AVG() could produce the same result as it did before the update. And you'd surely want to avoid updating the matview if the new value was the same as what was already stored in the matview (but not if it was equal

Re: [HACKERS] record identical operator

2013-09-23 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: I'm trying to figure out what situation you think we're in. Seriously, if you could apply the patch and show one example that demonstrates what you see to be a problem, that would be great. Here's an example to illustrate what I'm talking about when

Re: [HACKERS] record identical operator

2013-09-23 Thread Robert Haas
On Mon, Sep 23, 2013 at 9:21 PM, Stephen Frost sfr...@snowman.net wrote: Here's an example to illustrate what I'm talking about when it comes down to you can't claim that you'll produce exactly what the query will always, with these types: Your example demonstrates that if a given query can

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Martijn van Oosterhout
On Thu, Sep 19, 2013 at 06:31:38PM -0400, Steve Singer wrote: I think there is agreement that better (as in more obscure) operators than === and !== need to be picked and we need to find a place in the user-docs to warn users of the behaviour of this operators. Hannu has proposed *==*

Re: [HACKERS] record identical operator

2013-09-20 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: ... like just refreshing a view so that the new contents are the same as what you would see if you re-ran the query defining the matview. I've heard this notion a few times of wanting to make sure that what you get from running the query matches the

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Kevin Grittner
Steve Singer st...@ssinger.info wrote: On 09/12/2013 06:27 PM, Kevin Grittner wrote: Attached is a patch for a bit of infrastructure I believe to be necessary for correct behavior of REFRESH MATERIALIZED VIEW CONCURRENTLY as well as incremental maintenance of matviews. Here is attempt at a

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Stephen Frost
Steve, Thanks for providing a summary. * Steve Singer (st...@ssinger.info) wrote: The arguments for this patch are * We want the materialized view to return the same value as would be returned if the query were executed directly. This not only means that the values should be the same

Re: [HACKERS] record identical operator

2013-09-20 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: * Kevin Grittner (kgri...@ymail.com) wrote: ... like just refreshing a view so that the new contents are the same as what you would see if you re-ran the query defining the matview. I've heard this notion a few times of wanting to make sure that what

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: The issue here is that we're trying to make the mat view look like what the query would do when run at the same time, which is a bit of a losing battle, imv. If we're not doing that, I don't see the point of matviews. -- Kevin Grittner EDB:

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Stephen Frost
On Friday, September 20, 2013, Kevin Grittner wrote: Stephen Frost sfr...@snowman.net javascript:; wrote: The issue here is that we're trying to make the mat view look like what the query would do when run at the same time, which is a bit of a losing battle, imv. If we're not doing

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: On Friday, September 20, 2013, Kevin Grittner  wrote: Stephen Frost sfr...@snowman.net wrote: The issue here is that we're trying to make the mat view look like what the query would do when run at the same time, which is a bit of a losing battle, imv.

Re: [HACKERS] record identical operator

2013-09-20 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: The result of this is that with the patch, an incremental refresh of a matview will always match what the query returned at the time it was run (there is no *correctness* problem) but if someone uses a query with non-deterministic results they may

Re: [HACKERS] record identical operator

2013-09-20 Thread Andres Freund
On 2013-09-20 10:51:46 -0400, Stephen Frost wrote: I'm trying to figure out why that's a perfectly acceptable solution for users running views with GROUP BYs, but apparently it isn't sufficient for mat views? Err, because users wrote a GROUP BY? They haven't (neccessarily) in the cases of the

Re: [HACKERS] record identical operator

2013-09-20 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: * Kevin Grittner (kgri...@ymail.com) wrote: The result of this is that with the patch, an incremental refresh of a matview will always match what the query returned at the time it was run (there is no *correctness* problem) but if someone uses a query

Re: [HACKERS] record identical operator

2013-09-20 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2013-09-20 10:51:46 -0400, Stephen Frost wrote: I'm trying to figure out why that's a perfectly acceptable solution for users running views with GROUP BYs, but apparently it isn't sufficient for mat views? Err, because users wrote a

Re: [HACKERS] record identical operator

2013-09-20 Thread Andres Freund
On 2013-09-20 11:05:06 -0400, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2013-09-20 10:51:46 -0400, Stephen Frost wrote: I'm trying to figure out why that's a perfectly acceptable solution for users running views with GROUP BYs, but apparently it isn't

Re: [HACKERS] record identical operator

2013-09-20 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: Stephen Frost sfr...@snowman.net wrote: You mean 'at the time of the incremental refresh', right?  Yet that may or may not match running that query directly by an end-user as the plan that a user might get for the entire query could be different

Re: [HACKERS] record identical operator

2013-09-20 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2013-09-20 11:05:06 -0400, Stephen Frost wrote: Sure; my thinking was going back to what Hannu had suggested where we have a mechanism to see if the value was updated (using xmin or similar) and then update it in the mat view in that case,

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Steve Singer
On 09/20/2013 08:38 AM, Kevin Grittner wrote: Did you look at the record_eq and record_cmp functions which exist before this patch? Is there a reason we should do it one way for the default operators and another way for this alternative? Do you think we should change record_eq and record_cmp

Re: [HACKERS] record identical operator

2013-09-20 Thread Alvaro Herrera
Dimitri Fontaine escribió: My understanding is that if you choose citext then you don't care at all about the case, so that the two relations actually yield the same results for the right definition of same here: the citext one. For the record, I don't think citext means that the user doesn't

Re: [HACKERS] record identical operator

2013-09-19 Thread Dimitri Fontaine
Kevin Grittner kgri...@ymail.com writes: There are examples in the patch and this thread, but rather than reference back to those I'll add a new one.  Without the patch: Thanks much for doing that. The problem, as I see it, is that the view and the concurrently refreshed materialized view

Re: [HACKERS] record identical operator

2013-09-19 Thread Hannu Krosing
On 09/19/2013 10:54 AM, Dimitri Fontaine wrote: Kevin Grittner kgri...@ymail.com writes: There are examples in the patch and this thread, but rather than reference back to those I'll add a new one. Without the patch: Thanks much for doing that. The problem, as I see it, is that the view and

Re: [HACKERS] record identical operator

2013-09-19 Thread Kevin Grittner
Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Kevin Grittner kgri...@ymail.com writes: The problem, as I see it, is that the view and the concurrently refreshed materialized view don't yield the same results for the same query.  The rows are equal, but they are not the same. With the patch

Re: [HACKERS] record identical operator

2013-09-19 Thread Andres Freund
On 2013-09-19 05:33:22 -0700, Kevin Grittner wrote: Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Kevin Grittner kgri...@ymail.com writes: The problem, as I see it, is that the view and the concurrently refreshed materialized view don't yield the same results for the same query.  The

Re: [HACKERS] record identical operator

2013-09-19 Thread Kevin Grittner
Hannu Krosing ha...@2ndquadrant.com wrote: the patch solves the general problem of when the table changes, refresh After saying it like this, the problem could also be solved by including xmin(s) for rows from underlying table(s)in the matview. Would this be a better approach ? Now

Re: [HACKERS] record identical operator

2013-09-19 Thread Hannu Krosing
On 09/19/2013 02:41 PM, Kevin Grittner wrote: Hannu Krosing ha...@2ndquadrant.com wrote: the patch solves the general problem of when the table changes, refresh After saying it like this, the problem could also be solved by including xmin(s) for rows from underlying table(s)in the matview.

Re: [HACKERS] record identical operator

2013-09-19 Thread Robert Haas
On Wed, Sep 18, 2013 at 7:29 PM, Vik Fearing vik.fear...@dalibo.com wrote: On 09/19/2013 12:55 AM, Dimitri Fontaine wrote: We have, as a community, gone to a fair amount of trouble to make the concept of equality pluggable and allow multiple types of equality per type. To me it seems the

Re: [HACKERS] record identical operator - Review

2013-09-19 Thread Steve Singer
On 09/12/2013 06:27 PM, Kevin Grittner wrote: Attached is a patch for a bit of infrastructure I believe to be necessary for correct behavior of REFRESH MATERIALIZED VIEW CONCURRENTLY as well as incremental maintenance of matviews. Here is attempt at a review of the v1 patch. There has been a

Re: [HACKERS] record identical operator

2013-09-18 Thread Robert Haas
On Tue, Sep 17, 2013 at 8:23 AM, Kevin Grittner kgri...@ymail.com wrote: To have clean semantics, I think the operator should mean that the stored format of the row is the same. Regarding the array null bitmap example, I think it would be truly weird if the operator said that the stored

Re: [HACKERS] record identical operator

2013-09-18 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Therefore, I see no harm in having an operator that tests for are-these-values-identical-for-all-purposes. If that's useful for RMVC, then there may be other legitimate uses for it as well. And once we decide that's OK, I think we ought to

Re: [HACKERS] record identical operator

2013-09-18 Thread Andres Freund
On 2013-09-18 11:06:13 -0500, Merlin Moncure wrote: Ugh. This feels like a pretty ugly hack to deal with that. I haven't got any magical wand to address it, but making an SQL operator for 'are these really the same bytes' to deal with what is essentially implementation detail is _very_

Re: [HACKERS] record identical operator

2013-09-18 Thread Andres Freund
On 2013-09-18 11:50:23 -0400, Stephen Frost wrote: For my 2c on this, while this can be useful for *us*, and maybe folks hacking pretty close to PG, I can't get behind introducing this as an '===' or some such operator. I've missed why this can't be a simple function and why in the world we

Re: [HACKERS] record identical operator

2013-09-18 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: making an SQL operator for 'are these really the same bytes' to deal with what is essentially implementation detail is _very_ grotty. We already have some such operators, although Andres argues that comparing to that isn't fair because we at least know

Re: [HACKERS] record identical operator

2013-09-18 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Kevin Grittner kgri...@ymail.com wrote: To have clean semantics, I think the operator should mean that the stored format of the row is the same.  Regarding the array null bitmap example, I think it would be truly weird if the operator said that the

Re: [HACKERS] record identical operator

2013-09-18 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: Right.  Not only would the per-type solution make materialized views maintenance broken by default, requiring per-type work to make it work reasonably, with silent failures for any type you didn't know about, but no user-visible differences is a

Re: [HACKERS] record identical operator

2013-09-18 Thread Merlin Moncure
On Wed, Sep 18, 2013 at 10:59 AM, Stephen Frost sfr...@snowman.net wrote: * Andres Freund (and...@2ndquadrant.com) wrote: I think this really needs to have an obscure name. Like ==!!== or somesuch (is equal very much, but doesn't actually test for equality ;)) hah. What the heck is the use

Re: [HACKERS] record identical operator

2013-09-18 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: I think this really needs to have an obscure name. Like ==!!== or somesuch (is equal very much, but doesn't actually test for equality ;)) hah. What the heck is the use case for this being a user-oriented, SQL operator..? The materalized

Re: [HACKERS] record identical operator

2013-09-18 Thread Stephen Frost
* Merlin Moncure (mmonc...@gmail.com) wrote: Having matviews using SQL expressible features is a *good* thing. Fine, then it should be implemented *using SQL*, which is based on *values*, not on how the value is represented in bits and bytes. Having a user accessible operator is nice to have

Re: [HACKERS] record identical operator

2013-09-18 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: = and aren't listed above even though they do a byte-for-byte comparison because, well, I guess because we have chosen to treat two UTF8 strings which produce the same set of glyphs using different bytes as unequal.  :-/ I tend to side with Andres

Re: [HACKERS] record identical operator

2013-09-18 Thread Steve Singer
On 09/18/2013 11:39 AM, Stephen Frost wrote: * Kevin Grittner (kgri...@ymail.com) wrote: = and aren't listed above even though they do a byte-for-byte comparison because, well, I guess because we have chosen to treat two UTF8 strings which produce the same set of glyphs using different bytes

Re: [HACKERS] record identical operator

2013-09-18 Thread Stephen Frost
* Steve Singer (st...@ssinger.info) wrote: The problem is that there are datatypes (citext, postgis,...) that have defined = to return true when comparing two values that are different not just stored differently. If the definition of the type is that they're equal, then they're equal.

Re: [HACKERS] record identical operator

2013-09-18 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: If it's not actually *changing* (wrt its value), then I'm not at all impressed with the notion that it's going to get updated anyway. But PostgreSQL very specifically (and as far as I can tell *intentionally*) allows you to *change* a value and have it

Re: [HACKERS] record identical operator

2013-09-18 Thread Stephen Frost
Kevin, On Wednesday, September 18, 2013, Kevin Grittner wrote: Stephen Frost sfr...@snowman.net javascript:; wrote: If it's not actually *changing* (wrt its value), then I'm not at all impressed with the notion that it's going to get updated anyway. But PostgreSQL very specifically

Re: [HACKERS] record identical operator

2013-09-18 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: I don't think that means we should change our definition of equality to generally be are the bytes the same- clearly that'd lead to incorrect behavior in the NUMERIC case. Nobody is talking in any way, shape, or form about changing our concept of what

Re: [HACKERS] record identical operator

2013-09-18 Thread Hannu Krosing
On 09/18/2013 05:53 PM, Andres Freund wrote: On 2013-09-18 11:50:23 -0400, Stephen Frost wrote: For my 2c on this, while this can be useful for *us*, and maybe folks hacking pretty close to PG, I can't get behind introducing this as an '===' or some such operator. I've missed why this can't

Re: [HACKERS] record identical operator

2013-09-18 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net Kevin Grittner  wrote: Stephen Frost sfr...@snowman.net wrote: If it's not actually *changing* (wrt its value), then I'm not at all impressed with the notion that it's going to get updated anyway. But PostgreSQL very specifically (and as far as I can tell

Re: [HACKERS] record identical operator

2013-09-18 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: If an update happens with a conditional of: where col1 = 'Abc' When col1 is 'ABC' using citext, should we still issue the update? Absolutely not, because the update was requested in the case that the equality test was true. Sorry, as if this

Re: [HACKERS] record identical operator

2013-09-18 Thread Dimitri Fontaine
Kevin Grittner kgri...@ymail.com writes: change, but if '1.4' was stored in a column copied into a matview and they later updated the source to '1.40' the increase in accuracy would not flow to the matview.  That would be a bug, not a feature. Maybe the answer to that use case is to use the

Re: [HACKERS] record identical operator

2013-09-18 Thread Kevin Grittner
Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Kevin Grittner kgri...@ymail.com writes: change, but if '1.4' was stored in a column copied into a matview and they later updated the source to '1.40' the increase in accuracy would not flow to the matview.  That would be a bug, not a feature.

Re: [HACKERS] record identical operator

2013-09-18 Thread Hannu Krosing
On 09/18/2013 09:19 PM, Dimitri Fontaine wrote: Kevin Grittner kgri...@ymail.com writes: change, but if '1.4' was stored in a column copied into a matview and they later updated the source to '1.40' the increase in accuracy would not flow to the matview. That would be a bug, not a feature.

Re: [HACKERS] record identical operator

2013-09-18 Thread Hannu Krosing
On 09/18/2013 09:41 PM, Kevin Grittner wrote: Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Kevin Grittner kgri...@ymail.com writes: change, but if '1.4' was stored in a column copied into a matview and they later updated the source to '1.40' the increase in accuracy would not flow to the

  1   2   >