Re: Detoasting optionally to make Explain-Analyze less misleading

2024-04-10 Thread stepan rutz
First of all thanks for bringing this Feature to PostgreSQL. From a regular-user perspective (not everyone is a Pro) it is very misleading that ANALYZE doesn't do what it suggests it does. To run the query into some kind of /dev/null type of destination is feasible and that is what people end up

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-04-10 Thread Tom Lane
Matthias van de Meent writes: > Upthread at [0], Stepan mentioned that we should default to SERIALIZE > when ANALYZE is enabled. I suspect a patch in that direction would > primarily contain updates in the test plan outputs, but I've not yet > worked on that. > Does anyone else have a strong

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-04-10 Thread Matthias van de Meent
On Wed, 3 Apr 2024 at 23:50, Tom Lane wrote: > I've pushed this after a good deal of cosmetic polishing -- for > example, I spent some effort on making serializeAnalyzeReceive > look as much like printtup as possible, in hopes of making it > easier to keep the two functions in sync in future.

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-04-04 Thread Matthias van de Meent
On Wed, 3 Apr 2024 at 23:50, Tom Lane wrote: > > Matthias van de Meent writes: >> On Tue, 2 Apr 2024 at 17:47, Tom Lane wrote: >>> IIUC, it's not possible to use the SERIALIZE option when explaining >>> CREATE TABLE AS, because you can't install the instrumentation tuple >>> receiver when the

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-04-03 Thread Tom Lane
Matthias van de Meent writes: > On Tue, 2 Apr 2024 at 17:47, Tom Lane wrote: >> IIUC, it's not possible to use the SERIALIZE option when explaining >> CREATE TABLE AS, because you can't install the instrumentation tuple >> receiver when the IntoRel one is needed. I think that's fine because >>

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-04-03 Thread Matthias van de Meent
On Tue, 2 Apr 2024 at 17:47, Tom Lane wrote: > > Matthias van de Meent writes: > > Attached is v9, which is rebased on master to handle 24eebc65's > > changed signature of pq_sendcountedtext. > > It now also includes autocompletion, and a second patch which adds > > documentation to give users

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-04-02 Thread Tom Lane
Matthias van de Meent writes: > Attached is v9, which is rebased on master to handle 24eebc65's > changed signature of pq_sendcountedtext. > It now also includes autocompletion, and a second patch which adds > documentation to give users insights into this new addition to > EXPLAIN. I took a

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-03-12 Thread Matthias van de Meent
On Mon, 26 Feb 2024 at 21:54, stepan rutz wrote: > > Hi Matthias, thanks for picking it up. I still believe this is valuable > to a lot of people out there. Thanks for dealing with my proposal. > Matthias, Tom, Tomas everyone. > > Two (more or less) controversial remarks from side. > > 1.

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-02-26 Thread stepan rutz
Hi Matthias, thanks for picking it up. I still believe this is valuable to a lot of people out there. Thanks for dealing with my proposal. Matthias, Tom, Tomas everyone. Two (more or less) controversial remarks from side. 1. Actually serialization should be the default for "analyze" in explain,

Re: Detoasting optionally to make Explain-Analyze less misleading

2024-02-26 Thread Matthias van de Meent
Hi, I've taken the liberty to update this patch, and register it in the commitfest app to not lose track of progress [0]. The attached v8 patch measures scratch memory allocations (with MEMORY option), total time spent in serialization (with TIMING on, measures are inclusive of unseparated

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Tomas Vondra
On 11/2/23 22:33, Matthias van de Meent wrote: > On Thu, 2 Nov 2023 at 22:25, Tomas Vondra > wrote: >> >> >> >> On 11/2/23 21:02, Matthias van de Meent wrote: >>> On Thu, 2 Nov 2023 at 20:32, Tomas Vondra >>> wrote: On 11/2/23 20:09, stepan rutz wrote: > db1=# explain (analyze,

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Matthias van de Meent
On Thu, 2 Nov 2023 at 22:25, Tomas Vondra wrote: > > > > On 11/2/23 21:02, Matthias van de Meent wrote: > > On Thu, 2 Nov 2023 at 20:32, Tomas Vondra > > wrote: > >> On 11/2/23 20:09, stepan rutz wrote: > >>> db1=# explain (analyze, serialize) select * from test; > >>>

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Tomas Vondra
On 11/2/23 21:02, Matthias van de Meent wrote: > On Thu, 2 Nov 2023 at 20:32, Tomas Vondra > wrote: >> On 11/2/23 20:09, stepan rutz wrote: >>> db1=# explain (analyze, serialize) select * from test; >>> QUERY PLAN >>>

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Matthias van de Meent
On Thu, 2 Nov 2023 at 20:32, Tomas Vondra wrote: > On 11/2/23 20:09, stepan rutz wrote: > > db1=# explain (analyze, serialize) select * from test; > > QUERY PLAN > >

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread stepan rutz
Hi Thomas, indeed by doing less the code also becomes trivial and ExplainPropertyInteger can be used as a oneliner. My intention was to actually get the realistic payload-bytes from the wire-protocol counted by the serialization option. I am also adding the protocol bits and the length of the

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Tomas Vondra
On 11/2/23 20:09, stepan rutz wrote: > Hi Thomas, > > you are right of course. Thanks! > > I have attached a new version of the patch that supports the syntax like > suggested. The previous patch was insonsistent in style indeed. > > explain (analyze, serialize) > > and > > explain

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread stepan rutz
Hi Thomas, you are right of course. Thanks! I have attached a new version of the patch that supports the syntax like suggested. The previous patch was insonsistent in style indeed. explain (analyze, serialize) and explain (analyze, serialize binary) That doesn't make too much of a

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Tomas Vondra
Hi, On 9/15/23 22:09, stepan rutz wrote: > Hi, > > please see a revised version yesterday's mail. The patch attached now > provides the following: > > EXPLAIN(ANALYZE,SERIALIZE) > > and > > EXPLAIN(ANALYZE,SERIALIZEBINARY) > I haven't looked at the patch in detail yet, but this option name

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-15 Thread stepan rutz
Hi, please see a revised version yesterday's mail. The patch attached now provides the following: EXPLAIN(ANALYZE,SERIALIZE) and EXPLAIN(ANALYZE,SERIALIZEBINARY) and timing output. Both options perform the serialization during analyze and provide an additional output in the plan like this:

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-14 Thread stepan rutz
Hi Tom, Hi Matthias, you are right of course. I have looked at the code from printtup.c and made a new version of the patch. Thanks for the MemoryContextReset hint too (@Matthias) This time is called  EXPLAIN(ANALYZE,SERIALIZE) (hey, it also sounds nicer phonetically) If the option SERIALIZE

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-13 Thread Jehan-Guillaume de Rorthais
Hi Stepan & all, On Tue, 12 Sep 2023 17:16:00 +0200 stepan rutz wrote: ... > Attached a new patch. Hoping for feedback, Nice addition to EXPLAIN! On the feature front, what about adding the actual detoasting/serializing time in the explain output? That could be: => explain

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-12 Thread Tom Lane
Matthias van de Meent writes: > Hmm, maybe we should measure the overhead of serializing the tuples instead. > The difference between your patch and "serializing the tuples, but not > sending them" is that serializing also does the detoasting, but also > includes any time spent in the

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-12 Thread stepan rutz
Hi Matthias, thanks for your feedback. I wasn't sure on the memory-contexts. I was actually also unsure on whether the array   TupleTableSlot.tts_isnull is also set up correctly by the previous call to slot_getallattrs(slot). This would allow to get rid of even more code from this patch,

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-12 Thread Matthias van de Meent
On Tue, 12 Sept 2023 at 12:56, stepan rutz wrote: > > Hi, > > I have fallen into this trap and others have too. If you run > EXPLAIN(ANALYZE) no de-toasting happens. This makes query-runtimes > differ a lot. The bigger point is that the average user expects more > from EXPLAIN(ANALYZE) than what

Detoasting optionally to make Explain-Analyze less misleading

2023-09-12 Thread stepan rutz
Hi, I have fallen into this trap and others have too. If you run EXPLAIN(ANALYZE) no de-toasting happens. This makes query-runtimes differ a lot. The bigger point is that the average user expects more from EXPLAIN(ANALYZE) than what it provides. This can be suprising. You can force detoasting