Re: [HACKERS] how to understand these macro defines such as ObjectIdAttributeNumber in sysattr.h?

[土卜皿]

hi, Michael
  thanks a lot!   your reminding lead me go a step forward!
  the 7 attributes are system attributes of some table such as pg_class,
pg_type. but now I have a new question:

where and when using these system attributes?



2014-08-18 15:57 GMT+08:00 Michael Paquier :

> On Mon, Aug 18, 2014 at 4:42 PM, 土卜皿  wrote:
> >Recently, for understanding postgresql's optimizer,  I read and debug
> the
> > source code. these two days,  I read these macros in sysattr.h:
> >
> > ...
> > #define ObjectIdAttributeNumber (-2)
> > ...
> >
> > and the relative codes, but I still did not know why authors designed
> them
> > like that, especially the value '-2' ?
> >
> > what should I do if I want to see the thoughts behind them? thanks in
> > advance!
> Those values define the system attribute IDs associated to a relation.
> You can have a look at the top of heap.c to see where they come from,
> particularly the fields FormData_pg_attribute a1..a7.
> Hope this helps.
> Regards,
> --
> Michael
>

[HACKERS] how to understand these macro defines such as ObjectIdAttributeNumber in sysattr.h?

[土卜皿]

hi, all
   I am so sorry for asking some questions like this!
   Recently, for understanding postgresql's optimizer,  I read and debug
the source code. these two days,  I read these macros in sysattr.h:

...
#define ObjectIdAttributeNumber (-2)
...

and the relative codes, but I still did not know why authors designed them
like that, especially the value '-2' ?

what should I do if I want to see the thoughts behind them? thanks in
advance!

Re: [HACKERS] how to debug into InitPostgres() and InitCatalogCache()?

[土卜皿]

2014-08-05 22:08 GMT+08:00 土卜皿 :

> hi, all
>   I already can debug general postgres using "ddd" and "select
> pg_backend_pid();" ,  now, I want to study the details of the system
> catalog cache and system cache management, so I need to debug the function
> InitPostgres() and InitCatalogCache(), and I tried the following steps:
>
> [refer: How to debug postgresql during initdb ]
> <http://www.phacai.com/how-to-debug-postgresql-during-initdb>
>
> (1) in first terminal:
>
> ddd initdb
>
> and set a breakpoint on the next executable line after “PG_CMD_OPEN” in
> function bootstrap_template1() ,
>
> Then within gdb, use “r $myfolder/mydb”,
>
> and found the cursor stop at the breakpoint
>
> (2) in the second terminal:
>
> ps aux | grep "bin\/postgres"
>
> after get the posgtres server pid, start a new ddd without args, and
> within gdb, using "attach postgres.pid"
>
> and set a breakpoint as InitCatalogCache()
>
> (3) back to the first ddd window, hit the "cont" button. I hope it can
> stop at the second ddd's breakpoint, but nothing happened?
>
> I must make mistake in some step, or my understanding is fully wrong,
> please give me some advice, thanks a lot
>


  I am sorry that this is a simple question.  I found that the above steps
make the posgres reach boot_yyparse() of BootstrapModeMain(void), which
means postgres has already finished the initialization including
InitCatalogCache(), and it was waiting query command or other user command .

  For debug initialization including system catalog cache, I add some code
like:

 bool forDebug = true;

while (forDebug){
  forDebug = true;
}

in the InitPosgres()'s starting position.

>
>
>
>
>

[HACKERS] how to debug into InitPostgres() and InitCatalogCache()?

[土卜皿]

hi, all
  I already can debug general postgres using "ddd" and "select
pg_backend_pid();" ,  now, I want to study the details of the system
catalog cache and system cache management, so I need to debug the function
InitPostgres() and InitCatalogCache(), and I tried the following steps:

[refer: How to debug postgresql during initdb ]


(1) in first terminal:

ddd initdb

and set a breakpoint on the next executable line after “PG_CMD_OPEN” in
function bootstrap_template1() ,

Then within gdb, use “r $myfolder/mydb”,

and found the cursor stop at the breakpoint

(2) in the second terminal:

ps aux | grep "bin\/postgres"

after get the posgtres server pid, start a new ddd without args, and within
gdb, using "attach postgres.pid"

and set a breakpoint as InitCatalogCache()

(3) back to the first ddd window, hit the "cont" button. I hope it can stop
at the second ddd's breakpoint, but nothing happened?

I must make mistake in some step, or my understanding is fully wrong,
please give me some advice, thanks a lot!

Dillon

[HACKERS] whether I can see other alternative plantrees for one query?

[土卜皿]

hi, all
  NOTE: Version is 8.4   Fedora 20 X86_64

  for understanding optimizer's internals, I

*set debug_print_plan=on*

and created two tables as follows :


*create table Reserves (sid integer, bid integer,day date,rname
char(25));create table Sailors(sid integer,sname char(25),rating
integer,age real);*

and add 1,000,000 records for each.

and execute the cmd:




*select S.rating,count(*) from Sailors S where S.rating > 5
and S.age = 20 group by S.rating;*

but from the log, I only found the final selected planTree, so I want to
ask:
what should I do if I want to see the other alternative planTrees?  any
advice will be apprecitaed!

Dillon Peng

Re: [HACKERS] how to reach D5 in tuplesort.c 's polyphase merge algorithm?

[土卜皿]

hi,
   I got the same result after work_mem = 64,
but I can get to D5 and D6 after using bigger data sample (at least 10
records) as Tom said!




2014-07-19 6:35 GMT+08:00 土卜皿 :

>
> 2014-07-19 6:26 GMT+08:00 Tom Lane :
>
> =?UTF-8?B?5Zyf5Y2c55q/?=  writes:
>> >   for studying polyphase merge algorithm of tuplesort.c, I use ddd and
>> > apend a table, which has a schema as follows:
>> > ...
>> > and has 36684 records, and every record is like:
>> > id  code  article  name  department
>> > 31800266\NMachault77
>>
>> > and for getting into external sort, I type the following command:
>>
>> > select * from towns order by name desc;
>>
>> > but I found it need not reach D5 and D6 during sorting,
>>
>> That doesn't sound like enough data to force it to spill to disk at all;
>> at least not unless you turn down work_mem to some very small value.
>>
>>
>
> hi, Tom
>   thanks a lot!
>
>
>>
> work_mem you said remind me one more thing I did, I tried to change BLCKSZ
> = 8192/2, and successfully compiled, but I got a error when executing
> initdb
>
> Dillon
>

Re: [HACKERS] how to reach D5 in tuplesort.c 's polyphase merge algorithm?

[土卜皿]

2014-07-19 6:26 GMT+08:00 Tom Lane :

> =?UTF-8?B?5Zyf5Y2c55q/?=  writes:
> >   for studying polyphase merge algorithm of tuplesort.c, I use ddd and
> > apend a table, which has a schema as follows:
> > ...
> > and has 36684 records, and every record is like:
> > id  code  article  name  department
> > 31800266\NMachault77
>
> > and for getting into external sort, I type the following command:
>
> > select * from towns order by name desc;
>
> > but I found it need not reach D5 and D6 during sorting,
>
> That doesn't sound like enough data to force it to spill to disk at all;
> at least not unless you turn down work_mem to some very small value.
>
>

hi, Tom
  thanks a lot!


>
work_mem you said remind me one more thing I did, I tried to change BLCKSZ
= 8192/2, and successfully compiled, but I got a error when executing
initdb

Dillon

[HACKERS] how to reach D5 in tuplesort.c 's polyphase merge algorithm?

[土卜皿]

hi, all
  for studying polyphase merge algorithm of tuplesort.c, I use ddd and
apend a table, which has a schema as follows:

CREATE TABLE Towns (
   id SERIAL UNIQUE NOT NULL,
   code VARCHAR(10) NOT NULL, -- Only unique inside a department
   article TEXT,
   name TEXT NOT NULL, -- Names are not really unique, for instance
'Sainte-Croix'
   department VARCHAR(4) NOT NULL REFERENCES Departments (code),
   UNIQUE (code, department)
   -- UNIQUE (name, department) -- Not perfectly unique but almost
  );

and has 36684 records, and every record is like:
id  code  article  name  department
31800266\NMachault77

and for getting into external sort, I type the following command:

select * from towns order by name desc;

but I found it need not reach D5 and D6 during sorting,  I thought that the
reason is the amount of runs is 3 (too small) before merging, for generate
more runs, I shuffled the records
when inputting them and got the same result.

so that I want some help, what schema and records do I as least need for
reaching D5 and D6?  any advice will be appreciated!

BEST REGAERDS
Dillon

Re: [HACKERS] how many changes about backend mode from 7.2.2 to 8.4.0?

[土卜皿]

hi, tom
  thank you very much! this is what I wanted!

Dillon


2014-07-11 10:59 GMT+08:00 Tom Lane :

> =?UTF-8?B?5Zyf5Y2c55q/?=  writes:
> > I am sorry, maybe I should make my question clearer, I only want to know,
> > in 8.4.0 or newer version,
> >  whether I can debug posgres in the bare backend mode (in
>  usingddd-postgres
> > , he said
> that "
> > There are two ways to debug postgres (a) in the interactive mode and (b)
> in
> > the bare backend mode"
>
> Yeah, that still works as well or poorly as it ever did, though the
> startup process is a bit different: you need to say "postgres --single"
> to launch a standalone backend.
>
> But really, nobody does it that way anymore.  There is no advantage to it,
> unless maybe you're trying to debug a startup-time crash.  The standalone
> backend interface is very unpleasant to use compared to psql: no readline
> command editing, no nice formatting of output, no tab completion or help,
> etc etc.  For debugging of normal query operation it's far better to fire
> up a psql session and then gdb the connected backend process.
>
> There's more info here:
> https://wiki.postgresql.org/wiki/Developer_FAQ#gdb
>
> and if gdb isn't giving you useful symbolic information, see
> the setup tips here:
>
> https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD
>
> regards, tom lane
>

Re: [HACKERS] how many changes about backend mode from 7.2.2 to 8.4.0?

[土卜皿]

hi,
  thanks a lot!


2014-07-11 10:10 GMT+08:00 Michael Paquier :

> On Fri, Jul 11, 2014 at 10:50 AM, 土卜皿  wrote:
> > NOTE: fedora 17 x86_64, 7.2.2 can not be compiled in the env, I choose
> 8.4.0
> > because I think old version is easier to understand than newer version!
> Are you aware that Postgres 7.2 has been released in 2002? It is EOL
> (end-of-life) since 2005 by looking at the release notes.
>
I know the released time.


>
> > I want to study pg's external sort (in tuplesort.c )through 8.4.0's
> source
> > code, and use ddd to do it, according to usingddd-postgres 's
> description:
> If you are planning to do some development in the future with or for
> Postgres, you would get a better insight by looking at more recent
> code. Here are some guidelines for example to use git, which is really
> helpful as a base infrastructure when studying the code:
> https://wiki.postgresql.org/wiki/Working_with_GIT
>
> seemly the wiki you said has no information about debug

I am sorry, maybe I should make my question clearer, I only want to know,
in 8.4.0 or newer version,
 whether I can debug posgres in the bare backend mode (in  usingddd-postgres
<http://www-inst.eecs.berkeley.edu/~cs186/fa04/usingddd.pdf>, he said that "
There are two ways to debug postgres (a) in the interactive mode and (b) in
the bare backend mode"
), if yes, what should I do?  thanks in advance!

BEST REGARDS
Dillon

[HACKERS] how many changes about backend mode from 7.2.2 to 8.4.0?

[土卜皿]

hi, all
NOTE: fedora 17 x86_64, 7.2.2 can not be compiled in the env, I choose
8.4.0 because I think old version is easier to understand than newer
version!

I want to study pg's external sort (in tuplesort.c )through 8.4.0's source
code, and use ddd to do it, according to usingddd-postgres
 's description:

 In the backend mode, you don’t create a separate postmaster process and
don’t use psql at all. Instead, you start up postgres from the
command line and directly interact with it.

and after starting  postgres in ddd, a backend interactive interface works
as doc's Figure 5, and I found version 7.2.2's source postgres'c has
backend interactive interface, but my version is 8.4.0, which seemly has
not backend interactive interface, so I want to ask

(1) whether I must use another method to debug relative external sort code
(in tuplesort.c ), as some post said that I can execute "psql, select select
pg_backend_pid(), and gdb pid"  , any advice will be appreciated!

BEST REGARDS
Dillon Peng