Re: [HACKERS] Postgres code for a query intermediate dataset
Hi Mark & Atri, :) Thanks for reply. But, I think i confused you. I am talking about access using indexes. So, I assume that B+ tree store key-value pair where rohit is the key and all the versions are its value. Another way to think is I have a secondary index on emp. name and there are 4 rohit exist in DB. So, now B+ tree gives me 4 different tuple pointer for each Rohit. I want to know the code portion for this where i can see all 4 tuple pointer before each one have I/O access to fetch its tuple. Are the suggestions still valid? On Sun, Sep 14, 2014 at 10:53 AM, Mark Kirkwood < mark.kirkw...@catalyst.net.nz> wrote: > On 14/09/14 20:11, Rohit Goyal wrote: > >> Hi Mark, >> >> On Sun, Sep 14, 2014 at 8:57 AM, Mark Kirkwood >> mailto:mark.kirkw...@catalyst.net.nz>> >> wrote: >> > > Currently in Postgres, these intermediate versions all exist - >> however a given session can only see one of them. Also VACUUM is >> allowed to destroy versions that no other transactions can see. >> >> Exactly, one visible version is there per session. But, I want to test >> my algorithm in which i myself want to work on all the intermediate >> version and find the correct one for the session. >> >> So if I'm understanding you correctly, you would like to have some >> way for a session to see *all* these versions (and I guess >> preventing VACUUM from destroying them). >> >> yes and I also want to know the portion of code where i can find all the >> historical or intermediate versions >> > > Well that's going to be a bit of a learning curve for you :-), the concept > to get familiar with is snapshots (see src/backand/access/heap/heapam.c > to get started). Thanks I will read it. :) Can you please tel me some specifics from this c file, if you already know :) > > > >> >> It is certainly possible (or used to be via snapshot manipulation, I >> haven't looked at that code in a while sorry) to enable a session to >> see all the old versions, and is quite a cool idea (Postgres used to >> have this ability in older versions - called Time Travel). >> >> For pure practicality, this can be achieved without any code >> modifications using TRIGGERs and an extra table (as Gavin alludes to). >> >> Can you explain me more about how to starting working using trigger? >> > > I'm not sure we have specif examples in the docs for what you want to do, > but generally see http://www.postgresql.org/docs/9.4/static/plpgsql- > trigger.html > > Cheers > > Mark > -- Regards, Rohit Goyal
Re: [HACKERS] Postgres code for a query intermediate dataset
Hi Mark, On Sun, Sep 14, 2014 at 8:57 AM, Mark Kirkwood < mark.kirkw...@catalyst.net.nz> wrote: > On 14/09/14 05:36, Rohit Goyal wrote: > >> Hi All, >> >> I want to work on the code of intermediate dataset of select and update >> query. >> >> For example. >> >> Rohit's salary has been updated 4 times, so it has 4 different version >> of salary. >> >> I want to select salary of person named Rohit. Now suppose , in >> intermediate result, I found 4 different versions of the data. I want to >> know the code portion which i need to look for working on all 4 versions >> in dataset. :) >> >> >> > Hi Rohit, > > Currently in Postgres, these intermediate versions all exist - however a > given session can only see one of them. Also VACUUM is allowed to destroy > versions that no other transactions can see. > > Exactly, one visible version is there per session. But, I want to test my algorithm in which i myself want to work on all the intermediate version and find the correct one for the session. > So if I'm understanding you correctly, you would like to have some way for > a session to see *all* these versions (and I guess preventing VACUUM from > destroying them). > yes and I also want to know the portion of code where i can find all the historical or intermediate versions > > It is certainly possible (or used to be via snapshot manipulation, I > haven't looked at that code in a while sorry) to enable a session to see > all the old versions, and is quite a cool idea (Postgres used to have this > ability in older versions - called Time Travel). > > For pure practicality, this can be achieved without any code modifications > using TRIGGERs and an extra table (as Gavin alludes to). > Can you explain me more about how to starting working using trigger? > > Do tell us a bit more about what you are wanting to do! > > Cheers > > Mark > > -- Regards, Rohit Goyal
[HACKERS] Postgres code for a query intermediate dataset
Hi All, I want to work on the code of intermediate dataset of select and update query. For example. Rohit's salary has been updated 4 times, so it has 4 different version of salary. I want to select salary of person named Rohit. Now suppose , in intermediate result, I found 4 different versions of the data. I want to know the code portion which i need to look for working on all 4 versions in dataset. :) Thanks in advance!! Regards, Rohit Goyal -- Regards, Rohit Goyal
Re: [HACKERS] Least Active Transaction ID function
Hi Robert, On Thu, Jul 24, 2014 at 9:32 PM, Robert Haas wrote: > On Wed, Jul 23, 2014 at 3:53 PM, Rohit Goyal wrote: > > Hi All, > > > > On Wed, Jul 23, 2014 at 5:01 PM, Rohit Goyal > wrote: > >> > >> Hi All, > >> > >> I am doing programming with postgresql source code. I want to find out > the > >> function which can give me Least active transaction id currenty in the > >> system. > >> > >> Is there any function which can give me that? > >> > >> Regards, > >> Rohit Goyal > > > > 1> I know that somewhere there is an active transaction list in > postgresql. > > At any point of time, I want to get the smallest transaction present in > this > > active tx list or I want to get the transaction id before which all > > transaction smaller than that are committed/aborted. > > > > Is there any function which can give me this value? > > See the RecentXmin calculation in GetSnapshotData. > > This was really -2 helpful. 1. Can I use this xmin variable directly anytime anywhere in my code as it is a global variable. 2. What is the difference b/w recentXmin and RecentGlobalXmin. I read the description but any small detail can clear my mind. :) Thanks in advance!! > > 2> I found a function giving GetStableLatestTransactionId(), please tel > me > > what this function gives. I was not able to understand the description > given > > above it. > > I don't know how to help with this; the description seems clear to me. > > This is not important now, as you have already told me the variable and file for recentXmin.:) Regards, Rohit > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Regards, Rohit Goyal
Re: [HACKERS] Least Active Transaction ID function
Hi All, On Wed, Jul 23, 2014 at 5:01 PM, Rohit Goyal wrote: > Hi All, > > I am doing programming with postgresql source code. I want to find out the > function which can give me Least active transaction id currenty in the > system. > > Is there any function which can give me that? > > Regards, > Rohit Goyal > 1> I know that somewhere there is an active transaction list in postgresql. At any point of time, I want to get the smallest transaction present in this active tx list or I want to get the transaction id before which all transaction smaller than that are committed/aborted. Is there any function which can give me this value? 2> I found a function giving *GetStableLatestTransactionId()*, please tel me what this function gives. I was not able to understand the description given above it. Thanks for support in advance :)!! Regards, Rohit Goyal
[HACKERS] Least Active Transaction ID function
Hi All, I am doing programming with postgresql source code. I want to find out the function which can give me Least active transaction id currenty in the system. Is there any function which can give me that? Regards, Rohit Goyal
Re: [HACKERS] Shared Data Structure b/w clients
Hi Atri/All, I am very new in postgresql code. Can you please help in a bit detail ortel me how to create structure in shared memory(shared buffer). It would be really easy for me if you can give me a code snippet or any link to follow. Regards, Rohit Goyal On Tue, Jul 22, 2014 at 8:30 PM, Atri Sharma wrote: > > > On Tuesday, July 22, 2014, Rohit Goyal wrote: > >> Hi All, >> >> I am working on postgresql code and having some problem. :) >> >> I need to create shared data structure, so that different client and >> connection can update and share the state of those data structures in >> memory. I planned to use top memory context but it can give me shared >> structure within one session/terminal. >> >> Please tel me how postgresql do that and how i can do that? >> >> Regards, >> Rohit Goyal >> > > How about making it a part of shared mem, like shared buffers? > > Regards, > > Atri > > > -- > Regards, > > Atri > *l'apprenant* > > -- Regards, Rohit Goyal
[HACKERS] Shared Data Structure b/w clients
Hi All, I am working on postgresql code and having some problem. :) I need to create shared data structure, so that different client and connection can update and share the state of those data structures in memory. I planned to use top memory context but it can give me shared structure within one session/terminal. Please tel me how postgresql do that and how i can do that? Regards, Rohit Goyal
Re: [HACKERS] Error in running DBT2
On Tue, May 20, 2014 at 9:57 AM, Andrew Dunstan wrote: > > On 05/20/2014 03:39 AM, Rohit Goyal wrote: > >> Hi All, >> >> Just adding the actual error again. >> >> I run the the *dbt2-pgsql-build-db -w 1 * >> >> >> Output directory of data files: current directory >> >> *Generating data files for 1 warehouse(s)...* >> *Generating item table data...* >> *BEGIN* >> *ERROR: invalid byte sequence for encoding "UTF8": 0xf9* >> *CONTEXT: COPY item, line 1* >> *ROLLBACK* >> >> >> This pattern the above found many times. Please guide me through!!! >> >> >> >> > > Please stop posting this to postgresql-hackers. We've already told you > it's the wrong forum. > > cheers > > Hi Andrew, I am so sorry, it was by mistake. :). I was not able to analyse whether the issue belong to postgresql or dbt2. Regards, Rohit > andrew >
Re: [HACKERS] Error in running DBT2
Hi All, Just adding the actual error again. I run the the *dbt2-pgsql-build-db -w 1 * Output directory of data files: current directory *Generating data files for 1 warehouse(s)...* *Generating item table data...* *BEGIN* *ERROR: invalid byte sequence for encoding "UTF8": 0xf9* *CONTEXT: COPY item, line 1* *ROLLBACK* This pattern the above found many times. Please guide me through!!! Regards, Rohit Goyal On Tue, May 20, 2014 at 9:15 AM, Rohit Goyal wrote: > Hi All, > > > > On Tue, May 13, 2014 at 9:44 PM, Peter Geoghegan wrote: > >> >> On Tue, May 13, 2014 at 12:36 PM, Rohit Goyal wrote: >> >>> This pattern the above found many times. Please guide me through!!! >>> >> >> IIRC, people have been working around this by setting >> standard_conforming_strings to "off". It really ought to be fixed in a >> principled way, though -- the real issue here is that dbt2 has severe >> bit-rot. >> >> > Sorry for so asking so late. I thought the error is resolved. > > I changed standard_conforming_strings to "off" in *postgresql.conf.* and > then I run *dbt2-pgsql-build-db -w 1 script again. * > But, I am still facing the same error. Please tel me what else I can do to > resolve the issue. > > Regards, > Rohit Goyal > >> -- >> Peter Geoghegan >> > > > > -- > Regards, > Rohit Goyal > -- Regards, Rohit Goyal
Re: [HACKERS] Error in running DBT2
Hi All, On Tue, May 13, 2014 at 9:44 PM, Peter Geoghegan wrote: > > On Tue, May 13, 2014 at 12:36 PM, Rohit Goyal wrote: > >> This pattern the above found many times. Please guide me through!!! >> > > IIRC, people have been working around this by setting > standard_conforming_strings to "off". It really ought to be fixed in a > principled way, though -- the real issue here is that dbt2 has severe > bit-rot. > > Sorry for so asking so late. I thought the error is resolved. I changed standard_conforming_strings to "off" in *postgresql.conf.* and then I run *dbt2-pgsql-build-db -w 1 script again. * But, I am still facing the same error. Please tel me what else I can do to resolve the issue. Regards, Rohit Goyal > -- > Peter Geoghegan > -- Regards, Rohit Goyal
[HACKERS] Error in running DBT2
Hi All, I am using centOS6 and after all confugration, I run the below command *dbt2-run-workload -a pgsql -d 120 -w 1 -o /tmp/result -c 10* *Error:* Stage 3. Processing of results... Killing client... waiting for server to shut down done server stopped Traceback (most recent call last): File "/tmp/dbt2/bin/dbt2-post-process", line 14, in import rpy2.robjects as robjects ImportError: No module named rpy2.robjects Test completed. Results are in: /tmp/result Please guide me !! :) Regards, Rohit Goyal -- Regards, Rohit Goyal
Re: [HACKERS] Error in running DBT2
Hi All, I am runnig dbt2 with last postgresql kit. pg9.4. I tried everything again after setting up awhole new machine again with ubuntu. Still facing the same error. I run the the *dbt2-pgsql-build-db -w 1 * but, after some time, I faced this error */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 45: [: c: unexpected operator* */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 53: [: c: unexpected operator* *unknown stored function type: c* and script ends!! Please guide me probable solution to get over it. Regards, Rohit On Wed, May 14, 2014 at 12:43 AM, Rohit Goyal wrote: > Hi Peter, > > I tried the solution suggested by you. Please problem still persists. > > I run the the *dbt2-pgsql-build-db -w 1 * > > but, after some time, I faced this error > > */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 45: [: c: > unexpected operator* > */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 53: [: c: > unexpected operator* > *unknown stored function type: c* > > and script ends!! > > moreover, I see that DB has been created and also 9 tables are there in > Database dbt2. Please suggest how to proceed. > > Regards, > rohit Goyal > > > > On Tue, May 13, 2014 at 9:44 PM, Peter Geoghegan wrote: > >> >> On Tue, May 13, 2014 at 12:36 PM, Rohit Goyal wrote: >> >>> This pattern the above found many times. Please guide me through!!! >>> >> >> IIRC, people have been working around this by setting >> standard_conforming_strings to "off". It really ought to be fixed in a >> principled way, though -- the real issue here is that dbt2 has severe >> bit-rot. >> >> -- >> Peter Geoghegan >> > > > > -- > Regards, > Rohit Goyal > -- Regards, Rohit Goyal
Re: [HACKERS] Error in running DBT2
Hi Peter, I tried the solution suggested by you. Please problem still persists. I run the the *dbt2-pgsql-build-db -w 1 * but, after some time, I faced this error */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 45: [: c: unexpected operator* */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 53: [: c: unexpected operator* *unknown stored function type: c* and script ends!! moreover, I see that DB has been created and also 9 tables are there in Database dbt2. Please suggest how to proceed. Regards, rohit Goyal On Tue, May 13, 2014 at 9:44 PM, Peter Geoghegan wrote: > > On Tue, May 13, 2014 at 12:36 PM, Rohit Goyal wrote: > >> This pattern the above found many times. Please guide me through!!! >> > > IIRC, people have been working around this by setting > standard_conforming_strings to "off". It really ought to be fixed in a > principled way, though -- the real issue here is that dbt2 has severe > bit-rot. > > -- > Peter Geoghegan > -- Regards, Rohit Goyal
Re: [HACKERS] Error in running DBT2
Hi Peter, On Tue, May 13, 2014 at 9:44 PM, Peter Geoghegan wrote: > > On Tue, May 13, 2014 at 12:36 PM, Rohit Goyal wrote: > >> This pattern the above found many times. Please guide me through!!! >> > > IIRC, people have been working around this by setting > standard_conforming_strings to "off". It really ought to be fixed in a > principled way, though -- the real issue here is that dbt2 has severe > bit-rot. > > Can you please elaborate, where I can make this changes :) ? Regards, Rohit Goyal > >
[HACKERS] Error in running DBT2
Hi All, I run the the *dbt2-pgsql-build-db -w 1 * but, after some time, I faced this error */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 45: [: c: unexpected operator* */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 53: [: c: unexpected operator* *unknown stored function type: c* and script ends!! Moreover, in between the scripts running, i was looing at the ouput i found something like Output directory of data files: current directory *Generating data files for 1 warehouse(s)...* *Generating item table data...* *BEGIN* *ERROR: invalid byte sequence for encoding "UTF8": 0xf9* *CONTEXT: COPY item, line 1* *ROLLBACK* This pattern the above found many times. Please guide me through!!! Regards, Rohit Goyal
[HACKERS] Running DBT2 on postgresql
Hi All, Please help me in running DBT2 on postgresql. I am doing it for the first time. I am facing error while running dbt2 test. I installed dbt2 by following "install" file. Below some final lines of terminal. Install the project... -- Install configuration: "" -- Installing: /home/abhi/dbt2_install/bin/dbt2-client -- Installing: /home/abhi/dbt2_install/bin/dbt2-datagen -- Installing: /home/abhi/dbt2_install/bin/dbt2-driver -- Installing: /home/abhi/dbt2_install/bin/dbt2-transaction-test -- Installing: /home/abhi/dbt2_install/bin/dbt2-generate-report -- Installing: /home/abhi/dbt2_install/bin/dbt2-get-os-info -- Installing: /home/abhi/dbt2_install/bin/dbt2-post-process -- Installing: /home/abhi/dbt2_install/bin/dbt2-run-workload -- Installing: /home/abhi/dbt2_install/bin/dbt2-sysstats -- Installing: /home/abhi/dbt2_install/bin/dbt2-plot-transaction-rate -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-build-db -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-check-db -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-create-db -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-create-indexes -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-create-tables -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-db-stat -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-drop-db -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-drop-tables -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-load-db -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-plans -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-start-db -- Installing: /home/abhi/dbt2_install/bin/dbt2-pgsql-stop-db After that I am following readme_postgresql file. But, I can't understand "A really quick howto. Edit bin/pgsql/pgsql_profile.in and follow the notes for the DBT2PGDATA and DBDATA directory. DBT2PGDATA is where the database directory will be created and DBDATA is where the database table data will be generated. Set environment variables, see examples/dbt2_profile." Please explain me the parameters pls. I have set export DBT2INSTALLDIR=/home/abhi/dbt2_install # change where you want install dbt2 export DBT2PGDATA=/home/abhi/project/pgsql/DemoDir # your postgres database directory export DBT2DBNAME=dbt2 # keep as it is I set above parameter before installation. Regards, Rohit Goyal -- Regards, Rohit Goyal
Re: [HACKERS] Runing DBT2 on Postgresql
On Thu, Apr 24, 2014 at 6:57 AM, Peter Geoghegan wrote: > On Wed, Apr 23, 2014 at 2:33 AM, Rohit Goyal wrote: > > I am trying to install dbt2 on postgresql database. > > > > cmake(configure) command work fine and but make command(build) give an > error > > given below. I have no idea about how to solve it > > ld has become less tolerant of certain flag orderings over time in > certain distros. The following tweak may be used as a quick-and-dirty > work around: > > diff --git a/CMakeLists.txt b/CMakeLists.txt > index 6a128e3..f6a796b 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > @@ -11,6 +11,7 @@ SET(DBT2_CLIENT bin/dbt2-client) > SET(DBT2_DATAGEN bin/dbt2-datagen) > SET(DBT2_DRIVER bin/dbt2-driver) > SET(DBT2_TXN_TEST bin/dbt2-transaction-test) > +set(CMAKE_EXE_LINKER_FLAGS "-Wl,--no-as-needed") > > # > # Check for large file support by using 'getconf'. > > > -- > Peter Geoghegan > Hi Peter/All, I installed the dbt-2 benchmark by implementing the change you mentioned. Now, I am trying to follow the readme_postgresql for running test on postgresql, but facing an error in understanding changes in dbt2_profile. Can you explain my next step: Could you tel me what to write in how to set environment variables, see examples/dbt2_profile and proceed further. Please help me and give me some link to run the test. I cant find bin/pgsql/pgsql_profile.in file also and when i tried to "Create a 1 warehouse database by running bin/pgsql/dbt2-pgsql-build-db and put the data files in '/tmp/data': dbt2-pgsql-build-db -w 1" I got the error that dbt2-pgsql-build-db not found. Please guide!! -- Regards, Rohit Goyal
Re: [HACKERS] Runing DBT2 on Postgresql
Hi Everyone, Sorry, i got this now. :) Regards, Rohit Goyal On Thu, Apr 24, 2014 at 4:24 PM, Rohit Goyal wrote: > Hi Peter/All, > > Can you please elaborate a bit in details the steps? where are how to run > these steps? > > Regards, > Rohit Goyal > > > On Thu, Apr 24, 2014 at 6:57 AM, Peter Geoghegan wrote: > >> On Wed, Apr 23, 2014 at 2:33 AM, Rohit Goyal wrote: >> > I am trying to install dbt2 on postgresql database. >> > >> > cmake(configure) command work fine and but make command(build) give an >> error >> > given below. I have no idea about how to solve it >> >> ld has become less tolerant of certain flag orderings over time in >> certain distros. The following tweak may be used as a quick-and-dirty >> work around: >> >> diff --git a/CMakeLists.txt b/CMakeLists.txt >> index 6a128e3..f6a796b 100644 >> --- a/CMakeLists.txt >> +++ b/CMakeLists.txt >> @@ -11,6 +11,7 @@ SET(DBT2_CLIENT bin/dbt2-client) >> SET(DBT2_DATAGEN bin/dbt2-datagen) >> SET(DBT2_DRIVER bin/dbt2-driver) >> SET(DBT2_TXN_TEST bin/dbt2-transaction-test) >> +set(CMAKE_EXE_LINKER_FLAGS "-Wl,--no-as-needed") >> >> # >> # Check for large file support by using 'getconf'. >> >> >> -- >> Peter Geoghegan >> > > > > -- > Regards, > Rohit Goyal > -- Regards, Rohit Goyal
Re: [HACKERS] Runing DBT2 on Postgresql
Hi Peter/All, Can you please elaborate a bit in details the steps? where are how to run these steps? Regards, Rohit Goyal On Thu, Apr 24, 2014 at 6:57 AM, Peter Geoghegan wrote: > On Wed, Apr 23, 2014 at 2:33 AM, Rohit Goyal wrote: > > I am trying to install dbt2 on postgresql database. > > > > cmake(configure) command work fine and but make command(build) give an > error > > given below. I have no idea about how to solve it > > ld has become less tolerant of certain flag orderings over time in > certain distros. The following tweak may be used as a quick-and-dirty > work around: > > diff --git a/CMakeLists.txt b/CMakeLists.txt > index 6a128e3..f6a796b 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > @@ -11,6 +11,7 @@ SET(DBT2_CLIENT bin/dbt2-client) > SET(DBT2_DATAGEN bin/dbt2-datagen) > SET(DBT2_DRIVER bin/dbt2-driver) > SET(DBT2_TXN_TEST bin/dbt2-transaction-test) > +set(CMAKE_EXE_LINKER_FLAGS "-Wl,--no-as-needed") > > # > # Check for large file support by using 'getconf'. > > > -- > Peter Geoghegan > -- Regards, Rohit Goyal
[HACKERS] Runing DBT2 on Postgresql
Hi All, I am trying to install dbt2 on postgresql database. cmake(configure) command work fine and but make command(build) give an error given below. I have no idea about how to solve it I have set export LD_LIBRARY_PATH=home/abhi/project/lib:$LD_LIBRARY_PATH dbt2pgdata = /home/abhi/project/pgsql/DemoDir/ export PATH=$HOME/project/bin:$PATH CMakeFiles/bin/dbt2-client.dir/src/client.o: In function `startup': /home/abhi/project/dbt2/src/client.c:344: undefined reference to `pthread_create' CMakeFiles/bin/dbt2-client.dir/src/listener.o: In function `init_listener': /home/abhi/project/dbt2/src/listener.c:94: undefined reference to `pthread_attr_setstacksize' /home/abhi/project/dbt2/src/listener.c:99: undefined reference to `pthread_create' CMakeFiles/bin/dbt2-client.dir/src/db_threadpool.o: In function `db_threadpool_init': /home/abhi/project/dbt2/src/db_threadpool.c:226: undefined reference to `pthread_attr_setstacksize' /home/abhi/project/dbt2/src/db_threadpool.c:230: undefined reference to `pthread_create' CMakeFiles/bin/dbt2-client.dir/src/common.o: In function `get_think_time': /home/abhi/project/dbt2/src/common.c:149: undefined reference to `log' CMakeFiles/bin/dbt2-client.dir/src/libpq/dbc_common.o: In function `commit_transaction': /home/abhi/project/dbt2/src/libpq/dbc_common.c:25: undefined reference to `PQexec' /home/abhi/project/dbt2/src/libpq/dbc_common.c:26: undefined reference to `PQresultStatus' /home/abhi/project/dbt2/src/libpq/dbc_common.c:27: undefined reference to `PQerrorMessage' /home/abhi/project/dbt2/src/libpq/dbc_common.c:29: undefined reference to `PQclear' CMakeFiles/bin/dbt2-client.dir/src/libpq/dbc_common.o: In function `_connect_to_db': /home/abhi/project/dbt2/src/libpq/dbc_common.c:44: undefined reference to `PQconnectdb' /home/abhi/project/dbt2/src/libpq/dbc_common.c:45: undefined reference to `PQstatus' /home/abhi/project/dbt2/src/libpq/dbc_common.c:48: undefined reference to `PQerrorMessage' /home/abhi/project/dbt2/src/libpq/dbc_common.c:49: undefined reference to `PQfinish' CMakeFiles/bin/dbt2-client.dir/src/libpq/dbc_common.o: In function `_disconnect_from_db': /home/abhi/project/dbt2/src/libpq/dbc_common.c:58: undefined reference to `PQfinish' Please guide me through !! Regards, Rohit Goyal
Re: [HACKERS] HOT Update || want to use a different page for updated tuple
On Fri, Apr 4, 2014 at 2:03 PM, Heikki Linnakangas wrote: > On 04/04/2014 02:53 PM, Rohit Goyal wrote: > >> Hi All, >> >> I was comparing postgresql performance and was just curious about >> performance in case i want to store the updated index tuple version on a >> different page. >> I was looking into the code of heapam.c, but was not able to find loop >> which i should remove so that postgresql use a different buffer for >> updated >> index tuple version. >> > > Did you mean to: > > 1. Force the old and new tuple to always be stored on different pages? > > Hack heap_update so that it chooses a new page. I think you'll also need > to hack RelationGetBufferForTuple to not return the same buffer. > > Can you explain a bit more from inside the code where i have to make the changes :)? my focus is just to store the index tuple of an updated tuple onto a different page or on a different buffer. > 2. Disable the HOT optimization, so that HOT is not used even if the old > and new tuple are stored on the same page? > > In heap_update, force satisfies_hot variable to false. > If I do change only this variable to false. then does it means, i have disabled the hot update.? > > 3. Allow HOT to be used even though the old and new tuple are stored on > different pages? > > This is not feasible.. > > - Heikki > thanks in advance.!! :) Regards, Rohit Goyal -- Regards, Rohit Goyal
[HACKERS] HOT Update || want to use a different page for updated tuple
Hi All, I was comparing postgresql performance and was just curious about performance in case i want to store the updated index tuple version on a different page. I was looking into the code of heapam.c, but was not able to find loop which i should remove so that postgresql use a different buffer for updated index tuple version. Please guide me, about how i can achieve this. Regards, Rohit Goyal
[HACKERS] Disable hot-update functionality
Hi All, Is there any ways by which i can disable the hot-update functionality? -- Regards, Rohit Goyal
Re: [HACKERS] Fwd: Request for error explaination || Adding a new integer in indextupleData Structure
> > Hello, >> >> I started all the process again and configured my eclipse with raw >> postgresql code. First change i made in the code is >> >> I added *int i; *in indextupleData structure in itup.h. >> > > You should show us *exactly* where you added it. (Doing so is what "diff" > was developed for, so please use that or a similar tool.) > > >> >> I got the same error message. Please help me to understand and solve the >> issue. I want to add an integer in index tuple for btree. >> > > The data from IndexTupleData is written to disk, and then read back in > again. Did you initdb a new database cluster after you made your change? > If you did the initdb with the original code, and then tried to point your > new code at the old disk files, that is very unlikely to work, as format is > now different. > > Cheers, > > Jeff > Hi Jeff and Tom, Thanks you so much. I was making the mistake you mentioned in the last mail. :) Regards, Rohit Goyal -- Regards, Rohit Goyal
Re: [HACKERS] Fwd: Request for error explaination || Adding a new integer in indextupleData Structure
> > Rohit Goyal writes: >> > Hi All, >> > I was trying to modify indextupledata structure by adding an integer >> > variable. ButI faced an error message "psql: FATAL: could not find >> tuple >> > for opclass 10032". >> >> > Could anyone please help me in resolving this issue. >> >> You broke a system catalog index. Without seeing what you changed and >> where, it's impossible to say just how, but that's the bottom line. >> >> This is the first line which i want to write in the code. Can you tel me > how to add an integer in indextupledata structure if I have a fresh system > with no changes? > > In recent versions of PG, opclass 10032 is btree name_ops (unless you've >> also added/removed system catalog entries), which is a pretty plausible >> thing to be one of the first indexscanned fetches during relcache.c >> initialization, so I don't think there's any great significance in this >> particular error message. It's likely that you broke *all* indexscans >> not just one specific one. > > I am not sure how i can break all indexscan as this is the first line I > wrote in the code. > >> >> > regards, tom lane > > Hello, I started all the process again and configured my eclipse with raw postgresql code. First change i made in the code is I added *int i; *in indextupleData structure in itup.h. I got the same error message. Please help me to understand and solve the issue. I want to add an integer in index tuple for btree. Regards, Rohit Goyal
[HACKERS] Fwd: Request for error explaination || Adding a new integer in indextupleData Structure
Hi All, I was trying to modify indextupledata structure by adding an integer variable. ButI faced an error message "psql: FATAL: could not find tuple for opclass 10032". Could anyone please help me in resolving this issue. Regards, Rohit Goyal -- Regards, Rohit Goyal
Re: [HACKERS] Call flow of btinsert(PG_FUNCTION_ARGS)
On Tue, Nov 19, 2013 at 11:52 AM, Craig Ringer wrote: > On 11/19/2013 06:22 PM, Rohit Goyal wrote: > > Hi All, > > > > I was reading b tree index code and I wanted to the know the calling > > function which calls btinsert > > < > http://doxygen.postgresql.org/nbtree_8c.html#a2b69e4abd6e38fbb317a503b8cf6e00a > >(PG_FUNCTION_ARGS > > < > http://doxygen.postgresql.org/fmgr_8h.html#adf4dec9b7d23f1b4c68477affde8b7ff > >) > > function in nbtree.c file. > > It's specified as a member of the btree index access method in > include/catalog/pg_am.h . All invocations are done via index access > method lookups; see the documentation for pg_catalog.pg_am, index access > methods, etc. > > It's all abstracted so the core code doesn't know or care whether it's > using a b-tree, GiST, or something completely different. Relevant lines: > > $ git grep '\' > > backend/access/nbtree/nbtree.c:btinsert(PG_FUNCTION_ARGS) > > include/catalog/pg_am.h:DATA(insert OID = 403 ( btree 5 2 t f > t t t t t t f t t 0 btinsert btbeginscan btgettuple btgetbitmap btrescan > btendscan btmarkpos btrestrpos btbuild btbuildempty btbulkdelete > btvacuumcleanup btcanreturn btcostestimate btoptions )); > > include/catalog/pg_proc.h:DATA(insert OID = 331 ( btinsert >PGNSP PGUID 12 1 0 0 0 f f f f t f v 6 0 16 "2281 2281 2281 2281 2281 > 2281" _null_ _null_ _null_ _null_ btinsert _null_ _null_ _null_ )); > > > See http://www.postgresql.org/docs/current/static/catalog-pg-am.html > > In psql: > > regress=> \x > regress=> select * from pg_am where amname = 'btree'; > -[ RECORD 1 ]---+ > amname | btree > ... snip ... > aminsert| btinsert > ambeginscan | btbeginscan > ... snip ... > > > pg_catalog.pg_index rows refer to a vector of pg_opclass rows, one per > index column, and that in turns refers to pg_am via the opcmethod column. > > http://www.postgresql.org/docs/current/static/catalog-pg-index.html > http://www.postgresql.org/docs/current/static/catalog-pg-am.html > http://www.postgresql.org/docs/current/static/catalog-pg-opclass.html > > > > See: > > This query demonstrates the relationship. It'd be smarter to properly > separate out each column but I couldn't be bothered, so it just prints > one row per index column in multicolumn indexes, not preserving order. > > > regress=> select c.relname, oc.opcname, am.amname, am.aminsert > from pg_index i > inner join pg_class c on (i.indexrelid = c.oid) > inner join pg_opclass oc ON (oc.oid = ANY(i.indclass)) > inner join pg_am am ON (oc.opcmethod = am.oid); > > > All the executor knows is how to ask, eg "for this column of this index, > what opclass was specified, and for that opclass, what is the oid of the > function to perform the insert access-method operation". > > It has no idea it's calling btinsert. > > Try creating a GiST or GIN index and compare. It might help clarify things. > > > I want to understand the code flow for b tree index > > It'll take time, but stepping/jumping through it with a debugger will > probably be a good way to do that. Connect a psql session, run "SELECT > pg_backend_pid()", attach gdb to that pid, set breakpoints at the entry > points of interest, cont, then run a command in psql that'll trigger one > of the breakpoints. From there you can step through and watch what it > does, using "next" to skip over function invocations you aren't > interested in and "step" to enter ones you are. > > -- > Craig Ringer http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services > Please tel me how to attached gdb to pid and debug. Also what I understood from your reply is that everything is abstracted, so am doesnot care abt backend implementation. Now, I want to modify B tree indexing scheme. So, which files I should focus ? Regards, Rohit Goyal
Re: [HACKERS] Call flow of btinsert(PG_FUNCTION_ARGS)
On Tue, Nov 19, 2013 at 11:26 AM, Peter Geoghegan wrote: > On Tue, Nov 19, 2013 at 2:22 AM, Rohit Goyal wrote: > > > > I was reading b tree index code and I wanted to the know the calling > > function which calls btinsert(PG_FUNCTION_ARGS) function in nbtree.c > file. > > Moreover, my goal behind reading this function was to check how tuple is > > inserted in btree. > > I want to understand the code flow for b tree index > > Take a look at this: > > http://www.postgresql.org/docs/9.3/static/index-functions.html > > > -- > Peter Geoghegan > thanks for reply. :) Moreover, I am so confused about reading the nbtree folder c files or the file which you gave. For example, If I want to read how the data is inserted in B tree index. Should I read your link or the btinsert(PG_FUNCTION_ARGS) in nbtree.c file in nbtree folder. In future, if i want to modify btree indexing approach, then which files to look for.? -- Regards, Rohit Goyal
[HACKERS] Call flow of btinsert(PG_FUNCTION_ARGS)
Hi All, I was reading b tree index code and I wanted to the know the calling function which calls btinsert<http://doxygen.postgresql.org/nbtree_8c.html#a2b69e4abd6e38fbb317a503b8cf6e00a> (PG_FUNCTION_ARGS<http://doxygen.postgresql.org/fmgr_8h.html#adf4dec9b7d23f1b4c68477affde8b7ff>) function in nbtree.c file. Moreover, my goal behind reading this function was to check how tuple is inserted in btree. I want to understand the code flow for b tree index Please help!! -- Regards, Rohit Goyal
[HACKERS] Compile and test in netbeans
Hi All, I want to modify postgresql code for the first time. I tried to use netbeans to compile the code. Now, I wanted to print something on console when I create a table or an index. I am printing just for testing purpose to get a feel of code. Please tel me the procedure to run and print using printf(). -- Regards, Rohit Goyal
Re: [HACKERS] Information about Access methods
On Wed, Nov 13, 2013 at 2:42 AM, Craig Ringer wrote: > On 11/13/2013 06:36 AM, Rohit Goyal wrote: > > Hi, > > Thanks for document. It was really helpful. > > Now, as index folder contains basically interface for all index type, do > > I need to change it, if I want to modify only b tree index algorithm? > > > > Do I need to change only btree index files or Do i have to change index > > folder files like indexam.c > > Without knowing what you are trying to do, it's really hard to answer > that usefully. In general if you want to modify b-tree indexes you'd > only have to modify the b-tree index implementation. If you want to add > new capabilities or features to the indexing system in general then use > them in b-tree, you'd probably also have to modify the access method > interface. > > If you aren't just doing this for learning/research you should explain > in detail what you are trying to do. There is no point spending lots of > time creating a patch that would never get accepted into the PostgreSQL > core, so you have to maintain it forever more... > > -- > Craig Ringer http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services > Hi, Thanks for update. Actually, I am doing it for learning. I want to modify B tree approach for insert and update operation to perform in a more faster way. Could you please suggest something about abt update operation of B tree index. Regards, Rohit Goyal
Re: [HACKERS] Information about Access methods
Hi, Thanks for document. It was really helpful. Now, as index folder contains basically interface for all index type, do I need to change it, if I want to modify only b tree index algorithm? Do I need to change only btree index files or Do i have to change index folder files like indexam.c Please confirm. :) Regards, Rohit On Tue, Nov 12, 2013 at 1:22 PM, Craig Ringer wrote: > On 11/12/2013 07:00 PM, Rohit Goyal wrote: > > > > Inside access methods in Backend, I found "Index" folder and "Btree" > > folder but i don't know which code to look for. i want to change > > something in B tree implementation. > > > > Start here: > > http://www.postgresql.org/docs/current/static/indexam.html > > -- > Craig Ringer http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services > -- Regards, Rohit Goyal
[HACKERS] Information about Access methods
Hi, I am looking in Postgresql code for the first time. Specifically in Indexing scheme. I want to focus on B tree for my testing. Inside access methods in Backend, I found "Index" folder and "Btree" folder but i don't know which code to look for. i want to change something in B tree implementation. Please guide. Regards, Rohit Goyal
Re: [HACKERS] Fwd: Test of Algorithm || Indexing Scheme
Hi, Actually, I want to test an algorithm in which I will use store tuple_id in some other data structure and value of the key of index will contain some random created by me. Could you please tel me which file need to be change for it. you said something about usage of GiST opclass. Can you please elaborate ur opinion in detail. It would be of a great help to me. Regards, Rohit On Mon, Nov 11, 2013 at 6:16 PM, Rohit Goyal wrote: > Hi, > Can you pls tel me how can achieve which sound pretty exotic to you. > > Can u please share some relevant documents. I havw no clue from where to > start. > > Regards > Rohit > On 11.11.2013 14:12, Rohit Goyal wrote: > >> Hi, >> Thanks for reply. >> I actually want to make some changes in all operations on index like >> insert, update, delete. >> for example, i want store a new customized value for every key inserted in >> b tree instead of storing Tuple Id as value. >> > > That sounds pretty exotic, I can't imagine what good the index will do if > it doesn't contain tuple IDs. > > Can you also pls explain me more about appoach to follow to use my >> algorithm as GiST opclass. I dont have much knowledge abt this also? >> > > See user manual, and the examples in the server code and contrib. > > - Heikki > -- Regards, Rohit Goyal
Re: [HACKERS] Fwd: Test of Algorithm || Indexing Scheme
Hi, Can you pls tel me how can achieve which sound pretty exotic to you. Can u please share some relevant documents. I havw no clue from where to start. Regards Rohit On 11.11.2013 14:12, Rohit Goyal wrote: > Hi, > Thanks for reply. > I actually want to make some changes in all operations on index like > insert, update, delete. > for example, i want store a new customized value for every key inserted in > b tree instead of storing Tuple Id as value. > That sounds pretty exotic, I can't imagine what good the index will do if it doesn't contain tuple IDs. Can you also pls explain me more about appoach to follow to use my > algorithm as GiST opclass. I dont have much knowledge abt this also? > See user manual, and the examples in the server code and contrib. - Heikki
Re: [HACKERS] Fwd: Test of Algorithm || Indexing Scheme
Hi, Thanks for reply. I actually want to make some changes in all operations on index like insert, update, delete. for example, i want store a new customized value for every key inserted in b tree instead of storing Tuple Id as value. Can you also pls explain me more about appoach to follow to use my algorithm as GiST opclass. I dont have much knowledge abt this also? Regards, Rohit Goyal On Mon, Nov 11, 2013 at 12:31 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 11.11.2013 13:19, Rohit Goyal wrote: > >> Hi All, >> >> I want to implement and test my indexing approach. >> I would like to know which are the main files to look for b tree indexing >> scheme modification. >> >> It would be great, if can share some helpful links which are needed to >> understand how to modify and test an indexing scheme in postgresql. >> > > You didn't give any details on what kind of changes you want to make, so > hard to tell. The code for the btree access method is in > src/backend/access/nbtree. See the README in that directory for the gory > details. > > Also consider implementing your algorithm as a GiST opclass, instead of > changing the internals. > > - Heikki > -- Regards, Rohit Goyal
[HACKERS] Fwd: Test of Algorithm || Indexing Scheme
Hi All, I want to implement and test my indexing approach. I would like to know which are the main files to look for b tree indexing scheme modification. It would be great, if can share some helpful links which are needed to understand how to modify and test an indexing scheme in postgresql. Regards, Rohit Goyal
