Re: [HACKERS] Alpha release this week?
Alvaro Herrera wrote: Josh Berkus escribió: And snaphsots have more compile dependancies than releases do. As far as I know, a snapshot is identical to a release in that regard. If they are not, that's a bug and we can fix it before weekend. yeah - snapshots do have the same compile time dependencies as release tarballs have. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parallel pg_dump for 9.1
Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: On 3/29/10 7:46 AM, Joachim Wieland wrote: I actually assume that whenever people are interested in a very fast dump, it is because they are doing some maintenance task (like migrating to a different server) that involves pg_dump. In these cases, they would stop their system anyway. Actually, I'd say that there's a broad set of cases of people who want to do a parallel pg_dump while their system is active. Parallel pg_dump on a stopped system will help some people (for migration, particularly) but parallel pg_dump with snapshot cloning will help a lot more people. I doubt that. My thought about it is that parallel dump will suck enough resources from the source server, both disk and CPU, that you would never want to use it on a live production machine. Not even at 2am. And your proposed use case is hardly a broad set in any case. Thus, Joachim's approach seems perfectly sane from here. I certainly don't see that there's an argument for spending 10x more development effort to pick up such use cases. Another question that's worth asking is exactly what the use case would be for parallel pg_dump against a live server, whether the snapshots are synchronized or not. You will not be able to use that dump as a basis for PITR, so there is no practical way of incorporating any changes that occur after the dump begins. So what are you making it for? If it's a routine backup for disaster recovery, fine, but it's not apparent why you want max speed and to heck with live performance for that purpose. I think migration to a new server version (that's too incompatible for PITR or pg_migrate migration) is really the only likely use case. I really doubt that - on fast systems pg_dump is completely CPU bottlenecked and typical 1-2U typical hardware you get these days has 8-16 cores so simply dedicating a few cores to dumping the database during quieter times is very realistic. Databases are growing larger and larger and the single threaded nature of pg_dump makes it very hard to even stay withing reasonable time limits for doing the backup. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] I am interested in the MERGE command implementation as my gSoC project
To whom may concern, My name is Zhai Boxuan, a student from China. I am now a Master Student of National University of Singapore. And, before I came to Singapore, I have got another master degree in Wuhan University. In that period, I focus mainly on implementing a novel Object-oriented database model on the postgresql 7.4. I am have great interest on the projects provided by postgres in google Summer Code of this year. I think the MERGE command in TO DO list is a suitable topic for me. I have read some infor about the MERGE command, which has not been implemented yet in Postgres 8. I considered the problem and have a brief plan for the jobs. 1 We need to update the backend/parser/gram.y for adding the SQL style MERGE command in the parser. I can do this, since it is similar to what I was doing in China. One new “MergeStmt” structure should be designed to hold the transformed command information. The structure definitely need: one SelectStmt to hold the subquery of the source table, a list of expressions for the MATCH condition. Yet some other expression lists are needed for specifying the additional match and/or not match conditions. It is relatively easy to implement since we can reuse many components of the SELECT command. 2. In the Analyze.c file we need to add a function to transform this MergeStmt into a Query node. It is necessary to add a new command type for MERGE, which is a plannable command. We need to check the semantics correctness of the statement. What I am thinking about is to combine the target table and the source table as a whole SELECT query. If there is no NOT MATCH option, we can generate a normal query node of something like “SELECT * FROM target, source WHERE match-condition;” or , We have to do a cross join if we want to handle some NOT MATCH actions, which will do a query like “SELECT * FROM target, source;” The benefit is that we can almost fully reuse the rewriter and planner to transform this generated query as an executor-accepting structure. 3. A plan is need for the query. The planner should accept this new plannable command. However, as motioned above, the real work will be: do a traditional query plan on the formatted select query based on the target and source table. Then pack this plan with a outer planner node, which is designed for MERGE command specifically. 4. How to execute the query? I am still not very clear. The basic idea is for each returned tuple of the select query we generated above (the tuple contains all the attributes in both source and target table) we can test it with MATCH and/or NOT MATCH conditions, and do corresponding actions base the testing result. I believe there are some problems will encounter especially for the transaction things. And I am also not sure about whether the UPDATE, INSERT and DELETE operations for previous output tuple will affect the remaining join processing. Hope you can help me on improving this rough idea. Or, if you are not convenient, please kindly forward this letter to who may concern it. Thank you very much! Yours Boxuan -- Best Wishes Yours Boxuan -- Best Wishes Yours Boxuan
Re: [HACKERS] I am interested in the MERGE command implementation as my gSoC project
On 30 March 2010 08:26, Zhai Boxuan bxz...@gmail.com wrote: I have read some infor about the MERGE command, which has not been implemented yet in Postgres 8. I considered the problem and have a brief plan for the jobs. You may find these to be some help if you haven't already read them: http://archives.postgresql.org/pgsql-hackers/2008-04/msg01890.php http://archives.postgresql.org/pgsql-hackers/2008-04/msg01157.php Regards Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Streaming replication document improvements
Hi, The attached patch improves the documents about SR as follows: * Improve the description about the setup procedure of SR. o Add the link to the recovery.conf parameter if needed. o Add the example of recovery.conf setting. * Document what should be monitored for avoiding the disk full failure on the primary. * Clarify how to safely take an incrementally updated backups. * Define the recovery.conf parameters as an index term. * Describe how to use recovery.conf. Any comments are welcome. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center replication_doc_improvement_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
pending patch: Re: [HACKERS] HS/SR and smart shutdown
On Mon, Feb 1, 2010 at 11:49 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Jan 30, 2010 at 12:54 PM, Fujii Masao masao.fu...@gmail.com wrote: HOWEVER, I do believe this is an issue we could live with for 9.0 if it's going to lead to a whole lot of additional debugging of SR. But if it's an easy fix, it'll avoid a lot of complaints on pgsql-general. I think that the latter statement is right. Though we've not reached consensus on smart shutdown during recovery yet, I wrote the patch that changes its behavior: shut down the server (including the startup process and the walreceiver) as soon as all read-only connections have died. The code is also available in the 'replication' branch in my git repository. And, let's discuss whether something like the attached patch is required for v9.0 or not. I rebased the patch to HEAD. Is the patch still required for 9.0? If not, I'd remove the open item of the smart shutdown during recovery. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center new_smart_shutdown_20100330.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
pending patch: Re: [HACKERS] Streaming replication and pg_xlogfile_name()
On Wed, Mar 3, 2010 at 11:03 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Mar 2, 2010 at 10:52 PM, Fujii Masao masao.fu...@gmail.com wrote: It's not clear what it should return, a TLI corresponding the filename of the WAL segment the record was replayed from, so that you can use pg_xlogfile_name() to find out the filename of the WAL segment being replayed, or the accurate TLI of the record being replayed. I'm leaning towards the latter, it feels more correct and accurate, but you could argue for the former too. In any case, it needs to be well-defined. I agree with you that the latter is more correct and accurate. The simple fix is updating the lastPageTLI with the CheckPoint-ThisTimeLineID when replaying the shutdown checkpoint record. Though we might need to use new variable to keep the last applied timeline instead of the lastPageTLI. Here is the revised patch. I used new local variable instead of lastPageTLI to track the tli of last applied record. It is updated with the tli of the log page header when reading the page, and with the tli of the checkpoint record when replaying the checkpoint shutdown record that changes the tli. So pg_last_xlog_replay_location() can return the accurate tli of the last applied record. I rebased the patch to HEAD. Should I think that the patch has been rejected because it has remained pending for about one month? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center extend_format_of_recovery_info_funcs_v5.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UUIDs generated using ossp-uuid on windows not unique
On Tue, Jun 17, 2008 at 11:17 AM, Hiroshi Saito z-saito(at)guitar(dot)ocn(dot)ne(dot)jp wrote: Hi. Please this. http://winpg.jp/~saito/pg_work/OSSP_win32/msvc/ Ahh, a makefile - that makes things somewhat clearer :-) Thanks - I've updated the build machine so future releases will include this fix. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com It seems that link http://winpg.jp/~saito/pg_work/OSSP_win32/msvc/ is no longer available. I am working on uuid-ossp contrib module, I want to share my findings with you.Thanks. I have downloaded uuid-1.6.2.tar.gz from http://www.ossp.org/pkg/lib/uuid/ . It successfully built with MinGW gcc and made libuuid.a, I renamed it to uuid.lib to build uuid-ossp contrib module with visual studio 2005. uuid-ossp give the following linker error with VC2005 i.e. Linking... Creating library Release\uuid-ossp\uuid-ossp.lib and object Release\uuid-ossp\uuid-ossp.exp uuid.lib(uuid_time.o) : error LNK2019: unresolved external symbol ___divdi3 referenced in function _uuid_time_gettimeofday uuid.lib(uuid_time.o) : error LNK2019: unresolved external symbol ___moddi3 referenced in function _uuid_time_gettimeofday uuid.lib(uuid_str.o) : error LNK2019: unresolved external symbol ___umoddi3 referenced in function _fmtint uuid.lib(uuid_str.o) : error LNK2019: unresolved external symbol ___udivdi3 referenced in function _fmtint .\Release\uuid-ossp\uuid-ossp.dll : fatal error LNK1120: 4 unresolved externals MinGW libgcc.a contains these symbols and these routines are not implemented in c ( its assembly code to make it fast ) so I extracted _divdi3.o _moddi3.o _umoddi3.o _udivdi3.o (we can add these additional files to uuid.lib to make build easy) from libgcc.a and link it with uuid-ossp contrib. I have tested generated uuid-ossp contrib it gives the following results that seems fine i.e. edb=# select uuid_generate_v4(); uuid_generate_v4 -- 20ff31ce-77a8-4db6-ac7f-3b55380e46ac (1 row) edb=# edb=# select uuid_generate_v4() edb-# ; uuid_generate_v4 -- e58c46d5-7161-47ec-8006-f905c6b66a1f (1 row) Any thoughts on this ?. Best Regards, Asif Naeem EnterpriseDB Pakistan: http://www.enterprisedb.com _ Hotmail: Free, trusted and rich email service. https://signup.live.com/signup.aspx?id=60969 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parallel pg_dump for 9.1
On tis, 2010-03-30 at 08:39 +0200, Stefan Kaltenbrunner wrote: on fast systems pg_dump is completely CPU bottlenecked Might be useful to profile why that is. I don't think pg_dump has historically been developed with CPU efficiency in mind. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC
On mån, 2010-03-29 at 17:03 -0700, Josh Berkus wrote: The idea crudely is the following: I want to implement a image database system: This will have the following quality: This seems fine, but I believe it's been done before. You might want to search for existing projects, and make an improvement on an existing project rather than starting from scratch. You mean INGRES, the Interactive Graphics Retrieval System? ;-) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parallel pg_dump for 9.1
On Tue, 30 Mar 2010 13:01:54 +0200, Peter Eisentraut pete...@gmx.net wrote: On tis, 2010-03-30 at 08:39 +0200, Stefan Kaltenbrunner wrote: on fast systems pg_dump is completely CPU bottlenecked Might be useful to profile why that is. I don't think pg_dump has historically been developed with CPU efficiency in mind. Already done that (I had posted some WIP patches to speed up COPY, hopefully I'll have time to finish those one day ;) Most of the time spent in the postmaster process during COPY TO is in the datum - string functions. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parallel pg_dump for 9.1
Peter Eisentraut wrote: On tis, 2010-03-30 at 08:39 +0200, Stefan Kaltenbrunner wrote: on fast systems pg_dump is completely CPU bottlenecked Might be useful to profile why that is. I don't think pg_dump has historically been developed with CPU efficiency in mind. It's not pg_dump that is the problem - it is COPY that is the limit. In my specific case als the fact that a lot of the columns are bytea adds to the horrible CPU overhead (fixed in 9.0). Still our bulk load unload performance is still way slower on a per core comparision than a lot of other databases :( Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] keep ppport.h in sync on all branches
Some recent off-list discussion has suggested that we should keep plperl's copy of ppport.h in sync on all branches, just as we do with timezone data files. So when we update it on HEAD we should simultaneously update the back branches. It's a derived file, produced by the perl module Devel::PPPort, so there is really no maintenance effort involved. In general I think this is a good idea. The whole idea of ppport.h is to aid in backwards compatibility. Here is some info from (my version of) the Devel::PPPort docs: Perl’s API has changed over time, gaining new features, new functions, increasing its flexibility, and reducing the impact on the C namespace environment (reduced pollution). The header file written by this module, typically ppport.h, attempts to bring some of the newer Perl API features to older versions of Perl, so that you can worry less about keeping track of old releases, but users can still reap the benefit. [...] Currently, Perl versions from 5.003 to 5.10.0 are supported. (Of course, we don't want to support anything nearly as old as 5.003. Perl 5.6.0 was released in 2000, so I wouldn't be inclined to look kindly on complaints about perl releases before that, since it's well before the release date of the earliest live branch we have. But that doesn't concern us here.) That means if we keep this file up to date in all branches, we can be much less worried about whether backporting a patch, as for example Tom recently did here: http://git.postgresql.org/gitweb?p=postgresql.git;a=commit;h=ef0ec92a57e565a668851f23a894c4b96d60fec8, might break a build made with an older version of perl. As we recently did a significant upgrade of ppport.h on HEAD, I therefore propose to apply that change to all the live branches. Another point: as this is a derived file, I think it should be excluded from pgindent runs. Comments? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Questions about 9.0 release note
On Tue, Mar 30, 2010 at 1:37 AM, Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote: Hi, I have some questions about 9.0 release note. I'd like to work for some of them if required. Comments welcome. * Allow per-tablespace sequential and random page cost variables (seq_page_cost/(random_page_cost)) via ALTER TABLESPACE ... SET/RESET ^ ^ Are those parentheses around random_page_cost intentional? They seems to mean just (seq_page_cost and random_page_cost). * EXCLUDE constraints has no tags to be linked. The item in release note is pointing the following page, http://developer.postgresql.org/pgdocs/postgres/sql-createtable.html#SQL-CREATETABLE-DESCRIPTION but the actual description about EXCLUDE constraints are in the subentry of Parameters section. Can we add a tag to varlistentry for EXCLUDE? * EXCLUDE constraints is not indexed from the Index page. Should we have for it? Unique Constraints have a section for them: http://developer.postgresql.org/pgdocs/postgres/ddl-constraints.html#AEN2431 I think I agree with all of these suggestions. Also maybe it should say CREATE TABLE ... CONSTRAINT ... EXCLUDE rather than CREATE TABLE CONSTRAINT ... EXCLUDE. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC
As Jeff Davis pointed out, I followed the modification he suggested and now I want to have a basic matching only. I think atleast the fingerprint processing can be done in summer (if not the image processing). Is it a good GSoC project now? Waiting for a reply eagerly Thanks and regards, Anindya Jyoti Roy Sophomore CSE Btech IIT Kanpur On 30-Mar-10 6:33 AM, Jeff Davis wrote: On Tue, 2010-03-30 at 01:33 +0530, Anindya Jyoti Roy wrote: I have some idea of implementing am image database system, which will let you search against the image and fingerprint: The idea crudely is the following: I want to implement a image database system: This will have the following quality: 1 will store image along with the other attributes 2 the database search engine will be able to search for image also 3 it will list the matching images in the order of degree of match. 4 in this matching system I will likely use the system of dividing the image into important parts and match them. 5 The database will also contain fingerprints, that may be the primary key. 6 it will match the finger prints on the basis of the nodes and then making a bitmap of it and using the coordinates with some error it will match them 7 to include some accuracy in case of the angle of picture in case of search of equality we will introduce some logical amount of error. 8 this project can be coded by me as I have some previous experience in this type of project. 9 this will make a definitly good search engine as well as innovative and uncommon. If you plan to spend your time refining complex image processing algorithms, or learning about image processing as you go, I think the project will be impossible to finish in one summer. Not only that, you would need to find someone capable of reviewing such code. I believe you should focus on a couple simple image processing algorithms that you already understand very well. If you written image processing code in the past, I believe it could be a good project. The way I see it, you would need to: 1. Create a user-defined type for the fingerprint. 2. Create a user-defined function to transform a raster image into a fingerprint. 3. Code to index fingerprints using the GiST or GIN API. 4. Code to search for fingerprints that match a given fingerprint, using the GiST or GIN API and a custom operator. After you have the basics completed, you can introduce more sophisticated fingerprint representations and comparison algorithms. That would probably be an ongoing effort after the GSoC project is complete. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC
As Jeff Davis pointed out, I followed the modification he suggested and now I want to have a basic matching only. I think atleast the fingerprint processing can be done in summer (if not the image processing). Is it a good GSoC project now? Waiting for a reply eagerly Thanks and regards, Anindya Jyoti Roy Sophomore CSE Btech IIT Kanpur On 30-Mar-10 6:33 AM, Jeff Davis wrote: On Tue, 2010-03-30 at 01:33 +0530, Anindya Jyoti Roy wrote: I have some idea of implementing am image database system, which will let you search against the image and fingerprint: The idea crudely is the following: I want to implement a image database system: This will have the following quality: 1 will store image along with the other attributes 2 the database search engine will be able to search for image also 3 it will list the matching images in the order of degree of match. 4 in this matching system I will likely use the system of dividing the image into important parts and match them. 5 The database will also contain fingerprints, that may be the primary key. 6 it will match the finger prints on the basis of the nodes and then making a bitmap of it and using the coordinates with some error it will match them 7 to include some accuracy in case of the angle of picture in case of search of equality we will introduce some logical amount of error. 8 this project can be coded by me as I have some previous experience in this type of project. 9 this will make a definitly good search engine as well as innovative and uncommon. If you plan to spend your time refining complex image processing algorithms, or learning about image processing as you go, I think the project will be impossible to finish in one summer. Not only that, you would need to find someone capable of reviewing such code. I believe you should focus on a couple simple image processing algorithms that you already understand very well. If you written image processing code in the past, I believe it could be a good project. The way I see it, you would need to: 1. Create a user-defined type for the fingerprint. 2. Create a user-defined function to transform a raster image into a fingerprint. 3. Code to index fingerprints using the GiST or GIN API. 4. Code to search for fingerprints that match a given fingerprint, using the GiST or GIN API and a custom operator. After you have the basics completed, you can introduce more sophisticated fingerprint representations and comparison algorithms. That would probably be an ongoing effort after the GSoC project is complete. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] why table.name is translated to (name.*)::name?
Hello, I was noticed on little bit strange feature of PostgreSQL 8.4 and 9.0 we can use a non existing column name. What does mean? postgres=# create table h(a int, b int); CREATE TABLE Time: 2,604 ms postgres=# insert into h values(199,22); INSERT 0 1 Time: 0,970 ms postgres=# explain verbose select h.name from h; QUERY PLAN - Seq Scan on public.h (cost=0.00..42.10 rows=2140 width=32) Output: (h.*)::name (2 rows) Time: 0,727 ms postgres=# select h.name from h; name -- (199,22) (1 row) Time: 0,589 ms postgres=# Regards Pavel Stehule -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Questions about 9.0 release note
Also, where are we on using full names rather than first names only? I don't see the point in omitting the last names. Are we trying to obscure to outsiders who is really working on our code? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC
On Tue, Mar 30, 2010 at 12:56 AM, Anindya Jyoti Roy anind...@iitk.ac.in wrote: As Jeff Davis pointed out, I followed the modification he suggested and now I want to have a basic matching only. I think atleast the fingerprint processing can be done in summer (if not the image processing). Is it a good GSoC project now? I'm not sure. Can you provide a more detailed design? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UUIDs generated using ossp-uuid on windows not unique
MUHAMMAD ASIF anaeem...@hotmail.com writes: I have downloaded uuid-1.6.2.tar.gz from http://www.ossp.org/pkg/lib/uuid/ . It successfully built with MinGW gcc and made libuuid.a, I renamed it to uuid.lib to build uuid-ossp contrib module with visual studio 2005. uuid-ossp give the following linker error with VC2005 i.e. Linking... Creating library Release\uuid-ossp\uuid-ossp.lib and object Release\uuid-ossp\uuid-ossp.exp uuid.lib(uuid_time.o) : error LNK2019: unresolved external symbol ___divdi3 referenced in function _uuid_time_gettimeofday uuid.lib(uuid_time.o) : error LNK2019: unresolved external symbol ___moddi3 referenced in function _uuid_time_gettimeofday uuid.lib(uuid_str.o) : error LNK2019: unresolved external symbol ___umoddi3 referenced in function _fmtint uuid.lib(uuid_str.o) : error LNK2019: unresolved external symbol ___udivdi3 referenced in function _fmtint .\Release\uuid-ossp\uuid-ossp.dll : fatal error LNK1120: 4 unresolved externals Generally, gcc is supposed to arrange for -lgcc to be included automatically in links if it's needed. I have seen failures like this before on other platforms, and it generally had something to do with using gcc followed by a non-gnu linker; which indeed seems to describe what you did here. I would suggest adding -lgcc to your VC project; or maybe where it actually went missing was in your build of libuuid. MinGW libgcc.a contains these symbols and these routines are not implemented in c ( its assembly code to make it fast ) so I extracted _divdi3.o _moddi3.o _umoddi3.o _udivdi3.o (we can add these additional files to uuid.lib to make build easy) from libgcc.a and link it with uuid-ossp contrib. This seems entirely inappropriate; it makes unwarranted assumptions about which parts of libgcc are needed, and we couldn't ship any such thing anyhow for license reasons. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why table.name is translated to (name.*)::name?
2010/3/30 Pavel Stehule pavel.steh...@gmail.com: Hello, I was noticed on little bit strange feature of PostgreSQL 8.4 and 9.0 we can use a non existing column name. What does mean? postgres=# create table h(a int, b int); CREATE TABLE Time: 2,604 ms postgres=# insert into h values(199,22); INSERT 0 1 Time: 0,970 ms postgres=# explain verbose select h.name from h; QUERY PLAN - Seq Scan on public.h (cost=0.00..42.10 rows=2140 width=32) Output: (h.*)::name (2 rows) Time: 0,727 ms postgres=# select h.name from h; name -- (199,22) (1 row) Time: 0,589 ms postgres=# FYI this has caused me (and presumably a few other people) a bit of head-scratching, e.g.: http://archives.postgresql.org/pgsql-general/2010-03/msg00362.php I imagine it has some potential as a 'gotcha', as name is hardly an uncommon column name, but it's not an issue which can easily researched... Ian Barwick -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why table.name is translated to (name.*)::name?
Ian Barwick barw...@gmail.com writes: 2010/3/30 Pavel Stehule pavel.steh...@gmail.com: we can use a non existing column name. What does mean? FYI this has caused me (and presumably a few other people) a bit of head-scratching, e.g.: http://archives.postgresql.org/pgsql-general/2010-03/msg00362.php We could make that stop happening if we were willing to restrict the cases in which an I/O conversion would be applied, but I think the cure might be worse than the disease. It would be an entirely arbitrary restriction of a feature. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why table.name is translated to (name.*)::name?
2010/3/30 Tom Lane t...@sss.pgh.pa.us: Ian Barwick barw...@gmail.com writes: 2010/3/30 Pavel Stehule pavel.steh...@gmail.com: we can use a non existing column name. What does mean? FYI this has caused me (and presumably a few other people) a bit of head-scratching, e.g.: http://archives.postgresql.org/pgsql-general/2010-03/msg00362.php We could make that stop happening if we were willing to restrict the cases in which an I/O conversion would be applied, but I think the cure might be worse than the disease. It would be an entirely arbitrary restriction of a feature. it is confusing :(. It returns some data, but it have to returns syntax error. Regards Pavel Stehule regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC
Robert Haas robertmh...@gmail.com writes: On Tue, Mar 30, 2010 at 12:56 AM, Anindya Jyoti Roy anind...@iitk.ac.in wrote: As Jeff Davis pointed out, I followed the modification he suggested and now I want to have a basic matching only. I think atleast the fingerprint processing can be done in summer (if not the image processing). Is it a good GSoC project now? I'm not sure. Can you provide a more detailed design? Apply the following to fingerprint searches ? http://www.postgresql.org/docs/current/static/gist-implementation.html http://wiki.postgresql.org/wiki/Image:Prato_2008_prefix.pdf I guess that what remains to be defined is how you get those fingerprint, what the datatype is named, is it fixed size or varlena, what operators you want to make available, and which will have index support. That's GiST + GIN, right ? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Problems with variable cursorname in ecpg
The interpretation of the standard in the above way (DECLARE is declarative, ... It's not just interpretation, but also a regression if we were to change this. The uniqueness problem can only be solved with modifying the runtime library to keep track of the cursor names in the client. It would ruin the declarative nature of DECLARE but would increase compatibility with Informix, and we would also need to implement correct FREE cursorname behaviour, too. Which would also bring the consequence that the ECPG client library would need to forbid cursors and prepared statements with the same name as FREE can also free cursors and prepared statements. True. I think the current behaviour is the best we could achieve while keeping close standard conformance. I think we should make the error message/documentation a little bit clearer as people have stumbled over it. Having said that couldn't we keep the statement declarative only for statements that do not carry a variable? This will not break any onld program and besides using a variable that doesn't exist, because you're outside a function doesn't make sense either. This is probably something for 9.1 though if it indeed works. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org ICQ 179140304, AIM/Yahoo/Skype michaelmeskes, Jabber mes...@jabber.org VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] OK for ABI break of PlannerInfo in 8.4?
Marc Cousins pointed out here
http://archives.postgresql.org/pgsql-general/2010-03/msg01123.php
that the constraint_exclusion = partition feature added in 8.4
does not do what you'd expect for the target relation of an UPDATE
or DELETE. That's because expansion of an inheritance set is managed
differently for an UPDATE/DELETE target rel than for other cases.
I'm intending to apply the attached patch to fix this in HEAD.
I am tempted to back-patch it to 8.4 as well, but there is a potential
problem for external modules that may be touching PlannerInfo (eg,
planner hooks): the added field in that struct is an ABI break for them.
We can minimize the risk by adding the new field at the end rather than
in any more logical position; but it would still be a problem for
modules that palloc'd or copied a PlannerInfo struct. AFAICS though the
only real risk would be for relation_excluded_by_constraints to see a
garbage value of root-hasInheritedTarget and possibly make an
unexpected choice of what to do. I think that's probably a small enough
problem to be acceptable. Comments?
regards, tom lane
Index: src/backend/nodes/outfuncs.c
===
RCS file: /cvsroot/pgsql/src/backend/nodes/outfuncs.c,v
retrieving revision 1.384
diff -c -r1.384 outfuncs.c
*** src/backend/nodes/outfuncs.c 28 Mar 2010 22:59:32 - 1.384
--- src/backend/nodes/outfuncs.c 30 Mar 2010 15:18:27 -
***
*** 1559,1564
--- 1559,1565
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_FLOAT_FIELD(total_table_pages, %.0f);
WRITE_FLOAT_FIELD(tuple_fraction, %.4f);
+ WRITE_BOOL_FIELD(hasInheritedTarget);
WRITE_BOOL_FIELD(hasJoinRTEs);
WRITE_BOOL_FIELD(hasHavingQual);
WRITE_BOOL_FIELD(hasPseudoConstantQuals);
Index: src/backend/optimizer/plan/planner.c
===
RCS file: /cvsroot/pgsql/src/backend/optimizer/plan/planner.c,v
retrieving revision 1.266
diff -c -r1.266 planner.c
*** src/backend/optimizer/plan/planner.c 26 Feb 2010 02:00:45 - 1.266
--- src/backend/optimizer/plan/planner.c 30 Mar 2010 15:18:27 -
***
*** 307,312
--- 307,313
root-eq_classes = NIL;
root-append_rel_list = NIL;
root-rowMarks = NIL;
+ root-hasInheritedTarget = false;
root-hasRecursion = hasRecursion;
if (hasRecursion)
***
*** 749,754
--- 750,756
adjust_appendrel_attrs((Node *) parse,
appinfo);
subroot.init_plans = NIL;
+ subroot.hasInheritedTarget = true;
/* We needn't modify the child's append_rel_list */
/* There shouldn't be any OJ info to translate, as yet */
Assert(subroot.join_info_list == NIL);
Index: src/backend/optimizer/util/plancat.c
===
RCS file: /cvsroot/pgsql/src/backend/optimizer/util/plancat.c,v
retrieving revision 1.162
diff -c -r1.162 plancat.c
*** src/backend/optimizer/util/plancat.c 5 Jan 2010 21:53:58 - 1.162
--- src/backend/optimizer/util/plancat.c 30 Mar 2010 15:18:27 -
***
*** 580,586
/* Skip the test if constraint exclusion is disabled for the rel */
if (constraint_exclusion == CONSTRAINT_EXCLUSION_OFF ||
(constraint_exclusion == CONSTRAINT_EXCLUSION_PARTITION
! rel-reloptkind != RELOPT_OTHER_MEMBER_REL))
return false;
/*
--- 580,589
/* Skip the test if constraint exclusion is disabled for the rel */
if (constraint_exclusion == CONSTRAINT_EXCLUSION_OFF ||
(constraint_exclusion == CONSTRAINT_EXCLUSION_PARTITION
! !(rel-reloptkind == RELOPT_OTHER_MEMBER_REL ||
! (root-hasInheritedTarget
! rel-reloptkind == RELOPT_BASEREL
! rel-relid == root-parse-resultRelation
return false;
/*
Index: src/include/nodes/relation.h
===
RCS file: /cvsroot/pgsql/src/include/nodes/relation.h,v
retrieving revision 1.185
diff -c -r1.185 relation.h
*** src/include/nodes/relation.h 28 Mar 2010 22:59:33 - 1.185
--- src/include/nodes/relation.h 30 Mar 2010 15:18:27 -
***
*** 197,202
--- 197,204
double tuple_fraction; /* tuple_fraction passed to query_planner */
+ bool hasInheritedTarget; /* true if parse-resultRelation is an
+ * inheritance child rel */
bool hasJoinRTEs; /* true if any RTEs are RTE_JOIN kind */
bool hasHavingQual; /* true if havingQual was non-null */
bool hasPseudoConstantQuals; /* true if any RestrictInfo has
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL
Fujii Masao wrote: * Small code changes to handling of failedSources, inspired by your comment. No change in functionality. This is also available in my git repository at git://git.postgresql.org/git/users/heikki/postgres.git, branch xlogchanges I looked the patch and was not able to find any big problems until now. The attached small patch fixes the typo. Thanks. Committed with that typo-fix, and I also added a comment explaining how failedSources and retrying XLogPageRead() works. I'm now happy with the standby mode logic. It was a bigger struggle than I anticipated back in January/February, I hope others now find it intuitive as well. I'm going to work on the documentation of this, along the lines of the draft I posted last week. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OK for ABI break of PlannerInfo in 8.4?
Tom Lane wrote: Marc Cousins pointed out here http://archives.postgresql.org/pgsql-general/2010-03/msg01123.php that the constraint_exclusion = partition feature added in 8.4 does not do what you'd expect for the target relation of an UPDATE or DELETE. That's because expansion of an inheritance set is managed differently for an UPDATE/DELETE target rel than for other cases. I'm intending to apply the attached patch to fix this in HEAD. I am tempted to back-patch it to 8.4 as well, but there is a potential problem for external modules that may be touching PlannerInfo (eg, planner hooks): the added field in that struct is an ABI break for them. We can minimize the risk by adding the new field at the end rather than in any more logical position; but it would still be a problem for modules that palloc'd or copied a PlannerInfo struct. AFAICS though the only real risk would be for relation_excluded_by_constraints to see a garbage value of root-hasInheritedTarget and possibly make an unexpected choice of what to do. I think that's probably a small enough problem to be acceptable. Comments? Seems OK to me. It's worth noting though that if a module does do palloc+memcpy of PlannerInfo, and it's compiled against the new sources with the extra field, but used on an old server version, it will memcpy() from beyond the end of the struct. If you're seriously unlucky and the struct is at the end of allocated address space, that can segfault. Unfortunately there doesn't seem to be any alignment padding in the struct either. You could've stuck the new field there and avoided changing sizeof(PlannerInfo). -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] UUIDs generated using ossp-uuid on windows not unique
Hi MUHAMMAD-san. Sorry late reaction... unfortunately, My machine crashed and was not revitalized.:-( However, There is a thing of now a few. http://winpg.jp/~saito/pg_work/OSSP_win32/msvc/ I have a comment from Ralf S. Engelschall. == Sorry for being late with the release of OSSP uuid 2.0.0. Yes, it will be released the next days and will contain the Win32 support I already prepared a few months ago. == It seems that however, he is very busy. ... Regards, Hiroshi Saito - Original Message - From: MUHAMMAD ASIF anaeem...@hotmail.com On Tue, Jun 17, 2008 at 11:17 AM, Hiroshi Saito z-saito(at)guitar(dot)ocn(dot)ne(dot)jp wrote: Hi. Please this. http://winpg.jp/~saito/pg_work/OSSP_win32/msvc/ Ahh, a makefile - that makes things somewhat clearer :-) Thanks - I've updated the build machine so future releases will include this fix. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com It seems that link http://winpg.jp/~saito/pg_work/OSSP_win32/msvc/ is no longer available. I am working on uuid-ossp contrib module, I want to share my findings with you.Thanks. I have downloaded uuid-1.6.2.tar.gz from http://www.ossp.org/pkg/lib/uuid/ . It successfully built with MinGW gcc and made libuuid.a, I renamed it to uuid.lib to build uuid-ossp contrib module with visual studio 2005. uuid-ossp give the following linker error with VC2005 i.e. Linking... Creating library Release\uuid-ossp\uuid-ossp.lib and object Release\uuid-ossp\uuid-ossp.exp uuid.lib(uuid_time.o) : error LNK2019: unresolved external symbol ___divdi3 referenced in function _uuid_time_gettimeofday uuid.lib(uuid_time.o) : error LNK2019: unresolved external symbol ___moddi3 referenced in function _uuid_time_gettimeofday uuid.lib(uuid_str.o) : error LNK2019: unresolved external symbol ___umoddi3 referenced in function _fmtint uuid.lib(uuid_str.o) : error LNK2019: unresolved external symbol ___udivdi3 referenced in function _fmtint .\Release\uuid-ossp\uuid-ossp.dll : fatal error LNK1120: 4 unresolved externals MinGW libgcc.a contains these symbols and these routines are not implemented in c ( its assembly code to make it fast ) so I extracted _divdi3.o _moddi3.o _umoddi3.o _udivdi3.o (we can add these additional files to uuid.lib to make build easy) from libgcc.a and link it with uuid-ossp contrib. I have tested generated uuid-ossp contrib it gives the following results that seems fine i.e. edb=# select uuid_generate_v4(); uuid_generate_v4 -- 20ff31ce-77a8-4db6-ac7f-3b55380e46ac (1 row) edb=# edb=# select uuid_generate_v4() edb-# ; uuid_generate_v4 -- e58c46d5-7161-47ec-8006-f905c6b66a1f (1 row) Any thoughts on this ?. Best Regards, Asif Naeem EnterpriseDB Pakistan: http://www.enterprisedb.com _ Hotmail: Free, trusted and rich email service. https://signup.live.com/signup.aspx?id=60969= -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OK for ABI break of PlannerInfo in 8.4?
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Seems OK to me. It's worth noting though that if a module does do palloc+memcpy of PlannerInfo, and it's compiled against the new sources with the extra field, but used on an old server version, it will memcpy() from beyond the end of the struct. If you're seriously unlucky and the struct is at the end of allocated address space, that can segfault. Yeah, and in the converse case (copied version is too small) the reference from relation_excluded_by_constraints could segfault. But actually, neither will happen because palloc rounds the struct size up to a power of 2, so there will be some memory there; it just might contain garbage. Unfortunately there doesn't seem to be any alignment padding in the struct either. You could've stuck the new field there and avoided changing sizeof(PlannerInfo). Yeah, I looked for some pad space :-( The alternative fix for 8.4 would be to not add the field but instead grovel through app_info_list to see if the target relation is a child. I discarded this approach because the whole point of constraint_exclusion = partition is to not add overhead to simple non-inheritance queries. However, if we think the ABI break is actually worth worrying about then maybe that's the better fix. I'm inclined not to worry, in part because I'm not aware of any third-party planner hooks actually being used in production. Does anyone know of one? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha release this week?
On 3/29/10 5:04 PM, Magnus Hagander wrote: Last i heard from Dave on that topic is that there's no chance of that happening that quickly. He's on a plane now but I'm sure he'll confirm that when he lands. That means that we'll be doing the test-fest using Alpha4, materially. Which is annoying because it means we'll be catching a lot of bugs which are already fixed. However, it's pretty much impossible for me to coordinate 25 volunteers getting the *same* daily snapshot. And we need to test on Windows, since it has all kinds of special issues. I think my big goal for 9.1 is going to be to fix our testing procedure, or rather total lack of a procedure. We've got development systematized, now it's time for testing. I'm just sorry that the press of work kept me from really doing it this time around. I think we could be getting from alpha to beta in 6 weeks if we actually had a schedule and some real testing goals. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha release this week?
On Tue, Mar 30, 2010 at 4:43 PM, Josh Berkus j...@agliodbs.com wrote: On 3/29/10 5:04 PM, Magnus Hagander wrote: Last i heard from Dave on that topic is that there's no chance of that happening that quickly. He's on a plane now but I'm sure he'll confirm that when he lands. That means that we'll be doing the test-fest using Alpha4, materially. Which is annoying because it means we'll be catching a lot of bugs which are already fixed. However, it's pretty much impossible for me to coordinate 25 volunteers getting the *same* daily snapshot. And we need to test on Windows, since it has all kinds of special issues. I think my big goal for 9.1 is going to be to fix our testing procedure, or rather total lack of a procedure. We've got development systematized, now it's time for testing. I'm just sorry that the press of work kept me from really doing it this time around. I think we could be getting from alpha to beta in 6 weeks if we actually had a schedule and some real testing goals. At the risk of being blunt, AFAICT, the delay in getting to beta has little or nothing to do with testing and everything to do with the fact that streaming replication got committed with a long list of open items two months ago, and many of them haven't been fixed yet. Hot Standby has a few warts too, but I think Simon has done a better job cleaning up the loose ends there (with help from Tom and Heikki), no doubt because he got it committed two months sooner than SR. From a project management point of view, it seems to me that we shouldn't commit major patches late in the release cycle unless someone has the time to actually get them finished and stable. If Streaming Replication were any other patch, we would have reverted it a month ago. As it is, it looks like we're going to be waiting until Heikki has time to deal with the issues, because it doesn't look like any of the other committers are able/willing to help. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: Add JSON support
joeyadams3.14...@gmail.com (Joseph Adams) writes: I introduced myself in the thread Proposal: access control jails (and introduction as aspiring GSoC student), and we discussed jails and session-local variables. But, as Robert Haas suggested, implementing variable support in the backend would probably be way too ambitious a project for a newbie like me. I decided instead to pursue the task of adding JSON support to PostgreSQL, hence the new thread. Interesting... I had a discussion about much this sort of thing with a local LUG associate; he was interested in this from a doing CouchDB-ish things using PostgreSQL perspective. There were a couple perspectives there, which may be somewhat orthogonal to what you're trying to do. I'll mention them as they may suggest useful operations. 1. Buddy Myles pointed out a NYTimes project which does something pretty analagous... http://code.nytimes.com/projects/dbslayer This is a proxy that allows clients to submit requests via HTTP, returning responses in JSON form. Note that the HTTP request has the SQL query embedded into it. 2. CouchDB's interface is much the same, where clients submit HTTP requests and receive JSON responses back, but with the difference that the query is a stylized sorta-JSON form. I'd think that you could get quite a long ways on this, at least doing something like dbslayer without *necessarily* needing to do terribly much work inside the DB engine. Mapping a tuple, or a list of tuples, into a forest of JSON documents should be pretty straightforward; whether or not it's really desirable to operate a JSON-flavoured query inside PostgreSQL may be the difference between *this year's* GSOC and *next year's* :-). -- ...the Jedi learned early on what language the universe was programmed in. Then they took advantage of an accident of language to obscure this fact from the unwashed. They all affected an inverted lisp. so, a Jedi to be, you the Forth must use. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GSoC
On Tue, Mar 30, 2010 at 9:36 AM, Dimitri Fontaine dfonta...@hi-media.com wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Mar 30, 2010 at 12:56 AM, Anindya Jyoti Roy anind...@iitk.ac.in wrote: As Jeff Davis pointed out, I followed the modification he suggested and now I want to have a basic matching only. I think atleast the fingerprint processing can be done in summer (if not the image processing). Is it a good GSoC project now? I'm not sure. Can you provide a more detailed design? Apply the following to fingerprint searches ? http://www.postgresql.org/docs/current/static/gist-implementation.html http://wiki.postgresql.org/wiki/Image:Prato_2008_prefix.pdf I guess that what remains to be defined is how you get those fingerprint, what the datatype is named, is it fixed size or varlena, what operators you want to make available, and which will have index support. That's GiST + GIN, right ? Well, yeah. I think the fingerprinting and operator support are the real questions. My fear is that the student who is asking this question does not really have a good handle on that aspect of the project. Maybe I'm wrong. However the description that was given was: 2 the database search engine will be able to search for image also 3 it will list the matching images in the order of degree of match. 4 in this matching system I will likely use the system of dividing the image into important parts and match them. That's pretty vague. If someone came and said, I'm going to use XYZ system from the following academic papers, that would inspire a lot more confidence, at least for me. Also I think this item from the original email reflects a fundamental misunderstanding of how this would integrate into PostgreSQL: 5 The database will also contain fingerprints, that may be the primary key. Again, if the student had said, the XYZ system above will work well with GIN indexing because we can construct the posting lists like thus-and-so, or if they had said, it will work well with GIST because there is a similarity metric we can use to construct the penalty and picksplit functions, I would feel a lot better. But the description given is so general in terms of both what is to be done on the image processing side and what is to be done on the PostgreSQL side that I am afraid that the student is going to be in far too deep. Compare this description with the one from the student who wants to implement JSON support - that sounds a whole lot closer to something that someone (perhaps him) could sit down and code. My point here is not to discourage anyone or turn them off on trying to submit a GSoC project related to PostgreSQL. Indeed, I really hope they do. But it will benefit the project much more if the projects are small and successful than it will if they are large and not successful, or successful according to some metric but not actually producing code that will be widely used or merged into core. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown
On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I rebased the patch to HEAD. Is the patch still required for 9.0? If not, I'd remove the open item of the smart shutdown during recovery. I am by no means an expert on this area of the code, but in the interest of moving things along I reviewed this patch tonight. 1. I wonder if there is a problem if we receive SIGINT while in the PM_WAIT_READONLY state? Seems to me that might need to be added to the if statement beginning at line 2212, in pmdie(). 2. It appears to me that HandleChildCrash() needs to switch to PM_WAIT_BACKENDS state if it's in PM_WAIT_READONLY when the child crash occurs - i.e. the if statement beginning at line 2772 needs updating. Thoughts? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: Add JSON support
I'd think that you could get quite a long ways on this, at least doing something like dbslayer without *necessarily* needing to do terribly much work inside the DB engine. There's actually an HTTP framework tool for Postgres which already does something of the sort. It was introduced at pgCon 2 years ago ... will look for. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: Add JSON support
On Tue, Mar 30, 2010 at 8:58 PM, Josh Berkus j...@agliodbs.com wrote: I'd think that you could get quite a long ways on this, at least doing something like dbslayer without *necessarily* needing to do terribly much work inside the DB engine. There's actually an HTTP framework tool for Postgres which already does something of the sort. It was introduced at pgCon 2 years ago ... will look for. While it might be interesting to have/find/write a tool that puts an HTTP/JSON layer around the DB connection, it's pretty much entirely unrelated to the proposed project of creating a json type with PostgreSQL analagous to the xml type we already have, which is what the OP is proposing to do. Personally, I suspect that a JSON type is both a more interesting project to work on and a more useful result for this community. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown
On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I rebased the patch to HEAD. Is the patch still required for 9.0? If not, I'd remove the open item of the smart shutdown during recovery. I am by no means an expert on this area of the code, but in the interest of moving things along I reviewed this patch tonight. Thanks a lot! 1. I wonder if there is a problem if we receive SIGINT while in the PM_WAIT_READONLY state? Seems to me that might need to be added to the if statement beginning at line 2212, in pmdie(). 2. It appears to me that HandleChildCrash() needs to switch to PM_WAIT_BACKENDS state if it's in PM_WAIT_READONLY when the child crash occurs - i.e. the if statement beginning at line 2772 needs updating. Thoughts? Oh, those are my oversights. You are right! I modified the patch as you pointed out. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center new_smart_shutdown_20100331.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: Add JSON support
Robert Haas wrote: While it might be interesting to have/find/write a tool that puts an HTTP/JSON layer around the DB connection, it's pretty much entirely unrelated to the proposed project of creating a json type with PostgreSQL analagous to the xml type we already have, which is what the OP is proposing to do. Personally, I suspect that a JSON type is both a more interesting project to work on and a more useful result for this community. I agree. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL
On Wed, Mar 31, 2010 at 1:28 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Fujii Masao wrote: * Small code changes to handling of failedSources, inspired by your comment. No change in functionality. This is also available in my git repository at git://git.postgresql.org/git/users/heikki/postgres.git, branch xlogchanges I looked the patch and was not able to find any big problems until now. The attached small patch fixes the typo. Thanks. Committed with that typo-fix, and I also added a comment explaining how failedSources and retrying XLogPageRead() works. Great! Thanks a lot! This change affects various recovery patterns that we support now. For example, normal crash recovery, archive recovery, recovery using pg_standby, recovery in standby mode, and so on. So we would need to test whether all of those patterns work fine with the change. I'll do it as much as possible. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha release this week?
On Wed, Mar 31, 2010 at 7:01 AM, Robert Haas robertmh...@gmail.com wrote: At the risk of being blunt, AFAICT, the delay in getting to beta has little or nothing to do with testing and everything to do with the fact that streaming replication got committed with a long list of open items two months ago, and many of them haven't been fixed yet. Hot Standby has a few warts too, but I think Simon has done a better job cleaning up the loose ends there (with help from Tom and Heikki), no doubt because he got it committed two months sooner than SR. From a project management point of view, it seems to me that we shouldn't commit major patches late in the release cycle unless someone has the time to actually get them finished and stable. If Streaming Replication were any other patch, we would have reverted it a month ago. As it is, it looks like we're going to be waiting until Heikki has time to deal with the issues, because it doesn't look like any of the other committers are able/willing to help. I believe that anyone except Heikki me can deal with the following issues since they started with SR but actually are not tied up to SR ;) * dblink and walreceiver are not interruptible on win32 http://archives.postgresql.org/pgsql-hackers/2010-01/msg01672.php http://archives.postgresql.org/pgsql-hackers/2010-03/msg00413.php * smart shutdown during recovery gets stuck http://archives.postgresql.org/pgsql-hackers/2010-01/msg02044.php http://archives.postgresql.org/pgsql-hackers/2010-03/msg01208.php * pg_xlogfile_name() might report the wrong name http://archives.postgresql.org/pgsql-hackers/2010-01/msg01806.php Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] master in standby mode croaks
I discovered tonight that if you shut down a server, create recovery.conf with standby_mode = 'on', and start it back up again, you get this: LOG: database system was shut down at 2010-03-30 22:34:09 EDT LOG: entering standby mode FATAL: recovery connections cannot start because the recovery_connections parameter is disabled on the WAL source server LOG: startup process (PID 22980) exited with exit code 1 LOG: aborting startup due to startup process failure Now, you might certainly argue that this is a stupid thing to do (my motivation was to test some stuff) but certainly it's fair to say that error message is darn misleading, since in fact recovery_connections was NOT disabled. I believe this is the same start up from a shut down checkpoint problem that's been discussed previously so I won't belabor the point other than to say that I still think we need to fix this. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] sorry, too many standbys already vs. MaxWalSenders vs. max_wal_senders
After snapshotting my master using hot backup to create a workable slave instance, I created recovery.conf on the slave and tried to get it to connect to the master and stream WAL. This led to the message sorry, too many standbys already, which did not immediately clue me in as to what I needed to do to fix the problem. Grepping the source code for the error message revealed that the problem was that MaxWalSenders was zero. A few seconds further head-scratching revealed that this was the GUC max_wal_senders, which I duly increased from 0 to 1, after which it worked. I think perhaps this error message needs some adjustment. It should be reasonably possible to guess the name of the GUC that needs increasing based on the error message, and it currently isn't. Also I'd vote for making the variable name max_wal_senders rather than MaxWalSenders, but maybe that's being persnicketty. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown
On Tue, Mar 30, 2010 at 9:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I rebased the patch to HEAD. Is the patch still required for 9.0? If not, I'd remove the open item of the smart shutdown during recovery. I am by no means an expert on this area of the code, but in the interest of moving things along I reviewed this patch tonight. Thanks a lot! 1. I wonder if there is a problem if we receive SIGINT while in the PM_WAIT_READONLY state? Seems to me that might need to be added to the if statement beginning at line 2212, in pmdie(). 2. It appears to me that HandleChildCrash() needs to switch to PM_WAIT_BACKENDS state if it's in PM_WAIT_READONLY when the child crash occurs - i.e. the if statement beginning at line 2772 needs updating. Thoughts? Oh, those are my oversights. You are right! I modified the patch as you pointed out. Cool. This looks good to me now and I also tested it. I will commit it unless (a) someone objects or (b) someone else does it first. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parameter name standby_mode
On Tue, Mar 30, 2010 at 12:26 AM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Mar 3, 2010 at 9:41 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Feb 24, 2010 at 2:18 PM, Fujii Masao masao.fu...@gmail.com wrote: If standby_mode is enabled, and neither primary_conninfo nor restore_command are set, the standby would get stuck. How about forbidding (i.e., causing a FATAL message) this wrong setting? Here is the patch which forbids that wrong setting of recovery.conf. I think that this patch should be applied. Otherwise, if you wrongly set neither primary_conninfo nor restore_command in recovery.conf, the standby server would do nothing and get stuck because it doesn't know where to retrieve the WAL files from. Banning the incorrect setting makes sense to me. Does anyone commit the patch? Does anyone have a say? I just tested this and it seems to just sit there doing this over and over again: LOG: record with zero length at 0/3006B28 I'm not sure that we should forbid this configuration, but the current behavior doesn't seem right either. ISTM that, in the absence of a way to get any more WAL, it would be reasonable for the standby server to just start up and sit there in recovery mode but without actually advancing recovery, but the repeated log messages are pretty annoying. If we're connected in streaming mode and there is no activity on the primary, we don't emit logs of this type, so it doesn't seem like we should do that if there is no primary either. A related question is... do we ever reload recovery.conf? I tried adding the setting to recovery.conf and doing pg_ctl reload, and it says that it's reloading configuration files, but doesn't pick up the new setting. :-( ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sorry, too many standbys already vs. MaxWalSenders vs. max_wal_senders
On Wed, Mar 31, 2010 at 12:06 PM, Robert Haas robertmh...@gmail.com wrote: After snapshotting my master using hot backup to create a workable slave instance, I created recovery.conf on the slave and tried to get it to connect to the master and stream WAL. This led to the message sorry, too many standbys already, which did not immediately clue me in as to what I needed to do to fix the problem. Grepping the source code for the error message revealed that the problem was that MaxWalSenders was zero. A few seconds further head-scratching revealed that this was the GUC max_wal_senders, which I duly increased from 0 to 1, after which it worked. I think perhaps this error message needs some adjustment. It should be reasonably possible to guess the name of the GUC that needs increasing based on the error message, and it currently isn't. Agreed. How about the atteched patch? The patch treats differently the case where max_wal_senders is 0, and the following error message (better message?) is written only in this case. could not accept connection from the standby because max_wal_senders is 0 Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center sr_error_message_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parameter name standby_mode
On Wed, Mar 31, 2010 at 12:21 PM, Robert Haas robertmh...@gmail.com wrote: I just tested this and it seems to just sit there doing this over and over again: LOG: record with zero length at 0/3006B28 I'm not sure that we should forbid this configuration, but the current behavior doesn't seem right either. ISTM that, in the absence of a way to get any more WAL, it would be reasonable for the standby server to just start up and sit there in recovery mode but without actually advancing recovery, but the repeated log messages are pretty annoying. I'm concerned about that the configuration might prevent the standby from accepting connection from a client because it cannot get the WAL for making the database consistent. So that configuration seems to be reasonable only when starting the standby from the already-consistent database or with enough WAL files in pg_xlog. But it seems to me that the standby often starts from the inconsistent database without enough WAL in pg_xlog. A related question is... do we ever reload recovery.conf? I tried adding the setting to recovery.conf and doing pg_ctl reload, and it says that it's reloading configuration files, but doesn't pick up the new setting. :-( recovery.conf cannot be reloaded while the server is running. This restriction should be removed in the future release, I think. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Feature request - function-based deferrable uniques.
For now Postgres able to create deferrable uniques with following syntax:
...
and table_constraint is:
[ CONSTRAINT constraint_name ]
{ UNIQUE ( column_name [, ... ] ) index_parameters |
PRIMARY KEY ( column_name [, ... ] ) index_parameters |
CHECK ( expression ) |
FOREIGN KEY ( column_name [, ... ] ) REFERENCES reftable [ ( refcolumn [,
... ] ) ]
[ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] [ ON DELETE action ] [ ON
UPDATE action ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
So, deferrable uniques now can be based on column/columns list only. It will
be very useful if there will be possibility to specify functions in this list.
Is it possible?
Regards,
Dmitry
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha release this week?
Josh Berkus wrote: On 3/29/10 5:04 PM, Magnus Hagander wrote: Last i heard from Dave on that topic is that there's no chance of that happening that quickly. He's on a plane now but I'm sure he'll confirm that when he lands. That means that we'll be doing the test-fest using Alpha4, materially. Which is annoying because it means we'll be catching a lot of bugs which are already fixed. However, it's pretty much impossible for me to coordinate 25 volunteers getting the *same* daily snapshot. And we need to test on Windows, since it has all kinds of special issues. well this is actually the first release cycle where we even HAD alpha releases - as for the snapsho thing I cannot see why you could not download a snpashot on your own, put it somewhere (like a wiki page) and send out an email to all the volunteers and ask them to use at least that one if building from source... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
