Threads vs processes - The Apache Way (Re: [HACKERS] Path to PostgreSQL portabiliy)
Hi, Win32 threads support are both going to be a lot of work and maybe we'll need in the future one or both - is there any chance Postgres developers look at the Apache experience? Briefly, Apache 2 had the some problems as are discussed here (need to support Win, problems with Win32 fork, questionable cygwin etc) and they decided to solve it once and for all with their Apache Portable Runtime and Multi-Processing Modules. APR was already mentioned here - now how about MPMs? - Robert PS Relevant links: http://httpd.apache.org/docs-2.0/mpm.html http://httpd.apache.org/docs-2.0/new_features_2_0.html PS2 It took them some three years to release Apache 2 so it's probable not that easy - but you knew that already. PS3 And when talking about Win32 Postgres uses, don't forget there might be a large number of people who would use Posgtres embeded in accounting and many other packages - it can be single user Win98, but it might still need decent SQL backend (subqueries, user functions for all kind of CDROM catalogs etc). So when doing major rearchitecture of Postgres, it might be usefull to plan for a bit of modularity maybe like in Mozilla when you can drop UI and use just the layout engine or just the JavaScript etc. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Queries using rules show no rows modified?
On Fri, 2002-05-10 at 06:27, Tom Lane wrote: I'm also concerned about having an understandable definition for the OID returned for an INSERT query --- if there are additional INSERTs triggered by rules, does that mean you don't get to see the OID assigned to the single row you tried to insert? At least when there was actually no insert you don't and if there actually was more than 1 insert then INSERT 0 N seems quite reasonable to me. It may even be that returning a concatenation of results would be acceptable for current libs INSERT OID 1 INSERT 0 3 UPDATE 2 DELETE 2 You'll definitely get push-back if you propose that. But if we add up all the actions for the generated queries, we are quite likely to be returning an OID along with an insert count greater than one --- which is certainly confusing, as well as contrary to the existing documentation about how it works. Let's please quit worrying about can we install a hack today and instead try to figure out what a sensible behavior is. The problem seems to be that recent changes broke updatable views for a few users. So have these basic options: 1. revert the changes until we have a consensus on doing the right thing (seems best to me) 2. change clients (client libraries) for 7.2 cycle at least 3. not revert but install a hack today so that it seems like things still work ;) 4. figure out correct behaviour and do that for 7.2.2 5. do nothing and tell users to keep themselves busy with other things until there is consensus about new behaviour. option 5 seems to be worst, as it leaves users in a state with no clear view of what is going to happen - we have just changed one arguably broken behaviour for a new one and are probably going to change it again soon when we figure out what the right behaviour should be. I don't think it's likely to be hard to implement anything we might come up with, considering how tiny this API is. The sensible behaviour for updatable views would be to report ho many rows visible through this view were changed, but this can be hard to do in a generic way. - Hannu ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Two pieces of information about Cygwin installer
1) Cygwin latest CVS installer version supports command lines. 2) Cygwin setup.exe is not needed. According to Robert Collins, an appropriate setup.ini file can be used for automatic installation. Cheers, Jean-Michel POURE ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] the parsing of parameters
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: I have a little patch that actually allows SPI_prepare() to use UNKNOWN_OID in the passed in parameter type array and put's the choosen datatypes Oid back into there. The parser treats those parameters like single quoted literals of unknown type and chooses what would be the most useful datatype here. Any objections? For this particular application, at least, I do not see the value ... in fact this seems more likely to break stuff than help. If the application does not know what the datatypes are supposed to be, how is it going to call the prepared statement? Right now using UNKNOWN_OID in that place leads to a parse error, what makes me feel absolutely comfortable that there will be nobody using it today. So what kind of break are you talking about? You could possibly get away with that for a textual interface (always pass quoted literals), but it would surely destroy any chance of having a binary protocol for passing parameters to prepared statements. Right. And BTW, how do you propose that the client application passes the values in binary form anyway? Are you going to maintain that process for backwards compatibility when we change the internal representation of stuff (like we want to for numeric) or who? And what about byte ordering? User defined types? I think the backend is the only one who can convert into it's personal, binary format. Wouldn't anything else lead to security holes? Offhand I'm having a hard time visualizing why you'd want this at the SPI_prepare level, either ... what's the application? It propagates up to the SPI level. In fact it is down in the parser/analyzer. There are DB interfaces that allow a generic application to get a description of the result set (column names, types) even before telling the data types of all parameters. Our ODBC driver for example has it's own more or less complete SQL parser to deal with that case! I don't see THAT implementation very superior compared to the ability to ask the DB server for a guess. I thought that this PREPARE statement will be used by such interfaces in the future, no? Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Queries using rules show no rows modified?
Hiroshi Inoue wrote: Tom Lane wrote: Hiroshi Inoue [EMAIL PROTECTED] writes: Of cource it is nice to have a complete solution immediately but it doesn't seem easy. My patch is only a makeshift solution but fixes the most siginificant case(typical updatable views). I would like to devise a complete solution *before* we consider installing makeshift solutions (which will institutionalize wrong behavior). There seems to be some feeling here that in the presence of rewrites you only want to know that something happened. Are you suggesting that the returned tuple count should be the sum of all counts from insert, update, and delete actions that happened as a result of the query? We could certainly implement that, but it does not seem like a good idea to me. What should the backends return for complicated rewrites ? And how should/could clients handle the results ? It doesn't seem easy to me and it seems a flaw of rule system. Honestly I don't think that the psqlodbc driver can guarantee to handle such cases properly. However both Ron's case and Michael's one are ordinary updatable views. If we can't handle the case properly, we could never recommend users to use (updatable) views. The fact that our rule system is that powerful that you can have multi-action rules is a flaw ... awe. Do you think that if a trigger suppresses your original insert, but instead does 2 inserts somewhere else and another update and delete here and there, then 0 is the correct answer to the client? Well, that's what happens now, so it should irritate your client in exactly the same way. So not only our rule system, but our trigger system has a flaw too. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Queries using rules show no rows modified?
Tom Lane wrote: Hiroshi Inoue [EMAIL PROTECTED] writes: Of cource it is nice to have a complete solution immediately but it doesn't seem easy. My patch is only a makeshift solution but fixes the most siginificant case(typical updatable views). I would like to devise a complete solution *before* we consider installing makeshift solutions (which will institutionalize wrong behavior). There seems to be some feeling here that in the presence of rewrites you only want to know that something happened. Are you suggesting that the returned tuple count should be the sum of all counts from insert, update, and delete actions that happened as a result of the query? We could certainly implement that, but it does not seem like a good idea to me. IMHO the answer should only be a number if the rewritten querytree list consists of one query of the same command type. everything else has to lead into unknown. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Nested transactions RFC
Hi,
if it is acceptable for subtransactions to use up transaction numbers,
then here is a half baked RFC for a possible implementation.
If not, forget the rest of this message.
The proposed implementation works much like the current transaction
handling. It needs an additional system table
pg_subtrans (child XactId PRIMARY KEY, parent XactId).
BEGIN; -- starts a new (top level) transaction, say 100
INSERT row1; -- row1.xmin = 100
DELETE row2; -- row2.xmax = 100
BEGIN; -- starts a subtransaction, let's call it 200,
-- stores 100 on the parent transaction stack
-- (a local memory structure),
-- inserts (200, 100) into pg_subtrans
INSERT row3; -- row3.xmin = 200, row3.XMIN_IS_SUB = true
DELETE row4; -- row4.xmax = 200, row4.XMAX_IS_SUB = true
COMMIT; -- resets CurrentTransaction to 100 (pop from xact stack),
-- does *NOT* mark T200 as committed
BEGIN; -- starts a subtransaction, let's call it 300,
-- pushes 100 on the parent transaction stack,
-- inserts (300, 100) into pg_subtrans
BEGIN; -- starts a 3rd level subtransaction (400),
-- pushes 300 on the parent transaction stack,
-- inserts (400, 300) into pg_subtrans
...
COMMIT; -- resets CurrentTransaction to 300 (transaction stack),
-- does NOT mark T400 as committed
INSERT row5; -- row5.xmin = 300, row5.XMIN_IS_SUB = true
DELETE row6; -- row6.xmax = 300, row6.XMAX_IS_SUB = true
ROLLBACK; -- resets CurrentTransaction to 100 (transaction stack),
-- optionally removes (300, 100) from pg_subtrans,
-- marks T300 as aborted
COMMIT; -- marks T100 as committed
or
ROLLBACK; -- marks T100 as aborted
Visibility:
---
The checks for xmin and xmax are very similar. We look at xmin here:
Traditionally a tuple is visible, if xmin has committed before the
current snapshot was taken, or if xmin == CurrentTransaction().
A subtransaction is considered aborted, if it is marked aborted. Else
it is considered to be in the same state as its parent transaction
(which again can be a subtransaction).
The effects of tup.xmin are considered visible, if ...
(This is not a formal specification. It shall only illustrate the
difference to the existing implementation of HeapTupleSatisfiesXxx()
in tqual.c)
if (tup.XMIN_ABORTED) // flag set by prior visitor
return false;
if (tup.XMIN_COMMITTED) // flag set by prior visitor
return true;
// xmin neither known aborted nor known committed,
// could be active
// or finished and tup not yet visited
for (xmin = tup.xmin; IsValid(xmin); xmin = GetParentXact(xmin)) {
if (TransactionIdDidAbort(xmin)) {
tup.XMIN_ABORTED = true;
return false;
}/*if*/
if (IsCurrentTransaction(xmin)) {
// tup.xmin is one of my own subtransactions,
// it is already committed. So tup can be
// considered belonging to the current transaction.
tup.xmin = xmin;
tup.XMIN_IS_SUB = CurrentTransactionIsSub();
return true; // or rather check cmin ...
}/*if*/
if (TransactionIdDidCommit(xmin)) {
// xmin is a top level transaction
tup.xmin = xmin;
tup.XMIN_IS_SUB = false;
tup.XMIN_COMMITTED = true;
return true;
}/*if*/
if (!tup.XMIN_IS_SUB) {
// Don't try expensive GetParentXact()
break;
}/*if*/
}/*for*/
// tup.xmin still active
return false;
TransactionId GetParentXact(TransactionId xnum) uses pg_subtrans to
find the parent transaction of xnum. It returns InvalidTransaction,
if it doesn't find one.
Performance:
. Zero overhead, if nested transactions are not used.
. BEGIN SUB has to insert a pair of TransactionIds into pg_subtrans.
Apart from that it is not slower than BEGIN top level transaction.
. COMMIT SUB is faster than COMMIT.
. ROLLBACK SUB is much like ROLLBACK, plus (optionally) deletes one
entry from pg_subtrans.
. COMMIT and ROLLBACK of top level transactions don't care about
subtransactions.
. Access a tuple inserted/deleted by a subtransaction: Zero
overhead, if the subtransaction has been rolled back, otherwise the
parent transaction has to be looked up in pg_subtrans (possibly
recursive). This price has to be paid only once per tuple (well, once
for xmin and once for xmax). More accurate: once after the
inserting/deleting top level transaction has finished.
Problems:
-
. pg_subtrans grows by 8 bytes per subtransaction.
. Other pitfalls???
Administration:
---
As soon as a top level transaction has finished, its subtransaction
ids are replaced by the top level transaction id on the next access to
each tuple.
VACUUM (*not* VACUUM tablename) removes old entries from pg_subtrans.
An entry is old, if the parent transaction has finished, before VACUUM
started.
Re: [HACKERS] Threads vs processes - The Apache Way (Re: Path to PostgreSQL
Robert wrote: Hi, Win32 threads support are both going to be a lot of work and maybe we'll need in the future one or both - is there any chance Postgres developers look at the Apache experience? Briefly, Apache 2 had the some problems as are discussed here (need to support Win, problems with Win32 fork, questionable cygwin etc) and they decided to solve it once and for all with their Apache Portable Runtime and Multi-Processing Modules. APR was already mentioned here - now how about MPMs? I am starting to come to the conclusion that the PostgreSQL group is satisfied with cygwin, and the will to create a native Win32 version does not exist outside of a few organizations that are paying developers to create one. Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] pgAdmin2 to be included in Dev-C++
Le Vendredi 10 Mai 2002 13:13, mlw a écrit : I am starting to come to the conclusion that the PostgreSQL group is satisfied with cygwin, and the will to create a native Win32 version does not exist outside of a few organizations that are paying developers to create one. The more important is get a Windows version on the way. pgAdmin2, PostgreSQL Windows GUI, will soon be included in the Dev-C++ development environment, as per discussion with Colin Laplace. Native tools for Windows can have a huge success. Dev-C++ had 1.200.000 hits over the last years. Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. I suggest we focuss on providing a minimal PostgreSQL + Cygwin layer at first. This will give you the required user base to transform PostgreSQL into a multi-platform RDBMS. If we add together direct downloads on http://www.postgresql.org and from partner sites (Dec-C++ on http://www.bloodshed.net), we could well reach the number of 1.000.000 downloads a year under the Windows platform. Cheers, Jean-Michel ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Jean-Michel POURE wrote: Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. I suggest we focuss on providing a minimal PostgreSQL + Cygwin layer at first. This will give you the required user base to transform PostgreSQL into a multi-platform RDBMS. Sorry, I'm not interested in a cygwin version of PostgreSQL. I think it will do more harm than good. If we make it something that people want to try, and then they TRY it, they will find that is sucks, then we lose. It is very hard to remove the bad taste in ones mouth of a poor product. Think Yugo. I have no patience with designed to fail projects, certainly not with my time. PostgreSQL+cygwin is a loser. If I am going to invest my time and effort, I want it to be great. Put it this way. The run of the mill Windows developer will be using MSDN. With MSDN comes MSSQL. To the developer, it is largely free to setup MSSQL to do development work. OK, a conscientious developer will explore options. They will install various systems and try them. Given a cygwin+PostgreSQL system, MSSQL, MySQL, Oracle, DB2, etc. MSSQL will win. MSSQL will win over Oracle for cost and ease of setup. DB2 will lose, similarly to Oracle. MySQL will lose because it sucks. PostgreSQL+cygwin will lose because it will also suck. The idea is to sway Microsoft developers to open source, not give them ammunition of why they think it sucks. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Le Vendredi 10 Mai 2002 14:06, mlw a écrit : Sorry, I'm not interested in a cygwin version of PostgreSQL. I think it will do more harm than good. If we make it something that people want to try, and then they TRY it, they will find that is sucks, then we lose. It is very hard to remove the bad taste in ones mouth of a poor product. Think Yugo. Cygwin is very stable. Its community is relatively small but very actuve. We could well provide a unique installer to hide Cygwin from the user. This can be done compiling Cygwin.dll in a separate user space, as per discussion with Dave Page. I have no patience with designed to fail projects, certainly not with my time. PostgreSQL+cygwin is a loser. If I am going to invest my time and effort, I want it to be great. I agree a native Windows PostgreSQL would be better. OK, a conscientious developer will explore options. They will install various systems and try them. Given a cygwin+PostgreSQL system, MSSQL, MySQL, Oracle, DB2, etc. MSSQL will win. MSSQL will win over Oracle for cost and ease of setup. DB2 will lose, similarly to Oracle. MySQL will lose because it sucks. PostgreSQL+cygwin will lose because it will also suck. MySQL under Windows is based on Cygwin. MySQL sucks and has a 'huge success. So let's do it in three moves : - first move : gain a large audience providing a stable release of Cygwin + PostgreSQL. This could be done within days ... not weeks. This will be much better than MySQL. - second move : release a bundle of pgAdmin2 + PostgreSQL on http://www.postgresql.org, Bloodshed and other sites. - third move : based on 1.000.000 downloads and 100.000 users, feed the community with more developpers, more ideas and more Windows native source-code. So you wron't say I am alone. Rome ne s'est pas faite en une nuit. Cheers, Jean-michel ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] pgaccess
On Thu, May 09, 2002 at 06:33:58PM +0100, Nigel J. Andrews wrote: On Thu, 9 May 2002, Thomas Lockhart wrote: gborg is another way to organize, and of course www.pgaccess.org is a way too. It partly depends on how you see the future of pgaccess. If it stays tightly coupled to pgsql, then perhaps it may as way stay organized with pgsql. I was working on the assumption that PgAccess was tightly coupled to postgres [and versions of postgres] and since Teo was busy with other things and the PG commiters were happy to apply patches that I would be submitting patches to the postgres CVS. What we'll probably need is a note from teo to HACKERS, letting the CVS commiters know who is 'approved' to bless pgaccess patches: i.e. their patches should be commited, and they can bless third party patches. I see no reason why pgaccess needs a separate repository, I presume it can be fetched from the postgress CVS as a single entity. Although I haven't tried this. Works fine. Only tricky part would be providing the windows binary bits (dlls) that have traditionally resided on teo's site. BTW, I had been wondering what to call the Schema tab now that that label is required for schemas rather than design. If you check the archives, when I submitted that patch, I had the forsight to ask if anyone could come up with a better name, forseeing the collison that is happening today: no one came up with anything. I agree it needs renaming. How about one of 'Charting', 'Graphing', 'Diagrams', 'Graphics', 'PrettyPictures', 'BossBait' ... Ross ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] www.pgaccess.org - the official story (the way I saw it)
On Thu, May 09, 2002 at 10:24:00PM +0200, Iavor Raytchev wrote: nice summary of how we got here PROPOSAL What pgaccess needs is some fresh air - it needs a small and fresh team. It needs own web site, own cvs, own mailing list. So that the people who love it, write for it and really need it can be easy to identify and to talk to. This will not break its relationship to PostgreSQL in any way (see 3] above) I'd suggest keeping a copy of pgaccess in the main tree, as well, and pushing versions from the development CVS over on a regular basis. There are basically two types of development that will need to happen: adapting pgaccess to changes in PostgreSQL, and developing new features, on top of the stable release of PostgreSQL. I suggest having two branches at cvs.pgaccess.org: one that tracks HEAD of pgsql, one that uses the latest stable release. As features stablize on the second branch, we push them over to the pgsql branch, then into the pgsql tree, itself. Note that we might be able to write some pgaccess regression tests: at minimum, some sanity tests on the schema we store in the database. At postgresql release time, we'd make sure to get the latest, freshest code into the main tree, and distributions. At the end - I am not experienced how decisions are taken in an open source community - I have no idea what is next. Like this! Out in the open, on the mailing lists. This message of yours was exactly the right thing to post: you contacted the original maintainer, got the 'mantle' passed over to the new group, and continue on. It might be good to get a mailing list at the main site, rather than running our own: that way, people will find it, and Bruce or someone has an easy place to push patches he receives for our approval. May be one can write a summary what are the bad sides of the above proposal. And if there are no such really - we should just proceed and have this nice tool alive and running. Only bad thing would be to let the code in the main postgresql tree rot: either we keep it fresh, or we ask to have it pulled. Ross ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] www.pgaccess.org - the official story (the way I saw it)
Hi everybody, I think, that our job is to help this project to grow up to fit the needs of the people that are using it. In the last months I didn't notice any activity around it. And there are real expectations that are still unsatisfied. This project really needs the fresh air. I think, to have the pgaccess.org is something good, and we shold make this whole thing work. So let's do it! Let's take the last stable release, let's apply the patches, and let's put it on the pgaccess.org, where everybody can reach it easily. If we find some other patches we can easily apply them too. The source is very readable, not too complicated, even as a beginner in tcl I was able to make useful changes. Congratulations to Teo, he did a very good job. To have an enthusiastic group of developers around the pgaccess is good for the postgres teem too. Once again: LET'S DO IT! Levi. P.S: In the near future I'm planning to make the hungarian translation too. On 2002.05.09 22:24 Iavor Raytchev wrote: Hello everybody, The last message of Chris helped me a lot. Let me give a short summary why do we (www.pgaccess.org) do what we do. What are the motives behind and what is the goal. My company needed pgaccess exactly because of the nice visual 'schema'. The 'schema', however, did not behave well if you give it 20-30 tables, so we asked Teo if he plans to patch this. The last official update on the site of Teo is from January 2001. Since then - if there have been patches, they have remained somehow unannounced. Teo said he has no time and we fixed it. We sent Teo patches several times and he came back with the following e-mail - From: Constantin Teodorescu [mailto:[EMAIL PROTECTED]] Sent: Thursday, April 25, 2002 11:16 PM To: Iavor Raytchev Cc: Boyan Dzambazov; [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: Future PgAccess development Dear Iavor, Boyan, Bartus, Chris I am writing to you all because in the last days I have received from all of you different patches and enhancements to PgAccess: - Iavor Boyan in schema module - Bartus in function handling - Chris in report module Thank you all for your work and for developing PgAccess. For the moment, it's impossible to me to receive patches, maintain and push a new version (0.99) of PgAccess. I am involved in a lot of other projects and I have no free time. Furthermore, I am not familiar with the CVS and I have no free time to learn something new right now. I ask you to join your efforts, to exchange between all of you the patches that you have done and to try to set up a web site where PgAccess development could continue in future. I don't know anything about Sourceforge but it seems that they do such a thing. I want to stay close to the discussions concerning the future of PgAccess and I want to contribute with ideas, suggestions. But I feel that I will have no time to build up a new release and I think that your enhancements should be included in the next PostgreSQL release. I have also some changes in the query builder in order to support the outer and inner join capabilities in PostgreSQL 7.x. but they are not finished. Another important thing will be the changes that have to be done in order to support table (row) editing without OID's because 7.2.x versions allow table creation without OID's and table viewing is not working any more. Thank you all , I'm waiting for your answers, Teo To sum it up - - pgaccess has not been officially updated since January 2001 = there is no real interest in it or the interest is not public - the author has no time = the project has no leader - there are several people actively working on it = there is some interest - the author gives us the chance to bring life = if we like it we must get it So we did. We took the www.pgaccess.org domain (on the name of Teo). We set up a server. And we started searching for the latest pgaccess versioin to insert it into the cvs. First I thought Teo should have the latest version. He said - no, it should be with the PostgreSQL distribution. I went there, but it did not seem very fresh. Then I continued my investigation and wrote to the [EMAIL PROTECTED] - my goal was to really find all patches and intersted people and to bring the project to some useful place. Vince Vielhaber wrote back that I should ask the HACKERS. So I did. And now we are here. We heard a lot of opinions from different sides. I would make the following summary - 1] During the last 1 year there has not been an active interest in and/or development of pgaccess. Or if it has been - it has not been very official. 2] Currently there are at least four people who actively need pgaccess and write for it - Bartus, Chris, Boyan and myself. 3] To talk about pgaccess without talking about PostgreSQL is a nonsense -
[HACKERS] internal voting
Hello everybody, After Marc Fournier commented, it is time for pgaccess.org to make a decision. It is clear the project needs the following tools. - web site - mailing list(s) - cvs - bug tracking system It is clear, that there is a small new group with fresh desire to contribute in a dedicated way. It is clear, that pgaccess has only one meaning and this is PostgreSQL. It is clear, that the PostgreSQL core team is very supportive. It is clear, that pgaccess.org efforts can not result in anything good without a close collaboration with the PostgreSQL core team. Now, when we heard many different opinions, the question is - what is the best decision of organization. I would make the following summary, please, send your comments - SUMMARY 1] In terms of infrastructure, a separate web site, mailing list(s) and bug tracking system will increase the flexibility of the pgaccess team and will not create additional (and not very useful) burden for the PostgreSQL core team. The pgaccess is a tool - it is not an integral part of PostgreSQL and does not need day-to-day sharing. In the beginning it will be developed rather for the stable, than for the future versions of PostgreSQL. 2] It is clear that there must be one master copy of the CVS. The possibilities are two - this copy is kept with PostgreSQL or this copy is kept with pgaccess.org If the PostgreSQL core team can provide a CVS repository with similar flexibility to that it would have being based on the pgaccess.org server - I would vote for a PostgreSQL hosted CVS. This will be the naval cord between the two projects. 3] Still - the only thing that is not clear to me is - who is going to collect all patches and make one whole form them. As long as each of us works on a different thing - this should not be a big problem, but still - needs to be one person. Iavor -- www.pgaccess.org ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Cygwin Setup.exe future
Dear all, Here is a copy of a mail received from Robert Collins [EMAIL PROTECTED]. Jean-Michel POURE -Original Message- From: Jean-Michel POURE [mailto:[EMAIL PROTECTED]] Sent: Friday, May 10, 2002 6:30 PM Does setup.exe support uninstalling just like rpm -e package name does? Are dependencies taken into account during uninstall? Not currently, but it should. It will for the cygwin project eventually. Is Cygwin listed in each package dependency? No, as I said - it's optional (because cygwin itself is in base). As we are talking about the use of the codebase for debian, it doesn't really matter what the cygwin setup.ini files contain though. OK then. If I only want ot install PostgreSQL, it will only download the required dependencies, right? Does the installer check version dependency? It only downloads whats needed for what you install. i.e. If you install (say) ncurses, it will download libncurses automatically. And dependencies are transitive. If A requires B, and B requires C. A does not need to list C unless A is directly dependent on C. Setup will 'do the right thing'. Not currently, it's on the todo, as is 'provides:'. Why do you say setup.exe is horrible? Bad architecture? Bad GUI? Doesn't work? The last two days of MD5 related errors that I have not had time to look at? Bad GUI for sure. 1) There should be a small descrition of each package like in .DEB or .RPM packages. A single line is not enought. A Windows user does not know what he is downloading. We want to put popups when you mouse over the packages. Also we want more keyboard control, to assist partially disabled (or whatever the politically correct discription is) users. 2) Packages should be listed in an on-line database. With a full description and manuals. http://www.cygwin.com/packages. However, because setup.ini, like the debian Packages database is federated, this cannot be a complete list, only a list of the cygwin-ditribution's packages. 3) Cygwin installer should be accessible in the Control Panel directly or in Add/Remove software. Presently, it can only be access through the setup.exe There's no reason that it can't be. It'd only take a few registry entries. I've added this to the TODO list. However, the user would have to choose when to register setup.exe, because if the user chooses 'run from net' you wouldn't want the temporary copy of setup.exe to be registered with Add/Remove. 4) We need a setup.exe command line tool to implement limited installers that will not conflict with setup.exe. Example : if we release a limited Cygwin installer at PostgreSQL, we need to be sure it will not conflict with Cygwin. The setup.exe code base in HEAD is being heavily modified for reuse. It's been a long term goal to make setup.exe's code available without a full fork() being made of the code base. The first tool to appear will be a setup.ini linter, similar to lintian, which will use the setup.ini parser, but nothing else. The code is in C++, and is slowly becoming clean. (It started off life as a sort-of C++ using C methodology project, and that made it very hard to change.) What is the on-going work as for setup.exe? Could you describe shortly what is in the hub ? The [EMAIL PROTECTED] mailing list is the best place to discuss setup.exe. I think it's a little off-topic. Suffice to say, setup.exe is not a trivial application, and while a minimal version can be created quite easily, I really believe that contributing to/leveraging setup.exe will be much more time-effective. Rob Current WISHLIST and TODO's from CVS follow: (Some of these have been done, but not tested enough to remove from the list). TODO: * Chooser dialog needs work. * Mirrors list orer is snafued. * Don't downgrade if the curr version is = installed? * support rpm/deb files for reading the package from. (To allow the maintainers the use of rpm/deb tools to create packages.) * make a librar(y|ies) for setup and cygcheck to use containing 1) Something to translate POSIX - native. Currently called cygpath in setup, although this is probably a bad choice of name. 2) Something to return the list of installed packages. 3) Something to return the cygwin mount table. Currently, I have implemented a lightweight setmntent and getmntent using the code in 4) Something to parse a tar file name into package/version or altenatively, return that information from 2) 5) Something to return a list of files associated with a package. * When installing and enough packages default to visible, the RH scrollbar is sometimes hidden. * Mark versions as prev/curr/test in the GUI when clicking through them. * Remove *empty* directories on uninstalls * Correctly overwrite -r--r--r-- files. * Make setup.exe available through Add/Remove WISHLIST: * rsync:// support * Some way to download *all* the source * When clicking on a category that is
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Jean-Michel POURE wrote: Le Vendredi 10 Mai 2002 14:06, mlw a écrit : Sorry, I'm not interested in a cygwin version of PostgreSQL. I think it will do more harm than good. If we make it something that people want to try, and then they TRY it, they will find that is sucks, then we lose. It is very hard to remove the bad taste in ones mouth of a poor product. Think Yugo. Cygwin is very stable. Its community is relatively small but very actuve. We could well provide a unique installer to hide Cygwin from the user. This can be done compiling Cygwin.dll in a separate user space, as per discussion with Dave Page. Here are the problems with cygwin: (1) GNU license issues. (2) Does not work well with anti-virus software (3) Since OS level copy-on-write is negated, process creation is much slower. (4) Since OS level copy-on-write is negated, memory that otherwise would not be allocated to the process is forced to be allocated when the parent process data is copied. As a product manager, I would not commit to using a cygwin application in production. Do you know of any long-uptime systems using cygwin? PostgreSQL would need to run for months. I would view it as a risk. Lastly, a Windows program is expected to be a Windows program. Native paths need to be used, like C:\My Database, D:\My Postgres, or something like that. Native tools must be used to manage it. I have no patience with designed to fail projects, certainly not with my MySQL under Windows is based on Cygwin. MySQL sucks and has a 'huge success. Define Success So let's do it in three moves : - first move : gain a large audience providing a stable release of Cygwin + PostgreSQL. This could be done within days ... not weeks. This will be much better than MySQL. No interest in cygwin, sorry. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Hi everyone, Jean-Michel POURE wrote: snip - second move : release a bundle of pgAdmin2 + PostgreSQL on http://www.postgresql.org, Bloodshed and other sites. Don't know if it's useful to know, but a PostgreSQL project got setup on Sourceforge recently (no CVS), pretty much just so PostgreSQL could be included in the Database Foundry on the Sourceforge site. :) http://www.sf.net/projects/pgsql And then I started a new contract and haven't had time to do anything with it (oh well). Regards and best wishes, Justin Clift snip Rome ne s'est pas faite en une nuit. Cheers, Jean-michel ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster -- My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there. - Indira Gandhi ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Dear Mark, Agreed except for paths (see below). But now that we agree, why not move to Windows in three steps: 1) Release a minimal Cygwin + PostgreSQL installer, 2) Have 100.000 downloads or more Windows developpers, 3) Work as a team on a Windows port. By the way : Cygwin accepts both Windows AND Unix paths depending on installation options. Cygwin is able to understand C:\program files\postgresql\var\lib\pgsql, /cygdrive/../var/lib/pgsql or simply /var/lib/pgsql. Cheers, Jean-Michel Here are the problems with cygwin: (1) GNU license issues. (2) Does not work well with anti-virus software (3) Since OS level copy-on-write is negated, process creation is much slower. (4) Since OS level copy-on-write is negated, memory that otherwise would not be allocated to the process is forced to be allocated when the parent process data is copied. As a product manager, I would not commit to using a cygwin application in production. Do you know of any long-uptime systems using cygwin? PostgreSQL would need to run for months. I would view it as a risk. Lastly, a Windows program is expected to be a Windows program. Native paths need to be used, like C:\My Database, D:\My Postgres, or something like that. Native tools must be used to manage it. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Jean-Michel POURE wrote: Dear Mark, Agreed except for paths (see below). But now that we agree, why not move to Windows in three steps: 1) Release a minimal Cygwin + PostgreSQL installer, 2) Have 100.000 downloads or more Windows developpers, 3) Work as a team on a Windows port. By the way : Cygwin accepts both Windows AND Unix paths depending on installation options. Cygwin is able to understand C:\program files\postgresql\var\lib\pgsql, /cygdrive/../var/lib/pgsql or simply /var/lib/pgsql. The point you are missing is that a cygwin version of postgres is unacceptable. Doing an installer BEFORE commiting to making the system excellent is putting the cart before the horse. The LAST thing we want is 100,000+ Windows users downloading PostgreSQL and getting a cygwin version. The first time it doesn't work because of anti-virus software, they'll call it junk. When they test performance and see that it sucks, they'll remove the software. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] internal voting
On Fri, 10 May 2002 10:58:28 +0200 Iavor Raytchev [EMAIL PROTECTED] wrote: Hello everybody, After Marc Fournier commented, it is time for pgaccess.org to make a decision. It is clear the project needs the following tools. - web site - mailing list(s) - cvs - bug tracking system It is clear, that there is a small new group with fresh desire to contribute in a dedicated way. It is clear, that pgaccess has only one meaning and this is PostgreSQL. It is clear, that the PostgreSQL core team is very supportive. It is clear, that pgaccess.org efforts can not result in anything good without a close collaboration with the PostgreSQL core team. Now, when we heard many different opinions, the question is - what is the best decision of organization. I would make the following summary, please, send your comments - SUMMARY 1] In terms of infrastructure, a separate web site, mailing list(s) and bug tracking system will increase the flexibility of the pgaccess team and will not create additional (and not very useful) burden for the PostgreSQL core team. The pgaccess is a tool - it is not an integral part of PostgreSQL and does not need day-to-day sharing. In the beginning it will be developed rather for the stable, than for the future versions of PostgreSQL. 2] It is clear that there must be one master copy of the CVS. The possibilities are two - this copy is kept with PostgreSQL or this copy is kept with pgaccess.org If the PostgreSQL core team can provide a CVS repository with similar flexibility to that it would have being based on the pgaccess.org server - I would vote for a PostgreSQL hosted CVS. This will be the naval cord between the two projects. 3] Still - the only thing that is not clear to me is - who is going to collect all patches and make one whole form them. As long as each of us works on a different thing - this should not be a big problem, but still - needs to be one person. This looks all good to me, except I have one question: How will pgaccess be distributed? Personally, I like the idea that PG comes with pgaccess in the distribution, so I would hate to see that go away. Even though there are people that don't use pgaccess, it is always nice to have a default tool that comes with PG (yes, I know there is psql). --brett p.s. I am willing to help out as well... ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Le Vendredi 10 Mai 2002 15:33, mlw a écrit : The first time it doesn't work because of anti-virus software, they'll call it junk. When they test performance and see that it sucks, they'll remove the software. Dear Mark, PostgreSQL will work well if cygwin.dll is compiled in a separate workspace and installed under C:/program files/postgresql and hidden from users. I agree it will not be able to serve a 50 TPS system. Furthermore : MySQL sucks, Windoze sucks and Microsoft is violating our private rights everyday. So if you care for freedom, we are going to release this f** Cygwin minimal installer. Don't you think my friend? Noone will complain about it. Do you see demonstrations in the street against Microsoft? The answer is no. Therefore, I believe noone will complain about a minimal Cygwin + PostgreSQL installer. This will only be the beginning of a complete Windows port. Which can also be expressed as : Il faut laisser le temps au temps Il n'y a pas le feu au lac Cheers, Jean-Michel ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] FW: Cygwin PostgreSQL Information and Suggestions
Some comments from Jason Tishler the Cygwin-PostgreSQL maintainer... -Original Message- From: Jason Tishler [mailto:[EMAIL PROTECTED]] Sent: 10 May 2002 15:00 To: Dave Page Cc: [EMAIL PROTECTED] Subject: Cygwin PostgreSQL Information and Suggestions Dave, Would you forward this to pgsql-hackers since I'm not subscribed? On Thu, May 09, 2002 at 10:45:42PM +0100, Dave Page wrote: -Original Message- From: Jason Tishler [mailto:[EMAIL PROTECTED]] Sent: 09 May 2002 21:52 To: Dave Page On Thu, May 09, 2002 at 07:51:33PM +0100, Dave Page wrote: BTW Are you aware there is currently a rather busy thread about native Windows/Beos ports on -hackers... No, I'm not subscribed, but I just read all that I could find in the archives. [snip] ...which is currently drifting towards a cutdown Cygwin version? Maybe I'll be out of (another) job soon? :,) [snip] Personnally, I think (from a 'good for PostgreSQL' rather than 'good for Cygwin' perspective) that the way forward is a Cygwin based system but using a tailored downloader/installer that installs the system 'like a Windows app' (and quickly easily etc.) rather than the current way which is Windows 'being' *nix. I think that's very offputting for many potential users (as others have said on the -hackers thread). I agree with the above, but more can be done with Cygwin and its setup.exe that can give a fair amount of bang for the buck for some good short time gains too. I will give some details below. I also wanted to dispel some misinformation (IMO) that I perceived from the above mentioned posts and/or elaborate on some of the items: 1. Cygwin's setup.exe supports categories and dependencies. Hence, there is no reason to install all Cygwin packages in order to ensure properly PostgreSQL operation. Someone just has to determine what is the minimal set of packages necessary for PostgreSQL and I will update the setup.hint accordingly. The current setup.hint is as follows: sdesc: PostgreSQL Data Base Management System category: Database requires: ash cygwin readline zlib libreadline5 Sorry, but since I install all Cygwin packages plus about 30 additional ones I haven't desire to determine what are the minimal requirements. 2. Cygwin's setup.exe is customizable. There is a tool called upset that generates the setup.ini file that drives setup.exe. PostgreSQL could offer a customized setup. For example, this is what the XEmacs folks are doing. 3. Cygwin's setup.exe can run package specific postinstall scripts during the installation. Hence, someone could automate the steps enumerated (e.g., postmaster NT service installation, initdb, etc.) in my README: http://www.tishler.net/jason/software/postgresql/postgresql-7.2.1.README to ease the installation burden. 4. Cygwin PostgreSQL is perceived to have poor performance. I have never done any benchmarks regarding this issue, but apparently Terry Carlin (from the defunct Great Bridge) did: http://archives.postgresql.org/pgsql-cygwin/2001-08/msg00029.php Specifically, he indicates the following: BTW, Up through 40 users, PostgreSQL under CYGWIN using the TPC-C benchmark performed very much the same as Linux PostgreSQL on the exact hardware. 5. Cygwin PostgreSQL is perceived to have poor reliability. Unfortunately, I have not been able to gather data to concur or refute this perception due a sudden job change last summer. :,) However, there are reports such as the following on the pgsql-cygwin list: http://archives.postgresql.org/pgsql-cygwin/2002-04/msg00021.php IMO, the biggest reliability issue with Cygwin PostgreSQL is it's dependency on cygipc. There is some very recent work to create a Cygwin daemon to support features such as System V IPC. So soon the cygipc dependency and its problems will be going way. Those interested in a Windows PostgreSQL should possibly consider contributing in this area or other hard edges (due to Windows-isms) that would improve the reliability of Cygwin PostgreSQL. BTW, I have found the Cygwin core developers very responsive to PostgreSQL problems because it drives the Cygwin DLL harder than most other applications. 6. Satisfying the Cygwin license for binary distribution is very simple. Just include the source for the Cygwin DLL and all executables that are linked with it in your distribution package. It is really that easy. Jason ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Unbounded (Possibly) Database Size Increase - Test Case
Mark kirkwood [EMAIL PROTECTED] writes: Was the FSM size parameter set large enough to cover the amount of space you need the system to be able to recycle --- viz, the amount used between vacuum runs? As with most everything else in PG, the default value is not real large: 1 pages = 80MB. I thought I was generous here ...~ 960M free space map max_fsm_relations = 100# min 10, fsm is free space map max_fsm_pages = 12 # min 1000, fsm is free space map I think I need to count how many vacuums performed during the test, so I can work out if this amount should have been enough. I timed a vacuum now at 12 minutes. (So with 10 concurrent threads it could take a lot longer during the run ) Keep in mind also that you need enough FSM entries to keep track of partially-full pages. To really lock things down and guarantee no table growth you might need one FSM slot for every page in your relations. In practice you should be able to get away with much less than that: you certainly don't need entries for pages with no free space, and pages with only a little free space shouldn't be worth tracking either. But if your situation is 100% update turnover between vacuums then you could have a worst-case situation where all the pages have roughly 50% free space right after a vacuum, and if you fail to track them *all* then you're probably going to see some table growth in the next cycle. I believe that with a more reasonable vacuum frequency (vacuum after 10% to 25% turnover, say) the FSM requirements should be a lot less. But I have not had time to do any experimentation to arrive at a rule of thumb for vacuum frequency vs. FSM requirements. If you or someone could run some experiments, it'd be a big help. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] troubleshooting pointers
Tom Lane wrote: I don't much care for the thought of trawling every expression tree looking for functions-returning-set during plan shutdown, so the thought that comes to mind is to expect functions that want a shutdown callback to register themselves somehow. Adding a list of callbacks to ExprContext seems pretty reasonable, but you'd also need some link in ReturnSetInfo to let the function find the ExprContext to register itself with. Then FreeExprContext would call the callbacks. I've made changes which fix this and will send them in with a revised SRF patch later today. Summary of design: 1.) moved the execution_state struct and ExecStatus enum to executor.h 2.) added void *es member to ExprContext 3.) added econtext member to ReturnSetInfo 4.) set rsi-econtext on the way in at ExecMakeFunctionResult() 5.) set rsi-econtext-es on the way in at fmgr_sql() 6.) used econtext-es on the way out at ExecFreeExprContext() to call ExecutorEnd() if needed (because postquel_execute() never got the chance). One note: I changed ExecFreeExprContext() because that's where all the action was for SQL function calls. FreeExprContext() was not involved for the test case, but it looked like it probably should have the same changes, so I made them there also. Hmm ... another advantage of doing this is that the function would be able to find the ecxt_per_query_memory associated with the ExprContext. That would be a Good Thing. What does this allow done that can't be done today? We should also think about the fcache (FunctionCache) struct and whether that needs to tie into this. See the FIXME in utils/fcache.h. While I was at it, I added an fcache member to ExprContext, and populated it in ExecMakeFunctionResult() for SRF cases. I wasn't sure what else to do with it at the moment, but at least it is a step in the right direction. Joe ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] pgAdmin2 to be included in Dev-C++
Jean-Michel POURE wrote: Le Vendredi 10 Mai 2002 15:33, mlw a écrit : The first time it doesn't work because of anti-virus software, they'll call it junk. When they test performance and see that it sucks, they'll remove the software. Dear Mark, PostgreSQL will work well if cygwin.dll is compiled in a separate workspace and installed under C:/program files/postgresql and hidden from users. I agree it will not be able to serve a 50 TPS system. Then what is the point? Furthermore : MySQL sucks, Windoze sucks and Microsoft is violating our private rights everyday. So if you care for freedom, we are going to release this f** Cygwin minimal installer. Don't get me wrong, I would love it if Windows were no longer around. I think a cygwin version of PostgreSQL will not further your objective. Windows users will not be seeing the cream of the crop, they will be seeing a quick and dirty hack. In the words of Martin Luther King, Excellence is the best revenge. The risk you are taking is this: If you rush out a cygwin version of PostgreSQL there may be a lasting impression that PostgreSQL is of poor quality. How will Windows developers create C language function extensions? Using cygwin and gcc as well? These guys can't do crap without VisualStudio. Seriously, don't do it. Please don't do it. If we want to make a serious presence in the Windows market, it is better to take our time and do it well or not at all. Don't you think my friend? Noone will complain about it. Do you see demonstrations in the street against Microsoft? The answer is no. Therefore, I believe noone will complain about a minimal Cygwin + PostgreSQL installer. This will only be the beginning of a complete Windows port. I completely disagree. Let me ask you. Have you ever used Windows? I mean as your primary system? Have you ever thrilled at getting something new for your Windows system? (Like you do with you current system.) I'm not ashamed to admit I used to love Windows. Before Linux was usable, and before FreeBSD was unencumbered, it was the best system a user could get for the money. Windows was fun, especially if you had the SDK/DDK and knew how to use it. Think about Linux and Wine. Linux users do not like Wine applications, no matter how hidden they are. Franken-wine they are called, and fail quickly. Look at CorelDraw, a miserable failure. Cygwin on Windows is analogous to Wine on Linux. A native PostgreSQL on Windows would rock the Windows world. It would kick MSSQL's butt for many applications. I think you underestimate Windows and Windows users if you think a cygwin version will satisfy them. The mistake is thinking that they are the ignorant unwashed masses that so many UNIX people seem to think they are. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] troubleshooting pointers
Joe Conway [EMAIL PROTECTED] writes:
Tom Lane wrote:
Adding a list of callbacks to
ExprContext seems pretty reasonable, but you'd also need some link in
ReturnSetInfo to let the function find the ExprContext to register
itself with. Then FreeExprContext would call the callbacks.
I've made changes which fix this and will send them in with a revised
SRF patch later today. Summary of design:
1.) moved the execution_state struct and ExecStatus enum to executor.h
2.) added void *es member to ExprContext
3.) added econtext member to ReturnSetInfo
4.) set rsi-econtext on the way in at ExecMakeFunctionResult()
5.) set rsi-econtext-es on the way in at fmgr_sql()
6.) used econtext-es on the way out at ExecFreeExprContext() to call
ExecutorEnd() if needed (because postquel_execute() never got the chance).
Um. I don't like that; it assumes not only that ExecutorEnd is the only
kind of callback needed, but also that there is at most one function
per ExprContext that needs a shutdown callback. Neither of these
assumptions hold water IMO.
The design I had in mind was more like this: add to ExprContext a list
header field pointing to a list of structs along the lines of
struct exprcontext_callback {
struct exprcontext_callback *next;
void (*function) (Datum);
Datum arg;
}
and then call each specified function with given argument during
FreeExprContext. Probably ought to be careful to do that in reverse
order of registration. We'd also need to invent a RescanExprContext
operation to call the callbacks during a Rescan. The use of Datum
(and not, say, void *) as PG's standard callback arg type was settled on
some time ago --- originally for on_proc_exit IIRC --- and seems to have
worked well enough.
Hmm ... another advantage of doing this is that the function would be
able to find the ecxt_per_query_memory associated with the ExprContext.
That would be a Good Thing.
What does this allow done that can't be done today?
It provides a place for the function to allocate stuff that needs to
live over multiple calls, ie, until it gets its shutdown callback.
Right now a function has to use TransactionCommandContext for that,
but that's really too coarse-grained.
We should also think about the fcache (FunctionCache) struct and whether
that needs to tie into this. See the FIXME in utils/fcache.h.
While I was at it, I added an fcache member to ExprContext, and
populated it in ExecMakeFunctionResult() for SRF cases. I wasn't sure
what else to do with it at the moment, but at least it is a step in the
right direction.
Well, I was debating whether that's good or not. The existing fcache
approach is wrong (per cited FIXME); it might be better not to propagate
access of it into more places. Unless you can see a specific reason to
allow the function to have access to the fcache struct, I think I'm
inclined not to.
What's really more relevant here is that during the hypothetical new
RescanExprContext function, we ought to go around and clear any fcaches
in the context that have setArgsValid = true, so that they will be
restarted afresh during the next scan of the plan. (The fact that that
doesn't happen now is another shortcoming of the existing set-functions-
in-expressions code.) So this suggests making a callback function type
specifically to do that, and registering every fcache that is executing
a set function in the callback list...
regards, tom lane
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] Queries using rules show no rows modified?
Hiroshi Inoue [EMAIL PROTECTED] writes: What should the backends return for complicated rewrites ? Well, given that we have only two or three fields to work in, it obviously has to be a very simplified view of what happened. But we have to define *something*. And how should/could clients handle the results ? It doesn't seem easy to me and it seems a flaw of rule system. No, the problem is that the command tag API was designed without any thought for rule rewriting. But I don't think it's worth revising that API completely. Even if we did, we'd still have to define what behavior would be seen by clients that use the existing PQcmdTuples, etc, calls; so we'd still have to solve these same issues. Come on, guys, work with me a little here. I've thrown out several alternative suggestions already, and all I've gotten from either of you is refusal to think about the problem. I was thinking last night that it might help to break down the issue a little bit. We have either two or three result fields to think about: the tag name, the tuple count, and in the case of INSERT the inserted row OID. Let's consider each one independently. 1. The tag name: AFAICS, this ought *always* to match the type of the original command submitted by the client. Doing otherwise could confuse clients that are submitting multiple commands per query string. Besides, the only possible downside from making this requirement is that we couldn't send back an insertion OID when the original command was an update or delete. How likely is it that a client would expect to be able to get an insertion OID from such a command? 2. The inserted row OID: per above, will be supplied only if the original command was an INSERT. If the original insert command is not removed (no INSTEAD rule), then I think this result should clearly come from the execution of the original command, regardless of any additional INSERTs added by rules. If the original command is removed by INSTEAD, then we can distinguish three sub-cases: a. No INSERTs in rewriter output: easy, we must return 0. b. Exactly one INSERT in rewriter output: pretty easy to agree that we should return this command's result. c: More than one INSERT in rewriter output: we have a couple of possibilities here. It'd be reasonable to directly use the result of the last INSERT, or we could total the results of all the INSERTs (ie, if taken together they insert a sum total of one row, return that row OID; else return 0). Maybe there are other possible behaviors. Any thoughts? 3. The tuple count: this seems the most contentious issue. Again, if there is no INSTEAD rule I'd be strongly inclined to say we should just return the count from the original command, ignoring any commands added by rules. If there is an INSTEAD, we've discussed several possibilities: use result of last command in the rewritten series, use result of last command of same type as original command, sum up the results of all the rewritten commands, maybe some others that I forgot. Given Michael's concern about being able to tell that something happened, I'm inclined to go with the summing-up behavior in the INSTEAD cases. This would lead to the following boiled-down behavior: A. If original command is executed (no INSTEAD), return its tag as-is, regardless of commands added by rules. B. If original command is not executed, then return its tag name plus required fields defined as follows: tuple count is sum of tuple counts of all replacement commands. For an INSERT, if the replacement commands taken together inserted a grand total of exactly one tuple, return that tuple's OID; else return 0. This is not completely consistent in pathological cases: you could get a tuple OID returned even when the returned tuple count is greater than one, which is not a possible case currently. (This would happen given a rewrite consisting of a single-row INSERT plus additional update or delete actions that affect some rows.) But that seems pretty oddball. In all the simple cases I think this proposal gives reasonable behavior. A tighter definition for case B would use the sum of the tuple counts of only the replacement actions that are of the same type as the original command. This would eliminate the possible inconsistency between tuple count and insert OID results, and it's arguably saner than the above proposal: if it says UPDATE 4, that should mean that four rows were updated, not that something else happened to four rows. But it would not meet Michael's concern about using PQcmdTuples to tell that something happened. I could live with either definition. Thoughts, different proposals, alternative ways of breaking down the problem? regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Queries using rules show no rows modified?
Jan Wieck [EMAIL PROTECTED] writes: IMHO the answer should only be a number if the rewritten querytree list consists of one query of the same command type. everything else has to lead into unknown. I think you can easily generalize that to the statement that the result should be the sum of the rewritten operations of the same type as the original query; requiring there to be exactly one seems overly restrictive. Michael seems to feel that the tuple count should be nonzero if any of the replacement operations did anything at all. This does not make a lot of sense at the command tag level (UPDATE 4 might not mean that 4 tuples were updated) but if you look at the definition of PQcmdTuples (returns the number of rows affected by the SQL command) it's not so unreasonable. And I can see the point of wanting to know whether anything happened. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Queries using rules show no rows modified?
Hannu Krosing [EMAIL PROTECTED] writes: The problem seems to be that recent changes broke updatable views for a few users. So have these basic options: 1. revert the changes until we have a consensus on doing the right thing (seems best to me) Reverting is not an option, unless you want to also revert 7.2's change of execution order of ON INSERT rules; which I would resist as the new behavior is clearly better. But given that, both 7.2 and 7.2.1 have command-tag behavior that is making users unhappy ... in different ways. I think we should first concentrate on defining what we think the right behavior should be in the long term. Only after we know that can we devise a plan for getting there. I believe all the concrete suggestions that have been made so far could be implemented straight away in 7.2.2 (if there is a 7.2.2) ... but we might settle on something that represents a bigger change with more backwards-compatibility problems. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] the parsing of parameters
Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: For this particular application, at least, I do not see the value ... in fact this seems more likely to break stuff than help. If the application does not know what the datatypes are supposed to be, how is it going to call the prepared statement? Right now using UNKNOWN_OID in that place leads to a parse error, what makes me feel absolutely comfortable that there will be nobody using it today. So what kind of break are you talking about? What I mean is that I don't see how an application is going to use PREPARE/EXECUTE without knowing the data types of the values it has to send for EXECUTE. Inside SPI you could maybe do it, since the calling code can examine the modified argtype array, but there is no such back-communication channel for PREPARE. This holds for both textual and binary kinds of EXECUTE: how do you know what you are supposed to send? You could possibly get away with that for a textual interface (always pass quoted literals), but it would surely destroy any chance of having a binary protocol for passing parameters to prepared statements. Right. And BTW, how do you propose that the client application passes the values in binary form anyway? Same way as binary cursors work today, with the same ensuing platform and version dependencies. Maybe someday we'll improve on that, but that's a different project from supporting PREPARE/EXECUTE. I think the backend is the only one who can convert into it's personal, binary format. Wouldn't anything else lead to security holes? Good point; might need to restrict the operation to superusers. There are DB interfaces that allow a generic application to get a description of the result set (column names, types) even before telling the data types of all parameters. Our ODBC driver for example has it's own more or less complete SQL parser to deal with that case! I don't see THAT implementation very superior compared to the ability to ask the DB server for a guess. I thought that this PREPARE statement will be used by such interfaces in the future, no? Hmm. So your vision of PREPARE would allow the backend to reply with a list of parameter types. How would you envision that working exactly? regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Threads vs processes - The Apache Way (Re: Path to PostgreSQL
mlw [EMAIL PROTECTED] writes: Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. You are not going to get any buy-in with such ridiculous claims as that. If the total cost of a native Windows port were O(100 hours), it'd have been done long since. Add a couple zeroes on the end and I'd start to believe that you might have some grasp of the problem. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] the parsing of parameters
On Fri, May 10, 2002 at 11:17:39AM -0400, Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: For this particular application, at least, I do not see the value ... in fact this seems more likely to break stuff than help. If the application does not know what the datatypes are supposed to be, how is it going to call the prepared statement? Right now using UNKNOWN_OID in that place leads to a parse error, what makes me feel absolutely comfortable that there will be nobody using it today. So what kind of break are you talking about? What I mean is that I don't see how an application is going to use PREPARE/EXECUTE without knowing the data types of the values it has to send for EXECUTE. Inside SPI you could maybe do it, since the calling code can examine the modified argtype array, but there is no such back-communication channel for PREPARE. This holds for both textual and binary kinds of EXECUTE: how do you know what you are supposed to send? In my original PREPARE/EXECUTE patch (it works in 7.1): PREPARE name AS select * from tab where data=$1 USING text; EXECUTE name USING 'nice text data'; IMHO is possible think about EXECUTE name USING 'nice text'::text; or other cast methods. Karel -- Karel Zak [EMAIL PROTECTED] http://home.zf.jcu.cz/~zakkr/ C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] FW: Cygwin PostgreSQL Information and Suggestions
Dave Page [EMAIL PROTECTED] forwards: 4. Cygwin PostgreSQL is perceived to have poor performance. I have never done any benchmarks regarding this issue, but apparently Terry Carlin (from the defunct Great Bridge) did: http://archives.postgresql.org/pgsql-cygwin/2001-08/msg00029.php Specifically, he indicates the following: BTW, Up through 40 users, PostgreSQL under CYGWIN using the TPC-C benchmark performed very much the same as Linux PostgreSQL on the exact hardware. It should be noted that the benchmark Terry is describing fires up N concurrent backends and then measures the runtime for a specific query workload. So it's not measuring connection startup time, which is alleged by some to be Cygwin's weak spot. Nonetheless, I invite the Postgres-on-Cygwin-isn't-worth-our-time camp to produce some benchmarks supporting their position. I'm getting tired of reading unsubstantiated assertions. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] FW: Cygwin PostgreSQL Information and Suggestions
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Tom Lane Sent: Friday, May 10, 2002 12:31 PM To: Dave Page Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [HACKERS] FW: Cygwin PostgreSQL Information and Suggestions Dave Page [EMAIL PROTECTED] forwards: 4. Cygwin PostgreSQL is perceived to have poor performance. I have never done any benchmarks regarding this issue, but apparently Terry Carlin (from the defunct Great Bridge) did: http://archives.postgresql.org/pgsql-cygwin/2001-08/msg00029.php Specifically, he indicates the following: BTW, Up through 40 users, PostgreSQL under CYGWIN using the TPC-C benchmark performed very much the same as Linux PostgreSQL on the exact hardware. It should be noted that the benchmark Terry is describing fires up N concurrent backends and then measures the runtime for a specific query workload. So it's not measuring connection startup time, which is alleged by some to be Cygwin's weak spot. Nonetheless, I invite the Postgres-on-Cygwin-isn't-worth-our-time camp to produce some benchmarks supporting their position. I'm getting tired of reading unsubstantiated assertions. ... and it's worth remembering, too, that for some cases, connect time is completely unimportant: most of my work against PG is using shared, persistent connections from a web app (Zope); it could take 20 mins to make the initial connection and I'd still be happy. (Note to hackers: do not implement this 20min connect, though. :) ) - J. Joel BURTON | [EMAIL PROTECTED] | joelburton.com | aim: wjoelburton Knowledge Management Technology Consultant ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Threads vs processes - The Apache Way (Re: Path to PostgreSQL
Tom Lane wrote: mlw [EMAIL PROTECTED] writes: Without some buy-in from the core team, I'm not sure I am willing to spend my time on it. If someone would be willing to fund the 100 or so man-hours required to do it, then that would be a different story. You are not going to get any buy-in with such ridiculous claims as that. If the total cost of a native Windows port were O(100 hours), it'd have been done long since. Add a couple zeroes on the end and I'd start to believe that you might have some grasp of the problem. I was basing my estimates on a couple things. Please feel free to correct me where I'm wrong. Dann Corbit mentioned a number of, I think I recall, a couple hundred man-hours for their port. My approach would be to find all the global variables setup by postmaster, not all the globals, mind you. Just the ones initialized by postmaster. Move them to a structure. That structure would be capable of being copied to the child process. In the area where forking the postgres process happens, I would ifdef that area with an HAS_FORK The Windows portion would use CreateProcess. The Windows version of postgres would contact the postmaster and get its copy of the globals struct. The code to transfer ownership of sockets, files, and memory would have to be written also. I would only minimally change the back-end code, it would still be built with cygwin tools only directed not to link against the cygwin.dll. (The same goes for the utilities as well.) A thin port layer could then be constructed by either implementing sysv/UNIX replacements, or a more simple API as needed in the code, like your shared memory and semaphore APIs. Does that sound like an unworkable plan? ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Monitoring backend activities
Hi everybody, I'm a hookie in this discussion list. Well, my intent is to get some informations about PostgreSQL internals to work on a project. There is an excellent GPL'ed tool to work with Oracle called TOra. It is as good as TOAD and SQL Navigator from Quest Software. As a meaning of collaborate with the Open Source world i was thinking in port TOra to PostgreSQL. So, we'll have a great database and a great tool to manage it. Problem is: reading PostgreSQL documentation i didn't find any information about system tables having runtime informations as Oracle has. And one of the great features of TOra is the possibility to see in charts, in real-time, all kind of I/O operations, memory usage, queries being executed, etc... If i didn't make myself clear, please point your browser to http://www.globecom.se/tora/ and see what i am suggesting to adapt to PostgreSQL. I hope i did not disturb anybody here. And, keep doing your great job. We are in debt with you guys! Best regards, Daniel. __ Do You Yahoo!? Yahoo! Shopping - Mother's Day is May 12th! http://shopping.yahoo.com ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Monitoring backend activities
Hi everybody, I'm a hookie in this discussion list. Well, my intent is to get some informations about PostgreSQL internals to work on a project. There is an excellent GPL'ed tool to work with Oracle called TOra. It is as good as TOAD and SQL Navigator from Quest Software. As a meaning of collaborate with the Open Source world i was thinking in port TOra to PostgreSQL. So, we'll have a great database and a great tool to manage it. I think that would be rookie; the term hookie refers to what you're playing if you skip school. Problem is: reading PostgreSQL documentation i didn't find any information about system tables having runtime informations as Oracle has. And one of the great features of TOra is the possibility to see in charts, in real-time, all kind of I/O operations, memory usage, queries being executed, etc... The only problem I see is that TOra already seems quite well supported for PostgreSQL. I'm running it at the moment, and it works quite well... -- (concatenate 'string cbbrowne cbbrowne.com) http://www.cbbrowne.com/info/lsf.html Put simply, the antitrust laws in this country are basically a joke, protecting us just enough to not have to re-name our park service the Phillip Morris National Park Service. -- Courtney Love, Salon.com, June 14, 2000 -- (concatenate 'string cbbrowne ntlug.org) http://www.cbbrowne.com/info/rdbms.html Rules of the Evil Overlord #220. Whatever my one vulnerability is, I will fake a different one. For example, ordering all mirrors removed from the palace, screaming and flinching whenever someone accidentally holds up a mirror, etc. In the climax when the hero whips out a mirror and thrusts it at my face, my reaction will be ``Hmm...I think I need a shave.'' http://www.eviloverlord.com/ -- (reverse (concatenate 'string moc.enworbbc sirhc)) http://www.cbbrowne.com/info/linuxxian.html As of next Monday, MACLISP will no longer support list structure. Please downgrade your programs. msg16907/pgp0.pgp Description: PGP signature
Re: [HACKERS] the parsing of parameters
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: There are DB interfaces that allow a generic application to get a description of the result set (column names, types) even before telling the data types of all parameters. Our ODBC driver for example has it's own more or less complete SQL parser to deal with that case! I don't see THAT implementation very superior compared to the ability to ask the DB server for a guess. I thought that this PREPARE statement will be used by such interfaces in the future, no? Hmm. So your vision of PREPARE would allow the backend to reply with a list of parameter types. How would you envision that working exactly? I guess there's some sort of statement identifier you use to refer to something you've prepared. Wouldn't a function call returning a list of names or type oid's be sufficient? Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] the parsing of parameters
Jan Wieck [EMAIL PROTECTED] writes: Hmm. So your vision of PREPARE would allow the backend to reply with a list of parameter types. How would you envision that working exactly? I guess there's some sort of statement identifier you use to refer to something you've prepared. Wouldn't a function call returning a list of names or type oid's be sufficient? I was thinking of having the type names returned unconditionally, perhaps like a SELECT result (compare the new behavior of EXPLAIN). But if we assume that this won't be a commonly used feature, maybe a separate inquiry operation is better. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pgaccess
Nigel J. Andrews writes: BTW, I had been wondering what to call the Schema tab now that that label is required for schemas rather than design. Design? -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] the parsing of parameters
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Hmm. So your vision of PREPARE would allow the backend to reply with a list of parameter types. How would you envision that working exactly? I guess there's some sort of statement identifier you use to refer to something you've prepared. Wouldn't a function call returning a list of names or type oid's be sufficient? I was thinking of having the type names returned unconditionally, perhaps like a SELECT result (compare the new behavior of EXPLAIN). But if we assume that this won't be a commonly used feature, maybe a separate inquiry operation is better. I wouldn't mind. One way or the other is okay with me. Reminds me though of another feature we should have on the TODO. INSERT/UPDATE/DELETE ... RETURNING ... Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] pgaccess
On Fri, May 10, 2002 at 09:13:20PM +0200, Peter Eisentraut wrote: Nigel J. Andrews writes: BTW, I had been wondering what to call the Schema tab now that that label is required for schemas rather than design. Design? Thought about it, but it seems to 'active' for what's behind the tab: drawing pretty pictures. There's no way to draw arbitrary tables and create them, for example. Also, 'Design' is used a the button contrasting to 'New' and 'Open' for things like the Table tab. I think I'm leaning toward Diagram, since that's the verb as well as the noun. Hmm, on further inspection, all the tabs are plural nouns, so Designs or Diagrams, perhaps. Ross ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Making the regression tests locale-proof
Since locale support is now enabled by default, it is desirable that the regression tests can pass if the clusters locale is not C. As a first step I have included the following statements in pg_regress right after the database is created: alter database $dbname set lc_messages to 'C'; alter database $dbname set lc_monetary to 'C'; alter database $dbname set lc_numeric to 'C'; alter database $dbname set lc_time to 'C'; This gets rid of a boatload of failures related to number formatting. For that purpose I have changed the permissions on these options to USERSET. (I'm still debating making lc_messages SUSET, because otherwise users can screw with admins by changing the language of the log output all the time. Comments?) The remaining issue is the sort order. I think this can be solved for practical purposes by creating two expected files for each affected test, say char.out and char-locale.out. The regression test driver would try the first one, if that fails try the second one. The assumption here is that all locales will choose the same sort order as long as they're dealing only with the core 26 letters. This does not have to be true in theory, but I think it works for the vast majority of practical cases. We could also cut down the number of affected tests by making the select_implicit and select_having not use mixed-case strings in the test tables. Then we have only char, varchar, and select_views left. Comments? -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Native Win32, How about this?
A binary version of PostgreSQL for Windows should not use the cygwin dll. I know and understand there is some disagreement with this position, but in this I'm sure about this. The tools used to create the binary need not be Microsoft, many venders have used Borland or Watcom, the run of the mill user/developer does not care. The developers who do care won't mind the cygwin development environment as long as it produces a native Windows binary that does not play tricks such as fork(). Windows developers don't care too much about source code. The build environment will not be a problem. The issue is that the system must perform well and must be stable. I do not believe that cygwin can meet this requirement. Having done some research for these discussions, I think we know it has startup performance issues and unknown operational issues. FYI: My PHP project msession, can produce both a Windows version and a Cygwin version. It is threaded C++, but I have measured a performance improvements using the native Windows version over the cygwin version. I have since abandoned the cygwin version. I believe we can use the cygwin development environment, and direct gcc not to link with the cygwin dll. Last time I looked it was a command line option. This will produce a native windows application. No emulation, just a standard C runtime. Some of the hits will be file path manipulation, '/' vs '\', the notion of drive letters, and case insensitivity in file names. Unicode may be an issue, I haven't looked at that yet. Is that a must for the initial release? There will be a need for some emulation/api specification of things like semaphores, shared memory, file API (I would like to use Windows native CreateFile routines, as these should be pretty fast.), and so on. We will also have to breakup postgres and postmaster, and for the Windows version use CreateProcess. There are a number of ways to attack this, globals in a structure based in shared memory, globals in a .DLL exported to processes and shared, and so on. I think a huge time savings can be had by avoiding rewriting everything for the Microsoft build environment. As far as I know, and please correct me if I'm wrong, code produced by the cygwin gcc is freely distributable and need not be GPL. Once we have it working without fork() using the cygwin build environment, we will have a native Windows application, we can then further evaluate whether or not we want to expend the work to make a MSC version. Once the backend and most of the tools are built without requiring the cygwin.dll, installation is a breeze. Just dump it somewhere and run it. A couple simple programs can be written using msvc to monitor, start and stop PostgreSQL. The programs will be simple using the application wizard, just make a small dialog box application. Pgaccess will provide all the GUI stuff, and we may even be able to wrap the monitor code into pgaccess. The server install can be done with install shield. There is code that will run any program as an NT service. We can use that for server installations. We can use the MSVC wizard application to pop-up in the tool bar. Have I missed anything? Is this a realistic and attainable plan? ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Making the regression tests locale-proof
Peter Eisentraut [EMAIL PROTECTED] writes: The assumption here is that all locales will choose the same sort order as long as they're dealing only with the core 26 letters. This does not have to be true in theory, but I think it works for the vast majority of practical cases. Not for uppercase vs. lowercase versions of them. With no locale used (straight ASCII), you get A C b, with a locale you'll get A b C. -- Trond Eivind Glomsrød Red Hat, Inc. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Native Win32, How about this?
On Sat, 2002-05-11 at 02:25, mlw wrote: A binary version of PostgreSQL for Windows should not use the cygwin dll. I know and understand there is some disagreement with this position, but in this I'm sure about this. ... I believe we can use the cygwin development environment, and direct gcc not to link with the cygwin dll. Last time I looked it was a command line option. This will produce a native windows application. No emulation, just a standard C runtime. It seems that mingw (http://www.mingw.org/) does exactly this and provides needed headers/libs. And they have also non-cycwin minimal build environment (MSYS) that supplies make,sh and other stuff we might use for running initdb and other shell scripts. Some of the hits will be file path manipulation, '/' vs '\', the notion of drive letters, and case insensitivity in file names. Unicode may be an issue, I haven't looked at that yet. Is that a must for the initial release? Probably not. A couple simple programs can be written using msvc to monitor, start and stop PostgreSQL. The programs will be simple using the application wizard, just make a small dialog box application. dev-c++ has also wizards for easy making of trivial user interfaces http://sourceforge.net/projects/dev-cpp/ -- Hannu ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Making the regression tests locale-proof
On Sat, 2002-05-11 at 02:25, Peter Eisentraut wrote: The remaining issue is the sort order. I think this can be solved for practical purposes by creating two expected files for each affected test, say char.out and char-locale.out. The regression test driver would try the first one, if that fails try the second one. The assumption here is that all locales will choose the same sort order as long as they're dealing only with the core 26 letters. This does not have to be true in theory, but I think it works for the vast majority of practical cases. et_EE locale has the following order for core 26 letters _ are other letters ABCDEFGHIJKLMNOPQRS_Z_TUVWXY (notice position of Z) and I'm not sure if V and W are distinguished when sorting words that have anything after them. I've heard that in some other locales there are other veir behaviours (like sorting on or two of the same letters as equivalent) Hannu ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Unbounded (Possibly) Database Size Increase - Test
On Thu, 2002-05-09 at 14:21, Mark kirkwood wrote:
On Wed, 2002-05-08 at 01:45, Tom Lane wrote:
Which files grew exactly? (Main table, indexes, toast table, toast index?)
Here a listing (from another run - I dumped and reloaded before getting
any of that info last time...)
[:/data1/pgdata/7.2/base/23424803]$ du -sk .
4900806 .
-rw--- 1 postgres dba 1073741824 May 9 21:20 23424806.3
-rw--- 1 postgres dba 1073741824 May 9 21:19 23424806.2
-rw--- 1 postgres dba 1073741824 May 9 21:18 23424806.1
-rw--- 1 postgres dba 1073741824 May 9 21:16 23424806
-rw--- 1 postgres dba 12672 May 9 21:16 23424808
-rw--- 1 postgres dba 587505664 May 9 21:14 23424806.4
-rw--- 1 postgres dba 5914624 May 9 21:05 23424804
-rw--- 1 postgres dba 2441216 May 9 21:05 23424809
These files are for :
grow=# select relname,oid
grow-# from pg_class where oid in
('23424806','23424808','23424804','23424809');relname|
oid
---+--
pg_toast_23424804_idx | 23424808
pg_toast_23424804 | 23424806
grow_pk | 23424809
grow | 23424804
(4 rows)
so the big guy is the toast table and index
- BTW the table design is
CREATE TABLE grow (id integer,body text,CONSTRAINT grow_pk PRIMARY KEY
(id))
Was it not the case that lazy vacuum had problems freeing tuples that
have toasted fields ?
The row length is big ~ 14K. I am wondering if this behaviour will go
away if I use recompile with a 32K page size (also seem to recall I can
tell Pg not to toast certain column types)
--
Hannu
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] troubleshooting pointers
Tom Lane wrote:
Um. I don't like that; it assumes not only that ExecutorEnd is the only
kind of callback needed, but also that there is at most one function
per ExprContext that needs a shutdown callback. Neither of these
assumptions hold water IMO.
The design I had in mind was more like this: add to ExprContext a list
header field pointing to a list of structs along the lines of
struct exprcontext_callback {
struct exprcontext_callback *next;
void (*function) (Datum);
Datum arg;
}
and then call each specified function with given argument during
FreeExprContext. Probably ought to be careful to do that in reverse
order of registration. We'd also need to invent a RescanExprContext
operation to call the callbacks during a Rescan. The use of Datum
(and not, say, void *) as PG's standard callback arg type was settled on
some time ago --- originally for on_proc_exit IIRC --- and seems to have
worked well enough.
Well, I guess I set my sights too low ;-) This is a very nice design.
I have the shutdown callback working now, and will send a new patch in a
few minutes. I have not started RescanExprContext() yet, but will do it
when I address rescans in general.
What's really more relevant here is that during the hypothetical new
RescanExprContext function, we ought to go around and clear any fcaches
in the context that have setArgsValid = true, so that they will be
restarted afresh during the next scan of the plan. (The fact that that
doesn't happen now is another shortcoming of the existing set-functions-
in-expressions code.) So this suggests making a callback function type
specifically to do that, and registering every fcache that is executing
a set function in the callback list...
I also added FunctionCachePtr_callback struct and a member to
ExprContext. I have not yet created the registration or shutdown
functions, but again, I'll work on them as part of the rescan work.
I still have a couple of issues related to VIEWs that I need to figure
out, then I'll start the rescan work.
Thanks for the review and help!
Joe
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] troubleshooting pointers
Joe Conway [EMAIL PROTECTED] writes: ... I have not started RescanExprContext() yet, but will do it when I address rescans in general. I still have a couple of issues related to VIEWs that I need to figure out, then I'll start the rescan work. It's not unlikely that those issues are exactly due to not having rescan handled properly. What misbehavior are you seeing? regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Making the regression tests locale-proof
Peter Eisentraut [EMAIL PROTECTED] writes: For that purpose I have changed the permissions on these options to USERSET. (I'm still debating making lc_messages SUSET, because otherwise users can screw with admins by changing the language of the log output all the time. Comments?) Hm. Don't the regression tests already assume they are run by the superuser? They've got create/drop user commands in them. So I'd say SUSET is fine from the point of view of the tests, and I agree with your concern about making the logs unreadable. The assumption here is that all locales will choose the same sort order as long as they're dealing only with the core 26 letters. Nope. For instance, on HPUX I get this sort order in English: $ LANG=en_US.iso88591 sort testll eix ela ella ellm elm eln enx and this in Spanish: $ LANG=es_ES.iso88591 sort testll eix ela elm eln ella ellm enx because the Spanish treat LL as a single collating element. (Actually, my very-rusty recollection is that they sort LL the same as one L, which would mean that HPUX's behavior is not quite right here: it's treating LL as one symbol that sorts after L. Linux seems to have no clue that LL is special at all though...) We could also cut down the number of affected tests by making the select_implicit and select_having not use mixed-case strings in the test tables. Then we have only char, varchar, and select_views left. In practice we could perhaps use test data that doesn't hit any of the special cases in the popular languages. But I wonder whether this would not be shirking our responsibility as testers. Seems like if you avoid exercising these kinds of cases, you avoid finding corner-case bugs. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Unbounded (Possibly) Database Size Increase - Test
Hannu Krosing [EMAIL PROTECTED] writes: Was it not the case that lazy vacuum had problems freeing tuples that have toasted fields ? News to me if so. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Making the regression tests locale-proof
Tom Lane escribió: Peter Eisentraut [EMAIL PROTECTED] writes: The assumption here is that all locales will choose the same sort order as long as they're dealing only with the core 26 letters. Nope. For instance, on HPUX I get this sort order in English: [...] because the Spanish treat LL as a single collating element. (Actually, my very-rusty recollection is that they sort LL the same as one L, which would mean that HPUX's behavior is not quite right here: it's treating LL as one symbol that sorts after L. Linux seems to have no clue that LL is special at all though...) HPUX's behaviour is broken, because in spanish LL (as well as CH) stopped being a special symbol some five years ago (it used to be treated as one collating element sorted after L, so HPUX behaviour was right then). We could also cut down the number of affected tests by making the select_implicit and select_having not use mixed-case strings in the test tables. Then we have only char, varchar, and select_views left. Maybe it would be better to prepare various results, one for each of a subset of the locales supported (C, en_EN, some other western and maybe a couple multibyte?). That way at least you make sure the C library is working as expected. -- Alvaro Herrera (alvherre[a]atentus.com) No deja de ser humillante para una persona de ingenio saber que no hay tonto que no le pueda enseñar algo. (Jean B. Say) ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Making the regression tests locale-proof
Alvaro Herrera [EMAIL PROTECTED] writes: HPUX's behaviour is broken, because in spanish LL (as well as CH) stopped being a special symbol some five years ago (it used to be treated as one collating element sorted after L, so HPUX behaviour was right then). Well, this is an old release ;-) ... the localedef files are dated around 1996. (And you don't want to know how long it's been since I could speak passable Spanish.) In any case, the fact that the official rules have changed does not invalidate my point: there are systems on which the assumption Peter wants to make will fail. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Queries using rules show no rows modified?
Jan Wieck wrote: Hiroshi Inoue wrote: Tom Lane wrote: What should the backends return for complicated rewrites ? And how should/could clients handle the results ? It doesn't seem easy to me and it seems a flaw of rule system. Honestly I don't think that the psqlodbc driver can guarantee to handle such cases properly. However both Ron's case and Michael's one are ordinary updatable views. If we can't handle the case properly, we could never recommend users to use (updatable) views. The fact that our rule system is that powerful that you can have multi-action rules is a flaw ... awe. There's always a plus and a minus. For generic applications the powerfulness is a nuisance in a sense because it is difficult for them to understand the intension of complicated rewrites( and triggers as you pointed out). I don't think every application can handle every case. The main point may be how the applications can judge if they can handle individual cases. regards, Hiroshi Inoue http://w2422.nsk.ne.jp/~inoue/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [BUGS] Bug #659: lower()/upper() bug on -multibyte- DB
[Cc:ed to hackers]
(trying select convert(lower(convert('X', 'LATIN1')),'LATIN1','UNICODE');)
Ok, this is working now (I cann't reproduce why not at the first time).
Good.
Is it planned to implement it so that I can write lower()/ upper() for multibyte
according to SQL standard (without convert)?
SQL standard? The SQL standard says nothing about locale. So making
lower() (and others) locale aware is far different from the SQL
standard of point of view. Of course this does not mean locale
support is should not be a part of PostgreSQL's implementation of
SQL. However, we should be aware the limitation of locale support
(as well as multibyte support). They are just the stopgap util CREATE
CHARACTER SET etc. is implemnted IMO.
I could do it if you tell me where the final tolower()/toupper() happens.
(but not before middle of June).
For the short term solution making convert() hiding from users might
be a good idea (what I mean here is kind of auto execution of
convert()). The hardest part is there's no idea how we could find a
relationship bewteen particular locale and the encoding. For example,
you know that for de_DE locale using LATIN1 encoding is appropreate,
but PostgreSQL does not.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] troubleshooting pointers
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: ... I have not started RescanExprContext() yet, but will do it when I address rescans in general. I still have a couple of issues related to VIEWs that I need to figure out, then I'll start the rescan work. It's not unlikely that those issues are exactly due to not having rescan handled properly. What misbehavior are you seeing? Hmm, that might just be it. When I select from a view based on a function which returns a base type, I only get the first row. When I select from a view which is based on a function returning a composite type, it triggers an assertion. I've traced the latter down to a slot pointer which is reset to NULL somewhere. Haven't had the time to get much further. In both cases, selecting from the function directly works great. Thanks, Joe ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] bug? in current cvs with bigint datatype
I just did a fresh build from current cvs and found the following regression from 7.2: create table test (cola bigint); update test set cola = 100; In 7.3 the update results in the following error: ERROR: column cola is of type 'bigint' but expression is of type 'double precision' You will need to rewrite or cast the expression In 7.2 the update worked. (updated 0 rows in this case) It is interesting to note that if I use 'cola = 100' in a where clause instead of as an assignment (i.e. select * from test where cola = 100) this works in both 7.3 and 7.2. thanks, --Barry ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
