Re: [HACKERS] Two-phase commit issues
Tom Lane wrote: [ Shrug... ] I remain of the opinion that 2PC is a solution in search of a problem, because it does not solve the single point of failure issue (just moves same from the database to the 2PC controller). But some people want it anyway, and they aren't going to be satisfied that we are an enterprise grade database until we can check off this particular bullet point. As long as the implementation doesn't impose any significant costs when not being used (which AFAICS Heikki's method doesn't), I think we gotta hold our noses and do it. I thought the primary reason for having 2PC is to be able to participate in a heterogenous transaction, e.g. with a non-Postgres database/other types of resource managers? 2PC is mostly about how to make these cross-RM transactions [appear] atomic. Redundancy is not covered by 2PC protocol. -- dave ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Status of server side Large Object support?
Joe Conway wrote:
Not if the column is storage type EXTERNAL. See a past discussion here:
http://archives.postgresql.org/pgsql-general/2003-07/msg01447.php
what is the reasoning behind this syntax?
ALTER TABLE [ ONLY ] table [ * ]
ALTER [ COLUMN ] column SET STORAGE
{ PLAIN | EXTERNAL | EXTENDED | MAIN }
I find it nonintuitive and hard to remember. Perhaps something like this
is better (I know, it's probably too late):
ALTER [ COLUMN ] column SET STORAGE { INLINE | EXTERNAL }
ALTER [ COLUMN ] column SET COMPRESSION { YES | NO }
--
dave
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Embedded postgresql
[EMAIL PROTECTED] wrote: I would like to know if there are any discussions about creating an embedded version on postgresql. My thoughts go towards building/porting a sqlite equivalent of pg. Yes, there have been several. Peruse the archives: http://archives.postgresql.org/ -- dave ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Two-phase commit security restrictions
Heikki Linnakangas wrote: What kind of security restrictions do we want for prepared transactions? Who has the right to finish a transaction that was started by user A? At least the original user, I suppose, but who else? Under what account is the transaction manager typically going to run? A separate TM account perhaps? Do we need a GRANT TRANSACTION command to give permission to finish 2PC transcations? Another approach I've been thinking about is to allow anyone that knows the (user-supplied) global transaction identifier to finish the transaction, and hide the gids of running transactions from regular users. That way, the gid acts as a secret token that's only known by the transaction manager, much like the cancel key. Personally I prefer the last. It should be infeasible to crack as long as the gid is long enough (e.g. sufficiently random 128bit value or more) and the channel between the TM and Postgres is secure. The problem is, we cannot guarantee that a TM will generate a good random gid, or even a long enough one. (But then a good TM should assume that RM doesn't have any protection on global transactions and thus generate a good secret-like gid). Does the XA standard regulate about this security issue? -- dave ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Two-phase commit security restrictions
Alvaro Herrera wrote: Another approach I've been thinking about is to allow anyone that knows the (user-supplied) global transaction identifier to finish the transaction, and hide the gids of running transactions from regular users. That way, the gid acts as a secret token that's only known by the transaction manager, much like the cancel key. Personally I prefer the last. It should be infeasible to crack as long as the gid is long enough (e.g. sufficiently random 128bit value or more) and the channel between the TM and Postgres is secure. So it is possible for a user connected to the DB to send random commit or cancel commands, just in case she happens to hit a valid GID? It is not essentially different from someone trying to bruteforce a password. A 128bit value like a random GUID is as strong as a 16 char password comprising ASCII 0-255 characters. And I would argue that this is _not_ security through obscurity. Security through obscurity is relying on unpublished methods/algorithms. This is not. But I understand that everybody seems to be against this idea. -- dave ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] pg_dump and blobs
Why doesn't pg_dump include blob by default? I understand that older pg_dump didn't deal with blobs, and blobs are now kind of obsolete in favor of BYTEA/TEXT, but blobs are every bit a part of a database. Perhaps only exclude blobs when -t is specified? Then -b is required to include blob. Otherwise, -b is implied. Also, it would be really nice if there were a way that pg_dumpall could include blobs. This is my biggest Postgres annoyance nowadays, as I've recently been bitten by this. I though we are already able to escape all octet range from '\\000' to '\\377'? -- dave ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] FYI: Fujitsu
Neil Conway wrote: I've accepted an offer from Fujitsu Australia Software Technologies to work on PostgreSQL full-time for them for the next twelve months in Sydney, Australia. I'll be working with Gavin Sherry and two other full-time developers from FAST. I'm grateful to Fujitsu for giving me the opportunity to do this, and I look forward to doing what I can to improve PostgreSQL over the next year. Congratulations. It's nice to hear that Fujitsu keeps supporting Postgres. Hopefully other companies can follow suit :-) It seems commercial companies are more keen to support BSD-ish free software (Apache, Postgres, Perl, Ruby, etc.)? With the exception of Linux of course. -- dave ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] PostgreSQL as an application server
Jonathan M. Gardner wrote: Thoughts? Comments? Hasn't Oracle done something like this? Probably this is more suited to -general? I haven't done anything near this. I wonder how much more painful it is to debug the application, put it under version control, etc. Personally, I can't stand editing/debugging application if they are stored in something less flexible than a filesystem. Probably in the far future Postgres can provide an FTP interface like Zope... -- dave ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
[HACKERS] pg_ctl and failing postmaster
When postmaster fails to run due to e.g. datadir being set to 755, 'pg_ctl start' incorrectly reports postmaster successfully started. I'm not sure how to fix this though. Do a [shorter] wait for 'start' mode by default? Install a child handler (can you even do that with shell script?) This is postgresql 7.4.3 on Linux. -- dave ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] postmaster.opts, moving datadir around, and pg_ctl
I had datadir in /dir1 and I started postmaster with: postmaster -D /dir1 this recorded -D /dir1 in /dir1/postmaster.opts. Then I stopped postmaster and moved /dir1 to /dir2. I then started postmaster with: PG_DATA=/dir2 pg_ctl start It worked normally. But: PG_DATA=/dir2 pg_ctl restart failed when starting postmaster because it used datadir from postmaster.opts (/dir1). This worked though: pg_ctl restart -D /dir2 So there's a slight inconsistency between the start and restart command. Would it be nicer if postmaster doesn't record -D in postmaster.opts, or if pg_ctl strips it? Or is it entirely my fault for not adjusting postmaster.opts when I move datadir around? IMO it would be nice if datadir is portable (i.e. it doesn't contain its own full path). Also pardon my ignorance, but what is the purpose of having postmaster.opts inside datadir? This is postgresql 7.4.3 on Linux. -- dave ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] 7.5 release notes
Bruce Momjian wrote: I have completed the 7.5 release notes. You can view them in HTML on the developer web page. I have marked a few items with question marks that need to be addressed. I am looking for improvements, even minor ones. Either send in a patch or committers can modify the file directly. In E.1.1 Overview: ... This release supports Windows NT 4 and all later releases. It does not support earlier releases like Windows 95, 98, or ME because ... Win98 and WinME is released _after_ NT4. See http://www.computerhope.com/history/windows.htm Perhaps it's better to say: We only support NT-based Windows such as NT4, Win2k/XP/2003/later. Old 9x-based Windows versions such as Win95/98/Me are not supported because ... Btw, can 7.5 run on WinCE/XP Media Center/XP SP2/etc? -- dave ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] creating a cluster
Alvaro Herrera wrote: On Mon, Jun 21, 2004 at 09:16:35PM -0400, Alexander Cohen wrote: Does anyone have any new ways to create clusters without using initdb or bootstrap mode? I need to be able to create one without those 2 things. Any ideas? initdb'ing somewhere else and copying the resulting directory? Btw, I've been doing this for a binary distribution on Windows (Cygwin) and Linux. Primarily because initdb-ing + doing a bunch of SQL commands to the db takes a long time on Cygwin. Seems fine so far. -- dave ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [pgsql-advocacy] Not 7.5, but 8.0 ?
Tom Lane wrote: Granted, the script itself is faulty, but since some other OS projects (like Ruby, with the same x.y.z numbering) do guarantee they never will have double digits in version number component Oh? What's their plan for the release after 9.9.9? As for Ruby, it probably won't expect 9.9.9 in any foreseeable future. It takes +- 10 years to get to 1.8.1. Same with Python. But Perl will have 5.10.0. -- dave ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [pgsql-advocacy] Not 7.5, but 8.0 ?
This probably has been discussed and is probably a very minor point, but consider how many more years we want to be able to use the single digit.single digit major release numbering. Assuming 1 year between major releases (7.3.0 - 7.4.0 = +- 1 year), then we have 7.5-9.9 = 26 years = up until +- jul 2030. if we skip to 8.0 now, then we have up until 2023. Also we have 1 more chance to skip major number: 8.x - 9.0. Imagine what features will there be in 9.0 that is ground-breaking enough. Because after that, we don't have any more major number to jump into without going into 2 digits. I personally don't see the major number as a very magical thing. Look at Linux for example. People still see 2.6 as very different/ahead compared to 2.4... -- dave ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] [pgsql-advocacy] Not 7.5, but 8.0 ?
Dave Page wrote: From: David Garamond Sent: Sat 6/5/2004 9:28 AM Cc: postgresql advocacy; [EMAIL PROTECTED] Subject: Re: [HACKERS] [pgsql-advocacy] Not 7.5, but 8.0 ? Assuming 1 year between major releases (7.3.0 - 7.4.0 = +- 1 year), then we have 7.5-9.9 = 26 years = up until +- jul 2030. if we skip to 8.0 now, then we have up until 2023. Hi Dave, I might be missing the point, but why can't we go to double figures? MS Office has, HP-UX has, OS-X, Norton AV has, Madrake Linux has... Of course we can, I didn't say we can't. But double digits are sometimes undesirable because it can break some things. For example, a simple shell or Perl script might try to compare the version of two data directories by comparing the content of PG_VERSION stringwise. It then concludes that 7.10 is smaller than 7.4. Granted, the script itself is faulty, but since some other OS projects (like Ruby, with the same x.y.z numbering) do guarantee they never will have double digits in version number component than people might think the same too and thus the habit of stringwise version comparison continues. -- dave ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Call for 7.5 feature completion
Robert Treat wrote: Given that the cygwin version is currently labeled as not ready for production I would say you are right. The truth is that many will never declare win32 good for production simply because of the OS it runs on, but we still want to make it as solid as possible. People _do_ use postgresql+cygwin in production environments though (see the pgsql-cygwin archive). And I suspect people _will_ use 7.5 for win32 in production, despite the release notes and the website clearly saying it's not production ready. Why? 1) The version number is 7.5 and many people will presume the ports are more or less equal in quality/maturity since they have the same version number; 2) People don't read release notes. See the various reviews on the recently released Fedora Core 2, complaining about how it doesn't support MP3 or DVD playback, despite the [legal] issues having been known and documented since Red Hat 8. Strangely enough, these people (who don't read release notes) _do_ write public reviews. They will badmouth PostgreSQL, saying it's unstable, crashes a lot, MySQL being much much more rock solid, etc etc. I suggest we label the win32 port as 7.5 ALPHA or 7.5 DANGEROUS :-) -- dave ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] The features I'm waiting for.
scott.marlowe wrote: For me, the only features I'm likely to use in the upcoming releases are nested transactions. While PITR is a great selling point, and the Windows Port is something I do look forward to, having to do half my job programming windows boxes, nested transactions are a feature I can genuinely use in my daily (maybe weekly??? :-) life. While a focus on things that make postgresql more market acceptable are important, the things that make it more feature complete to me as a user are the things I'd gladly wait an extra month or two for. But I'm not programming any of the code, so I'm just sayin'... I'm sure everybody has their own favorite feature. But I can say quite confidently that the upcoming release contains the most number of highly anticipated features ever. Nested transaction, 2-phase commit, Windows port... I mean these are all major stuffs. They are paving the way of deployments of Postgres in new areas and applications. Plus don't forget all the other sweet goodies like autovacuum and PITR. But the next release could also be the buggies version ever, due to the number of these new features. :-) -- dave ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] What can we learn from MySQL?
Bruce Momjian wrote: My question is, What can we learn from MySQL? I don't know there is anything, but I think it makes sense to ask the question. MySQL was my first introduction to SQL databases (I had dabbled with Clipper and Foxpro years earlier, but only for a couple of months and had forgotten most of it by then). So practically all I knew about SQL and RDBMS I got from the MySQL manual. IIRC, MySQL has a chapter for beginners, on how to create your first database and tables, how to insert a record, etc. I see that the Pg manual already has that. Good. The problem is that, since MySQL was my only SQL database I knew for a long time, I didn't know that an RDBMS can be [much] more than what MySQL was/is. I could only do simple SELECTs (no JOINs, let alone subselect since MySQL doesn't support it) but found it sufficient, since I did most of the hard work from Perl/PHP (for example, doing an adjacency tree query by several SELECTs and combining the results myself from the client side). I didn't know squat about stored procedures or triggers or check constraints. I had no idea what a foreign key is -- and when MySQL manual says it's not necessary, slow, and evil, I believed it. I never bothered checking out other databases until I started reading more about transactions, reliability, Date/Codd, and other more theoretical stuffs. Only then I started trying out Interbase, Firebird, SAPDB, DB2, Oracle, and later Pg. So in my opinion, as long as the general awareness about RDBMS (on what tasks/responsibilities it should do, what features it generally has to have, etc) is low, people will be looking at MySQL as good enough and will not be motivated to look around for something better. As a comparison, I'm always amazed by people who use Windows 95/98/Me. They find it normal/good enough that the system crashes every now and then, has to be rebooted every few hours (or every time they install something). They don't know of anything better. So perhaps the direction of advocacy should be towards increasing that awareness? -- dave ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] subversion vs cvs
Sailesh Krishnamurthy wrote: I've had plenty of pain with cvs in terms of directories not being first-class etc .. but I don't really contribute to pgsql so you guys probably don't have the same experience. I was just curious as it looks like eventually subversion (or arch :-) will be an alternative to cvs. Eventually it (either subversion, or arch, or something else) will. You just have to be patient :-) The movement will be very slow, we'll probably see Apache 1.3.x disappear first before we see CVS disappear. It _is_ frustrating to have to use something new, especially something so frequently used like source control tool. -- dave ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: subversion vs cvs (Was: Re: [HACKERS] linked list rewrite)
Dustin Sallings wrote: On Mar 24, 2004, at 11:45, David Garamond wrote: So one might ask, what *will* motivate a die-hard CVS user? A real-close Bitkeeper clone? :-) Since it's illegal for anyone who uses Bitkeeper's free license to contribute to another project, does anyone know if there are any features in Bitkeeper missing from arch (specifically tla) that matter to developers? Or is there anything that may be a better match than arch? From what I read here and there, BitKeeper excels primarily in merging (good merging is apparently a very complex and hard problem) and GUI stuffs. Unfortunately, I have never and will never use Bitkeeper unless someone buys me a license for some reason. The distributed model seems like the only way to go for the open source development of the future. Not necessarily. For small to medium projects, a centralized model might work better. -- dave ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Michael Glaesemann wrote: Just to speak up (as an avid lurker), I agree with Jeroen that this distinction is quite subtle and may cause confusion. Some may even expect the two to resolve to the same site, as a lot of popular sites own .com/.net/.org, all resolving to the same site. Speaking of .com vs .net vs .org, anyone remember the mysql.com vs mysql.org fiasco? Anyway, if I can vote, I'll vote for postgresql.net (for the lack of better choices). I agree with Tom that pgfoundry is kind of random. It's not apparent at all that it's a PostgreSQL entity. Besides, Tom Marc is already listed as the registrant of several domains including postgresql.com. Why not use them? Also, we're targetting the developers right? Please do not consider ourselves as being too stupid to differentiate between postgresql.org and postgresql.net... If people don't like to type long names, we can always do automatic redirection between projname.postgresql.net, postgresql.org/projects/projname, projname.projects.postgresql.org, etc. Or even perhaps use tinyurl :-) -- dave ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Sigh, 7.3.6 rewrap not right
Steve Crawford wrote: Please, don't call it 7.3.6. Streamlining releases is terrible. 7.3.7 or 7.3.6.1 or SOMETHING other than 7.3.6, and just let 7.3.6 be a brown paper bag release (like 6.4.1 was). There were no code-change differences in this rewrap, so I see no real need to change the version number. I have to agree with Lamar et. al. The _code_ may not have changed but the product did and the version number should reflect that. I second this. As someone has said, we should probably use the -rc mechanism in the future (changing the versioning from 7.3.6 into 7.3.6.1 has a greater chance of breaking things). Allow at least one week before the final -rc turns into final. The last -rc will be byte-to-byte identical with the final, we just rename it. *If* the final turns out to contain some stupid mistake, we'll just have to make 7.3.7... Once something is released, it should not change at all. -- dave ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-hackers-win32] [HACKERS] What's left?
Dann Corbit wrote: But for now I suggest that the default prefix on Windows is C:\Program Files\PostgreSQL More properly: %ProgramFiles%\PostgreSQL Another suggestion: %ProgramFiles%\PGDG\PostgreSQL (or even %ProgramFiles%\PGDG\PostgreSQL 7.5). Apache2 uses %ProgramFiles%\Apache Group\Apache2. Note: Many software uses the %ProgramFiles%\VendorName\ProductName convention, but apparently Microsoft itself puts stuffs right under %ProgramFiles% (%ProgramFiles%\Microsoft Money, \Internet Explorer, \Windows Media Player, etc). And then, if they don't like that, let them put it wherever they darn well please. -- dave ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] What's planned for 7.5?
Andrew Sullivan wrote: On Mon, Jan 19, 2004 at 08:12:28AM -0500, Jan Wieck wrote: and cons, none is the one size that fits all magic solution. To select Does anyone realy believe that there can be a one size fits all solution? Heck, even Oracle and IBM offer a couple of different systems, depending on what you need. (That also suggests that any replication system need not always be shipped with the basic distribution, but could instead be integrated into a larger, postgresql_plus_enterprise_features.tgz or something like that.) I don't, but consider Linux which can be configured to run on devices as small as a wristwatch and as large as the the big irons... -- dave ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] And ppl complain about *our* beta cycles ...
Marc G. Fournier wrote: From the Firebird FAQ: The first beta was released on January 29, 2003. We are hoping to be close to a full release some time around Easter 2003. They are at RC8 right now ... running a *wee* bit behind scheduale :) Yes, they're pretty late. Last time I read, the only major issues preventing their final release is around the installer. The 1.5 codebase itself has been stabilized for quite a while. Practically all work is now done to the 2.0 branch/HEAD. They have several goodies in store for the 2.0 release (e.g.: incremental backup). I think the Firebird team wants to make sure that their first public release to the world is really good. (Okay, they have some 1.0.x releases too but these were mainly bugfixes for IB 6.0, its predecessor; and there are *lots* of bugs being fixed). FB 1.5 is the first version that uses the new C++ codebase. Anyway, I'm certainly not among those who complain. I believe Postgres is already moving ahead faster than any other open source and semi-commercial database projects that I know of. You guys are doing a great great job. :-) -- dave ---(end of broadcast)--- TIP 8: explain analyze is your friend
