Re: [HACKERS] Dead Space Map for vacuum
On Thu, 28 Dec 2006, Heikki Linnakangas wrote: ITAGAKI Takahiro wrote: Hello, NTT staffs are working on TODO item: | Create a bitmap of pages that need vacuuming We call the bitmap Dead Space Map (DSM), that allows VACUUM to scan only pages that need vacuuming or freezing. We'd like to discuss the design on hackers and make agreements with community. Great! I agree! We implemented the basic parts of it and measured the performance. As expected, VACUUM took shorter time when the fraction of updates are low. DSM is useful for large but not so heavily-updated databases. The overhead of managing DSM seemed to be negligible small in our CPU-loaded tests. There are a lot of choices in implementation. We followed the descriptions in TODO list and past discussions in some parts, but did not in other parts for some reasons. I would appreciate your comments and suggestions. I experimented with a different DSM design last winter. I got busy with other things and never posted it AFAIR, but the idea was to store a bitmap in the special area on every 32k heap page. That had some advantages: * doesn't require a new dedicated shared memory area that needs to be allocated and tuned. * doesn't introduce a (single) new global lwlock that might become hotspot. * WAL logging is quite simple, since the bitmaps are on normal pages handled by buffer manager. I too have experimented with this idea. 4 DSM pages means 2 bits per block. What were you using the other bit for? When I discussed this some time ago with Jan Weick, we recommended we track blocks in the FSM with this extra bit. One problem this approach may have is contention for the DSM pages and the granularity of the lock associated with it. Locking a DSM page to update one bit will affect writes to 32K pages. This might be less of a problem than I think because of the very low cost of setting the bit. | In the event of a system crash, the bitmap would probably be invalidated. We bought it for simplicity. Currently, all DSM are lost after crash. All tables will be untracked status. Full-scanned vacuum is needed after the lost of DSM. If you flush the DSM to disk at checkpoint, it should be easy to bring it up-to-date on WAL replay. Having to do full-scanning vacuum after crash can be a nasty surprise. I agree. One interesting insight is, we have 4 bits left over (those not used for the 4 DSM blocks). We might use those status bits for some purpose. For example, when a page is dirtied and we update the DSM page, we set a DSM dirtied bit. Checkpoint would then only flush DSM pages dirtied since the last check point. This is important in the presense of lots of read only tables. | One complexity is that index entries still have to be vacuumed, and doing | this without an index scan (by using the heap values to find the index entry) | might be slow and unreliable, especially for user-defined index functions. Indexes are still needed to be full-scanned at the present moment. We are also researching a retail index vacuum method, but it requires more works. Yeah, that's an old story :(. BTW: Yesterday I realized that bitmap indexes could be retail vacuumed safely. You'll still need to visit all bitmaps to find the dead bit, but you only need to check the bitmap page that corresponds the tid of the dead tuple. Cool. The problem is solving it for the other AMs, particularly btree. | http://archives.postgresql.org/pgsql-hackers/2004-03/msg00957.php | Maintain 2 bits per block that tell if the block has been vaccumed of all | dead tuples since the last time it was dirtied, and if all its tuples are | completely frozen. We use 1 bit per block, so we cannot separate pages that need either vacuum or freeze only. The reason is that we cannot determine where to record before/after updated tuples. If the transaction is commited, before-version should be recorded into needs-vacuum bitmap and after-version into needs-freeze bitmap. But on rollback, it is inverted. We cannot judge which we should do until the end of transaction. Yeah, that makes the DSM less useful. Are you thinking of freezing more aggressively because of that? Or doing a full-scanning vacuum every now and then to freeze? I don't see any problem with moving to two status bits per block in the DSM-in-heap model, so we can track this better. What do you think? Thanks, Gavin ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Load distributed checkpoint
Gregory Stark wrote: Bruce Momjian [EMAIL PROTECTED] writes: I have a new idea. ...the BSD kernel...similar issue...to smooth writes: Linux has a more complex solution to this (of course) which has undergone a few generations over time. Older kernels had a user space daemon called bdflush which called an undocumented syscall every 5s. More recent ones have a kernel thread called pdflush. I think both have various mostly undocumented tuning knobs but neither makes any sort of guarantee about the amount of time a dirty buffer might live before being synced. Earlier in this thread (around the 7th) was a discussion of /proc/sys/vm/dirty_expire_centisecs and /proc/vm/dirty_writeback_centisecs which seem to be the tunables that matter here. Googling suggests that dirty_expire_centisecs specifies that data which has been dirty in memory for longer than this interval will be written out next time a pdflush daemon wakes up and dirty_writeback_centisecs expresses the interval between those wakeups It seems to me that the sum of the two times does determine the maximum time before the kernel will start syncing a dirtied page. Bottom line, though is that it seems both postgresql and the OS's are trying to delay writes in the hopes of collapsing them; and that the actual delay is the sum of the OS's delay and postgresql's delay. I think Kevin Grittner's experimentation earlier in the thread did indeed suggest that getting writes to the OS faster and let it handle the collapsing of the writes was an effective method of reducing painful checkpoints. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Dead Space Map for vacuum
On 12/28/06, ITAGAKI Takahiro wrote: | [TODO item] Allow data to be pulled directly from indexes | Another idea is to maintain a bitmap of heap pages where all rows are | visible to all backends, and allow index lookups to reference that bitmap | to avoid heap lookups It is not done yet, but we can use DSM for this purpose. If the corresponding bit in DSM is '0', all tuples in the page are frozen and visible to all backends. We don't have to look up frozen pages only for visibiliby checking. Does that really work in the absence of a retail index vacuum method? What if the heap is already vacuumed, frozen and the bit for that page in the DSM is set to '0', but the index still contains entries that haven't been removed by a vacuum yet? Jochem ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Dead Space Map for vacuum
Heikki Linnakangas [EMAIL PROTECTED] wrote: We use 1 bit per block, so we cannot separate pages that need either vacuum or freeze only. The reason is that we cannot determine where to record before/after updated tuples. If the transaction is commited, before-version should be recorded into needs-vacuum bitmap and after-version into needs-freeze bitmap. But on rollback, it is inverted. We cannot judge which we should do until the end of transaction. Yeah, that makes the DSM less useful. Are you thinking of freezing more aggressively because of that? Or doing a full-scanning vacuum every now and then to freeze? I recommend to use VACUUM FREEZE with the patch, but what you say is surely reasonable. I think that backends record all of the changes into 1st bitmap, and then, VACUUM moves some bits from 1st bitmap to 2nd one. Assumimg we had two bitmaps; (1)invisible-from-someone-bitmap and (2)freeze-needed-bitmap. (Actually, two bits would be packed into one compound bitmap, but for illustrative purposes.) We have to record changes into (1) at first, because we can find a page needs either VACUUM or FREEZE only after commits or rollbacks. VACUUM scans pages using (1). Newly inserted pages are also scanned at first vacuum, even if they requires only freeze. If the vacuum find that all of tuples in a page are vacuumed completely and old enough to be visible by all open transactions, delete the page from (1). Then, all are also frozen, the page is not in both bitmaps. Otherwise, put it into (2). We can use (1) for the index only scanning, because all of the tuples not recorded in (1) are visible to all backends, regardless of whether they are also recorded in (2). Regards, --- ITAGAKI Takahiro NTT Open Source Software Center ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Load distributed checkpoint
On Wed, Dec 27, 2006 at 10:54:57PM +, Simon Riggs wrote: On Wed, 2006-12-27 at 23:26 +0100, Martijn van Oosterhout wrote: On Wed, Dec 27, 2006 at 09:24:06PM +, Simon Riggs wrote: On Fri, 2006-12-22 at 13:53 -0500, Bruce Momjian wrote: I assume other kernels have similar I/O smoothing, so that data sent to the kernel via write() gets to disk within 30 seconds. I assume write() is not our checkpoint performance problem, but the transfer to disk via fsync(). Well, its correct to say that the transfer to disk is the source of the problem, but that doesn't only occur when we fsync(). There are actually two disk storms that occur, because of the way the fs cache works. [Ron referred to this effect uplist] As someone looking from the outside: fsync only works on one file, so presumably the checkpoint process is opening each file one by one and fsyncing them. Yes Does that make any difference here? Could you adjust the timing here? Thats the hard bit with io storm 2. When you fsync a file you don't actually know how many blocks you're writing, plus there's no way to slow down those writes by putting delays between them (although its possible your controller might know how to do this, I've never heard of one that does). Any controller that sophisticated would likely also have a BBU and write caching, which should greatly reduce the impact of at least the fsync storm... unless you fill the cache. I suspect we might need a way to control how much data we try and push out at a time to avoid that... As for settings, I really like the simplicity of the Oracle system... Just try to ensure recovery takes about X amount of seconds. I like the idea of a creeping checkpoint even more; only writing a buffer out when we need to checkpoint it makes a lot more sense to me than trying to guess when we'll next dirty a buffer. Such a system would probably also be a lot easier to tune than the current bgwriter, even if we couldn't simplify it all the way to seconds for recovery. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Dirty pages in freelist cause WAL stuck
On Tue, Dec 19, 2006 at 05:53:06PM +0900, ITAGAKI Takahiro wrote: Simon Riggs [EMAIL PROTECTED] wrote: Another connected thought is the idea of a having a FullBufferList - the opposite of a free buffer list. When VACUUM/INSERT/COPY fills a block we notify the buffer manager that this block needs writing ahead of other buffers, so that the bgwriter can work more effectively. That seems like it would help with both this current patch and the additional thoughts above. Do you mean that bgwriter should take care of buffers in freelist, not only ones in the tail of LRU? We might need activity control of bgwriter. Buffers are reused rapidly in VACUUM or bulk insert, so bgwriter is not sufficient if its settings are same as usual. Actually, if I understand the code, the LRU stuff actually only hits the free list. Also, the only thing that runs the clock sweep (which is what maintains the LRU-type info) is a backend requesting a page and not finding one on the free list. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] effective_cache_size vs units
On Wed, Dec 27, 2006 at 09:39:22AM +0100, Benny Amorsen wrote: TL == Tom Lane [EMAIL PROTECTED] writes: TL Anyone against making it case-insensitive, speak now or hold your TL peace. SI-units are inherently case-sensitive. The obvious example is that now you will allow people to specify an amount in millibytes, while interpreting it in megabytes. Yes, and I can't think of a single reason why we'd let people specify anything in millibytes, or kilobits. Truth is, I bet many (if not most) DBAs barely know that case matters in the units. And even those that do are likely to prefer ease of use to pedantics. As for the comments about SI datatypes and MySQL-isms, there's a hell of a big difference between muddying the line in a config file for ease of use and stomping on data that's actually stored in the database. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] Allow the identifier length to be increased via a
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Dhanaraj M wrote: I am sending the patch for the following TODO item: Allow the identifier length to be increased via a configure option You should use pg_config.h, not mangle postgres_ext.h like that. Or maybe generate postgres_ext.h from an hypotetical postgres_ext.h.in (but I wouldn't do that, really). I'm wondering how this got into the TODO list. It seems rather pointless, and likely to create client compatibility problems (if not, why is NAMEDATALEN exported at all?) Will this TODO item be removed from the list? Or I shall proceed with the suggestions given. Thanks Dhanaraj ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_standby and build farm
On Wed, 2006-12-27 at 20:09 +, Simon Riggs wrote: The reason for the -m option was performance. Recovery is I/O-bound, with 50% of the CPU it does use coming from IsRecordValid() - which is where the CRC checking takes place. (I can add an option to recover without CRC checks, if anyone wants it, once I've rejigged the option parsing for recovery.conf.) Make that 70% of the CPU, for long running recoveries, but the CPU only gets as high as 20% on my tests, so still I/O bound. Should be able to use hard links, i.e. ln -f -s /archivepath/%f %p instead. I'll test that tomorrow then issue a new version. The ln works, and helps, but not that much. I'll remove the -m option and replace it with an -l option. Must be careful to use the -f option. The majority of the I/O comes from writing dirty buffers, so enabling the bgwriter during recovery would probably be more helpful. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Tom Lane wrote: Several of the buildfarm machines are exhibiting repeatable signal 11 crashes in what seem perfectly ordinary queries. This started about four days ago so I suppose it's got something to do with my operator-families patch :-( ... but I dunno what, and none of my own machines show the failure. Can someone provide a stack trace? no stack trace yet however impala at least seems to be running out of memory (!) with 380MB of RAM and some 800MB of swap(and no other tasks) during the regression run. Maybe something is causing a dramatic increase in memory usage that is causing the random failures (in impalas case the OOM-killer actually decides to terminate the postmaster) ? Stefan ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pg_standby and build farm
Thanks for the response, Simon. I'm continuing to use your script, so if there's anything I can help you with (testing, ideas, etc), just let me know. Doug On Thu, 2006-12-28 at 13:40 +, Simon Riggs wrote: On Wed, 2006-12-27 at 20:09 +, Simon Riggs wrote: The reason for the -m option was performance. Recovery is I/O-bound, with 50% of the CPU it does use coming from IsRecordValid() - which is where the CRC checking takes place. (I can add an option to recover without CRC checks, if anyone wants it, once I've rejigged the option parsing for recovery.conf.) Make that 70% of the CPU, for long running recoveries, but the CPU only gets as high as 20% on my tests, so still I/O bound. Should be able to use hard links, i.e. ln -f -s /archivepath/%f %p instead. I'll test that tomorrow then issue a new version. The ln works, and helps, but not that much. I'll remove the -m option and replace it with an -l option. Must be careful to use the -f option. The majority of the I/O comes from writing dirty buffers, so enabling the bgwriter during recovery would probably be more helpful.
Re: [HACKERS] Bitmap index thoughts
On 12/27/06 3:10 AM, Gavin Sherry [EMAIL PROTECTED] wrote: On Wed, 27 Dec 2006, Heikki Linnakangas wrote: Gavin Sherry wrote: On Tue, 26 Dec 2006, Heikki Linnakangas wrote: for typical bitmap index use cases and most of the needed pages should stay in memory, but could we simplify this? Why do we need the auxiliary heap, couldn't we just store the blk+offset of the LOV item directly in the b-tree index item? The problem is, the b-tree code is very much tied to the heap. I don't want to modify the b-tree code to make bitmap indexes work (better). What's really tempting is to just manage our own balanced tree within the bitmap index file(s) itself. It would start from the metapage and simply spill to other 'special' index pages when necessary. The problem is, we do not have b-tree code generic enough that it would allow us to do this trivially -- consider concurrency and WAL in particular, which we currently get for free. I guess this is why I've been ignoring this issue :-). Maybe we could reuse the code in ginbtree.c. Looks like Teodor Oleg had the same problem :). Modifying the nbtree code doesn't seem that difficult either. AFAICS, the only places where the heap is from within nbtree code is in index building and uniqueness checks. I'll take a look and think about it. I will take a look at it as well. I would also like to add eventually the values for bm_last_tid_location in all bitmap page to the lov heap. Combining this with the attribute values, we can easily jump into a specific bitmap page. This will be useful during vacuuming and insertion after vacuuming -- we can jump into a page to modify a bit. I am not sure if this will affect the use of the btree code without the heap. But if we can get rid of the lov heap, that would be great. And instead of having separate LOV pages that store a number of LOV items, how about storing each LOV item on a page of it's own, and using the rest of the page to store the last chunk of the bitmap. That would eliminate one page access, but more importantly, maybe we could then get rid of all the bm_last_* attributes in BMLOVItemData that complicate the patch quite a bit, while preserving the performance. That's an interesting approach. We would still need a concept of last_word, at the very least, and probably last_comp_word for convenience. Why? The two last words of the bitmap vector, stored in the LOV item, provide a few things: they buffer the addition of new matches in the vector and they ease the calculation of the current position of the end of the vector. Jie, do you want to add more? I think we could do this differently by calculating everything based on the other data stored in the lov item and data page (last_tid_location and num_hrl_words_used, from memory). But, I'm not sure. Jie? Sure. When appending a new bit into an existing bitmap, the last two words are needed to compute new HRL words, and they are the only words affected by this new appending bit. So we need a way to know these two words easily. Like Gavin said, separating them from other words is for convenience. Also, we won't be able to get rid of bm_last_setbit. We need bm_last_setbit to tell us how many zeros are there between two bit 1s. There is no other way to know this. Thanks, Jie ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Deadline-Based Vacuum Delay
Galy Lee [EMAIL PROTECTED] writes: So I am thinking another way to perform vacuum. For example vacuum can be refined in a maintenance time frame like VACUUM IN 6 HOURS, and then vacuum operation will be performed within the window. The delay time is adjusted internally to spread the disk I/O over the time frame. And you will manage that how? The total amount of work to be done is quite unpredictable. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Tom Lane wrote: Several of the buildfarm machines are exhibiting repeatable signal 11 crashes in what seem perfectly ordinary queries. no stack trace yet however impala at least seems to be running out of memory (!) with 380MB of RAM and some 800MB of swap(and no other tasks) during the regression run. Maybe something is causing a dramatic increase in memory usage that is causing the random failures (in impalas case the OOM-killer actually decides to terminate the postmaster) ? No, most all the failures I've looked at are sig11 not sig9. It is interesting that the failures are not as consistent as I first thought --- the machines that are showing failures actually fail maybe one time in two. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Dead Space Map for vacuum
Heikki Linnakangas [EMAIL PROTECTED] writes: I experimented with a different DSM design last winter. I got busy with other things and never posted it AFAIR, but the idea was to store a bitmap in the special area on every 32k heap page. That had some advantages: * doesn't require a new dedicated shared memory area that needs to be allocated and tuned. * doesn't introduce a (single) new global lwlock that might become hotspot. I agree with doing something along that line (not necessarily with putting the bits into existing heap pages, but anyway with keeping the information on disk not in shared memory). Our experience with the existing FSM design has been uniformly negative; so I don't wish to add still another shared memory area that requires tricky manual size tuning. I think sooner or later we'll need to redesign FSM so it's not kept in a fixed-size shared memory area. Let's not make that mistake twice. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Bitmap index thoughts
On 12/27/06 3:16 AM, Gavin Sherry [EMAIL PROTECTED] wrote: On Wed, 27 Dec 2006, Heikki Linnakangas wrote: Jie Zhang wrote: The bitmap data segment sounds good in terms of space. The problem is that one bitmap is likely to occupy more pages than before, which may hurt the query performance. We could have segments of say 1/5 of page. When a bitmap grows larger than that, the bitmap would be moved to a page of its own. That way we wouldn't get unnecessary fragmentation with large bitmaps, but small bitmaps would be stored efficiently. Yes. I see. This sounds good. I will think more about this. Thanks, Jie ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] TODO: Add a GUC to control whether BEGIN inside a transcation should abort the transaction.
Hello, Is this really a TODO or is this someone being overzealous with the TODO list? Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] TODO: Particularly, move GPL-licensed /contrib/userlock and /contrib/dbmirror/clean_pending.pl.
Isn't this done? -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Load distributed checkpoint
ITAGAKI Takahiro wrote: Bruce Momjian [EMAIL PROTECTED] wrote: 566.973777 327.158222 - (1) write() 560.773868 - (2) sleep 544.106645 - (3) fsync() OK, so you are saying that performance dropped only during the write(), and not during the fsync()? Interesting. Almost yes, but there is a small drop in fsync. (560-540) I would like to know the results of a few tests just like you reported them above: 1a) write spread out over 30 seconds 1b) write with no delay 2a) sleep(0) 2b) sleep(30) 3) fsync I would like to know the performance at each stage for each combination, e.g. when using 1b, 2a, 3, performance during the write() phase was X, during the sleep it was Y, and during the fsync it was Z. (Of course, sleep(0) has no stage timing.) I'm thinking about generalizing your idea; Adding three parameters (checkpoint_write, checkpoint_naptime and checkpoint_fsync) to control sleeps in each stage. 1) write() spread out over 'checkpoint_write' seconds. 2) sleep 'checkpoint_naptime' seconds between write() and fsync(). 3) fsync() spread out over 'checkpoint_fsync' seconds. If three parameter are all zero, checkpoints behave as the same as now. If checkpoint_write = checkpoint_timeout and other two are zero, it is just like my proposal before. As you might expect, I intend the above only for development purpose. Additinal three parameters are hard to use for users. If we can pull out some proper values from the tests, we'd better to set those values as default. I assume we can derive them from existing checkpoint_timeout. Great idea, though I wouldn't bother with checkpoint_fsync. I think Simon's previous email spelled out the problems of trying to delay fsyncs() --- in most cases, there will be one file with most of the I/O, and that fsync is going to be the flood. Basically, I think the variability of table access is too great for the fsync delay to ever be tunable by users. To summarize, if we could have fsync() only write the dirty buffers that happened as part of the checkpoint, we could delay the write() for the entire time between checkpoints, but we can't do that, so we have to make it user-tunable. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Load distributed checkpoint
On Thu, Dec 28, 2006 at 12:50:19PM -0500, Bruce Momjian wrote: To summarize, if we could have fsync() only write the dirty buffers that happened as part of the checkpoint, we could delay the write() for the entire time between checkpoints, but we can't do that, so we have to make it user-tunable. What about the mmap/msync(?)/munmap idea someone mentioned? -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Joshua D. Drake wrote: Hello, Is this really a TODO or is this someone being overzealous with the TODO list? Add a GUC to control whether BEGIN inside a transcation should abort the transaction. Well, right now, BEGIN inside a transaction just issues a warning: test= BEGIN; BEGIN test= BEGIN; WARNING: there is already a transaction in progress BEGIN test= SELECT 1; ?column? -- 1 (1 row) test= COMMIT; COMMIT I think you can make the case that this should be an error, or at least that's how it got on the TODO list. I can always remove it if people don't want the item completed. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] TODO: GNU TLS
Hello, What is the consideration here? I read the thread and it appears that OpenSSL is not compatible with GPL? But we don't care about that right? The OpenSSL looks pretty BSDish to me, expect the advertising clause (is that what caused XFree86.org to fork?). Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] TODO: Particularly, move GPL-licensed
Joshua D. Drake wrote: Isn't this done? -- Yes, thanks, removed. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TODO: GNU TLS
* Joshua D. Drake ([EMAIL PROTECTED]) wrote: What is the consideration here? I read the thread and it appears that OpenSSL is not compatible with GPL? But we don't care about that right? The OpenSSL looks pretty BSDish to me, expect the advertising clause (is that what caused XFree86.org to fork?). OpenSSL isn't compatible with the GPL. We do care because GPL applications link against libpq and therefore can end up linking against OpenSSL. I don't believe it was the OpenSSL advertising clause that caused the XFree86.org fork. My vaugue recollection is that XFree86 changed their license to include something like an advertising clause and that's what cause the split. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
Joshua D. Drake wrote: Hello, What is the consideration here? I read the thread and it appears that OpenSSL is not compatible with GPL? But we don't care about that right? The OpenSSL looks pretty BSDish to me, expect the advertising clause (is that what caused XFree86.org to fork?). Folks, please restate the topic in the email body, rather than requiring people to magically remember the subject. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Load distributed checkpoint
Jim C. Nasby wrote: On Thu, Dec 28, 2006 at 12:50:19PM -0500, Bruce Momjian wrote: To summarize, if we could have fsync() only write the dirty buffers that happened as part of the checkpoint, we could delay the write() for the entire time between checkpoints, but we can't do that, so we have to make it user-tunable. What about the mmap/msync(?)/munmap idea someone mentioned? I see that as similar to using O_DIRECT during checkpoint, which had poor performance. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
1 (1 row) test= COMMIT; COMMIT I think you can make the case that this should be an error, or at least that's how it got on the TODO list. I can always remove it if people don't want the item completed. Well I can tell you that my customers who are postgresql users ;) would howl in fury if we did that. They are already significantly irritated that certain errors are so strict. E.g., postgres=# BEGIN; BEGIN postgres=# ALTER TABLE baz ADD COLUMN bar text; ERROR: relation baz does not exist postgres=# SELECT * FROM foo; ERROR: current transaction is aborted, commands ignored until end of transaction block You do not need to argue with me about the purpose :), I understand why it is just really frustrating for many users. I would say that a GUC variable for such behavior as listed in the TODO is overzealous. We should either enforce it, or not. As we do not now, there is no reason imo to change it. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO: GNU TLS
On Thu, 2006-12-28 at 13:01 -0500, Stephen Frost wrote: * Joshua D. Drake ([EMAIL PROTECTED]) wrote: What is the consideration here? I read the thread and it appears that OpenSSL is not compatible with GPL? But we don't care about that right? The OpenSSL looks pretty BSDish to me, expect the advertising clause (is that what caused XFree86.org to fork?). OpenSSL isn't compatible with the GPL. The original discussion stated that well placed attorneys in the market feel that the FSF is trying to reach beyond the hands of god on this one and that we don't need to worry about it (my words, for actual quote read thread ;)). We do care because GPL applications link against libpq and therefore can end up linking against OpenSSL. \ Our concern is whether PostgreSQL is legally linked, not if something else is. /me is trying to think very hard of a program that is GPL that links to to PostgreSQL that would have a problem I don't believe it was the OpenSSL advertising clause that caused the XFree86.org fork. My vaugue recollection is that XFree86 changed their license to include something like an advertising clause and that's what cause the split. That's what I meant, that Xfree86.org added an advertising clause which caused the split. Sincerely, Joshua D. Drake Thanks, Stephen -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TODO: GNU TLS
On Thu, 2006-12-28 at 13:02 -0500, Bruce Momjian wrote: Joshua D. Drake wrote: Hello, What is the consideration here? I read the thread and it appears that OpenSSL is not compatible with GPL? But we don't care about that right? The OpenSSL looks pretty BSDish to me, expect the advertising clause (is that what caused XFree86.org to fork?). Folks, please restate the topic in the email body, rather than requiring people to magically remember the subject. http://archives.postgresql.org/pgsql-patches/2006-05/msg00040.php Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Tom Lane wrote: Several of the buildfarm machines are exhibiting repeatable signal 11 crashes in what seem perfectly ordinary queries. no stack trace yet however impala at least seems to be running out of memory (!) with 380MB of RAM and some 800MB of swap(and no other tasks) during the regression run. Maybe something is causing a dramatic increase in memory usage that is causing the random failures (in impalas case the OOM-killer actually decides to terminate the postmaster) ? No, most all the failures I've looked at are sig11 not sig9. hmm - still weird and I would not actually consider impala a resource starved box (especially when compared to other buildfarm-members) so there seems to be something strange going on. I have changed the overcommit settings on that box for now - let's see what the result of that will be. It is interesting that the failures are not as consistent as I first thought --- the machines that are showing failures actually fail maybe one time in two. or some even less - dove seems to be one of the affected boxes too - I increased the build frequency since yesterday but it has not yet failed again ... Stefan ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Strange pgsql crash on MacOSX
Shane Ambler [EMAIL PROTECTED] writes:
Tom Lane wrote:
Hm, so the question is: is it our bug or Apple's? If you kept the
busted history file, would you be willing to send me a copy?
The zip file attached has the psql_history file that crashes when
quiting but doesn't appear to contain the steps I done when it first
crashed.
So the answer is: it's Apple's bug, or at least not ours. libedit
contains a typo that causes it to potentially fail when saving strings
exceeding 256 bytes. Check out this code (around line 730 in history.c):
len = strlen(ev.str) * 4;
if (len = max_size) {
char *nptr;
max_size = (len + 1023) 1023;
nptr = h_realloc(ptr, max_size);
I think the intent of the max_size recalculation is to select the next
1K boundary larger than len, but it actually produces a number *less*
than 1K. Probably (len + 1023) ~1023 was meant ... but even that
is wrong if len is exactly a multiple of 1024, because it will fail to
round up. So the buffer is realloc'd too small, and that results in
a potential memory clobber if the history entry is less than 1K, and a
guaranteed clobber if it's more.
The source code available from Apple shows that they got this code from
NetBSD originally
/* $NetBSD: history.c,v 1.25 2003/10/18 23:48:42 christos Exp $*/
so this may well be a pretty generic *BSD bug. Anyone clear on who to
report it to? I have no idea if libedit is an independent project...
regards, tom lane
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: ... Maybe something is causing a dramatic increase in memory usage that is causing the random failures (in impalas case the OOM-killer actually decides to terminate the postmaster) ? No, most all the failures I've looked at are sig11 not sig9. hmm - still weird and I would not actually consider impala a resource starved box (especially when compared to other buildfarm-members) so there seems to be something strange going on. Actually ... one way that a memory overconsumption bug could manifest as sig11 would be if it's a runaway-recursion issue: usually you get sig11 when the machine's stack size limit is exceeded. This doesn't put us any closer to localizing the problem, but at least it's a guess about the cause? I wonder whether there's any way to get the buildfarm script to report a stack trace automatically if it finds a core file left behind in the $PGDATA directory after running the tests. Would something like this be adequately portable? if [ -f $PGDATA/core* ] then echo bt | gdb $installdir/bin/postgres $PGDATA/core* fi Obviously it'd fail if no gdb available, but that seems pretty harmless. The other thing that we'd likely need is an explicit ulimit -c unlimited for machines where core dumps are off by default. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO: GNU TLS
* Joshua D. Drake ([EMAIL PROTECTED]) wrote: On Thu, 2006-12-28 at 13:01 -0500, Stephen Frost wrote: OpenSSL isn't compatible with the GPL. The original discussion stated that well placed attorneys in the market feel that the FSF is trying to reach beyond the hands of god on this one and that we don't need to worry about it (my words, for actual quote read thread ;)). Yeah, that's nice. We do care because GPL applications link against libpq and therefore can end up linking against OpenSSL. \ Our concern is whether PostgreSQL is legally linked, not if something else is. /me is trying to think very hard of a program that is GPL that links to to PostgreSQL that would have a problem Looking at it from the point of view that psql if fine and therefore there isn't a problem is *very* short-sighted. Some of the packages currently in Debian/unstable which are licensed under the GPL and linked against libpq4 (a few of which have already provided exceptions for linking against OpenSSL): amarok http://packages.debian.org/changelogs/pool/main/a/amarok/current/copyright aolserver4-nspostgres http://packages.debian.org/changelogs/pool/main/a/aolserver4-nspostgres/current/copyright asterisk (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/a/asterisk/current/copyright bacula (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/b/bacula/current/copyright bandwidthd http://packages.debian.org/changelogs/pool/main/b/bandwidthd/current/copyright courier http://packages.debian.org/changelogs/pool/main/c/courier/current/copyright cvm http://packages.debian.org/changelogs/pool/main/c/cvm/current/copyright cvsnt http://packages.debian.org/changelogs/pool/main/c/cvsnt/current/copyright cyphesis-cpp http://packages.debian.org/changelogs/pool/main/c/cyphesis-cpp/current/copyright exim4 (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/e/exim4/current/copyright gambas http://packages.debian.org/changelogs/pool/main/g/gambas/current/copyright gnokii http://packages.debian.org/changelogs/pool/main/g/gnokii/current/copyright gnugk (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/g/gnugk/current/copyright gnustep-dl2 http://packages.debian.org/changelogs/pool/main/g/gnustep-dl2/current/copyright gphotocoll http://packages.debian.org/changelogs/pool/main/g/gphotocoll/current/copyright grass http://packages.debian.org/changelogs/pool/main/g/grass/current/copyright guile-pg http://packages.debian.org/changelogs/pool/main/g/guile-pg/current/copyright guile-simplesql http://packages.debian.org/changelogs/pool/main/g/guile-simplesql/current/copyright jabberd2 http://packages.debian.org/changelogs/pool/main/j/jabberd2/current/copyright libdbd-pg-perl http://packages.debian.org/changelogs/pool/main/libd/libdbd-pg-perl/current/copyright postgis http://packages.debian.org/changelogs/pool/main/p/postgis/current/copyright And quite a few others, I'm sure, I just got tired of looking. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
Some of the packages currently in Debian/unstable which are licensed under the GPL and linked against libpq4 (a few of which have already provided exceptions for linking against OpenSSL): Sounds to me like the list below needs to add an OpenSSL exception and that PostgreSQL needs to make mention in the docs somewhere of the potential issue. Sincerely, Joshua D. Drake amarok http://packages.debian.org/changelogs/pool/main/a/amarok/current/copyright aolserver4-nspostgres http://packages.debian.org/changelogs/pool/main/a/aolserver4-nspostgres/current/copyright asterisk (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/a/asterisk/current/copyright bacula (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/b/bacula/current/copyright bandwidthd http://packages.debian.org/changelogs/pool/main/b/bandwidthd/current/copyright courier http://packages.debian.org/changelogs/pool/main/c/courier/current/copyright cvm http://packages.debian.org/changelogs/pool/main/c/cvm/current/copyright cvsnt http://packages.debian.org/changelogs/pool/main/c/cvsnt/current/copyright cyphesis-cpp http://packages.debian.org/changelogs/pool/main/c/cyphesis-cpp/current/copyright exim4 (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/e/exim4/current/copyright gambas http://packages.debian.org/changelogs/pool/main/g/gambas/current/copyright gnokii http://packages.debian.org/changelogs/pool/main/g/gnokii/current/copyright gnugk (which has provided an OpenSSL exception) http://packages.debian.org/changelogs/pool/main/g/gnugk/current/copyright gnustep-dl2 http://packages.debian.org/changelogs/pool/main/g/gnustep-dl2/current/copyright gphotocoll http://packages.debian.org/changelogs/pool/main/g/gphotocoll/current/copyright grass http://packages.debian.org/changelogs/pool/main/g/grass/current/copyright guile-pg http://packages.debian.org/changelogs/pool/main/g/guile-pg/current/copyright guile-simplesql http://packages.debian.org/changelogs/pool/main/g/guile-simplesql/current/copyright jabberd2 http://packages.debian.org/changelogs/pool/main/j/jabberd2/current/copyright libdbd-pg-perl http://packages.debian.org/changelogs/pool/main/libd/libdbd-pg-perl/current/copyright postgis http://packages.debian.org/changelogs/pool/main/p/postgis/current/copyright And quite a few others, I'm sure, I just got tired of looking. Thanks, Stephen -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Load distributed checkpoint
Bruce Momjian [EMAIL PROTECTED] writes: Jim C. Nasby wrote: What about the mmap/msync(?)/munmap idea someone mentioned? I see that as similar to using O_DIRECT during checkpoint, which had poor performance. That's a complete nonstarter on portability grounds, even if msync gave us the desired semantics, which it doesn't. It's no better than fsync for our purposes. To my mind the problem with fsync is not that it gives us too little control but that it gives too much: we have to specify a particular order of writing out files. What we'd really like is a version of sync(2) that tells us when it's done but doesn't constrain the I/O scheduler's choices at all. Unfortunately there's no such API ... regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Joshua D. Drake [EMAIL PROTECTED] writes: I would say that a GUC variable for such behavior as listed in the TODO is overzealous. We should either enforce it, or not. As we do not now, there is no reason imo to change it. Not only is it overzealous, but the proposal to have one reflects a failure to learn from history. GUC variables that change transaction-boundary semantics are a bad idea, period: see autocommit. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
* Joshua D. Drake ([EMAIL PROTECTED]) wrote: Some of the packages currently in Debian/unstable which are licensed under the GPL and linked against libpq4 (a few of which have already provided exceptions for linking against OpenSSL): Sounds to me like the list below needs to add an OpenSSL exception and that PostgreSQL needs to make mention in the docs somewhere of the potential issue. Gee, thanks. Perhaps one might consider the license a reason why some might prefer GNUTLS and would like to see the option? I'm pretty confident that Debian, at least, would switch to using GNUTLS for Postgres were it available. It's certainly something we'd like to see supported by upstream as an alternative to OpenSSL. Given the patch is available it's possible we could just apply it locally (as was done w/ OpenLDAP) but I really don't think anyone would like to see that. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
Gee, thanks. Perhaps one might consider the license a reason why some might prefer GNUTLS and would like to see the option? I'm pretty confident that Debian, at least, would switch to using GNUTLS for Postgres were it available. It's certainly something we'd like to see supported by upstream as an alternative to OpenSSL. Given the patch is available it's possible we could just apply it locally (as was done w/ OpenLDAP) but I really don't think anyone would like to see that. I am not the one you need to convince :). I honestly don't care that much. I am just trying to help clean up the TODO list. If you want the GNU TLS patch accepted, you should probably start a thread about that problem specifically. Currently, Tom Lane does not like how invasive the patch is. So someone is going to have to convince him this is a good idea or some of the other committers. Sincerely, Joshua D. Drake Thanks, Stephen -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
On Thu, 2006-12-28 at 13:52 -0500, Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: I would say that a GUC variable for such behavior as listed in the TODO is overzealous. We should either enforce it, or not. As we do not now, there is no reason imo to change it. Not only is it overzealous, but the proposal to have one reflects a failure to learn from history. GUC variables that change transaction-boundary semantics are a bad idea, period: see autocommit. Nod. Let's get this TODO removed. Sincerely, Joshua D. Drake regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] TODO: GNU TLS
* Joshua D. Drake ([EMAIL PROTECTED]) wrote: I am not the one you need to convince :). I honestly don't care that much. I am just trying to help clean up the TODO list. If you want the GNU TLS patch accepted, you should probably start a thread about that problem specifically. Given the thread subject I figured that was in scope... :) Currently, Tom Lane does not like how invasive the patch is. So someone is going to have to convince him this is a good idea or some of the other committers. Yeah, I saw that, but I think Martijn answers that quite well: More than half the patch is simply moving the OpenSSL related stuff from fe/be-secure.c to fe/be-secure-openssl.c. If you create the -openssl versions first you can more easily see that the changes are in fact quite minor. Unfortunatly diff can't represent the change copy N lines from file A to file B very efficiently. And following on to that wrt code maintenance: Hmm, I see your point. I guess that's an unavoidable side-effect of the process :(. However, judging from the CVS logs, these have not been files with a high change rate. I think it's worth it but I can imagine other people see that differently. As for Brunce's comment here: Of course, we are trading a BSD license with advertizing clause with an LGPL license. I guess it makes sense. I don't see that we're *trading* anything here if we support both, we're providing users with the choice of which they'd prefer. I'd agree with Martijn from above- the benefit is worth the (hopefully low) maintenance cost. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
I don't see that we're *trading* anything here if we support both, we're providing users with the choice of which they'd prefer. I'd agree with Martijn from above- the benefit is worth the (hopefully low) maintenance cost. Well for my honest opinion, I think we should pick *one* and stick with it. I don't care which. Same for readline. Either spoon up libedit in ice cream glory or use just readline :) Sincerely, Joshua D. Drake Thanks, Stephen -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_standby and build farm
On Thu, 2006-12-28 at 08:45 -0500, Doug Knight wrote: Thanks for the response, Simon. I'm continuing to use your script, so if there's anything I can help you with (testing, ideas, etc), just let me know. My earlier comments about mv were not correct; when fully retesting things, I noted another error instead and have corrected that. pg_standby now supports cp, mv and ln - all 3 now tested on Linux. Posting new version to patches; no signal handling (yet). Another other comments gratefully received. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
Joshua D. Drake [EMAIL PROTECTED] writes: Currently, Tom Lane does not like how invasive the patch is. If GNUTLS really wants to take market share from OpenSSL, why don't they provide a more nearly compatible API? I don't see why we should have to jump through so many hoops in order to satify someone else's license fetish. (And it is a fetish, because only a compulsively narrow-minded reading of the licenses yields the conclusion that there's a problem.) regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TODO: GNU TLS
* Tom Lane ([EMAIL PROTECTED]) wrote: Joshua D. Drake [EMAIL PROTECTED] writes: Currently, Tom Lane does not like how invasive the patch is. If GNUTLS really wants to take market share from OpenSSL, why don't they provide a more nearly compatible API? I don't see why we should have They do but it's GPL. Not to mention that the OpenSSL API isn't exactly a shining star in the software world... I really don't feel this has got anything to do with 'market share' and I'm not advocating Postgres drop support for OpenSSL. I disagree that the only way Postgres should support multiple libraries for a given component is if they provide the same API- we wouldn't have much in the way of authentication options if that was really the case. The patch appears large because of things being moved around and not becuase it is tremendously invasive. Also, this area hasn't required a lot of maintenance in the past and I doubt adding GNUTLS support would change that. to jump through so many hoops in order to satify someone else's license fetish. (And it is a fetish, because only a compulsively narrow-minded reading of the licenses yields the conclusion that there's a problem.) While I appriciate that RH isn't concerned I don't feel their interpretation is the only possible one which could come out of a court case. Not to mention that just finding out would be expensive in its own right. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
On Thu, Dec 28, 2006 at 10:13:14AM -0800, Joshua D. Drake wrote: On Thu, 2006-12-28 at 13:01 -0500, Stephen Frost wrote: * Joshua D. Drake ([EMAIL PROTECTED]) wrote: What is the consideration here? I read the thread and it appears that OpenSSL is not compatible with GPL? But we don't care about that right? The OpenSSL looks pretty BSDish to me, expect the advertising clause (is that what caused XFree86.org to fork?). OpenSSL isn't compatible with the GPL. With few exceptions, you cannot derive or include GPL software in your non-GPL software. The GPL works very hard to maintain this position to protect the freedom of the user. The GPL cannot control how OpenSSL is distributed, though. The OpenSSL license controls this. I don't see any place in the (short and sweet!) OpenSSL license that prevents it from being using in GPL software. Are you reading some particular point in the OpenSSL license that I am not? PostgreSQL isn't GPL software anyways, and there is certainly nothing in the OpenSSL license preventing it from being used in PostgreSQL. If the argument is that the 'whole derived product' must fit the outter most provided license, then I think you should consider that PostgreSQL should not include *ANY* GPL software, as any user of PostgreSQL cannot be guaranteed of the generous freedoms provided by the PostgreSQL license. Some components are covered by GPL, which is restrictive compared to PostgreSQL. Down this path is the impractical, and silly conclusion that all software must be licensed under the exact same license to be distributed. An all GPL distribution, for example. While those with a political agenda such as Richard Stallman would cheer at this result, those people do not have the power to force this will on us. The Free Software Foundation provides an LGPL that has fewer restrictions than GPL, out of recognition that their political goals are not practical for all uses. LGPL software may be linked with GPL software without invalidating the GPL rights of the user. GPL applies to the GPL part, and LGPL applies to the LGPL part. All is well in the world. In conclusion - I'll restate. The only license that can restrict the distribution of OpenSSL, is the OpenSSL license. The GPL is not relevant in determining where OpenSSL may be distributed to. Anybody who believes OpenSSL is a problem, must be aware of some software distribution for which the OpenSSL licensing terms are unreasonable. I'm not sure who that would be. They ask for attribution. They ask that their name not be used to promote another product. They ask that their name not be used in the name of another product. All of these terms seem fair to me. The original discussion stated that well placed attorneys in the market feel that the FSF is trying to reach beyond the hands of god on this one and that we don't need to worry about it (my words, for actual quote read thread ;)). I agree with Tom - if they really want people to use GNUTLS, why did they make it have such a different interface? I recently had to choose between the two for a product at work, and GNUTLS seemed to fall short in several areas. It was a race between GNUTLS seeming to having superior documentation vs OpenSSL seeming to have superior functionality. For my rather complicated requirements, OpenSSL won out (function is more important than documentation), and the product using it is about 90% complete. It includes such ugliness as OpenSSL/C code that needs to load the encrypted private key and client/server x509 certificates from a Java Keystore (JKS)... Total fun... :-) Cheers, mark -- [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] __ . . _ ._ . . .__. . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/|_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] TODO: GNU TLS
On Thu, Dec 28, 2006 at 02:48:56PM -0500, Stephen Frost wrote: I disagree that the only way Postgres should support multiple libraries for a given component is if they provide the same API- we wouldn't have much in the way of authentication options if that was really the case. I don't believe that was said. However, using two very different APIs for the exact same purpose, providing the exact same functionality would seem to require a business case. If fear of litigation over what seems to be a non-existent point is the only business case, the position deserves to be challenged. There are other elements that could be included in the business case. For example, the documentation for GNUTLS seems to be significantly better. I don't like fear mongering. It smells like FUD. :-) Cheers, mark -- [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] __ . . _ ._ . . .__. . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/|_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Joshua D. Drake wrote: On Thu, 2006-12-28 at 13:52 -0500, Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: I would say that a GUC variable for such behavior as listed in the TODO is overzealous. We should either enforce it, or not. As we do not now, there is no reason imo to change it. Not only is it overzealous, but the proposal to have one reflects a failure to learn from history. GUC variables that change transaction-boundary semantics are a bad idea, period: see autocommit. Nod. Let's get this TODO removed. OK, removed. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Load distributed checkpoint
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Jim C. Nasby wrote: What about the mmap/msync(?)/munmap idea someone mentioned? I see that as similar to using O_DIRECT during checkpoint, which had poor performance. That's a complete nonstarter on portability grounds, even if msync gave us the desired semantics, which it doesn't. It's no better than fsync for our purposes. To my mind the problem with fsync is not that it gives us too little control but that it gives too much: we have to specify a particular order of writing out files. What we'd really like is a version of sync(2) that tells us when it's done but doesn't constrain the I/O scheduler's choices at all. Unfortunately there's no such API ... Yea, we used to use sync() but that did all files, not just the PostgreSQL ones. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
* [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: In conclusion - I'll restate. The only license that can restrict the distribution of OpenSSL, is the OpenSSL license. The GPL is not relevant in determining where OpenSSL may be distributed to. The issue is not the distribution of OpenSSL but rather the distribution of GPL applications which link against OpenSSL. Because of the GPL the resulting application can not have any *additional* restrictions on it (meaning it can be linked against libpq without any problem because libpq's license doesn't add any restrictions, but can't be against OpenSSL because the OpenSSL license adds the advertising clause which isn't in the GPL). *That's* the issue here, not whatever it is you were arguing against. There are a few ways to resolve this- add GNUTLS support to PostgreSQL (GNUTLS is LGPL and so won't cause a problem with GPL or other licenses in general) or get every GPL application author which ends up using OpenSSL to provide an exception (which Debian's been working on, actually, with some success), or get GPLv3 to allow advertising clauses and get everyone to switch to it (not exactly likely to happen...), or get OpenSSL to drop the advertising clause (I've been told they would if they could but that much of the code is authored by an individual who now works for a competitor and now has very little interest in helping out the OpenSSL project in any way...). If you feel the advertising clause is fine, then it's the GPL that's at fault here for not allowing it. If you disagree with the advertising clause then it's the fault of the OpenSSL license. Personally, I don't really care which way you want to look at it. On another note, personally I feel it's a good thing to support multiple libraries when the cost of doing so is reasonably low. I had kind of half-expected people would agree with that sentiment and so the licenseing issue it would resolve (for at least Debian) is that much more reason. I didn't really expect a reaction of there isn't a licenseing issue so we shouldn't add support for another library. I could understand if people don't care about the licensing aspect but I don't really get this one-size-fits-all mentality when it comes to an SSL library. We don't seem to feel that way about authentication mechanisms, operating systems, or even client libraries. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: ... Maybe something is causing a dramatic increase in memory usage that is causing the random failures (in impalas case the OOM-killer actually decides to terminate the postmaster) ? No, most all the failures I've looked at are sig11 not sig9. hmm - still weird and I would not actually consider impala a resource starved box (especially when compared to other buildfarm-members) so there seems to be something strange going on. Actually ... one way that a memory overconsumption bug could manifest as sig11 would be if it's a runaway-recursion issue: usually you get sig11 when the machine's stack size limit is exceeded. This doesn't put us any closer to localizing the problem, but at least it's a guess about the cause? that sounds like a possibility though I'm not too optimistic this is indeed the cause of the problem we see. I wonder whether there's any way to get the buildfarm script to report a stack trace automatically if it finds a core file left behind in the $PGDATA directory after running the tests. Would something like this be adequately portable? if [ -f $PGDATA/core* ] then echo bt | gdb $installdir/bin/postgres $PGDATA/core* fi hmmm - not sure I like that that much Obviously it'd fail if no gdb available, but that seems pretty harmless. The other thing that we'd likely need is an explicit ulimit -c unlimited for machines where core dumps are off by default. there are other issues with that - gdb might be available but not actually producing reliable results on certain platforms (some commercial unixes,windows). The thing we might might want to do is the buildfarm script overriding keep_error_builds=0 conditionally in some cases (like detecting a core). That way we will at least have a useful buildtree for later examination(which would be removed even if we get a one-time stacktrace and keep_error_builds is disabled) Stefan ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO: GNU TLS
Stephen Frost wrote: * [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: In conclusion - I'll restate. The only license that can restrict the distribution of OpenSSL, is the OpenSSL license. The GPL is not relevant in determining where OpenSSL may be distributed to. The issue is not the distribution of OpenSSL but rather the distribution of GPL applications which link against OpenSSL. Because of the GPL the resulting application can not have any *additional* restrictions on it (meaning it can be linked against libpq without any problem because libpq's license doesn't add any restrictions, but can't be against OpenSSL because the OpenSSL license adds the advertising clause which isn't in the GPL). *That's* the issue here, not whatever it is you were arguing against. Stephen, you write as if there were no legal disagreement about this. But there is, as you well know. My understanding is that most of the non-FSF lawyers who have looked at this think it's not a problem. I am not a lawyer, and AFAIK neither are you. Maybe we all need to stop playing Perry Mason and take some well informed legal advice. cheers andrew ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Load distributed checkpoint
Tom Lane wrote: To my mind the problem with fsync is not that it gives us too little control but that it gives too much: we have to specify a particular order of writing out files. What we'd really like is a version of sync(2) that tells us when it's done but doesn't constrain the I/O scheduler's choices at all. Unfortunately there's no such API ... The problem I see with fsync is that it causes an immediate I/O storm as the OS tries to flush everything out as quickly as possible. But we're not in a hurry. What we'd need is a lazy fsync, that would tell the operating system let me know when all these dirty buffers are written to disk, but I'm not in a hurry, take your time. It wouldn't change the scheduling of the writes, just inform the caller when they're done. If we wanted more precise control of the flushing, we could use sync_file_range on Linux, but that's not portable. Nevertheless, I think it would be OK to have an ifdef and use it on platforms that support it, if it gave a benefit. As a side note, with full_page_writes on, a checkpoint wouldn't actually need to fsync those pages that have been written to WAL after the checkpoint started. Doesn't make much difference in most cases, but we could take that into account if we start taking more control of the flushing. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
Stephen, you write as if there were no legal disagreement about this. But there is, as you well know. My understanding is that most of the non-FSF lawyers who have looked at this think it's not a problem. I am not a lawyer, and AFAIK neither are you. Maybe we all need to stop playing Perry Mason and take some well informed legal advice. But Perry Mason was so cool! cheers andrew -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TODO: GNU TLS
* [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: On Thu, Dec 28, 2006 at 02:48:56PM -0500, Stephen Frost wrote: I disagree that the only way Postgres should support multiple libraries for a given component is if they provide the same API- we wouldn't have much in the way of authentication options if that was really the case. I don't believe that was said. However, using two very different APIs for the exact same purpose, providing the exact same functionality would seem to require a business case. If fear of litigation over what seems to be a non-existent point is the only business case, the position deserves to be challenged. There are other elements that could be included in the business case. For example, the documentation for GNUTLS seems to be significantly better. I havn't done serious comparisons of the two since the license issue matters to me, honestly, so this can be taken with a grain of salt, but... OpenSSL has more features and some niceties like the TLS_CACERTDIR (I've asked for this in GNUTLS, actually, so it might have it soon) They can each be faster than the other in some instances (I'm not sure which though, I've heard of 15% differences in each direction depending on architecture though) GNUTLS has a nicer/better API GNUTLS has a smaller memory footprint OpenSSL is working to get NIST certification/approval (it had it, but then lost it for some reason and they're working to get that fixed) GNUTLS has better documentation Actually, from a comparison done for libcurl (which supports both): GnuTLS vs OpenSSL While these two libraries offer similar features, they are not equal. Both libraries have features the other one lacks. libcurl does not (yet) offer a standardized stable ABI if you decide to switch from using libcurl-openssl to libcurl-gnutls or vice versa. The GnuTLS support is very recent in libcurl and it has not been tested nor used very extensively, while the OpenSSL equivalent code has been used and thus matured for more than seven (7) years. GnuTLS - LGPL licensened - supports SRP - lacks SSLv2 support - lacks MD2 support (used by at least some CA certs) - lacks the crypto functions libcurl uses for NTLM OpenSSL - Original BSD licensened - lacks SRP - supports SSLv2 - older and more widely used - provides crypto functions libcurl uses for NTLM - libcurl can do non-blocking connects with it in 7.15.4 and later That was from May 15, 2006: http://curl.mirrors.cyberservers.net/legal/distro-dilemma.html Regarding SSLv2 support, the GNUTLS page has this: Support for TLS 1.1, TLS 1.0 and SSL 3.0 protocols * Since SSL 2.0 is insecure it is not supported. * TLS 1.2 is supported in the experimental branch. I don't like fear mongering. It smells like FUD. :-) While I didn't intend it as fear mongering I can understand that might be the impression whenever licenses and possible license violations are brought up. I don't know of anyone who's actually attempting to prosecute this nor do I generally expect someone to in the future. Even so though, it wouldn't be a PostgreSQL issue anyway but rather an issue for someone distributing OpenSSL and some GPL application which linked against it (ie: a distribution like Debian). Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Dead Space Map for vacuum
Gavin Sherry wrote: On Thu, 28 Dec 2006, Heikki Linnakangas wrote: ITAGAKI Takahiro wrote: I experimented with a different DSM design last winter. I got busy with other things and never posted it AFAIR, but the idea was to store a bitmap in the special area on every 32k heap page. That had some advantages: * doesn't require a new dedicated shared memory area that needs to be allocated and tuned. * doesn't introduce a (single) new global lwlock that might become hotspot. * WAL logging is quite simple, since the bitmaps are on normal pages handled by buffer manager. I too have experimented with this idea. 4 DSM pages means 2 bits per block. What were you using the other bit for? When I discussed this some time ago with Jan Weick, we recommended we track blocks in the FSM with this extra bit. I only used 1 bit, just like in Itagaki's approach. The bitmap in the special area only occupied half a page, the rest was available for normal heap tuples. That might seem strange, but it avoids having to allocate 2 pages for small tables with just a handful of rows. It also spreads out the lock contention a bit. I think I wasn't clear in my explanation. If a table had less than 32k pages, it had just one bitmap with each bit representing a heap page. The bitmap was stored in the special area of the first page. If a table was larger, the bitmap on the 1st page represented pages 1-32k. On the 32768th page there's another bitmap that represents the next 32k pages, and so on. In other words, the DSM bitmap of page number X was always on page number (X / 32768). Vacuum had to check all the bitmaps. BTW: Yesterday I realized that bitmap indexes could be retail vacuumed safely. You'll still need to visit all bitmaps to find the dead bit, but you only need to check the bitmap page that corresponds the tid of the dead tuple. Cool. The problem is solving it for the other AMs, particularly btree. Yeah.. | http://archives.postgresql.org/pgsql-hackers/2004-03/msg00957.php | Maintain 2 bits per block that tell if the block has been vaccumed of all | dead tuples since the last time it was dirtied, and if all its tuples are | completely frozen. We use 1 bit per block, so we cannot separate pages that need either vacuum or freeze only. The reason is that we cannot determine where to record before/after updated tuples. If the transaction is commited, before-version should be recorded into needs-vacuum bitmap and after-version into needs-freeze bitmap. But on rollback, it is inverted. We cannot judge which we should do until the end of transaction. Yeah, that makes the DSM less useful. Are you thinking of freezing more aggressively because of that? Or doing a full-scanning vacuum every now and then to freeze? I don't see any problem with moving to two status bits per block in the DSM-in-heap model, so we can track this better. What do you think? Well, the obvious drawback is that two bits take more space than one bit. But it feels ok to me as well. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] TODO: GNU TLS
* Andrew Dunstan ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: The issue is not the distribution of OpenSSL but rather the distribution of GPL applications which link against OpenSSL. Because of the GPL the resulting application can not have any *additional* restrictions on it (meaning it can be linked against libpq without any problem because libpq's license doesn't add any restrictions, but can't be against OpenSSL because the OpenSSL license adds the advertising clause which isn't in the GPL). *That's* the issue here, not whatever it is you were arguing against. Stephen, you write as if there were no legal disagreement about this. I was trying to explain the issue as I understood it. I'm happy to admit that not everyone feels this is an issue (in fact, I've said as much elsewhere in this thread). That doesn't mean there aren't some who *do* feel it's an issue though. But there is, as you well know. My understanding is that most of the non-FSF lawyers who have looked at this think it's not a problem. I am not a lawyer, and AFAIK neither are you. Maybe we all need to stop playing Perry Mason and take some well informed legal advice. I'm certainly not a lawyer and I'd be astounded if anyone felt I represented myself as such. I don't have opinions from any lawyers beyond Tom's comments previously from RH's legal team and FSF's comments on the issue. I don't know where the 'most of the non-FSF lawyers' claim comes from, if you're aware of others who have commented on it I'd be happy to listen to them. I do know that this has been an issue for Debian for quite some time and it seems rather unlikely that Debian's position on it will change. SPI does have a pro-bono lawyer but I don't know that this question has been posed to him, probably because the general consensus among the Debian Powers that Be is that it is an issue and we try to not bother our pro-bono lawyer too much (being, uh, pro-bono and all). Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Tom Lane wrote: Actually ... one way that a memory overconsumption bug could manifest as sig11 would be if it's a runaway-recursion issue: usually you get sig11 when the machine's stack size limit is exceeded. This doesn't put us any closer to localizing the problem, but at least it's a guess about the cause? I wonder whether there's any way to get the buildfarm script to report a stack trace automatically if it finds a core file left behind in the $PGDATA directory after running the tests. Would something like this be adequately portable? if [ -f $PGDATA/core* ] then echo bt | gdb $installdir/bin/postgres $PGDATA/core* fi gdb has a batch mode which can be useful: if [ -f $PGDATA/core* ] then gdb -ex bt --batch $installdir/bin/postgres $PGDATA/core* fi -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] TODO: GNU TLS
Stephen Frost wrote: My understanding is that most of the non-FSF lawyers who have looked at this think it's not a problem. I am not a lawyer, and AFAIK neither are you. Maybe we all need to stop playing Perry Mason and take some well informed legal advice. I'm certainly not a lawyer and I'd be astounded if anyone felt I represented myself as such. I don't have opinions from any lawyers beyond Tom's comments previously from RH's legal team and FSF's comments on the issue. I don't know where the 'most of the non-FSF lawyers' claim comes from, if you're aware of others who have commented on it I'd be happy to listen to them. I said that was my understanding, not that I had direct knowledge of it. But maybe I'm wrong. I do know that this has been an issue for Debian for quite some time and it seems rather unlikely that Debian's position on it will change. SPI does have a pro-bono lawyer but I don't know that this question has been posed to him, probably because the general consensus among the Debian Powers that Be is that it is an issue and we try to not bother our pro-bono lawyer too much (being, uh, pro-bono and all). I have a sneaking suspicion that there are some hidden agendas in all this. I agree with this comment from Steve Langasek at http://lists.debian.org/debian-legal/2003/01/msg00022.html : Sure, code can be rewritten to use gnutls natively. But I don't understand why anyone would consider this a useful expenditure of developer resources when the necessary OpenSSL compat glue could simply be made available under the LGPL. If this is such an issue, why hasn't somebody done that? cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
On Thu, Dec 28, 2006 at 05:16:58PM -0500, Andrew Dunstan wrote: I agree with this comment from Steve Langasek at http://lists.debian.org/debian-legal/2003/01/msg00022.html : Sure, code can be rewritten to use gnutls natively. But I don't understand why anyone would consider this a useful expenditure of developer resources when the necessary OpenSSL compat glue could simply be made available under the LGPL. If this is such an issue, why hasn't somebody done that? Maybe because the OpenSSL interface is terrible? I'm not sure if it's actually possible to emulate the OpenSSL interface, given the way the libraries work internally are completely different. There is an OpenSSL compatability layer, but postgres won't compile with it because it's nowhere near complete enough, even for the simple things postgres wants to do. However, in this case we're talking about code that has already been written. So the work has already been done... Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Alvaro Herrera wrote:
Tom Lane wrote:
I wonder whether there's any way to get the buildfarm script to report a
stack trace automatically if it finds a core file left behind in the
$PGDATA directory after running the tests. Would something like this
be adequately portable?
if [ -f $PGDATA/core* ]
then
echo bt | gdb $installdir/bin/postgres $PGDATA/core*
fi
gdb has a batch mode which can be useful:
if [ -f $PGDATA/core* ]
then
gdb -ex bt --batch $installdir/bin/postgres $PGDATA/core*
fi
here's a quick untested patch for buildfarm that Stefan might like to try.
cheers
andrew
--- run_build.pl.orig 2006-12-28 17:32:14.0 -0500
+++ run_build.pl.new2006-12-28 17:58:51.0 -0500
@@ -795,6 +795,29 @@
$dbstarted=undef;
}
+
+sub get_stack_trace
+{
+ my $bindir = shift;
+ my $pgdata = shift;
+
+ # no core = no result
+ return () unless -f $pgdata/core;
+
+ # no gdb = no result
+ system gdb --version /dev/null 21;
+ my $status = $? 8;
+ return () if $status;
+
+ my @trace = `gdb -ex bt --batch $bindir/postgres $pgdata/core 21`;
+
+ unshift(@trace,
+ \n\n== stack trace
==\n);
+
+ return @trace;
+
+}
+
sub make_install_check
{
my @checkout = `cd $pgsql/src/test/regress $make installcheck 21`;
@@ -814,6 +837,11 @@
}
close($handle);
}
+ if ($status)
+ {
+ my @trace =
get_stack_trace($installdir/bin,$installdir/data);
+ push(@checkout,@trace);
+ }
writelog('install-check',[EMAIL PROTECTED]);
print make installcheck log ===\n,@checkout
if ($verbose 1);
@@ -839,6 +867,11 @@
}
close($handle);
}
+ if ($status)
+ {
+ my @trace =
get_stack_trace($installdir/bin,$installdir/data);
+ push(@checkout,@trace);
+ }
writelog('contrib-install-check',[EMAIL PROTECTED]);
print make contrib installcheck log ===\n,@checkout
if ($verbose 1);
@@ -864,6 +897,11 @@
}
close($handle);
}
+ if ($status)
+ {
+ my @trace =
get_stack_trace($installdir/bin,$installdir/data);
+ push(@checkout,@trace);
+ }
writelog('pl-install-check',[EMAIL PROTECTED]);
print make pl installcheck log ===\n,@checkout
if ($verbose 1);
@@ -892,6 +930,13 @@
}
close($handle);
}
+ if ($status)
+ {
+ my @trace =
+
get_stack_trace($pgsql/src/test/regress/install$installdir/bin,
+
$pgsql/src/test/regress/tmp_check/data);
+ push(@makeout,@trace);
+ }
writelog('check',[EMAIL PROTECTED]);
print make check logs ===\n,@makeout
if ($verbose 1);
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TODO: GNU TLS
* Andrew Dunstan ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: I do know that this has been an issue for Debian for quite some time and it seems rather unlikely that Debian's position on it will change. SPI does have a pro-bono lawyer but I don't know that this question has been posed to him, probably because the general consensus among the Debian Powers that Be is that it is an issue and we try to not bother our pro-bono lawyer too much (being, uh, pro-bono and all). I have a sneaking suspicion that there are some hidden agendas in all this. I'm certainly not aware of any personally. I doubt Debian in general does either since this isn't exactly a fun thing for us to have to deal with. I agree with this comment from Steve Langasek at http://lists.debian.org/debian-legal/2003/01/msg00022.html : Unfortunately, the glue hasn't been made available under the LGPL. While I agree with Steve generally (and in fact have been discussing this whole bit with him on IRC), in this case he's right but the point is moot- it *could* be done, but it *hasn't* been done. The options are to go ask the original author about relicensing it (which I think has actually been done already) or rewriting it (which apparently hasn't been done). Sure, code can be rewritten to use gnutls natively. But I don't understand why anyone would consider this a useful expenditure of developer resources when the necessary OpenSSL compat glue could simply be made available under the LGPL. If this is such an issue, why hasn't somebody done that? Based on what I've seen happen to date it appears that projects would rather just include GNUTLS support directly than write a wrapper to support the OpenSSL API using GNUTLS. Indeed, that's exactly the approach Martijn took as well. My guess as to why this would be is that it's simply not *that* difficult to do and maintain, and in the end perhaps some prefer the GNUTLS API over the OpenSSL API, or feel that more things are moving in that direction. I don't know, I can't speak for them so I'm really just speculating, but the empirical evidence is that projects support GNUTLS and there doesn't exist a non-GPL OpenSSL API for GNUTLS yet. I understand that at least some GPL projects do use the GPL OpenSSL API for GNUTLS but it's not common. (fe: I know exim4, elinks, mutt, samba, curl/libcurl, and others support GNUTLS directly while the only project I've heard of using the wrapper is slrn, cupsys used the compat layer at one point but then changed to using GNUTLS directly). Maybe people feel that using a compat layer is uglier than using GNUTLS directly? Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
On Thu, Dec 28, 2006 at 03:56:48PM -0500, Stephen Frost wrote: * [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: In conclusion - I'll restate. The only license that can restrict the distribution of OpenSSL, is the OpenSSL license. The GPL is not relevant in determining where OpenSSL may be distributed to. The issue is not the distribution of OpenSSL but rather the distribution of GPL applications which link against OpenSSL. Because of the GPL the resulting application can not have any *additional* restrictions on it (meaning it can be linked against libpq without any problem because libpq's license doesn't add any restrictions, but can't be against OpenSSL because the OpenSSL license adds the advertising clause which isn't in the GPL). I don't see the problem. If I redistribute PostgreSQL with GPL software that I author, I am supposed to keep a copy of the PostgreSQL license with the derived works. Respecting the license for every component of software is regular business. By the words you describe above, the GPL doesn't require that you include a copy of the PostgreSQL license either. Are you saying that this makes GPL incompatible with PostgreSQL? It's silliness. If you redistribute OpenSSL, you honour the OpenSSL requirements. That's the *only* requirement by copyright law. It doesn't matter if it is GPL on top, or not. You always honour each license. The *only* thing GPL-with-GPL does is reduce complexity. *That's* the issue here, not whatever it is you were arguing against. I think you might only be listening to one side. There are a few ways to resolve this- add GNUTLS support to PostgreSQL (GNUTLS is LGPL and so won't cause a problem with GPL or other licenses in general) or get every GPL application author which ends up using OpenSSL to provide an exception (which Debian's been working on, actually, with some success), or get GPLv3 to allow advertising clauses and get everyone to switch to it (not exactly likely to happen...), or get OpenSSL to drop the advertising clause (I've been told they would if they could but that much of the code is authored by an individual who now works for a competitor and now has very little interest in helping out the OpenSSL project in any way...). 1) Adding GNUTLS support to PostgreSQL does not eliminate any PostgreSQL obligation to honour the OpenSSL distribution license. Any PostgreSQL distribution that includes OpenSSL must honour the OpenSSL distribution license, just as any PostgreSQL distribution that includes GNUTLS must honour either the GPL or LGPL license. Nothing changes. It's about distribution. If PostgreSQL includes OpenSSL support, it is a derived works when distributed with OpenSSL. It is a misunderstanding to believe that support for many interfaces allows you to avoid any licensing issues. It is a popular misunderstanding. 2) Explicitly stating an OpenSSL exception is not a legal requirement. It is not possible for any derived product to except conditions for OpenSSL. OpenSSL defines its *own* license. You cannot modify it, which means that the GPL cannot reduce its significance, nor can an explicit exception claus increase its significance. OpenSSL distribution rights are defined by the OpenSSL license. Full stop. If you wish to explicitly point out that you don't mind if your product is linked against OpenSSL (which should be obvious by the fact that you included support for it in your program), you are free to do so. Maybe it'll keep the lawyers a little hungrier. It's not necessary, and it *cannot* have legal effect. Exception clause or not, every author of a derived works that makes use of it, should understand their *obligation* to honour any and all licenses for any derived software. GPL, LGPL, OpenSSL, Apache, whatever. The exception clause is making this obvious. It has no legal weight. If you feel the advertising clause is fine, then it's the GPL that's at fault here for not allowing it. If you disagree with the advertising clause then it's the fault of the OpenSSL license. Personally, I don't really care which way you want to look at it. No. The GPL is allowed to do whatever it wants. What it wants, is to achieve Richard Stallman's vision of software communism. What the GPL cannot do is say whether you can, or cannot use OpenSSL. Only OpenSSL can say whether you can or cannot use OpenSSL. On another note, personally I feel it's a good thing to support multiple libraries when the cost of doing so is reasonably low. I had kind of half-expected people would agree with that sentiment and so the licenseing issue it would resolve (for at least Debian) is that much more reason. I didn't really expect a reaction of there isn't a licenseing issue so we shouldn't add support for another library. I could understand if people don't care about the licensing aspect but I don't really get this one-size-fits-all mentality when it
Re: [HACKERS] Strange pgsql crash on MacOSX
I wrote: The source code available from Apple shows that they got this code from NetBSD originally /*$NetBSD: history.c,v 1.25 2003/10/18 23:48:42 christos Exp $*/ so this may well be a pretty generic *BSD bug. Anyone clear on who to report it to? I have no idea if libedit is an independent project... Some digging in the NetBSD CVS shows that they found both parts of this bug more than two years ago: http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libedit/history.c.diff?r1=1.25r2=1.27f=h so the short and sweet answer is that Apple is behind the times. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Deadline-Based Vacuum Delay
On 12/28/06, Tom Lane [EMAIL PROTECTED] wrote: Galy Lee [EMAIL PROTECTED] writes: So I am thinking another way to perform vacuum. For example vacuum can be refined in a maintenance time frame like VACUUM IN 6 HOURS, and then vacuum operation will be performed within the window. The delay time is adjusted internally to spread the disk I/O over the time frame. And you will manage that how? The total amount of work to be done is quite unpredictable. specially for something you already can do with cron (*nix) or job scheduler (windows) -- regards, Jaime Casanova Programming today is a race between software engineers striving to build bigger and better idiot-proof programs and the universe trying to produce bigger and better idiots. So far, the universe is winning. Richard Cook ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] TODO: GNU TLS
* [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: I don't see the problem. If I redistribute PostgreSQL with GPL software that I author, I am supposed to keep a copy of the PostgreSQL license with the derived works. Respecting the license for every component of software is regular business. By the words you describe above, the GPL doesn't require that you include a copy of the PostgreSQL license either. Are you saying that this makes GPL incompatible with PostgreSQL? What in the world are you talking about...? I truely can't follow your train of thought in the least. It's silliness. If you redistribute OpenSSL, you honour the OpenSSL requirements. That's the *only* requirement by copyright law. It doesn't matter if it is GPL on top, or not. You always honour each license. The GPL says you can't combine GPL code with code which has additional requirements and then distribute the result. The *only* thing GPL-with-GPL does is reduce complexity. I don't seem to be able to explain it to you, even what the issue is, ignoring the question about if it's actually an issue or not. *That's* the issue here, not whatever it is you were arguing against. I think you might only be listening to one side. I'm honestly not sure *what* you're listening to, it doesn't seem to be me... There are a few ways to resolve this- add GNUTLS support to PostgreSQL (GNUTLS is LGPL and so won't cause a problem with GPL or other licenses in general) or get every GPL application author which ends up using OpenSSL to provide an exception (which Debian's been working on, actually, with some success), or get GPLv3 to allow advertising clauses and get everyone to switch to it (not exactly likely to happen...), or get OpenSSL to drop the advertising clause (I've been told they would if they could but that much of the code is authored by an individual who now works for a competitor and now has very little interest in helping out the OpenSSL project in any way...). 1) Adding GNUTLS support to PostgreSQL does not eliminate any PostgreSQL obligation to honour the OpenSSL distribution license. Any PostgreSQL distribution that includes OpenSSL must honour the OpenSSL distribution license, just as any PostgreSQL distribution that includes GNUTLS must honour either the GPL or LGPL license. Nothing changes. It's about distribution. If PostgreSQL includes OpenSSL support, it is a derived works when distributed with OpenSSL. It is a misunderstanding to believe that support for many interfaces allows you to avoid any licensing issues. It is a popular misunderstanding. As I tried to make clear previously, this hasn't got anything to do with PostgreSQL distributing anything. Nor does it have anything to do with PostgreSQL obligations. Here, I'll try again: Debian -- \ Applications - exim4 \ Links against: - libpq \ Links against: - libssl (OpenSSL) exim4 is under the GPL libpq is under BSD (no advertising clause) libssl is under BSD (advertising clause) So, Debian is distributing an application (exim4 w/ libpq libssl) which includes GPL code (exim4) combined with code under another license (BSD w/ advertising clause) which *adds additional restrictions* (the advertising clause) over those in the GPL, which is against the terms of the GPL. It's *Debian's* problem, but *PostgreSQL* can solve it by providing the option to link against GNUTLS instead. 2) Explicitly stating an OpenSSL exception is not a legal requirement. It is not possible for any derived product to except conditions for OpenSSL. OpenSSL defines its *own* license. You cannot modify it, which means that the GPL cannot reduce its significance, nor can an explicit exception claus increase its significance. OpenSSL distribution rights are defined by the OpenSSL license. Full stop. In the case above, exim4 *can* provide an exception because it's the *GPL* of *exim4* which is being violated by the advertising clause in the *OpenSSL* license. Which exim4 upstream has *done*, and which can be seen in their license (linked to previously in this thread). There are a number of other GPL'd applications which have done this, mostly (if not completely) at the prodding of Debian (bacula being another which comes to mind). If you wish to explicitly point out that you don't mind if your product is linked against OpenSSL (which should be obvious by the fact that you included support for it in your program), you are free to do so. Maybe it'll keep the lawyers a little hungrier. It's not necessary, and it *cannot* have legal effect. Programs may not directly link against OpenSSL and so may not directly include support for it (uhm, exactly the case when something links against libpq which then links against OpenSSL), but which ends up still being part of the distributed application. Exception clause or not, every author of a derived works that makes
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Andrew Dunstan wrote: here's a quick untested patch for buildfarm that Stefan might like to try. Note that not all core files are named core. On some Linux distros, it's configured to be core.PID by default. And you can even change it to weirder names, but I haven't seen those anywhere by default, so I guess supporting just the common ones is appropiate. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Recent SIGSEGV failures in buildfarm HEAD
Alvaro Herrera [EMAIL PROTECTED] writes: Andrew Dunstan wrote: here's a quick untested patch for buildfarm that Stefan might like to try. Note that not all core files are named core. On some Linux distros, it's configured to be core.PID by default. And on some platforms, cores don't drop in the current working directory ... but until we have a problem that *only* manifests on such a platform, I wouldn't worry about that. We do need to look for 'core*' not just 'core', though. Don't forget the ulimit point either ... on most Linuxen there won't be any core at all without twiddling ulimit. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] WITH support
Hello, What is the status of supporting the WITH keyword? -- Edwin S. Ramirez, Senior Developer Information Technology Department - ITDC Mount Sinai Medical Center [EMAIL PROTECTED] * 646-217-3112 ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] WITH support
Edwin Ramirez wrote: Hello, What is the status of supporting the WITH keyword? I see these TODO items: * Add SQL99 WITH clause to SELECT * Add SQL:2003 WITH RECURSIVE (hierarchical) queries to SELECT Are they the same item? Someone has said they will do RECURSIVE for 8.3, but there is no guarantee. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
On Thursday 28 December 2006 15:44, Bruce Momjian wrote:
Joshua D. Drake wrote:
On Thu, 2006-12-28 at 13:52 -0500, Tom Lane wrote:
Joshua D. Drake [EMAIL PROTECTED] writes:
I would say that a GUC variable for such behavior as listed in the
TODO is overzealous. We should either enforce it, or not. As we do
not now, there is no reason imo to change it.
Not only is it overzealous, but the proposal to have one reflects a
failure to learn from history. GUC variables that change
transaction-boundary semantics are a bad idea, period: see autocommit.
Nod. Let's get this TODO removed.
OK, removed.
I thought this was needed for spec compliance? If we have no plans to even
attempt to support it, istm that ought to be noted someplace.
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Robert Treat [EMAIL PROTECTED] writes: I thought this was needed for spec compliance? BEGIN isn't in the spec at all ... Now you could point to the spec for START TRANSACTION, which saith 1) If a start transaction statement statement is executed when an SQL-transaction is currently active, then an exception condition is raised: invalid transaction state - active SQL-transaction. However, seeing that the spec doesn't think that an exception necessarily aborts the transaction, I think the case for saying that ERROR is more spec-compliant here than WARNING is mighty thin. And if you want to hold our feet to the fire about whether errors abort transactions or not, this particular point is not one thousandth part of what would need to change. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] TODO: GNU TLS
Stephen Frost [EMAIL PROTECTED] writes: So, Debian is distributing an application (exim4 w/ libpq libssl) which includes GPL code (exim4) combined with code under another license (BSD w/ advertising clause) which *adds additional restrictions* (the advertising clause) over those in the GPL, which is against the terms of the GPL. Stephen, let me explain *exactly* why I think this is horsepucky. libjpeg, my other major open-source project, has always been shipped under a BSD-ish license that includes an advertising clause; I quote: : (2) If only executable code is distributed, then the accompanying : documentation must state that this software is based in part on the work of : the Independent JPEG Group. Curiously, every single GPL-license web browser in the world uses libjpeg. Until I see a widespread willingness to remove JPEG support in GPL-licensed software, and/or somebody providing a pure-GPL replacement for libjpeg, I am not going to take this argument seriously. There is exactly zero meaningful difference between the libjpeg license terms and the OpenSSL terms, but where is the pushback on libjpeg? I have not seen any, in all the years I worked on that project. (At one point RMS did make a half-hearted attempt to get me to relicense libjpeg as GPL, but I have never seen any indication whatsoever that anyone else cared.) regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PATCHES] [BUGS] BUG #2846: inconsistent and confusing
Bruce Momjian [EMAIL PROTECTED] writes: I wasn't excited about doing one isinf() call to avoid three, so I just made a fast isinf() macro: /*We call isinf() a lot, so we use a fast version in this file */ #define fast_isinf(val) (((val) DBL_MIN || (val) DBL_MAX) isinf(val)) This is *not* going in the right direction :-( regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] [BUGS] BUG #2846: inconsistent and
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I wasn't excited about doing one isinf() call to avoid three, so I just made a fast isinf() macro: /*We call isinf() a lot, so we use a fast version in this file */ #define fast_isinf(val) (((val) DBL_MIN || (val) DBL_MAX) isinf(val)) This is *not* going in the right direction :-( Well, then show me what direction you think is better. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] TODO: GNU TLS
On Thu, Dec 28, 2006 at 09:34:05PM -0500, Stephen Frost wrote: * [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote: By the words you describe above, the GPL doesn't require that you include a copy of the PostgreSQL license either. Are you saying that this makes GPL incompatible with PostgreSQL? What in the world are you talking about...? I truely can't follow your train of thought in the least. I will try again. It is a difficult subject for many. GPL software derived from PostgreSQL must honour the restrictions defined by the PostgreSQL (BSD) license. GPL software derived from OpenSSL must honour the restrictions defined by the OpenSSL license. What is the difference? Do you see it? You speak of compatibility as if it means that the above are different in some technical way. They are NOT different. Just because the GPL = the PostgreSQL license, does not allow you to disobey the PostgreSQL license restrictions. You *cannot* release your entire derived GPL product as GPL, if it is distributed with PostgreSQL. The PostgreSQL component retains the PostgreSQL licensing restrictions, The GPL restrictions do not supercede or replace the PostgreSQL component and there is NOTHING the GPL can do to change this. It's silliness. If you redistribute OpenSSL, you honour the OpenSSL requirements. That's the *only* requirement by copyright law. It doesn't matter if it is GPL on top, or not. You always honour each license. The GPL says you can't combine GPL code with code which has additional requirements and then distribute the result. The GPL *cannot* say this. The GPL *cannot* define how OpenSSL can or cannot be used. Only the OpenSSL license can define how OpenSSL is allowed to be distributed or used once distributed. You are looking at it from the wrong direction. The GPL can prevent *GPL-derived* works from being non-GPL. The GPL CANNOT prevent one from deriving from non-GPL works. Take some time to read this a few times if you need to. The difference is HUGE. It would be ridiculous for it to be different. If it was different, it would require all software to have exactly equivalent licenses in order to be integrated. Insanity. Any lawyer who would claim this should lose their license to practice law. I'll state again - there is no such thing as *compatibility* in the sense you are using it. If they were fully compatible, they would have the exact same effect. The PostgreSQL license is *not* compatible with GPL in the sense that they can be interchanged. Read the following snip from the PostgreSQL license, which is incompatible with the GPL license: Permission to use, copy, modiy, and distribute ... provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies. A distribution that includes GPL software and PostgreSQL software *must* include both the GPL license, and the PostgreSQL license. This is an additional incompatible restriction with the GPL. The advertising clause of the OpenSSL license is a second such restriction. It is not the first such restriction as some people are trying to make it appear. 1) Adding GNUTLS support to PostgreSQL does not eliminate any PostgreSQL obligation to honour the OpenSSL distribution license. Any PostgreSQL distribution that includes OpenSSL must honour the OpenSSL distribution license, just as any PostgreSQL distribution that includes GNUTLS must honour either the GPL or LGPL license. Nothing changes. It's about distribution. If PostgreSQL includes OpenSSL support, it is a derived works when distributed with OpenSSL. It is a misunderstanding to believe that support for many interfaces allows you to avoid any licensing issues. It is a popular misunderstanding. As I tried to make clear previously, this hasn't got anything to do with PostgreSQL distributing anything. Nor does it have anything to do with PostgreSQL obligations. Here, I'll try again: Debian -- \ Applications - exim4 \ Links against: - libpq \ Links against: - libssl (OpenSSL) exim4 is under the GPL libpq is under BSD (no advertising clause) libssl is under BSD (advertising clause) So, Debian is distributing an application (exim4 w/ libpq libssl) which includes GPL code (exim4) combined with code under another license (BSD w/ advertising clause) which *adds additional restrictions* (the advertising clause) over those in the GPL, which is against the terms of the GPL. It's *Debian's* problem, but *PostgreSQL* can solve it by providing the option to link against GNUTLS instead. There is no problem above. Debian must honour all licenses for all software components. There is no need for an exemption, or an OpenSSL exception clause. If OpenSSL is included within Debian, than the Debian distribution should include an attribution to it in the Debian documentation, and the OpenSSL LICENSE file should be stored somewhere in the file system. If
Re: [HACKERS] TODO: GNU TLS
[EMAIL PROTECTED] wrote: I will try again. It is a difficult subject for many. GPL software derived from PostgreSQL must honour the restrictions defined by the PostgreSQL (BSD) license. GPL software derived from OpenSSL must honour the restrictions defined by the OpenSSL license. What is the difference? Do you see it? You speak of compatibility as if it means that the above are different in some technical way. They are NOT different. Just because the GPL = the PostgreSQL license, does not allow you to disobey the PostgreSQL license restrictions. You *cannot* release your entire derived GPL product as GPL, if it is distributed with PostgreSQL. The PostgreSQL component retains the PostgreSQL licensing restrictions, The GPL restrictions do not supercede or replace the PostgreSQL component and there is NOTHING the GPL can do to change this. I think the issue revolves around the conditions that GPL stipulates about linking against libraries requiring the entire product to be *distributed* as GPL, even if components have differing licenses. This is the so-called viral clause that gets much attention! Now as Tom pointed out, I dunno why OpenSSL suddenly gets so much attention, but anyway, just trying to clarify why *in principle* that Stephen F is talking about a valid *possible* interpretation of the licensing maze... Cheers Mark ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
