Re: [HACKERS] win32 socket definition

2010-01-10 Thread James Mansion
Tom Lane wrote: There's another copy of ListenSocket[] in the BackendParameters struct. I also wonder about postmaster.c's habit of using -1 for empty slots in ListenSocket ... how safe is that for Win64? On Windows, it should be INVALID_SOCKET. James -- Sent via pgsql-hackers mailing

Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2009-12-02 Thread James Mansion
Marko Kreen wrote: Note - my proposal would be to get rid of HAVE_INLINE, which means we are already using inline functions unconditionally on platforms that matter (gcc). Keeping duplicate code for obsolete compilers is pointless. Microsoft C doesn't matter? I seem to remember that when

Re: [HACKERS] Listen / Notify rewrite

2009-11-13 Thread James Mansion
Josh Berkus wrote: Payloads are also quite useful even in a lossy environment, where you understand that LISTEN is not a queue. For example, I'd like to be using LISTEN/NOTIFY for cache invalidation for some applications; if it misses a few, or double-counts them, it's not an issue. However,

Re: [HACKERS] Could postgres be much cleaner if a future release skipped backward compatibility?

2009-10-25 Thread James Mansion
Tom Lane wrote: Actually, I think any attempt to do that would result in a fork, and a consequent splintering of the community. We can get away Perhaps the answer might be a pre-emptive simplifying fork to postgres-NG, perhaps taking a lead from MySQL and Samba. I'm not sure that you would

Re: [HACKERS] slow commits with heavy temp table usage in 8.4.0

2009-08-09 Thread James Mansion
Tom Lane wrote: the function time and the commit time a lot better. But I'm not sure if the use-case is popular enough to deserve such a hack. For some OLTP workloads, it makes a lot of sense to spool tuples of primary key plus new fields into a temp table and then doing a single update or

Re: [HACKERS] idea: global temp tables

2009-04-30 Thread James Mansion
Kevin Grittner wrote: contexts. I don't think the updates to the system tables have the same magnitude of performance hit as creating these tables, especially if write barriers are on. Wouldn't it be cleaner just to defer creation of real files to support the structures associated with a

Re: [HACKERS] libpq WSACleanup is not needed

2009-01-20 Thread James Mansion
Andrew Chernow wrote: The only problem is how to detect the first connection. In a threaded environment you'd have to perform locking in connectdb, which is probably not going to fly. Well, if you do an atomic test for a flag being zero, and if so then enter a critsec, do the init iff you're

Re: [HACKERS] libpq WSACleanup is not needed

2009-01-18 Thread James Mansion
Magnus Hagander wrote: The use-case of rapidly creating and dropping connections isn't particularly common, I think. And there is a perfectly functioning workaround - something that we should perhaps document in the FAQ or somewhere in the documentation? Would it be accetable to do initialise

Re: [HACKERS] libpq WSACleanup is not needed

2009-01-18 Thread James Mansion
Andrew Chernow wrote: m$ docs indicate that wsastartup can't be called from dllmain :( OK, fair cop. Says it in the MSDN online version but not in the SDK 6.1 version. :-( Some helper(s) must start threads I guess. Re the counting and doing it on first/last socket - of course WSAStartup

Re: [HACKERS] Significantly larger toast tables on 8.4?

2009-01-04 Thread James Mansion
Peter Eisentraut wrote: c. Are there any well-known pitfalls/objections which would prevent me from changing the algorithm to something more efficient (read: IO-bound)? copyright licenses and patents Would it be possible to have a plugin facility? I guess the most likely candidate

Re: [HACKERS] About CMake

2008-12-31 Thread James Mansion
Andrew Dunstan wrote: Quite so. CMake outputs MSVC Project files, as I understand it. If you know of another cross-platform build tool that will do that then speak up. I think the wxWidgets team have one, and I think scons has some support for doing that, though I haven't tried that part of

[HACKERS] Elide null updates

2008-12-16 Thread James Mansion
I saw on a summary for 8.4 that there's a generic function for use as a before row trigger that will elide null updates (ie replacement with an identical row). I can see that this is great - but I was wondering if it should be more integrated and turned on in the db schema. Trivially, doing

Re: [HACKERS] Elide null updates

2008-12-16 Thread James Mansion
Andrew Dunstan wrote: I don't follow what you're saying. If an update is skipped by a trigger, nothing new is written to disk, and there should be nothing to vacuum from it. That's why this trigger can speed up certain update queries enormously. OK I'll try again. Suppose we do an update.

Re: [HACKERS] Mostly Harmless: Welcoming our C++ friends

2008-12-14 Thread James Mansion
Kurt Harriman wrote: B) let the build farm do a nightly build with a C++ compiler merely as a test to verify that no C++ compilation errors are introduced, but continue to use C 'officially' for builds and releases; or This was the intent of my suggestion. There can be

Re: [HACKERS] Mostly Harmless: Welcoming our C++ friends

2008-12-06 Thread James Mansion
Kurt Harriman wrote: The foremost opposing argument seems to have been that there should be no attempt to alleviate the existing reserved word problem without automatic enforcement to guarantee that never in the future can new occurrences be introduced. Is there anything in the source that

Re: [HACKERS] Core team statement on replication in PostgreSQL

2008-06-01 Thread James Mansion
David Fetter wrote: This part is a deal-killer. It's a giant up-hill slog to sell warm standby to those in charge of making resources available because the warm standby machine consumes SA time, bandwidth, power, rack space, etc., but provides no tangible benefit, and this feature would have

Re: [HACKERS] replication hooks

2008-06-01 Thread James Mansion
Marko Kreen wrote: There is this tiny matter of replicating schema changes asynchronously, but I suspect nobody actually cares. Few random points about that: I'm not sure I follow you - the Sybase 'warm standby' replication of everything is really useful for business continuity. The

Re: [HACKERS] Core team statement on replication in PostgreSQL

2008-06-01 Thread James Mansion
Aidan Van Dyk wrote: The whole single-threaded WAL replay problem is going to rear it's ugly head here too, and mean that a slave *won't* be able to keep up with a busy master if it's actually trying to apply all the changes in real-time. Is there a reason to commit at the same points that the

Re: [HACKERS] triggers on prepare, commit, rollback... ?

2008-05-19 Thread James Mansion
Tom Lane wrote: * Trigger on rollback: what's that supposed to do? The current transaction is already aborted, so the trigger has no hope of making any database changes that will ever be visible to anyone. It can however affect state in the backend doing the rollback, which can be useful.

Re: [HACKERS] pgkill on win32

2008-04-22 Thread James Mansion
Magnus Hagander wrote: The problem is when winsock operations are interrupted by APCs. See: http://archives.postgresql.org/pgsql-hackers-win32/2004-04/msg00013.php Whoa! Indeed, that's a bit sucky because they really aren't documented as interruptible. In this case though I see not

Re: [HACKERS] pgkill on win32

2008-04-22 Thread James Mansion
Magnus Hagander wrote: You interested in trying to code up a patch to verify that? ;) Practical reality says that I won't get to this before the next version of Windows is released. I don't want to promise something I can't deliver. If there were any desire to provide a MT-aware

[HACKERS] pgkill on win32

2008-04-20 Thread James Mansion
I'm wondering if the mechanism used for sending signals between postmaster processes on Win32 is much more heavyweight that is necessary. Is there a reason not to call OpenThread on the target postmaster's thread id, and then use QueueUserAPC to execute a 'signal handler' method on it? (Or

Re: [HACKERS] pgkill on win32

2008-04-20 Thread James Mansion
Magnus Hagander wrote: Yes. We used to use APCs, but touching anything remotely related to Winsock from an APC is not supported... We had a lot of trouble with it By implication you'd be doing socket'y stuff from the signal handler on UNIX? Scary. I was assuming it would be used to signal an

Re: [HACKERS] Cached Query Plans

2008-04-13 Thread James Mansion
Would it be possible to store plans with an indication of the search path that was used to find tables, and for temp tables some snapshot of the statistics for the table if any? My suspicions are that: * where you have a lot of short-lived connections then actually they will often use the

[HACKERS] notify with payload (pgkill, notify)

2008-04-02 Thread James Mansion
Is the intent to replace most uses of (pg)kill with a general purpose messaging system between the processes, or (just) to address notify per se? (Presumably with 'fire-and-forget' and also rpc semantics? And pub-sub? And some sort of write to an fd protected by an atomic flag to elide multiple

Re: [HACKERS] notify with payload (pgkill, notify)

2008-04-02 Thread James Mansion
Andrew Dunstan wrote: If you have an alternative suggestion them you need to make it now. well, I'm not sure about the non-collapsing business but no matter. We are not intending to use FDs for message passing. They will be stored in shared memory. See previous discussions for details: I'm

[HACKERS] pgkill

2008-03-31 Thread James Mansion
I was looking at the notify processing in async.c and I noticed that kill is called whether or not the process has been signalled already, and whether or not 'this' process has signalled the process. It seems unecessary to me - especially if we are on Win32 and the pgkill is implemented as a

Re: [HACKERS] first time hacker ;) messing with prepared statements

2008-03-31 Thread James Mansion
Tom Lane wrote: PFC [EMAIL PROTECTED] writes: Do the parse tree store fully qualified schema.table or schema.function ? They store OIDs. So, what happens if we reference a temporary table or something else that requires resolution down a search path? I believe Microsoft and

Re: [HACKERS] POSIX shared memory support

2008-03-31 Thread James Mansion
Tom Lane wrote: Yeah, I would be far more interested in this patch if it avoided needing SysV shmem at all. The problem is to find an adequate substitute for the nattch-based interlock against live children of a dead postmaster. (confused) Why can't you use mmap of /dev/zero and inherit the

Re: [HACKERS] first time hacker ;) messing with prepared statements

2008-03-30 Thread James Mansion
PFC wrote: Hello, So, I embarked (yesterday) on a weekend project to add a new feature to Postgres... I use PHP with persistent connections and always have been bothered that those very small AJAX queries (usually simple selects returning 1 row) take more CPU in postgres to parse

Re: [HACKERS] Noob Hints on testing and debugging?

2008-03-13 Thread James Mansion
Tom Lane wrote: (eg how to run it up and feed it SQL ideally without running a postmaster and execing a back end) Why would you consider that ideal? Such a scenario would have approximately zip to do with the real-world environment your patch would face. Your points what? If I'm

[HACKERS] Proposal for db level triggers

2008-03-13 Thread James Mansion
Background: Firebird 2.1 allows: CREATE TRIGGER name ON ( CONNECT | DISCONNECT | TRANSACTION START | TRANSACTION COMMIT | TRANSACTION ROLLBACK ) I want to allow syntax: CREATE TRIGGER name event event (3 variations, application can specify any number of them): AFTER CONNECT AFTER

Re: [HACKERS] Proposal for db level triggers

2008-03-13 Thread James Mansion
James Mansion wrote: In usage: AFTER START clears counters and flags. UPDATE triggers on data set counters and flags. BEFORE COMMIT examines the counters and flags and performs any final validation or adjustments (or external events such as sending a MoM message) I'd like to point out also

[HACKERS] Build problem with VS2008 Express Edition

2008-03-12 Thread James Mansion
Hi, I managed to get most of 8.3 built with VS2008EE last night. Ii had to change some references to msbuild to vsbuild, which I guess is expected but one compile issue surprised me. I had to change #ifdef IPV6_V6ONLY at backend/libpq/pqcomm.c:386 to: #if defined(IPV6_V6ONLY)

[HACKERS] Noob Hints on testing and debugging?

2008-03-12 Thread James Mansion
Are there any hints and tips anywhere on practical issues for testing and debugging changes to a backend? (eg how to run it up and feed it SQL ideally without running a postmaster and execing a back end) I'm using VS2008 on XP by choice, and I'd like to investigate supporting something

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread James Mansion
Kenneth Marshall wrote: conversion process themselves. Accepting random input puts a performance hit on everybody following the standard. Why is that necessarily the case? Why not have a liberal parser and a configurable switch that determines whether non-standard forms are liberally

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread James Mansion
Mark Mielke wrote: I recall there being a measurable performance difference between the most liberal parser, and the most optimized parser, back when I wrote one for PostgreSQL. I don't know how good the one in use for PostgreSQL 8.3 is. As to whether the cost is noticeable to people or not -

Re: [HACKERS] wishlist for 8.4

2008-02-18 Thread James Mansion
Richard Huxton wrote: It would be nice to have a more dynamic language built-in. I'm not aware of any BSD-licensed dynamic languages though. Lua is MIT. And I believe there's even a pl/Lua around. The small size of the interpreter and ease of C interfacing would make it ideal, or at least

Re: [HACKERS] PostgreSQL 8.4 development plan

2008-02-06 Thread James Mansion
Tom Lane wrote: There is, although I think a large fraction of it will get bounced as needs more work, which should reduce the pressure. We'll just be trying to give feedback to let the patch authors move forward, which will not take as much time as actually committing would take. The current

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread James Mansion
Mark Mielke wrote: Joshua D. Drake wrote: Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try yourself? Because users possibly want to do that - use it? Some of us have better things to do

Re: [HACKERS] Sorting Improvements for 8.4

2007-12-27 Thread James Mansion
Ron Mayer wrote: Or do you mean being able to perform parts of the query plan fully in parallel? If this, then one would need a lot more than ParallelSort... I wouldn't recommend that - it seems like a Hard Problem. Isn't it the case that the implicit unions from processing

Re: [HACKERS] .NET or Mono functions in PG

2007-12-03 Thread James Mansion
Tom Lane wrote: It is also pretty well established that if pltcl or plperl cause the backend to become multithreaded, things break horribly. I strongly Isn't that something that needs to be fixed? Its one thing not to allow for multiple threads to be in your code, but not having a threaded

Re: [HACKERS] .NET or Mono functions in PG

2007-12-01 Thread James Mansion
Magnus Hagander wrote: I did look at this at some earlier point as well. One big problem at that time was that once you embedded mono, yuo had all sorts of threads running in your backend ;-) Is that necessarily a problem? You have to compile with a thread-capable libc and take some care

Re: [HACKERS] Win32 shared memory speed

2007-11-11 Thread James Mansion
Magnus Hagander wrote: IIRC, there hasn't been any direct benchmark for it (though I've wanted to do that but had no time), but it's been the olnly real explanation put forward for the behaviour we've seen. And it does make sense given the thread-centric view of the windows mm. /Magnus

Re: [HACKERS] Proposal: real procedures again (8.4)

2007-10-29 Thread James Mansion
Josh Berkus wrote: Not only would they be generally useful for SP programming, but multisets would eliminate one of the big hurdles in re-writing T-SQL stored procedures in PG, and thus make it easier to port from SQL Server. You don't hear a lot of demand for multisets on the mailing lists

Re: [HACKERS] INSERT/SELECT and excessive foreign key checks

2007-08-19 Thread James Mansion
Andrew Dunstan wrote: Could we achieve the same thing in a more general way by having a per-FK tiny (say 10?) LRU cache of values checked. Then it wouldn't only be restricted to constant expressions. Of course, then the trigger would need to keep state, so it might well be too complex (e.g.

Re: [HACKERS] Another idea for index-only scans

2007-08-18 Thread James Mansion
Decibel! wrote: The advantage to Bruce's idea is that it sounds pretty simple to implement. While it wouldn't be of use for many general cases, it *would* be useful for read-only tables, ie: old partitions. Wouldn't the mostcommon case by foreign key checks against tables that essentially

[HACKERS] Connection and transaction triggers

2007-08-05 Thread James Mansion
I've been looking at the new trigger features planned for Firebird 2.1 and I'm a bit jealous. I'm interested in 'stateful' proc packages that can benefit from accumulating data from updates during a transaction and then performing an action at the end - perhaps doing a bulk insert in a