Re: [HACKERS] Debian readline/libedit breakage

2011-02-14 Thread Markus Wanner
nd this issue by pre-loading readline via LD_PRELOAD for psql. Personally, I'm a bit suspicious about that solution (technically as well as from a licensing perspective), but it's probably the simplest way to let only psql link against readline. Regards Markus Wanner [1]: Changelog

Re: [HACKERS] Debian readline/libedit breakage

2011-02-14 Thread Markus Wanner
Martin, On 02/14/2011 02:08 PM, Martin Pitt wrote: > thanks Markus for CC'ing me, I'm not on -hackers@. Sure. > Note that the recent discussions happened on bug 608442, in particular > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=608442#30 Thanks for this poi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
to think of the replication between the master and the one standby that's sending an ACK first as "synchronous". (Even if those may well be different standbies for different transactions). Hope to have brought some light into this discussion. Regards Markus Wanner -

Re: [HACKERS] I am confused after reading codes of PostgreSQL three week

2011-03-18 Thread Markus Wanner
rested in and follow the backend doing its work. That's particularly helpful in trying to find a certain spot of interest. Of course, it doesn't help much in getting the big picture. Good luck on your journey through the code base. Regards Markus Wanner -- Sent via pgsql-hackers mail

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
isk". But then continues to say that "[the master is] waiting for acknowledgment from a slave after having performed a commit", so this indeed sounds like the transaction is visible to other sessions before the slave ACKs. So, semi-sync may show temporary inconsistencies in cas

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
t the client. It's just the commit *before* receiving the slave's ACK, which might make a transaction visible that's not durable, yet. But I guess that simplified implementation for them... Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
place, you don't have anything special to do upon recovery. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
s fsynced. If you have synchronous_commit mode enabled then > you'll block until your xlog is fsynced and that will implicitly mean > the other transactions you saw were also fsynced. Somewhat, yes. And for exactly that reason, most users run with synchronous_commit enabled. They do

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
Simon, On 03/18/2011 05:19 PM, Simon Riggs wrote: >>> Simon Riggs wrote: In PostgreSQL other users cannot observe the commit until an acknowledgement has been received. On other nodes as well? To me that means the standby needs to hold back COMMIT of an ACKed transaction, until receiv

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
On 03/18/2011 05:27 PM, Kevin Grittner wrote: > Basically, what Heikki addresses. It has to be committed after > crash and recovery, and deal with replicas which may or may not have > been notified and may or may not have applied the transaction. Huh? I'm not quite following here. Committing ad

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-18 Thread Markus Wanner
pletely biased. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-19 Thread Markus Wanner
owever, that even if there's no such guarantee, it's highly unlikely for a user (or application) to ever notice this during normal operation. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.p

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Markus Wanner
On 03/22/2011 09:33 PM, Robert Haas wrote: > We might have a version of synchronous replication that works this way > some day, but it's not the version were shipping with 9.1. The slave > acknowledges the WAL records when they hit the disk (i.e. fsync) not > when they are applied; WAL apply can l

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Markus Wanner
On 03/23/2011 12:52 PM, Robert Haas wrote: > Yes. What this won't do is let you build a big load-balancing network > (at least not without great caution about what you assume). This sounds too strong to me. Session-aware load balancing is pretty common these days. It's the default mode of PgBou

[HACKERS] dynamically allocating chunks from shared memory

2010-07-02 Thread Markus Wanner
the dynshmem patch first, then imessages). Comments? Greetings from Oxford, and thanks to Joachim Wieland for providing me the required Internet connectivity ;-) Markus Wanner [1]: Postgres-R: internal messages http://archives.postgresql.org/message-id/4886db0b.1090...@bluegap.ch [2]: Mentioned Cluster

[HACKERS] bg worker: overview

2010-07-13 Thread Markus Wanner
eused. Please don't hesitate to ask questions, if you need help. (I don't dare to add these patches to the commit fest, as this refactoring doesn't have any immediate benefit for Postgres itself, at the moment.) Regards Markus Wanner P.S.: git adicts, everything's up her

[HACKERS] bg worker: patch 1 of 6 - permanent process

2010-07-13 Thread Markus Wanner
This patch turns the existing autovacuum launcher into an always running process, partly called the coordinator. If autovacuum is disabled, the coordinator process still gets started and keeps around, but it doesn't dispatch vacuum jobs. The coordinator process now uses imessages to communicate

[HACKERS] bg worker: patch 3 of 6 - sockets

2010-07-13 Thread Markus Wanner
This patch adds the capability for the coordinator to listen on sockets while waiting for imessages to arrive. Before the coordinator just slept until a signal arrives, notifying the coordinator about an internal message. Major caveat here: I'm using pselect(), which might still not be portabl

[HACKERS] bg worker: patch 6 of 6 - ooo messages

2010-07-13 Thread Markus Wanner
Finally, this patch adds the capability to cache out-of-order messages for workers within the coordinator process. Unlike the cache of jobs, which basically are messages as well, these messages don't trigger a job, but might provide additional information or data payload to a worker that's alre

Re: [HACKERS] bg worker: overview

2010-07-14 Thread Markus Wanner
Hi, On 07/13/2010 08:45 PM, Kevin Grittner wrote: > You could submit them as Work In Progress patches Okay, I added them. I guess they get more attention that way. Regards Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: ht

Re: [HACKERS] cvs to git migration - keywords

2010-07-15 Thread Markus Wanner
s completely, just as sort of a cleanup commit. And no, that shouldn't pose any problem with outstanding patches, except you are fiddling with the tag itself. In which case you deserve to get a conflict. ;-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] bg worker: overview

2010-07-15 Thread Markus Wanner
Hi, On 07/15/2010 03:45 PM, Dimitri Fontaine wrote: > We've been talking about this topic on -performance: Thank for pointing out this discussion, I'm not following -performance too closely. > So, do you think we could use your work as a base for allowing custom > daemon code? Daemon code? That

Re: [HACKERS] bg worker: overview

2010-07-15 Thread Markus Wanner
to something like a 1 second precision or something, I don't remember exactly). (There's nothing that needs to be merged with autovacuum. It already is merged, if you want). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] imessages up-date

2010-07-15 Thread Markus Wanner
Hi, On 07/15/2010 10:37 PM, Alvaro Herrera wrote: > BTW I think this patch series makes sense, though I haven't looked at it > in detail. I guess it means I'll have to have a look at the IMessages > stuff as well. Yes, only after adding these patches to the commit fest, I realized that I'dd have

Re: [HACKERS] SHOW TABLES

2010-07-16 Thread Markus Wanner
> I have to agree with Simon here. \d is ridiculous for the common user. +1 Regards Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] imessages up-date

2010-07-16 Thread Markus Wanner
Hi, On 07/16/2010 04:01 PM, Kevin Grittner wrote: > Since these two patches were posted before the commit fest started, > and are prerequisites for six properly submitted patches, I'm going > with the "spirit of the law" and saying it's OK to add them. Does > the application allow that? Yes, it

Re: [HACKERS] bg worker: overview

2010-07-17 Thread Markus Wanner
Hello Simon, On 07/17/2010 12:30 PM, Simon Riggs wrote: The code itself merely reflects your design, so what I would really like to see is a full explanation of this. Are the descriptive mails I sent for each patch going into the right direction and just need to be extended, in your opinion?

Re: [HACKERS] bg worker: overview

2010-07-17 Thread Markus Wanner
Sorry, hit send too early. On 07/17/2010 01:47 PM, Markus Wanner wrote: I think that I commented the source code pretty extensively, however, that's a subjective feeling. Take this phrase. I'm under the impression, that I commented the source code pretty well. Scratch th

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-20 Thread Markus Wanner
uff: I don't mind much *which* allocator to use. I also looked at jemalloc, but haven't been able to integrate it into Postgres. So I've extended my experiment with wamalloc and turned it into something usable for Postgres. Regards Markus Wanner -- Sent via pgsql

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-20 Thread Markus Wanner
*** src/backend/storage/ipc/imsg.c dc149eef487eafb43409a78b8a33c70e7d3c2bfa (and, well, the dynshmem stuff ends just before that line. Those were two .diff files attached, IIRC). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-20 Thread Markus Wanner
ctive of a developer). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-21 Thread Markus Wanner
e of shared memory. (Given SLRU's ability to spill to disk, it could even be used to 'balance' out anomalies to some extent). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-21 Thread Markus Wanner
Hi, first of all, thanks for your feedback, I enjoy the discussion. On 07/21/2010 07:25 PM, Robert Haas wrote: Given what you're trying to do, it does sound like you're going to need some kind of an algorithm for space management; but you'll be managing space within the SLRU rather than within

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-22 Thread Markus Wanner
at's the case) and last but not least, I got told they still aren't fully portable. Another nice thing about imessages compared to unix pipes is, that it's a zero-copy approach. Hope that makes my opinions and decisions clearer. Thank you for sharing your concerns and for explai

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-22 Thread Markus Wanner
passing systems. Not sure how much that applies to imessages and/or how it's used in bgworkers or Postgres-R, though. ] That much about why using a unicast message-passing system. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-22 Thread Markus Wanner
;d be your favorite allocator? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-22 Thread Markus Wanner
Hi, On 07/22/2010 08:31 PM, Alvaro Herrera wrote: FWIW I don't think you should be thinking in "replacing imessages with SLRU". I rather think you should be thinking in how can you implement the imessages API on top of SLRU. Well, I'm rather comparing SLRU with the dynamic allocator. So far I

Re: [HACKERS] [RRR] CommitFest 2010-07 week one progress report

2010-07-23 Thread Markus Wanner
intained, but aren't currently needed or wanted. But do we want to use the CF application for that? How do you prefer to proceed with these patches? It's also worth noting that Simon requested more and better documentation. But I simply can't promise to write anything soon. Rega

Re: [HACKERS] bg worker: overview

2010-07-24 Thread Markus Wanner
connection pool is better outside of code. Let's think PGQ and a internal task scheduler first, if we think at any generalisation. To be honest, I still don't quite grok the concept behind PGQ. So I cannot really comment on this. Regards Markus Wanner -- Sent via pgsql-hackers mailing

Re: [HACKERS] bg worker: overview

2010-07-25 Thread Markus Wanner
Hey Dimitri, On 07/24/2010 07:26 PM, Dimitri Fontaine wrote: Trying to figure out how it would fit the PGQ and pgagent needs. But maybe user defined daemons should be sub-coordinators (I used to think about them as "supervisors") able to talk to the coordinator to get a backend connected to some

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-26 Thread Markus Wanner
ll, it cannot steal memory from the SLRU pools. Or am I mistaken? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-26 Thread Markus Wanner
egin to take OOM situations and their counter-measures into account. In a way, fixing every separate pool to its specific size just is the very simples rule-set I can think of. The dynamic allocator buys you more flexibility, but choosing good limits and rules between the sub-systems is anothe

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-07-26 Thread Markus Wanner
that's just my thinking. Thank you for your inputs. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] english parser in text search: support for multiple words in the same position

2010-08-02 Thread Markus Wanner
es a token "wikipedia.org/search?q=sushant" buy you in terms of text searching? Or even result highlighting? I wouldn't expect anybody to want to search for a full URL, do you? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] english parser in text search: support for multiple words in the same position

2010-08-02 Thread Markus Wanner
imply making the same string turn up twice in the tokenizer's output doesn't sound like the right solution to me. Especially considering that the query parser uses the very same tokenizer. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] parallel quicksort

2010-08-09 Thread Markus Wanner
Hoping this could be useful, I wanted to put out what I had so far and see how far away this is from something workable. Not to mention that there are probably some improvements that could be make to the parallel quicksort algorithm. Thanks for sharing. Regards Markus Wanner -- Sent via pgs

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
locate unused memory of one subsystem to another one *on the fly*. Not just for performance, but also for ease of use for the admin and the developer, IMO. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
re was only one process, period, and all backends were threads inside that. No shared memory as such, at all. That's how the threaded model normally is used, yes. And with that model, allocation of shared memory is very easy. It has none of the pre-allocation requirements we are currently facing

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
ea. That memory there is plain wasted at that moment. That's my point and the problem the allocator I posted tries to solve. I fail to see how malloc could help here. malloc() only allocates process-local memory. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
g the OOM error when trying to allocate for its very first entry). Thanks for bringing this discussion to live again. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
On 08/09/2010 08:33 PM, Bruce Momjian wrote: Robert Haas wrote: You probably wouldn't do either of those things. You'd just allocate small chunks here and there for whatever you need them for. Well, then we do that with shared memory then --- my point is that it is the same problem with threa

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
On 08/09/2010 08:49 PM, Bruce Momjian wrote: Markus Wanner wrote: That's what my patch allows you to do, yes. Currently you are bound to pre-allocate shared memory at startup. Or how would you allocate small chunks from shared memory at the moment? We don't --- we allocate it all

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
on of a dynamic allocator. I'm approaching the problem from another perspective: trying to implement a dynamic allocator on top of a fixed size memory pool, first. Once we have that, we may start to think about dynamically adding or removing underlying segments. Regards Markus Wanner -

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
emory allocator implemented in the C library". Think of my patch as the high(er)-level variant ;-) It's certainly doable using processes and shared memory. Yes. My patch shows one way of how to go a step into that direction. Regards Markus Wanner [1]: http://www.cs.utah.edu/flux/moss

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
, it's not. Agreed, it's going to be a difficult compromise and it possibly is very hard to find a good one automatically. However, I doubt our current approach with hard limits between subsystems is the best compromise. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] dynamically allocating chunks from shared memory

2010-08-09 Thread Markus Wanner
ument for it. Fair enough. (Patch is already marked as "returned with feedback" on the commitfest app, thanks again for additional feedback) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Deadlock bug

2010-08-25 Thread Markus Wanner
Simon, On 08/25/2010 11:53 AM, Simon Riggs wrote: ..we want to ensure that the PK value.. ..or any other possibly referenced attributes? Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Version Numbering

2010-08-25 Thread Markus Wanner
Hi, On 08/21/2010 10:11 PM, Peter Geoghegan wrote: We changed 8.5 to 9.0 explicitly because doing so was good marketing, That's exactly how I see this as well. The current scheme allows for some flexibility for marketing purposes while still being self-consistent and logical in numbering.

Re: [HACKERS] Deadlock bug

2010-08-25 Thread Markus Wanner
On 08/25/2010 04:57 PM, Tom Lane wrote: It strikes me that a possibly useful simplification of the idea is a lock type that allows HOT updates and not non-HOT ones; or more precisely not ones that change any indexed columns --- if the row ends up having to go off-page for lack of space, that need

Re: [HACKERS] Deadlock bug

2010-08-26 Thread Markus Wanner
Hi, On 08/25/2010 10:35 PM, Simon Riggs wrote: If the row is "key share" locked (as opposed to "tuple share" locks we already have), then an UPDATE would only work if it was a non-HOT UPDATE. I think you meant it the other way around: an UPDATE on a "key share" locked tuple only works if it *

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-26 Thread Markus Wanner
Hi, thanks for your feedback on this, it sort of got lost below the discussion about the dynamic shared memory stuff, IMO. On 08/26/2010 04:39 AM, Robert Haas wrote: It's not clear to me whether it's better to have a single coordinator process that handles both autovacuum and other things, or

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-26 Thread Markus Wanner
e plugin? If possible, you can use IMessage as a private protocol freely in the plugin. Am I missing something? Well, what problem(s) are you trying to solve with such a thing? I've no idea what direction you are aiming at, sorry. However, it's certainly different or extending bg work

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-26 Thread Markus Wanner
ssing"? It covers all of the API stability and the security issues I've raised. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-26 Thread Markus Wanner
Itagaki-san, On 08/26/2010 01:02 PM, Itagaki Takahiro wrote: OK, I see why you proposed coordinator hook (yeah, I call it hook :) rather than adding user-defined processes. I see. If you call that a hook, I'm definitely not a hook-hater ;-) at least not according to your definition. Howev

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-26 Thread Markus Wanner
lots of TCP connections, each of which queues an unknown amount of data). As is evident, all of these decisions are rather Postgres-R centric. However, I still think the simplicity and the level of generalization of imessages, dynamic shared memory and to some extent even the background wor

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-26 Thread Markus Wanner
On 08/26/2010 09:22 PM, Tom Lane wrote: Not having to have a hard limit on the space for unconsumed messages? Ah, I see. However, spilling to disk is unwanted for the current use cases of imessages. Instead the sender needs to be able to deal with out-of-(that-specific-part-of-shared)-memory

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-27 Thread Markus Wanner
ell, I on the other hand am utterly uncomfortable with having a separate solution for memory allocation per sub-system (and it definitely is an inherent problem to lots of our subsystems). Given the ubiquity of dynamic memory allocators, I don't really understand your discomfor

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-30 Thread Markus Wanner
Hi, On 08/27/2010 10:46 PM, Robert Haas wrote: What other subsystems are you imagining servicing with a dynamic allocator? If there were a big demand for this functionality, we probably would have been forced to implement it already, but that's not the case. We've already discussed the fact th

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-30 Thread Markus Wanner
g chunk). (2) I fear reducing the overall robustness of the system, and Well, that applies to pretty much every new feature you add. (3) I'm uncertain what other systems would be able leverage a dynamic allocator of the sort you propose. Okay, that's up to me to show evidences (or at le

Re: [HACKERS] bg worker: patch 1 of 6 - permanent process

2010-08-30 Thread Markus Wanner
Neither do I. Thus I'm probably going to try my approach. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] cheaper snapshots redux

2011-08-24 Thread Markus Wanner
Hello Dimitri, On 08/23/2011 06:39 PM, Dimitri Fontaine wrote: > I'm far from familiar with the detailed concepts here, but allow me to > comment. I have two open questions: > > - is it possible to use a distributed algorithm to produce XIDs, >something like Vector Clocks? > >Then each

Re: [HACKERS] cheaper snapshots redux

2011-08-24 Thread Markus Wanner
case an imessage was left there for a longer amount of time, it blocked creation of new imessages, because the ring-buffer cycled around once and its head arrived back at the unconsumed imessage. IIUC (which might not be the case) the same issue applies for snapshots. Regards Markus Wanner

Re: [HACKERS] cheaper snapshots redux

2011-08-24 Thread Markus Wanner
Robert, On 08/25/2011 04:59 AM, Robert Haas wrote: > True; although there are some other complications. With a > sufficiently sophisticated allocator you can avoid mutex contention > when allocating chunks, but then you have to store a pointer to the > chunk somewhere or other, and that then requ

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
to be worth testing. If the whole thing > crashes and burns I hope I'll at least learn enough along the way to > design something better... That's always a good motivation. In that sense: happy hacking! Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
buffer that it will have wrapped around and clobbered the portion > the reader was interested in. That needs to be rare. Yeah. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
ath we already have: subxids.overflowed. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-02 Thread Markus Wanner
obal addresses, I'm wondering what the use case for storing them is. I'm even wondering if 'fe80::1%1'::inet = 'fe80::1%2'::inet shouldn't simply yield true. After all, it's the same (non-global) address. Kind Regards Markus Wanner -- Sent via pgsql-hac

Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-03 Thread Markus Wanner
On 06/03/2016 12:14 AM, Tom Lane wrote: > Markus Wanner writes: >> I'm even wondering if 'fe80::1%1'::inet = 'fe80::1%2'::inet shouldn't >> simply yield true. After all, it's the same (non-global) address. > > Surely not? If the zone_ids

Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-03 Thread Markus Wanner
ugh. TEXT would be just fine to store a textual scope id. And it makes it utterly clear that there's no magic involved. Kind Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] IPv6 link-local addresses and init data type

2016-06-28 Thread Markus Wanner
;m starting to question if it's really wise to add the scope id to the INET6 type... > [2] - > http://stackoverflow.com/questions/24932172/what-length-can-a-network-interface-name-have Note that the scope id doesn't necessarily have to be a network interface name. Concluding there&#x

[HACKERS] pinging Dano

2009-12-16 Thread Markus Wanner
Hi, who is the main editor named "Dano" of the wiki page about Parallel Query Execution (http://wiki.postgresql.org/wiki/Parallel_Query_Execution), please speak up. Is there any code or patch available ATM? What "discussion with Tom and Simon" is that page referring t

Re: [HACKERS] determine snapshot after obtaining locks for first statement

2009-12-17 Thread Markus Wanner
eck fails. Meaning having to degrade the real snapshot acquired after the first lock to a candidate snapshot for the second tuple lock we try. Maybe "candidate snapshots" is a good short name for such a feature? Another line of thought: isn't this like READ COMMITTED for just t

Re: [HACKERS] Cancelling idle in transaction state

2010-01-02 Thread Markus Wanner
ve them. Aha, yes I see. That sounds more controllable (and should probably default to no timeout). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Testing with concurrent sessions

2010-01-05 Thread Markus Wanner
ted) and basically controls processes, issues queries and checks results and ordering constraints (e.g. transaction X must commit and return a result before transaction Y). I'm still under the impression that this testing framework needs cleanup. However, others already showed interest as

Re: [HACKERS] Testing with concurrent sessions

2010-01-05 Thread Markus Wanner
Hi, Kevin Grittner wrote: Where would I find this (and any related documentation)? Sorry, if that didn't get clear. I'm trying to put together something I can release real soon now (tm). I'll keep you informed. Regards Markus Wanner -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Serializable Isolation without blocking

2010-01-06 Thread Markus Wanner
because of the different lifecycle of SIREAD hints than locks), I think basing that on existing table level locks isn't a good idea. How about storing the SIREAD info in shared memory and using dynamic granularity based on the conflict rate and available memory? *duck* As this seems to be an o

Re: [HACKERS] Serializable Isolation without blocking

2010-01-07 Thread Markus Wanner
wn section above. Is there any such plan of development for predicate locking? To me that seems to be the harder problem (due to being more general). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Serializable Isolation without blocking

2010-01-07 Thread Markus Wanner
cks are kept in the tuple header until the end of the transaction (and even beyond). Is this really a problem? > I suppose these more persistent > write locks should be kept out of the DEFAULT lock method, too I fail to understand that part. What's the DEFAULT lock method? Regards

Re: [HACKERS] Serializable Isolation without blocking

2010-01-07 Thread Markus Wanner
lizability. In a way, both are optimizations of that trivial implementation. My point is that due to that dependency, the conceptual design of a solution for predicate locking (with acceptable performance) should at least be considered before going into details with true serializability. Regards

Re: [HACKERS] Serializable Isolation without blocking

2010-01-08 Thread Markus Wanner
AD may or may not be implemented atop existing or non-existent locking structures, IMO. I've made my points about implementing SIREAD atop table level or row level locking structures. With (non-existent) predicate locking, I'm still unsure. It might help to implement SIREAD atop such a p

Re: [HACKERS] Serializable Isolation without blocking

2010-01-08 Thread Markus Wanner
retty interesting. However, unlike others, it scales with the number of concurrently held locks. And with the current trend towards multi-multi-core platforms, that might get worse and worse (as concurrency must increase to efficiently use these cores). Regards Markus Wanner -- Sent via pgsq

Re: [HACKERS] RFC: PostgreSQL Add-On Network

2010-01-08 Thread Markus Wanner
GAN's fault. I'm with Josh here, get a PGAN up and running. Make it accessible via web to promote extensibility of Postgres and availability of extensions. Whether the first incarnation of the PGAN client works on Windows or Linux doesn't matter for now. Regards Markus W

Re: [HACKERS] Serializable Isolation without blocking

2010-01-08 Thread Markus Wanner
that adding them to the tuple header on disk is not an option. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Serializable Isolation without blocking

2010-01-08 Thread Markus Wanner
said for row or table level locking, for obvious reasons). However, it certainly depends a lot on the implementation of predicate locking, which we don't have, yet, so that's all speculation... Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Serializable Isolation without blocking

2010-01-08 Thread Markus Wanner
Hi, Kevin Grittner wrote: > I had this flagged as needing a response, but it fell through the > cracks yesterday. Apologies for the delayed response. No problem. > Markus Wanner wrote: > When the Cahill paper talks about predicate locking, it *is* talking > about what to l

Re: [HACKERS] Serializable Isolation without blocking

2010-01-09 Thread Markus Wanner
what. Anybody? A write to a non-last (or non-head) version would lead to having multiple "last" (or rather multiple head) versions, which is not something that's ever supposed to happen in Postgres, IIRC. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] synchronized snapshots

2010-01-09 Thread Markus Wanner
Hi Joachim Wieland wrote: > Since nobody objected to the idea in general, I have implemented it. Great! I hope to get some spare cycles within the next few days to review it. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] Testing with concurrent sessions

2010-01-14 Thread Markus Wanner
Hi, Markus Wanner wrote: Sorry, if that didn't get clear. I'm trying to put together something I can release real soon now (tm). I'll keep you informed. Okay, here we go: dtester version 0.0. This emerged out of Postgres-R, where I don't just need to test multiple cli

Re: [HACKERS] Testing with concurrent sessions

2010-01-14 Thread Markus Wanner
actually doing anything). Second: at the very end of pg_dtester.py, you find the line: reporter = StreamReporter() Try a CursesReporter() instead, it gives much nicer output! Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Testing with concurrent sessions

2010-01-15 Thread Markus Wanner
Hi, Quoting "Kevin Grittner" : I haven't quite gotten it to work yet; I'll start over with 3.0 and see how it goes. Let's stick to 2.x versions, first... I'll also attach the results of the 2.6 attempt. Thanks, that looks already pretty promising. ;-) A few other issues in testing so far

<    1   2   3   4   5   6   >