Re: [HACKERS] logging in high performance systems.

2011-11-24 Thread Theo Schlossnagle
to logging so I can solve a critical production issue we seem to be running into more and more. Thanks for you time. -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] logging in high performance systems.

2011-11-23 Thread Theo Schlossnagle
og lines per second with negligible impact on performance. https://github.com/postwait/postgres/commit/c8f5a346c7b2c3eba9f72ea49077dc72f03a2679 Thoughts? Feedback? As can be seen, the patch is pretty tiny. -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -- Sent via pgsql-

Re: [HACKERS] double and numeric conversion

2010-03-03 Thread Theo Schlossnagle
lting in things like: 1.23 turning into 1.2300 in the numeric returned. This are significantly faster (as expected) than the type -> string -> numeric conversions. On Mar 3, 2010, at 5:01 AM, Yeb Havinga wrote: > Theo Schlossnagle wrote: >> I didn't look deeply at the pos

Re: [HACKERS] double and numeric conversion

2010-03-02 Thread Theo Schlossnagle
On Mar 1, 2010, at 4:35 PM, Tom Lane wrote: > Theo Schlossnagle writes: >> I'm writing some extension and I have a hot code path that has a lot of >> double (C type) data and needs to output NUMERIC tuple data. The current >> methods I can find in the code to conve

[HACKERS] double and numeric conversion

2010-03-01 Thread Theo Schlossnagle
ve profile my stuff and I'm spending (wasting) all my time in that conversion. Is there a more efficient method of converting a double into a postgres numeric value? Best regards, Theo -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -- Sent via pgsql-hackers mailing list

[HACKERS] question about implementing XA-ish functions

2009-12-18 Thread Theo Schlossnagle
my cleanup handler fires, I detect whether the txn was committed or rolledback and rightly mark my work as committed or rolled back. Thoughts? -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle p: +1.443.325.1357 x201 f: +1.410.872.4911 -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] [patch] executor and slru dtrace probes

2009-12-09 Thread Theo Schlossnagle
e. There is (of course) some performance overhead when they are enabled, but that is intentionally performed by the operator, so it seems like a non-issue. Now, there was some indication that there was a better place to probe that would be more comprehensive -- that should be addressed. -- Theo

Re: [HACKERS] concurrent index builds unneeded lock?

2009-07-11 Thread Theo Schlossnagle
On Jul 11, 2009, at 6:50 AM, Greg Stark wrote: On Sat, Jul 11, 2009 at 6:17 AM, Theo Schlossnagle wrote: On Jul 11, 2009, at 12:12 AM, Tom Lane wrote: Theo Schlossnagle writes: I would think it would be txns that would be reading that table, but I'm thinking it is a b

Re: [HACKERS] concurrent index builds unneeded lock?

2009-07-10 Thread Theo Schlossnagle
On Jul 11, 2009, at 12:12 AM, Tom Lane wrote: Theo Schlossnagle writes: I would think it would be txns that would be reading that table, but I'm thinking it is a bit to aggressive. Am I reading the code wrong there? I'm thinking it should be more selective about vxids it choose

[HACKERS] concurrent index builds unneeded lock?

2009-07-10 Thread Theo Schlossnagle
wrong there? I'm thinking it should be more selective about vxids it chooses to block on. I'd expect it to block on vxids touching the same table only. Thoughts? -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle p: +1.443.325.1357 x201 f: +1.410.872.4911 -- Sent vi

Re: [HACKERS] Review: DTrace probes (merged version) ver_03

2008-07-24 Thread Theo Schlossnagle
correlating query execution to disk I/O to be induced. -- Theo Schlossnagle Esoteric Curio -- http://lethargy.org/ OmniTI Computer Consulting, Inc. -- http://omniti.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

[HACKERS] DTrace probes.

2008-05-19 Thread Theo Schlossnagle
convert it to something else: https://labs.omniti.com/trac/project-dtrace/wiki/Applications#PostgreSQL Best regards, Theo -- Theo Schlossnagle Esoteric Curio -- http://lethargy.org/ OmniTI Computer Consulting, Inc. -- http://omniti.com/ -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] [PATCHES] Doc update for pg_start_backup

2007-06-29 Thread Theo Schlossnagle
rage to take a hard snapshot and expose that as a LUN to mount elsewhere on attached to the same SAN. Many confuse this for being "free". Regardless of how the snap is taken you have to pay for it.. either at snap time, at read time or at release time. Nothing's free. /

Re: [HACKERS] New feature request: FlashBack Query

2007-02-20 Thread Theo Schlossnagle
for undoing of at least a week changes. You're living in a dream world. Do you know any Oracle DBs who keep enough rollback segments to go back a week? Ours go for a good 6 hours sometimes :-D // Theo Schlossnagle // Esoteric Curio: http://www.lethargy.org/~jesus/ --

Re: [HACKERS] XML export

2007-02-10 Thread Theo Schlossnagle
e else, short of the moon, let me know. Integrated, native XML support can only help PostgreSQL. IMO, I want this in core. Agreed. In the server would be more useful to more people I think. It would be really convenient to be able to have "no effort" XML results sets to q

Re: [HACKERS] Proposal: Commit timestamp

2007-02-04 Thread Theo Schlossnagle
On Feb 4, 2007, at 1:36 PM, Jan Wieck wrote: On 2/4/2007 10:53 AM, Theo Schlossnagle wrote: As the clock must be incremented clusterwide, the need for it to be insync with the system clock (on any or all of the systems) is obviated. In fact, as you can't guarantee the synchron

Re: [HACKERS] Proposal: Commit timestamp

2007-02-04 Thread Theo Schlossnagle
k (on any or all of the systems) is obviated. In fact, as you can't guarantee the synchronicity means that it can be confusing -- one expects a time-based clock to be accurate to the time. A counter-based clock has no such expectations. // Theo Schlossnagle // CTO -- http://w

Re: [HACKERS] Proposal: Commit timestamp

2007-02-03 Thread Theo Schlossnagle
On Feb 3, 2007, at 5:09 PM, Jan Wieck wrote: On 2/3/2007 4:58 PM, Theo Schlossnagle wrote: I don't have any such paper and the proof of concept will be the implementation of the system. I do however see enough resistance against this proposal to withdraw the commit timestamp at

Re: [HACKERS] Proposal: Commit timestamp

2007-02-03 Thread Theo Schlossnagle
On Feb 3, 2007, at 4:38 PM, Jan Wieck wrote: On 2/3/2007 4:05 PM, Theo Schlossnagle wrote: On Feb 3, 2007, at 3:52 PM, Jan Wieck wrote: On 2/1/2007 11:23 PM, Jim Nasby wrote: On Jan 25, 2007, at 6:16 PM, Jan Wieck wrote: If a per database configurable tslog_priority is given, the

Re: [HACKERS] Proposal: Commit timestamp

2007-02-03 Thread Theo Schlossnagle
semantics of a Lamport timestamp. As such, any algorithms that use lamport timestamps as a basis or assumption for the proof of their correctness will not translate (provably) to this system. How are your counter semantically equivalent to Lamport timestamps? // Theo Schlossnagle // CTO

Re: [HACKERS] Proposal: Commit timestamp

2007-01-26 Thread Theo Schlossnagle
is rigorous dissection of replication design hasn't happened, but I didn't see it referenced anywhere in this thread. Can you point me to it? I've reviewed many of these papers and would like to better understand what you are aiming at. Best regards, Theo Schlossnag

Re: [HACKERS] About PostgreSQL certification

2007-01-24 Thread Theo Schlossnagle
. We manage one of the larger postgres instances out there -- I know its pros and cons well. // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/ ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [HACKERS] About PostgreSQL certification

2007-01-23 Thread Theo Schlossnagle
ll PostgreSQL adoption. That's a good thing. -- Theo // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usene

Re: [HACKERS] About PostgreSQL certification

2007-01-23 Thread Theo Schlossnagle
On Jan 23, 2007, at 5:04 PM, Mark Kirkwood wrote: Theo Schlossnagle wrote: On Jan 23, 2007, at 4:33 PM, David Fetter wrote: On Tue, Jan 23, 2007 at 11:52:08AM -0200, Iannsp wrote: Hello, I did like to know what you think about the postgresql certifications provided for PostgreSQL CE http

Re: [HACKERS] About PostgreSQL certification

2007-01-23 Thread Theo Schlossnagle
ught it meant something that companies "out there" could rely on. Many other certifying entities have the same approach. // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/ ---(end of broadca

Re: [HACKERS] TODO: GNU TLS

2006-12-29 Thread Theo Schlossnagle
like specifically choosing a forum that will have the most disagreement. // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/ ---(end of broadcast)--- TIP 1: if posting/rea

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Theo Schlossnagle
On Oct 21, 2006, at 4:40 PM, Simon Riggs wrote: On Sat, 2006-10-21 at 15:17 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 3:12 PM, Simon Riggs wrote: On Sat, 2006-10-21 at 09:00 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 6:08 AM, Martijn van Oosterhout wrote: On Sat, Oct

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Theo Schlossnagle
On Oct 21, 2006, at 3:12 PM, Simon Riggs wrote: On Sat, 2006-10-21 at 09:00 -0400, Theo Schlossnagle wrote: On Oct 21, 2006, at 6:08 AM, Martijn van Oosterhout wrote: On Sat, Oct 21, 2006 at 10:37:51AM +0100, Simon Riggs wrote: Turning off WAL is a difficult topic. Without it you have no

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-21 Thread Theo Schlossnagle
have PITR -- sans snapshots) // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Theo Schlossnagle
On Oct 20, 2006, at 4:24 PM, Simon Riggs wrote: On Fri, 2006-10-20 at 13:18 -0400, Theo Schlossnagle wrote: Not sure who cares, so xzilla indicated I should drop a note here. I just made the xlogdump stuff work for 8.1 (trivial) and fixed a few other small issues that caused it to not work

Re: [HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Theo Schlossnagle
On Oct 20, 2006, at 1:58 PM, Tom Lane wrote: Theo Schlossnagle <[EMAIL PROTECTED]> writes: Is it possible to create tables in fashion that will not write info to the WAL log -- knowingly and intentionally making them unrecoverable? Use temp tables? temp tables won't wo

[HACKERS] xlogdump fixups and WAL log question.

2006-10-20 Thread Theo Schlossnagle
AS SELECT * from database> NO LOGGING; (NO LOGGING being the only part we're currently missing) Is something like this possible? Cheers ;-) Theo // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/

Re: [HACKERS] Upgrading a database dump/restore

2006-10-11 Thread Theo Schlossnagle
On Oct 11, 2006, at 5:06 PM, Josh Berkus wrote: What type of help did you envision? The answer is likely yes. I don't know, whatever you have available. Design advice, at the very least. Absolutely. I might be able to contribute some coding time as well. Testing time too. //

Re: [HACKERS] Upgrading a database dump/restore

2006-10-11 Thread Theo Schlossnagle
What type of help did you envision? The answer is likely yes. On Oct 11, 2006, at 5:02 PM, Josh Berkus wrote: Theo, Would you be able to help me, Zdenek & Gavin in work on a new pg_upgrade? -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco // Theo Schlossnagle // CTO --

Re: [HACKERS] Index Tuning Features

2006-10-11 Thread Theo Schlossnagle
499,999 of the queries were planned perfectly fine by the planner). To fix my one query, that is crucially important to my business, it is a much more sane approach to hint the system to change its plan than it is to have to upgrade my binaries. // Theo Schlossnagle // CTO -- http://www.omn

Re: [HACKERS] Upgrading a database dump/restore

2006-10-11 Thread Theo Schlossnagle
On Oct 11, 2006, at 9:36 AM, Tom Lane wrote: Theo Schlossnagle <[EMAIL PROTECTED]> writes: The real problem with a "dump" of the database is that you want to be able to quickly switch back to a known working copy in the event of a failure. A dump is the furthest possible thin

Re: [HACKERS] Upgrading a database dump/restore

2006-10-11 Thread Theo Schlossnagle
e database (indexes, etc.) and in doing so, you (1) spend the better part of a week running pg_restore and (2) ANALYZE stats change, so your system's behavior changes in hard-to- understand ways. Best regards, Theo // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniT

Re: [HACKERS] New version of money type

2006-09-16 Thread Theo Schlossnagle
trous for performance (joining). I'd imagine that if the above wasn't done cleverly, that performance problem would be repeated. // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/

[HACKERS] The enormous s->childXids problem

2006-09-15 Thread Theo Schlossnagle
I ask with respect to the suitability as general solution and as the suitability for my acute issue (of a 5 million row setof returned from that). Will it break anything? Best regards, Theo // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -

Re: [HACKERS] Interesting tight loop

2006-09-14 Thread Theo Schlossnagle
On Sep 14, 2006, at 8:19 AM, Gregory Stark wrote: Theo Schlossnagle <[EMAIL PROTECTED]> writes: We don't use savepoint's too much. Maybe one or two across out 1k or so pl/pgsql procs. Well if they're in a loop... We use dbi-link which is plperl. Perhaps tha

Re: [HACKERS] Interesting tight loop

2006-09-14 Thread Theo Schlossnagle
On Sep 14, 2006, at 7:03 AM, Gregory Stark wrote: Theo Schlossnagle <[EMAIL PROTECTED]> writes: But the interesting thing is that there were 4.6 million elements in the s->childXids list. Which is why it took so damn long. I can't quite figure out how I induced this st

[HACKERS] Interesting tight loop

2006-09-13 Thread Theo Schlossnagle
childXids would be more appropriate. Does the order of the childXids chained off the current transaction state matter? Most of the placed I could find that reference it seem to just attempt to find an Xid in there. O(1) sounds a lot better than O(n) :-D Best regards, Theo // Theo Schl

Re: [HACKERS] Getting a move on for 8.2 beta

2006-09-02 Thread Theo Schlossnagle
going with this? From my experience accepting incremental patches is a _bad_ idea unless you have a VCS that has really makes it _easy_ to manage them. Sounds like more work than worth on the postgres project as it is now. Additionally, what problem is accepting incremental patches supp

Re: [HACKERS] Prepared statements considered harmful

2006-08-31 Thread Theo Schlossnagle
a ton of other needed features, I rarely see the attitude that they are _unwanted_. Instead I see the "if it is important to you, go build it" attitude which is what I would expect in an open source project. // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-19 Thread Theo Schlossnagle
thout any probes at all in the code - I guess a configure option "--without-dtrace" on by default on those platforms would do it. Absolutely. As they are all proposed as preprocessor macros, this would be trivial to accomplish. // Theo Schlossnagle // CTO -- http://www.om

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-19 Thread Theo Schlossnagle
On Jun 19, 2006, at 6:41 PM, Robert Lor wrote: Theo Schlossnagle wrote: Heh. Syscall probes and FBT probes in Dtrace have zero overhead. User-space probes do have overhead, but it is only a few instructions (two I think). Besically, the probe points are replaced by illegal

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-19 Thread Theo Schlossnagle
es are not enabled. The reason that Robert proposes user-space probes (I assume) is that tracing C functions can be too granular and not conveniently expose the "right" information to make tracing useful. // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer

Re: [HACKERS] Compiling PL/Perl and Pl/Python on x86_64

2006-05-22 Thread Theo Schlossnagle
pt 32-bit intel. The bigger puzzle is why you could link against non-PIC code in shared objects on 32-bit x86. (I know the answer, but it has no real merit). If you want things dynamically loadable, they must be PIC. -- // Theo Schlossnagle // Principal Engineer -- http://www.omniti.com/~

Re: [HACKERS] Solaris ASM problem

2006-04-29 Thread Theo Schlossnagle
Kris Jurka wrote: On Fri, 28 Apr 2006, Theo Schlossnagle wrote: Kris Jurka wrote: Anyway the test exits with Stuck spinlock (80618e9) detected at ./s_lock.c:355. on a linux gcc build this exits with Stuck spinlock (0x5013ad) detected at ./s_lock.c:402. This seems like a different

Re: [HACKERS] Solaris ASM problem

2006-04-29 Thread Theo Schlossnagle
Kris Jurka wrote: On Fri, 28 Apr 2006, Theo Schlossnagle wrote: What platform is that? (OS rev, architecture and word size)? I tested the changes I submitted on Solaris 10 amd64. $ uname -a SunOS albert 5.9 Generic_112234-03 i86pc i386 i86pc $ cc -V cc: Sun WorkShop 6 update 2 C 5.3

Re: [HACKERS] Solaris ASM problem

2006-04-29 Thread Theo Schlossnagle
Kris Jurka wrote: On Fri, 28 Apr 2006, Theo Schlossnagle wrote: The file that uses the spinlocks: /src/backend/storage/lmgr/s_lock.c can be compiled standalone with -DS_LOCK_TEST To get the test to compile I had to link in tas.o as the attached patch shows. Unfortunately this doesn&#

Re: [HACKERS] Solaris ASM problem

2006-04-29 Thread Theo Schlossnagle
ified the build system to properly handle the preprocessing, it seems we have a problem with the ASM instructions. Theo? Comments? What platform is that? (OS rev, architecture and word size)? I tested the changes I submitted on Solaris 10 amd64. -- // Theo Schlossnagle // Principal Enginee