Re: [HACKERS] gettimeofday is at the end of its usefulness?

2017-01-13 Thread Florian Weimer
* Tom Lane: > Florian Weimer <f...@deneb.enyo.de> writes: >> * Tom Lane: >>> On Linux (RHEL6, 2.4GHz x86_64), I find that gettimeofday(), >>> clock_gettime(CLOCK_MONOTONIC), and clock_gettime(CLOCK_REALTIME) >>> all take about 40ns. Of cou

Re: [HACKERS] gettimeofday is at the end of its usefulness?

2016-12-29 Thread Florian Weimer
* Tom Lane: > On Linux (RHEL6, 2.4GHz x86_64), I find that gettimeofday(), > clock_gettime(CLOCK_MONOTONIC), and clock_gettime(CLOCK_REALTIME) > all take about 40ns. Of course gettimeofday() only has 1us resolution, > but the other two have perhaps 10ns resolution (I get no duplicate > readings

[HACKERS] B-tree index row size limit

2016-10-09 Thread Florian Weimer
The index row size limit reared its ugly head again. My current use of PostgreSQL is to load structured data into it but from sources I don't have control over, to support a wide range of queries whose precise nature is not yet known to me. (Is this called a data warehouse?) Anyway, what

Re: [HACKERS] SSL renegotiation

2015-03-23 Thread Florian Weimer
, mostly. If rekeying is strictly required after processing just a few petabytes, the cipher is severely broken and should no longer be used. -- Florian Weimer / Red Hat Product Security -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-22 Thread Florian Weimer
* David G. Johnston: ​enables or disables data durability ​promise of ACID. ? “fsync = on” only works if the storage stack doesn't do funny things. Depending on the system, it might not be sufficient. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] SSL renegotiation

2015-02-23 Thread Florian Weimer
think you could remove renegotiation from PostgreSQL as long as you offer something better than RC4 in the TLS handshake. -- Florian Weimer / Red Hat Product Security -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-23 Thread Florian Weimer
* Ants Aasma: CRC has exactly one hardware implementation in general purpose CPU's I'm pretty sure that's not true. Many general purpose CPUs have CRC circuity, and there must be some which also expose them as instructions. and Intel has a patent on the techniques they used to implement it.

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Florian Weimer
, but not the thing you proposed. But there have been no concrete counter-proposals, and some of the responses did not take into account the inherent complexities of round-trip avoidance. So I'm not sure how to move this topic forward. -- Florian Weimer / Red Hat Product Security Team -- Sent via

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Florian Weimer
are looking for general async response processing. (In order to implement your proposal, we'd have to add quite a few additional APIs to include the flag argument, but that's a minor detail.) -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Florian Weimer
On 04/22/2014 07:03 PM, Claudio Freire wrote: On Tue, Apr 22, 2014 at 8:19 AM, Florian Weimer fwei...@redhat.com wrote: Feedback in this thread was, we want something like this in libpq, but not the thing you proposed. But there have been no concrete counter-proposals, and some

Re: [HACKERS] atexit_callback can be a net negative

2014-03-07 Thread Florian Weimer
should not joggle. Can anyone see a flaw in that? There's the PID reuse problem. Forking twice (with a delay) could end up with the same PID as MyProcPid. Comparing the process start time would protect against that. Checking getppid() would have the same theoretical problem. -- Florian

Re: [HACKERS] atexit_callback can be a net negative

2014-03-07 Thread Florian Weimer
pthread_atfork currently requires linking against pthread (although this is about to change), and it might incur the pthread-induced overhead on some configurations. -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] atexit_callback can be a net negative

2014-03-07 Thread Florian Weimer
On 03/07/2014 04:10 PM, Tom Lane wrote: Florian Weimer fwei...@redhat.com writes: On 03/07/2014 03:57 PM, Tom Lane wrote: It's not a reason not to do something about the much larger chance of this happening in a direct child process, which certainly won't have a matching PID. Indeed

Re: [HACKERS] RFC: Async query processing

2014-01-05 Thread Florian Weimer
subsequent requests. Purely from a perspective of latency avoidance, sub-queries, WITH or stored procedures can achieve the same thing, and work even if the intermediate result has to undergo some transformation. :-) -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing

Re: [HACKERS] RFC: Async query processing

2014-01-05 Thread Florian Weimer
was to avoid any server-side changes, so that applications do not need fallback code for talking to old servers. -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] RFC: Async query processing

2014-01-03 Thread Florian Weimer
. But not for short bursts, which will be the most heavily used case I believe (most apps create short bursts of inserts and not continuous streams at full bandwidth). Loading data into the database isn't such an uncommon task. Not everything is OLTP. -- Florian Weimer / Red Hat Product Security Team

Re: [HACKERS] RFC: Async query processing

2014-01-03 Thread Florian Weimer
reusing them when possible might avoid that. Again, due to the variance in lengths of runs, the staging tables are not always beneficial. I understand that pipelining introduces complexity. But solving the issues described above is no picnic, either. -- Florian Weimer / Red Hat Product

Re: [HACKERS] RFC: Async query processing

2013-12-18 Thread Florian Weimer
On 11/04/2013 02:51 AM, Claudio Freire wrote: On Sun, Nov 3, 2013 at 3:58 PM, Florian Weimer fwei...@redhat.com wrote: I would like to add truly asynchronous query processing to libpq, enabling command pipelining. The idea is to to allow applications to auto-tune to the bandwidth-delay product

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Florian Weimer
. With ext4 and XFS on plain/LVM/md block devices, this issue should really be a thing of the past. I think the kernel folks would treat this as bugs nowadays, too. -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-14 Thread Florian Weimer
On 11/14/2013 07:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Shouldn't this be a separate setting? It's useful for storage which requires rewriting a partially written sector before it can be read again. -- Florian Weimer / Red Hat Product Security Team

[HACKERS] RFC: Async query processing

2013-11-03 Thread Florian Weimer
implementation of PQASYNC_RESULT, we could do that as well, but this might be too much magic. I thought I'd ask for comments before starting coding because this looks a bit more complicated than I expected. :) -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Why the asprintf patch is still breaking the buildfarm

2013-10-23 Thread Florian Weimer
that C99 and POSIX conflict, and glibc implements neither behavior). -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reasons not to like asprintf

2013-10-23 Thread Florian Weimer
, as required by POSIX. It's probably better to move it after the error logging. -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] get rid of SQL_ASCII?

2013-09-06 Thread Florian Weimer
, retrieving, and working with and how they're doing it. Practically speaking, the escaping gets in the way, and there isn't full feature parity with TEXT. Regular expression matching seems to be missing, for instance. But apart from that, yes, BYTEA would be the more appropriate choice. -- Florian

Re: [HACKERS] Enabling Checksums

2013-04-18 Thread Florian Weimer
* Greg Smith: The TCP/IP checksum spec is at https://tools.ietf.org/html/rfc793 ; its error detection limitations are described at http://www.noahdavids.org/self_published/CRC_and_checksum.html ; and a good article about optimizing its code is at

Re: [HACKERS] [GENERAL] Floating point error

2013-03-06 Thread Florian Weimer
. -- Florian Weimer / Red Hat Product Security Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] posix_fadvise missing in the walsender

2013-03-01 Thread Florian Weimer
* Jeff Janes: Does the kernel really read a data block from disk into memory in order to immediately overwrite it? I would have thought it would optimize that away, at least if the writes are sized and aligned to 512 or 1024 bytes blocks (which WAL should be). With Linux, you'd have to use

Re: [HACKERS] Synchronous commit not... synchronous?

2012-11-03 Thread Florian Weimer
* Daniel Farina: The idea of canceling a COMMIT statement causing a COMMIT seems pretty strange to me. Canceling commits is inherently racy, so I'm not sure if this behavior so strange after all. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Covering Indexes

2012-08-25 Thread Florian Weimer
* Jeff Janes: I don't see the virtue of this in this case. Since the index is not unique, why not just put the index on (a,b,c,d) and be done with it? AFAICT, SQLite 4 encodes keys in a way that is not easily reversed (although the encoding is injective, so it's reversible in principle).

Re: [HACKERS] [PATCH] Documentation: remove confusing paragraph about backslash escaping

2012-02-27 Thread Florian Weimer
. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] WIP: URI connection string support for libpq

2012-02-24 Thread Florian Weimer
of URIs). -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] WIP: URI connection string support for libpq

2012-02-24 Thread Florian Weimer
-of-band mechanism. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] initdb and fsync

2012-02-04 Thread Florian Weimer
* Tom Lane: I wonder whether it wouldn't be sufficient to call sync(2) at the end, anyway, rather than cluttering the entire initdb codebase with fsync calls. We tried to do this in the Debian package mananger. It works as expected on Linux systems, but it can cause a lot of data to hit the

Re: [HACKERS] Page Checksums

2012-01-24 Thread Florian Weimer
. And yes, I would for sure turn such functionality on if it were present. Same here. I already use page-level checksum with Berkeley DB. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Florian Weimer
was coming and could push updated Java and Perl client libraries well before the server-side change hit our internal repository, but I really don't want to have to pay attention to such details. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Florian Weimer
negotiation and you don't expose the binary format, then all clients can use the most efficient format automatically. If I understand things correctly, this is where the JDBC driver is heading (that is, automatic use of binary format). -- Florian Weimerfwei...@bfk.de BFK edv

Re: [HACKERS] Page Checksums

2012-01-23 Thread Florian Weimer
interesting how small the camp of those using checksummed filesystems is. Don't checksumming file systems currently come bundled with other features you might not want (such as certain vendors)? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße

Re: [HACKERS] checkpoint writeback via sync_file_range

2012-01-11 Thread Florian Weimer
to help to smoothen the checkpoints. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] optimizing repeated MVCC snapshots

2012-01-05 Thread Florian Weimer
* Robert Haas: and (c) architectures (like 32-bit x86) where ordinary 64-bit operations aren't atomic but special instructions (cmpxchg8b) can be used to get that behavior. FILD and FIST are atomic, too, and are supported by more micro-architectures. -- Florian Weimerfwei

Re: [HACKERS] Making TEXT NUL-transparent

2011-12-23 Thread Florian Weimer
* Florian Pflug: On Nov24, 2011, at 10:54 , Florian Weimer wrote: Or is it not only about being able to *store* NULs in a text field? No, the entire core should be NUL-transparent. That's unlikely to happen. Yes, with the type input/output functions tied to NUL-terminated strings

Re: [HACKERS] Page Checksums + Double Writes

2011-12-22 Thread Florian Weimer
significantly because it avoids read-modify-write cycles with a cold cache? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql

Re: [HACKERS] Large number of open(2) calls with bulk INSERT into empty table

2011-12-06 Thread Florian Weimer
system lookup performance, you need a larger directory, which presumably means a large number of tables. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201

[HACKERS] Large number of open(2) calls with bulk INSERT into empty table

2011-11-27 Thread Florian Weimer
, too), but I suspect that it could be an issue with other systems. This happens with PostgreSQL 9.1.0. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201

[HACKERS] Wire protocol: type-specific opt-in to binary format

2011-11-24 Thread Florian Weimer
protocol, it could be a regular run-time parameter. Do you think this would make sense? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via

[HACKERS] Making TEXT NUL-transparent

2011-11-24 Thread Florian Weimer
parameters). I haven't even looked how difficult it would be to implement this. Do you think it's worth the trouble? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe

Re: [HACKERS] Making TEXT NUL-transparent

2011-11-24 Thread Florian Weimer
* Pavel Stehule: Hello 2011/11/24 Florian Weimer fwei...@bfk.de: Occasionally, we get bitten by embedded NUL bytes in TEXT values.  We take care of generating proper UTF-8, but this additional restriction sometimes slips by.  It would be really helpful if PostgreSQL could store such TEXT

Re: [HACKERS] Making TEXT NUL-transparent

2011-11-24 Thread Florian Weimer
are broken. I can't recall any other system which enforces UTF-8 well-formedness, but does not permit embedded NULs. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49

Re: [HACKERS] Making TEXT NUL-transparent

2011-11-24 Thread Florian Weimer
? The source system does not enforce that constraint, so from time to time, such data slips through. I don't know why it's there in the first place, and I have no control over the original data source. Usually, it's okay to silently strip NUL bytes; it's just a bit awkward. -- Florian Weimer

Re: [HACKERS] Wire protocol: type-specific opt-in to binary format

2011-11-24 Thread Florian Weimer
* Florian Pflug: On Nov24, 2011, at 10:03 , Florian Weimer wrote: I would like to add functionality which allows a client to tell the server which types can be sent in binary format. The immediate goal is to suppress hex quoting for BYTEA values, but it seems to make sense to make

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-24 Thread Florian Weimer
for management functions). -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Wire protocol: type-specific opt-in to binary format

2011-11-24 Thread Florian Weimer
the client library (libpq, JDBC, ...), not users of that library, and should thus be handled on the protocol level, not via a GUC. How is this different from the bytea_output GUC? libpq doesn't hide that at all, but the JDBC driver does---similar to the text/binary distinction. -- Florian

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-24 Thread Florian Weimer
. with no local socket support). Ah, okay, your proposal looked like something which couldn't work with JDBC *at all* because of invalid URI syntax (but admittedly, I haven't checked that yet). -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Florian Weimer
it be possible to add something like psql -d postgresql+ssh://fweimer@db5/var/run/postgresql/.s.PGSQL.5432 similar to what Subversion supports? (This might have security implications when used from untrusted PHP scripts.) -- Florian Weimerfwei...@bfk.de BFK edv-consulting

Re: [HACKERS] strict aliasing

2011-11-15 Thread Florian Weimer
is not strict-aliasing-safe and you compile with -f-strict-aliasing, GCC may silently produce wrong code. (Same with -fwrapv, by the way.) -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133

Re: [HACKERS] How to define global variable in postgresql

2011-08-19 Thread Florian Weimer
Hello. How can we define a global variable in postgresql? Do you mean session-private, but persistent across transactions? Configuration parameters can be abused for this purpose. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100

Re: [HACKERS] synchronized snapshots

2011-08-15 Thread Florian Weimer
* Simon Riggs: I don't see the need to change the BEGIN command, which is SQL Standard. We don't normally do that. Some language bindings treat BEGIN specially, so it might be difficult to use this feature. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http

Re: [HACKERS] libedit memory stomp is apparently fixed in OS X Lion

2011-08-09 Thread Florian Weimer
under the system library exception on MacOS X. Curiously, the GPL favors proprietary operating systems in this area. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax

Re: [HACKERS] reducing the overhead of frequent table locks, v4

2011-07-11 Thread Florian Weimer
. The earliest citation seems to be W.C. Hsiesh, W. E. Weihl, Scalable reader-writer locks for parallel systems., MIT-LCS-TR-521, published in 1991. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D

Re: [HACKERS] POSIX question

2011-06-20 Thread Florian Weimer
with PROT_NONE and subsequent update with mprotect does this on Linux. (It's not clear to me what this is trying to solve, though.) -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Florian Weimer
do), doesn't this make it increasingly unlikely that it's going to be triggered in the wild? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99

Re: [HACKERS] Debian readline/libedit breakage

2011-02-14 Thread Florian Weimer
with LGPL code. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Range Types - efficiency

2011-02-10 Thread Florian Weimer
on the table, rather than restricting itself to the tuples of interest. This is quite similar to LIKE and regexp matches. The backend has a kludge to use the index in such cases. It did not seem extensible to me last time I looked, unfortunately. -- Florian Weimerfwei

Re: [HACKERS] WIP: Range Types

2011-01-07 Thread Florian Weimer
* Jeff Davis: On Tue, 2011-01-04 at 14:18 +, Florian Weimer wrote: * Jeff Davis: 4. For the GiST penalty function, and perhaps some picksplit algorithms, it might be nice to know the length of a range, or do some other kinds of math. It introduces a lot of complexity to try

Re: [HACKERS] WIP: Range Types

2011-01-04 Thread Florian Weimer
. So I was thinking that the user might need to specify a function that converts the subtype into a float that approximates a value's position in the total order. Doesn't the eqsel hint already provide this information? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH

Re: [HACKERS] [PATCH] Custom code int(32|64) = text conversions out of performance reasons

2010-12-03 Thread Florian Weimer
be helpful to break down the dependency chain for large input values. The int8 version should probably work in 1e9 chunks and use a zero-padding variant of the 32-bit code. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100

Re: [HACKERS] crash-safe visibility map, take three

2010-12-03 Thread Florian Weimer
not sure how much that would interfere with the executor architecture.) -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers

Re: [HACKERS] Idle git question: how come so many objects?

2010-12-03 Thread Florian Weimer
compressed by diffing them to some other object (which might, in turn, be a diff). -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Florian Weimer
of an atomic operation on i386 and amd64, you are guaranteed to observe all prior writes performed by the thread which did the atomic operation, too. Explicit fencing is only necessary if you need synchronization without atomic operations. -- Florian Weimerfwei...@bfk.de BFK edv-consulting

Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-10-20 Thread Florian Weimer
* Terry Laurenzo: Agreed. BSON was born out of implementations that either lacked arbitrary precision numbers or had a strong affinity to an int/floating point way of thinking about numbers. I believe that if BSON had an arbitrary precision number type, it would be a proper superset of

Re: [HACKERS] .gitignore files, take two

2010-10-11 Thread Florian Weimer
* James Cloos: I'm reading this a bit late, but... We (Xorg) found that ignoring: *~ *.bak *.patch in addition to the files generated by building is very helpful. I tend to put those into .git/info/exclude. They are somewhat developer-specific, after all. -- Florian Weimer

Re: [HACKERS] UTF16 surrogate pairs in UTF8 encoding

2010-08-23 Thread Florian Weimer
, I really question the sense of this. Usually, people want CESU-8 to preserve ordering between languages such as C# and Java and their database, and conversion destroys this property. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße

Re: [HACKERS] antisocial things you can do in git (but not CVS)

2010-07-23 Thread Florian Weimer
accidentally push a merge commit, either because they forgot to squash locally, or because of a conflict between their local git repo's master branch and origin/master. Can we forbid this? It's possible to do this with some scripting on the server side. -- Florian Weimerfwei...@bfk.de

Re: [HACKERS] [BUGS] Invalid YAML output from EXPLAIN

2010-06-08 Thread Florian Weimer
* Greg Smith: Florian Weimer wrote: It has been claimed before that YAML is a superset of JSON, so why can't the YAML folks use the existing JSON output instead? Because JSON just crosses the line where it feels like there's so much markup that people expect a tool is necessary to read

Re: [HACKERS] [BUGS] Invalid YAML output from EXPLAIN

2010-06-07 Thread Florian Weimer
before that YAML is a superset of JSON, so why can't the YAML folks use the existing JSON output instead? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721

Re: [HACKERS] Confusion over Python drivers

2010-02-14 Thread Florian Weimer
* Jeff Davis: Agreed. Ultimately, the conversion has to be done somewhere, but I don't believe the driver is the place for it. Type conversions are always going to be imperfect, and this has some important consequences: * The type conversion system will be endlessly tweaked to improve it *

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync

2010-02-14 Thread Florian Weimer
* Tom Lane: Which options would that be? I am not aware that there any for any of the recent linux filesystems. Shouldn't journaling of metadata be sufficient? You also need to enforce ordering between the directory update and the file update. The file metadata is flushed with fsync(), but

Re: [HACKERS] Confusion over Python drivers

2010-02-08 Thread Florian Weimer
* Jeff Davis: I have written up a set of guidelines for driver development based on what I learned working on ruby-pg: http://wiki.postgresql.org/wiki/Driver_development Interesting, thanks. I'm contemplating to create a new language binding for libpq (or, to be more precise, turn an

Re: [HACKERS] Confusion over Python drivers

2010-02-08 Thread Florian Weimer
* Andrew McNamara: Any other suggestions before I turn the above into a roadmap page on the wiki? I got sick of the constant stream of escaping bugs impacting on psycopg and pyPgSQL, and wrote my own DB-API driver, using the more modern libpq/binary/protocol 3 APIs where ever possible. The

Re: [HACKERS] MySQL-ism help patch for psql

2010-01-21 Thread Florian Weimer
, or \? for general help with psql commands ERROR: unrecognized configuration parameter databases NOTICE: use \l to list databases, or \? for general help with psql commands (I hope that this is less controversial, too.) -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH

Re: [HACKERS] Update on true serializable techniques in MVCC

2009-12-18 Thread Florian Weimer
, no? They are detected. In this context, serialization failure means that PostgreSQL generates a history which lacks one-copy serializability, without reporting any errors. (In the general case, the unique constraint violation which bugs me personally is a different beast and does result in an error.) -- Florian

Re: [HACKERS] Update on true serializable techniques in MVCC

2009-12-16 Thread Florian Weimer
a unique constraint violation caused by a phantom from an application bug. (We currently faking this by retrying a fixed number of times and bailing out if the error returned by PostgreSQL looks like a unique constraint violation.) -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH

Re: [HACKERS] YAML Was: CommitFest status/management

2009-12-07 Thread Florian Weimer
instantiations (and it's not possible to switch this off, or it's rather involved to do so). Plain JSON doesn't have this issue. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe

Re: [HACKERS] YAML Was: CommitFest status/management

2009-12-07 Thread Florian Weimer
* Alvaro Herrera: Florian Weimer escribió: * Dimitri Fontaine: Well we have JSON and agreed it was a good idea to have it. Now JSON is a subset of YAML and some would prefer another YAML style (me included). YAML is rather difficult to parse, and most parsers assume a trusted document

Re: [HACKERS] Block-level CRC checks

2009-12-01 Thread Florian Weimer
, since that will happen very quickly anyway. What about putting the whole visibility information out-of-line, into its own B-tree, indexed by page number? -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201

Re: [HACKERS] Add YAML option to explain

2009-08-28 Thread Florian Weimer
* Greg Sabino Mullane: I thought the consensus was that we didn't want to get into supporting more formats. What does YAML provide that JSON does not? Readability and easy editing. All the power of JSON without the annoying quotes, braces, and brackets. But YAML is much more difficult to

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

2009-06-18 Thread Florian Weimer
such as this one? SELECT c from sbtest where id between $1 and $2 order by c (See the start of the thread.) -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721

Re: [HACKERS] Multicolumn index corruption on 8.4 beta 2

2009-06-09 Thread Florian Weimer
* Simon Riggs: On Tue, 2009-06-09 at 13:40 +0200, Floris Bos / Maxnet wrote: fsync = off That's a bad plan if you care about your database. It shouldn't introduce this type of corruption, though. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http

Re: [HACKERS] Multicolumn index corruption on 8.4 beta 2

2009-06-09 Thread Florian Weimer
running), the documentation is seriously wrong here. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] [PATCH v2] Add bit operations util header

2009-06-03 Thread Florian Weimer
* Jeremy Kerr: +#if defined(__GNUC__) \ + (defined(__ppc__) || defined(__powerpc__) || \ + defined(__ppc64__) || defined (__powerpc64__)) If you require GCC anyway, you can use __builtin_clz instead. (It's been available since GCC 4.1 at least.) -- Florian Weimer

Re: [HACKERS] [PATCH v2] Add bit operations util header

2009-06-03 Thread Florian Weimer
to test the compiler *and* the version as well? This builtin is not architecture-specific, so you'd save the architecture check. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe

Re: [HACKERS] Managing multiple branches in git

2009-06-03 Thread Florian Weimer
its output, so it doesn't care about file modification times). -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing

Re: [HACKERS] renaming storage parameters

2009-02-09 Thread Florian Weimer
* Alvaro Herrera: Euler Taveira is arguing in an autovacuum thread that we should give storage parameters a different name; his argument is that autovacuum_enabled is not really a parameter that relates to storage. He is proposing relation parameters. They also apply to indices, right? I

Re: [HACKERS] Block-level CRC checks

2008-10-31 Thread Florian Weimer
), and I think the text book algorithms don't assume this behavior, either. -- Florian Weimer[EMAIL PROTECTED] BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql

Re: [HACKERS] Block-level CRC checks

2008-10-02 Thread Florian Weimer
). -- Florian Weimer[EMAIL PROTECTED] BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Block-level CRC checks

2008-10-01 Thread Florian Weimer
it to disk (= torn page). The CRC would not match, even though the page is actually valid. The non-logged hint bit writes are somewhat dangerous anyway. Maybe it's time to get rid of this peculiarity, despite the performance impact? -- Florian Weimer[EMAIL PROTECTED] BFK edv

Re: [HACKERS] Block-level CRC checks

2008-10-01 Thread Florian Weimer
. And for large blocks, Adler32 isn't that bad compared to CRC32 from a error detection POV, so maybe you could use that. I've seen faults which were uncovered by page-level checksumming, so I'd be willing to pay the performance cost. 8-/ -- Florian Weimer[EMAIL PROTECTED] BFK edv

Re: [HACKERS] Is mdextend really safe?

2008-08-20 Thread Florian Weimer
fragmented files on the file systems I use, and I don't see this with PostgreSQL. (It's one of my gripes with Berkeley DB.) However, I looked at the code recently and couldn't figure out *why* PostgreSQL's observed behavior is this way. 8-( -- Florian Weimer[EMAIL PROTECTED] BFK

Re: [HACKERS] Protection from SQL injection

2008-05-04 Thread Florian Weimer
* Thomas Mueller: What do you think about it? Do you think it makes sense to implement this security feature in PostgreSQL as well? Can't this be implemented in the client library, or a wrapper around it? A simple approximation would be to raise an error when you encounter a query string that

Re: [HACKERS] Cached Query Plans

2008-04-15 Thread Florian Weimer
in front of it, which breaks with temporary tables). -- Florian Weimer[EMAIL PROTECTED] BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Read-ahead and parallelism in redo recovery

2008-02-29 Thread Florian Weimer
does not result in lots of random I/O (with 8.3 and full page writes active; this is a relatively recent change). In the end, I wouldn't be surprised if for most loads, cache warming effects dominated recovery times, at least when the machine is not starved on RAM. -- Florian Weimer

  1   2   >