[HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Greg Stark
e_io_concurrency of 1000 actually means 7485 prefetches). At some point those i/o are going to start completing before Postgres even has a chance to start processing the data. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] Current int & float overflow checking is slow.

2017-10-24 Thread Greg Stark
We already know this integer overflow checking is non-standard and compilers keep trying to optimize them out. Our only strategy to defeat that depends on compiler flags like -fwrapv that vary by compiler and may or may not be working on less well tested compiler. So if there's a nice readable an

Re: [HACKERS] On markers of changed data

2017-10-11 Thread Greg Stark
imum size regardless of how long a time range they span so if you keep one changelist for every 10 checkpoints or every 100 checkpoints you could reduce the storage needs and only lose the time precision. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] On markers of changed data

2017-10-10 Thread Greg Stark
: look up all the records (or just the last record) that modified a given block. Instead of having to scan all the wal you would only need to scan the checkpoint eras that had touched that block. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] generated columns

2017-10-03 Thread Greg Stark
like password hashes or personal data with details censored without giving them access to the unhashed password or full personal info. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 64-bit queryId?

2017-10-01 Thread Greg Stark
On 1 October 2017 at 16:40, Tom Lane wrote: > Greg Stark writes: >> Indeed. It's simple enough to export stats to prometheus with queryid >> as the key. Then even if the query ages out of the database stats you >> have graphs and derivative metrics going further back. &

Re: [HACKERS] 64-bit queryId?

2017-10-01 Thread Greg Stark
ld make that impossible. (I have other advances in pg_stat_statements I would love to see. It would be so much more helpful if pg_stat_statements also kept a few examples of query parameters such as the most recent set, the set that caused the longest execution, maybe the set with the largest of

Re: [HACKERS] The case for removing replacement selection sort

2017-09-09 Thread Greg Stark
e with string sorting in an interesting encoding/locale -- say /usr/share/dict/polish in pl_PL for example. It's certainly true that people do sort text as well as numbers. Also, people often sort on keys of more than one column -- greg -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] JIT compiling expressions/deform + inlining prototype v2.0

2017-09-05 Thread Greg Stark
r every 8 columns up until the first variable size column (or I suppose you could even continue in the case where the variable size column is null). -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

[HACKERS] CSV Logging questions

2017-09-04 Thread Greg Stark
ant and be sure they would all have the same data. That would also mean that the timestamps would be in sync and we could probably eliminate the occurrences of the wrong format appearing in the wrong logs. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] log_destination=file

2017-09-01 Thread Greg Stark
are more concerning. For all we know there are people out there who tried to use the logging collector and found it didn't work well on some system -- perhaps it interacted with systemd or something else on the system -- and they switched back to just using stderr. I don't know how to flu

[HACKERS] signed logging format for pid in log_line_prefix?

2017-09-01 Thread Greg Stark
Both the text and csv logging seem to use %d on for logging the server pid: appendStringInfo(buf, "%d", MyProcPid); Am I missing something or wouldn't this mean we print pids with large values as negative numbers? Isn't that strange? Wouldn't we rather use %u here?

Re: [HACKERS] generated columns

2017-08-31 Thread Greg Stark
2 │ 5.08 │ └───┴───┘ (1 row) Time: 1.997 ms Yours looks better :) -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] MAIN, Uncompressed?

2017-08-25 Thread Greg Stark
hat it's worth I think a good start would be to give people more visibility into what the tuptoaster heuristic is actually doing to their data and that will encourage people to give feedback about when they're surprised and are frustrated by the existing UI. -- greg -- Sent via pgs

Re: [HACKERS] Funny WAL corruption issue

2017-08-11 Thread Greg Stark
different wal stream position. I think you could actually hack xlogdump to ignore this condition and keep outputting and you'll see whether the records that follow appear to be old wal log data. I haven't actually tried this though. -- greg -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] More optimization effort?

2017-07-21 Thread Greg Stark
operties and constraints of our existing data types and operators or for that matter how easy it would be to figure out what theorems we want proven to be able to use an index. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-21 Thread Greg Stark
On 20 July 2017 at 14:19, Tom Lane wrote: > Greg Stark writes: > >> Would it be useful to keep in one of the memory checking assertion builds? > > Why? Code that expects to continue accessing a relcache entry's tupdesc > after closing the relcache entry is broken, ind

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-20 Thread Greg Stark
oding that requires it. (And if so, there ought to > be a regression test incorporating that.) Would it be useful to keep in one of the memory checking assertion builds? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://ww

Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

2017-07-16 Thread Greg Stark
nly disappear at some unspecified date" is a lot less convincing than "will disappear in 2021 in Postgres 14". The specific year and version number is irrelevant but picking and naming a specific one makes it a lot easier to follow through come that date. -- greg -- Sent via pgsql-

Re: [HACKERS] building libpq.a static library

2017-07-12 Thread Greg Stark
by inventing a foo-shared.o and foo-static.o but that introduces its own weirdness. I don't know what the downsides would be of creating a static library out of objects built with -fPIC. It might just be a small performance penalty which might be no big deal for libpq. That may b

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Greg Stark
u never absolutely need a way to list partitions without the comments and permissions). At least that doesn't require the user to learn a new flag and how it interacts with everything else. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Greg Stark
even find other uses for it. That kind of abstraction would be more promising for the future than having yet another C api that is used for precisely one purpose and whose definition is "provide the data needed for this usage". -- greg -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-07-06 Thread Greg Stark
ge is going into the log and that tuple will never be modified again you could imagine just replacing the tuple with the LSN of the deletion and letting anyone who really needs it fetch it from the xlog. That would be a completely different model from the way Postgres works though. More like a log-s

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

2017-07-03 Thread Greg Stark
On 2 July 2017 at 18:33, Tom Lane wrote: > system("cp -a ...") call in favor of something more portable. If we're ok with using Perl there's File::Copy::Recursive::dircopy() which does exactly that. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] Notes on testing Postgres 10b1

2017-06-09 Thread Greg Stark
nd which would also work properly with the jsonb operators? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-14 Thread Greg Stark
ardware. That seems a bit far-fetched. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A design for amcheck heapam verification

2017-05-01 Thread Greg Stark
have a whole family of hash functions to pick for bloom filters. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication in the same cluster

2017-05-01 Thread Greg Stark
require two or three layers of messy nested PG_TRY and extra transactions. Perhaps there's a cleaner way to structure it and I should look again. I don't recall ever having a plan to do anything in recovery. I think we did talk about why it was hard to mark hash indexes invalid during r

Re: [HACKERS] multithreading in Batch/pipelining mode for libpq

2017-04-22 Thread Greg Stark
On 21 April 2017 at 21:31, Ilya Roublev wrote: > What I need is to make a huge amount of inserts This may be a silly question but I assume you've already considered using server-side COPY? That's the most efficient way to load a lot of data currently. -- greg -- Sent via

Re: [HACKERS] Highly Variable Planning Times

2017-04-20 Thread Greg Stark
obably end up deploying the new parser and building a new index with CREATE INDEX CONCURRENTLY using the new parser and then dropping the old index. I'm not sure if it's possible to do a FTS parser for handling arbitrary CSS selectors but if you managed that that would be a very valuable ad

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-15 Thread Greg Stark
the scram negotiation. Under our md5 auth that gives them your password, under scram they won't get the password which is a huge improvement but they would still have the raw unencrypted data from your traffic. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-04-10 Thread Greg Stark
mend switching to perl or python before trying to code up nesting arbitrary sql in nested expressions. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-04-10 Thread Greg Stark
iciently. I think it would behave more sensibly when you're adding or removing a partition, especially if you want to add many partitions in parallel using multiple transactions. An oidvector of children would effectively mean you could only be doing one partition creation or deletion at a time

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-22 Thread Greg Stark
't have any left. Then at least we'll know when the bits are available again. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] SortSupport for macaddr type

2017-03-19 Thread Greg Stark
of object code. The latter consists of only 10 > instructions, or 24 bytes of object code. I wonder if there's something that could be optimized out of the normal cmp function but we're defeating some compiler optimizations with all our casts and aliasing. -- greg -- Sent via

Re: [HACKERS] [POC] hash partitioning

2017-03-03 Thread Greg Stark
f having uneven partitions if you have a data distribution skew -- which can happen even if you have a good hash function. In a degenerate case you could have a partition for a single hash of a particularly common value then a reasonable number of partitions for the remaining hash ranges. --

Re: [HACKERS] UPDATE of partition key

2017-02-25 Thread Greg Stark
the target of the chain isn't available. That may be something we'll need in the future for other cases too. Throwing an error means the user has to retry their query but that's at least something they can do. Even if they don't do it automatically the ultimate user will prob

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Greg Stark
On 23 February 2017 at 13:27, Greg Stark wrote: > => SELECT "?column"? FROM (select 1+1 as "?column?", 1+1) AS x; Oops, I missed the typo there: => SELECT "?column?" FROM (select 1+1 AS "?column?", 1+1) AS x; ERROR: 42702: column reference "

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Greg Stark
s "?column?", 1+1) AS x; ^ HINT: Perhaps you meant to reference the column "x.?column?" or the column "x.?column?". -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Sum aggregate calculation for single precsion real

2017-02-17 Thread Greg Stark
is small enough. That would actually result in accurate results which neither float4 nor float8 guarantee.) -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-02-16 Thread Greg Stark
impact of breaking the ctid chain. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-11 Thread Greg Stark
ugging scripts it would be handy to have some way to tell whether the \if expression actually evaluated to true or false but that wouldn't be in the prompt I don't think. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-11 Thread Greg Stark
kimming through it for the first time I thought this was more baroque than I was expecting. I was expecting something like a { for each level of nested if you're in so you can see how many deep you are. I didn't expect to see anything more complex than that. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSoC 2017

2017-01-27 Thread Greg Stark
o have a single column that uses different units for different rows. You can aggregate across them without converting as long as you have an appropriate where clause or group by clause -- GROUP BY units_of(debit_amount) for example. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] GSoC 2017

2017-01-27 Thread Greg Stark
secs which is just as valid as preventing you from adding CAD to USD. Then you could imagine having a few entirely optional helper functions that could automatically provide conversion factors using units.dat or currency exchange rates. But even if you don't use these helper functions they

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-01-27 Thread Greg Stark
ragmas usually work. But it would probably not be easy to reconcile with how GUCs work. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Checksums by default?

2017-01-25 Thread Greg Stark
ons is how they can convince MySQL to continue and ignore the corruption. A typical response was "We slightly modified innochecksum and added option -f that means if the checksum of a page is wrong, rewrite it in the InnoDB page header." Which begs the question... -- greg -- Sent via pgsql-hack

Re: [HACKERS] pg_ls_dir & friends still have a hard-coded superuser check

2017-01-25 Thread Greg Stark
I tend to agree. But in the past when this came up people pointed out you could equally do things this way and still grant all the access you wanted using SECURITY DEFINER. Arguably that's a better approach because then instead of auditing the entire monitor script you only need to audit this one w

Re: [HACKERS] Online enabling of page level checksums

2017-01-23 Thread Greg Stark
On Jan 22, 2017 11:13 AM, "Magnus Hagander" wrote: Yes, this means the entire db will end up in the transaction log since everything is rewritten. That's not great, but for a lot of people that will be a trade they're willing to make since it's a one-time thing. Yes, this background process migh

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Greg Sabino Mullane
we dramatically increase the pool of people with checksums enabled. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201701211522 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B9

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Greg Sabino Mullane
% for nothing. It is not for nothing, it is for increasing reliability by detecting (and pinpointing!) corruption as early as possible. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201701211513 http://biglumber.com/

Re: [HACKERS] RustgreSQL

2017-01-08 Thread Greg Stark
nguages ran in a separate Java process. I think the solution to the problem you're describing is the project formerly known as NaCl https://en.wikipedia.org/wiki/Google_Native_Client -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

Re: [HACKERS] [PATCH] guc-ify the formerly hard-coded MAX_SEND_SIZE to max_wal_send

2017-01-08 Thread Greg Stark
On 8 January 2017 at 17:26, Greg Stark wrote: > On 5 January 2017 at 19:01, Andres Freund wrote: >> That's a bit odd - shouldn't the OS network stack take care of this in >> both cases? I mean either is too big for TCP packets (including jumbo >> frames).

Re: [HACKERS] [PATCH] guc-ify the formerly hard-coded MAX_SEND_SIZE to max_wal_send

2017-01-08 Thread Greg Stark
The first 128k goes out, then the rest queues up until the first ack comes back. Then the next 128kB goes out again without waiting... I think this is what Nagle is supposed to actually address but either it may be off by default these days or our usage pattern may be defeating it in some way. --

Re: [HACKERS] RustgreSQL

2017-01-08 Thread Greg Sabino Mullane
orting effort to any language would be immense. C++ would be the least painful option, probably. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201701080905 http://biglumber.com/x/web?pk=2529DF6AB8F7940

Re: [HACKERS] pg_stat_activity.waiting_start

2017-01-07 Thread Greg Stark
w how long. Perhaps not precisely for individual waits but in aggregate we need the totals to be right so as long as the measurements are accurate that would be sufficient. So an accurate but imprecise measurement +/- 10ms with low overhead is better than a precise measurement with high overhead.

Re: [HACKERS] Cluster wide option to control symbol case folding

2017-01-02 Thread Greg Stark
think things are currently organized that way. If you're only concerned with server-side functions it wouldn't be hard to have a specific pl language that was case sensitive though it might be tricky to do to pl/pgsql due to the way pl/pgsql depends on the sql parser. -- greg --

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-27 Thread Greg Stark
On Dec 24, 2016 5:44 PM, "Tom Lane" wrote: I think we'd need at least an order of magnitude cheaper to consider putting timing calls into spinlock or lwlock paths, and that's just not available at all, let alone portably. For spinlocks we could conceivably just bite the bullet and use a raw rdt

Re: [HACKERS] gettimeofday is at the end of its usefulness?

2016-12-26 Thread Greg Stark
On Dec 26, 2016 10:35 PM, "Tom Lane" wrote: So it seems like the configure support we'd need is to detect whether clock_gettime is available (note on Linux there's also a library requirement, -lrt), and we would also need a way to provide a platform-specific choice of clockid; we at least need e

Re: [HACKERS] Declarative partitioning - another take

2016-12-16 Thread Greg Stark
Just poking around with partitioning. I notice that "\d parent" doesn't list all the partitions, suggesting to use \d+ but a plain "\d" does indeed list the partitions. That seems a bit strange and also probably impractical if you have hundreds or thousands of partitions. Has this come up in previo

Re: [HACKERS] jsonb problematic operators

2016-12-12 Thread Greg Stark
or will miss the alternate spelling. There may even be security implications for that though to be honest that seems unlikely in this case. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Separate connection handling from backends

2016-12-06 Thread Greg Stark
be scans of the list of active transactions but never of all connections whether they're in a transaction or not. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Mail thread references in commits

2016-11-30 Thread Greg Stark
RLs with dirty words in them. Or with words like "terrorist" in them. Or with some javascript/html injection attack of some sort... -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UNDO and in-place update

2016-11-25 Thread Greg Stark
in the same database at the same time. I'm a bit confused because I distinctly remembered an UNDO log back in the 8i days as well but apparently that's just me imagining things. UNDO segments were introduced in 9i. This explained a bunch http://satya-dba.blogspot.ie/2009/09/undo-tabl

Re: [HACKERS] UNDO and in-place update

2016-11-24 Thread Greg Stark
;ve discussed before of having a "most frequent xmin" in the header and then a bit indicating the xmin is missing from the row header. Except in their case they don't need it for the actual xmin/xmax because their visibility is done per-block, only the transient lock state -- greg

[HACKERS] Re: Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-15 Thread Greg Stark
e might be able to maintain it by editing the table using SQL updates and/or other tools without having to teach them a particular input format. The trick would then be to have a preprocessing step in the build which loaded the CSV/TSV files into hash tables and replaced all the strings or other t

Re: [HACKERS] Physical append-only tables

2016-11-14 Thread Greg Stark
tering a bit better using BRIN" is a bad idea. It's just the bit about turning a table append-only to deal with update-once data that I think is overreach. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] JIT compiler for expressions

2016-10-30 Thread Greg Stark
This sounds amazing. My only comment is that LLVM 3.7 is kind of old in the accelerated world of LLVM. If you have patches to LLVM you need you won't have much success submitting them as patches on 3.7. The current stable release is 3.9 and the development snapshots are 4.0. I know LLVM moves qui

Re: [HACKERS] emergency outage requiring database restart

2016-10-27 Thread Greg Stark
On Thu, Oct 27, 2016 at 9:53 PM, Merlin Moncure wrote: > I think we can rule out faulty storage Nobody ever expects the faulty storage -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] pg_hba_file_settings view patch

2016-10-26 Thread Greg Stark
ly. Tools like pgadmin which want to use the view could check for such users and display a warning or error rather than inaccurate information. If there's any support for my recommendation I'm still happy to pick up the patch again and commit it. -- greg -- Sent via pgsql-hackers maili

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-10-22 Thread Greg Stark
ldn't easily be put into a script which I would have said was desirable -- except I suspect there are situations where Postgres database scripts need to do a resetxlog. I'm not sure I can think of any examples offhand but I wouldn't be too surprised. -- greg -- Sent via pgsq

Re: [HACKERS] LLVM Address Sanitizer (ASAN) and valgrind support

2016-10-20 Thread Greg Stark
On Oct 20, 2016 5:27 PM, "Noah Misch" wrote: > > On Wed, Oct 19, 2016 at 11:08:39AM +0100, Greg Stark wrote: > > > The MEMPOOL_FREE doesn't take any size argument and mcxt.c doesn't > > have convenient access to a size argument. It could call the > >

Re: [HACKERS] [COMMITTERS] packing/alignment annotation for ItemPointerData redux

2016-10-19 Thread Greg Stark
Sorry -- with the obvious error fixed: $ /usr/bin/clang-4.0 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wall -c clang-bug.c clang-bug.c:55:9: w

Re: [HACKERS] [COMMITTERS] packing/alignment annotation for ItemPointerData redux

2016-10-19 Thread Greg Stark
Ah. Here we go: $ /usr/bin/clang-4.0 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wall -c clang-bug.c clang-bug.c:54:9: error: use of undeclared

Re: [HACKERS] [COMMITTERS] packing/alignment annotation for ItemPointerData redux

2016-10-19 Thread Greg Stark
able to reproduce them with a minimal test case yet. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] packing/alignment annotation for ItemPointerData redux

2016-10-19 Thread Greg Stark
^~~ Which seems to indicate that clang may not understand the "pg_attribute_aligned(2)" or perhaps it does and just doesn't take it into account when generating these warnings. I'm sure there are other people testing clang -- isn't it the default on M

Re: [HACKERS] LLVM Address Sanitizer (ASAN) and valgrind support

2016-10-19 Thread Greg Stark
t sure what the net effect is. The MEMPOOL_FREE doesn't take any size argument and mcxt.c doesn't have convenient access to a size argument. It could call the GetChunkSpace method but that will include the allocation overhead and in any case isn't this memory already marked noaccess b

Re: [HACKERS] pg_dump getBlobs query broken for 7.3 servers

2016-10-12 Thread Greg Stark
On Mon, Oct 10, 2016 at 9:52 PM, Greg Stark wrote: > > The code is here: > > https://github.com/gsstark/retropg > > The build script is called "makeall" and it applies patches from the > "old-postgres-fixes" directory though some of the smarts are in the &

Re: [HACKERS] pg_dump getBlobs query broken for 7.3 servers

2016-10-10 Thread Greg Stark
On Mon, Oct 10, 2016 at 3:36 AM, Jim Nasby wrote: > FWIW, Greg Stark did a talk at PG Open about PG performance going back to at > least 7.4. He did discuss what he had to do to get those versions to compile > on modern tools, and has a set of patches that enable it. Unfortunately his

Re: [HACKERS] Is it time to kill support for very old servers?

2016-10-07 Thread Greg Stark
e to run a modern psql against old servers so you can run a benchmark script. For another there may be binary-only applications or drivers out there that are using V2 for whatever reason. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Greg Stark
n > before. I have a machine sitting idle now too if you have specific ideas of what to benchmark. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-10-01 Thread Greg Stark
years ago but we didn't and if Amit's patch makes hash indexes recoverable today then go for it. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Order of operations in SubPostmasterMain()

2016-09-29 Thread Greg Stark
if it hasn't been called yet. I'm sure there are problems with doing that in general but for the specific errors that can happen before pq_init it might be feasible since they obviously can't have very much shared state yet to have corrupted. -- greg -- Sent via pgsql-hackers mai

Re: [HACKERS] LLVM Address Sanitizer (ASAN) and valgrind support

2016-09-28 Thread Greg Stark
d initialized memory. But I was unable to test it because msan didn't work for me at all. This seems to be the way of things with llvm. It's great stuff but there's always 10% that is broken because there's some cool new thing that's better. -- greg -- Sent via pgsql-ha

Re: [HACKERS] LLVM Address Sanitizer (ASAN) and valgrind support

2016-09-27 Thread Greg Stark
Hm, the danger there is once I start refactoring things I could get bogged down... I would love to remove all the #ifdef's and have the macros just be no-ops if they're compiled out for example... -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] Hash Indexes

2016-09-24 Thread Greg Stark
o unlearn a million instructions warning not to use this feature and c) The fear of breaking existing users use cases and databases would be less and pg_upgrade would be an ignorable problem at least until the day comes for the big cutover of the default to the new opclass. -- greg -- Sent via

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-09 Thread Greg Stark
nywhere else where there are unintentional dependencies. I haven't followed the thread closely but I'm puzzled why you would need to use bit twiddling to set a floating point number. Isn't there a perfectly good way to calculate the value you want using ldexp() and other standard C librar

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-09 Thread Greg Stark
reduce the random i/o from switching tapes. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-07 Thread Greg Stark
suggested. This is just a large unsorted list that we need to iterate throught. Just allocate chunks of a few megabytes and when it's full allocate a new chunk and keep going. There's no need to get tricky with estimates and resizing and whatever. -- greg -- Sent via pgsql-hackers m

Re: [HACKERS] pg_sequence catalog

2016-09-04 Thread Greg Stark
ame page, parse the same page header, follow nearby line pointers...? I'm not sure how effective all that caching is today but it doesn't seem impossible to imagine getting that all optimized away. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] autonomous transactions

2016-09-03 Thread Greg Stark
ver. Autonomous transactions will certainly need to be read-write so the question then is what problems led to the restriction in parallel query and are they any more tractable with autonomous transactions? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Greg Stark
ucture for parallel query so there's at least some shared problem space there. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Greg Stark
rs during an autonomous transaction? What happens if you use a pl language in the autonomous transaction and if it tries to use non-transactional information such as prepared statements? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subs

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

2016-08-30 Thread Greg Stark
endency on the code structure in some far-away scan. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix some corner cases that cube_in rejects

2016-08-29 Thread Greg Stark
suspect they mostly become valid since float8in will handle NaN and the like. Incidentally, I so wish this module were named "vector" instead of cube. I don't think I was confused by it for ages and still find it confuses me for a few moments before I remember what it does. -- greg

[HACKERS] Re: [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-23 Thread Greg Stark
ce shows that separating DDL and making it static while the DML is dynamic is just a better design. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Are these supported??

2016-08-17 Thread Greg Stark
On Wed, Aug 17, 2016 at 4:22 PM, Robert Haas wrote: > We've supported having joins in a DELETE since PostgreSQL 8.1. Er, yes. Though he does say he's trying to use the same syntax as select... -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] Are these supported??

2016-08-17 Thread Greg Stark
;t want the extra "timestamp" keyword before the column there -- in the examples that's part of the literal being used to have it be read as a timestamp. https://www.postgresql.org/docs/8.4/static/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT -- greg -- Sent via pgsql-hackers

Re: [HACKERS] Why we lost Uber as a user

2016-08-17 Thread Greg Stark
ve the same use cases as physical backups. They both provide you with exactly what you had prior to the recovery. No more or less. That's what you want when recovering from a disaster. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] bigint txids vs 'xid' type, new txid_recent(bigint) => xid

2016-08-16 Thread Greg Stark
ldn't be a problem for any real production api. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   3   4   5   6   7   8   9   10   >