Re: [HACKERS] FDW and parallel execution

2017-04-11 Thread PostgreSQL - Hans-Jürgen Schönig
4718.76..294718.76 > rows=2284376 width=8) > -> Foreign Scan on orders_fdw > (cost=0.00..294718.76 rows=2284376 width=8) > -> Hash (cost=32605.64..32605.64 rows=1500032 > width=8) >->

Re: [HACKERS] remove checkpoint_warning

2016-07-12 Thread PostgreSQL - Hans-Jürgen Schönig
is about to go up in flames, which is of course not true. maybe we could do "consider increasing to ensure good performance" or so ... regards, hans -- Hans-Jürgen Schönig Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgr

Re: [HACKERS] WIP: Data at rest encryption

2016-06-15 Thread PostgreSQL - Hans-Jürgen Schönig
but this requires a value as low as 32 MB of shared buffers or so. many thanks, hans -- Hans-Jürgen Schönig Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt Web: http://www.postgresql-support.de, http://www.cybertec.at -- Sent via pgsql-hackers mailing l

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread PostgreSQL - Hans-Jürgen Schönig
> On 18 Aug 2015, at 11:19, Albe Laurenz wrote: > > Hans-Jürgen Schönig wrote: >> in addition to that you have the “problem” of transactions. if you failover >> in the middle >> of a transaction, strange things might happen from the application point of >> vie

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread PostgreSQL - Hans-Jürgen Schönig
> On 18 Aug 2015, at 10:32, Albe Laurenz wrote: > > Victor Wagner wrote: >> Rationale >> = >> >> Since introduction of the WAL-based replication into the PostgreSQL, it is >> possible to create high-availability and load-balancing clusters. >> >> However, there is no support for failov

Re: [HACKERS] Priority table or Cache table

2014-05-16 Thread Hans-Jürgen Schönig
On 20 Feb 2014, at 01:38, Tom Lane wrote: > Haribabu Kommi writes: >> I want to propose a new feature called "priority table" or "cache table". >> This is same as regular table except the pages of these tables are having >> high priority than normal tables. These tables are very useful, where a

Re: [HACKERS] Small GIN optimizations (after 9.4)

2014-02-06 Thread PostgreSQL - Hans-Jürgen Schönig
i think there is one more thing which would be really good in GIN and which would solve a ton of issues. atm GIN entries are sorted by item pointer. if we could sort them by a "column" it would fix a couple of real work issues such as ... SELECT ... FROM foo WHERE "tsearch_query" ORDER B

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Hans-Jürgen Schönig
On Jan 8, 2014, at 9:27 PM, Bruce Momjian wrote: > On Wed, Jan 8, 2014 at 05:39:23PM +, Simon Riggs wrote: >> On 8 January 2014 09:07, Heikki Linnakangas wrote: >> >>> I'm going to say right off the bat that I think the whole notion to >>> automatically disable synchronous replication when

Re: [HACKERS] Backup throttling

2013-08-21 Thread PostgreSQL - Hans-Jürgen Schönig
On Aug 21, 2013, at 10:57 AM, Andres Freund wrote: > On 2013-08-21 08:10:42 +0200, PostgreSQL - Hans-Jürgen Schönig wrote: >> >> On Aug 19, 2013, at 9:11 PM, Andres Freund wrote: >> >>> On 2013-08-19 20:15:51 +0200, Boszormenyi Zoltan wrote: >>>> 2

Re: [HACKERS] Backup throttling

2013-08-20 Thread PostgreSQL - Hans-Jürgen Schönig
was just disk wait caused by a full speed pg_basebackup. regarding the client side implementation: we have chosen this way because it is less invasive. i cannot see a reason to do this on the server side because we won't have 10 pg_basebackup-style tools making use of this feature anywa

[HACKERS] "Bloom filter" for 9.2 ...

2012-07-03 Thread Hans-Jürgen Schönig
hello, some time ago oleg and teodor have posted a PostgreSQL version of bloom filters. as this appears to be a useful thing for many people i have ported this prototype to PostgreSQL 9.2. it seems to work as expected on OS X and Linux. as it is a contrib module it lacks xlog support. maybe some

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Hans-Jürgen Schönig
On Jul 1, 2012, at 5:44 PM, Magnus Hagander wrote: > On Sun, Jul 1, 2012 at 1:02 PM, Boszormenyi Zoltan wrote: >> Hi, >> >> attached is a patch that does $SUBJECT. >> >> It's a usability enhancement, to take a backup, write >> a minimalistic recovery.conf and start the streaming >> standby in o

Re: [HACKERS] Combine non-recursive and recursive CTEs?

2012-06-15 Thread PostgreSQL - Hans-Jürgen Schönig
On Jun 16, 2012, at 8:27 AM, Magnus Hagander wrote: > I'm not sure if this is something I don't know how to do, or if it's > something we simply can't do, or if it's something we could do but the > syntax can't handle :-) > > Basically, I'd like to combine a recursive and a non-recursive CTE in >

Re: [HACKERS] Getting rid of cheap-startup-cost paths earlier

2012-05-22 Thread PostgreSQL - Hans-Jürgen Schönig
On May 22, 2012, at 9:57 AM, Simon Riggs wrote: > On 22 May 2012 06:50, Tom Lane wrote: > >> Currently, the planner keeps paths that appear to win on the grounds of >> either cheapest startup cost or cheapest total cost. It suddenly struck >> me that in many simple cases (viz, those with no LI

Re: [HACKERS] Odd out of memory problem.

2012-03-26 Thread Hans-Jürgen Schönig
hello, does the problem show up on 2% of all problems after 2 weeks or so? we had a similar problem on UNIX as well. it even materialized on 100 identical boxes (on 2% of them). it pops up randomly and never stops … i checked some code paths. some of those messages are direct output via stderr (

Re: [HACKERS] pg_prewarm

2012-03-10 Thread Hans-Jürgen Schönig
On Mar 9, 2012, at 2:34 PM, Robert Haas wrote: > On Fri, Mar 9, 2012 at 5:42 AM, Hans-Jürgen Schönig > wrote: >> we had some different idea here in the past: what if we had a procedure / >> method to allow people to save the list of current buffers / cached blocks >&g

Re: [HACKERS] pg_prewarm

2012-03-09 Thread Hans-Jürgen Schönig
we had some different idea here in the past: what if we had a procedure / method to allow people to save the list of current buffers / cached blocks to be written to disk (sorted). we could then reload this "cache profile" on startup in the background or people could load a certain cache content

Re: [HACKERS] index-only scans

2011-10-11 Thread PostgreSQL - Hans-Jürgen Schönig
On Oct 7, 2011, at 8:47 PM, Joshua D. Drake wrote: > > On 10/07/2011 11:40 AM, Tom Lane wrote: >> Robert Haas writes: >>> Please find attached a patch implementing a basic version of >>> index-only scans. >> >> I'm making some progress with this, but I notice what seems like a >> missing featur

Re: [HACKERS] What is known about PostgreSQL HP-UX support?

2011-10-10 Thread PostgreSQL - Hans-Jürgen Schönig
On Oct 10, 2011, at 4:21 AM, Alex Goncharov wrote: > [ Thanks all for the very productive discussion in the thread > "libpq, PQdescribePrepared -> PQftype, PQfmod, no PQnullable" > which I originated. Very useful. Now on something different. ] > > About two years ago, I had to research some P

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 19, 2011, at 5:16 PM, Tom Lane wrote: > Greg Stark writes: >> That said, to help in the case I described you would have to implement >> the tapesort algorithm on the GPU as well. > > I think the real problem would be that we are seldom sorting just the > key values. If you have to push

Re: [HACKERS] help with plug-in function for additional (partition/shard) visibility checks

2011-09-02 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 2, 2011, at 2:59 PM, Hannu Krosing wrote: > On Fri, 2011-09-02 at 14:51 +0200, PostgreSQL - Hans-Jürgen Schönig > wrote: >> hello … >> >> the goal of the entire proxy thing is to make the right query go to the >> right node / nodes. >> we determine t

Re: [HACKERS] help with plug-in function for additional (partition/shard) visibility checks

2011-09-02 Thread PostgreSQL - Hans-Jürgen Schönig
the proxy what to append - and: this "what" would be under your full control. what do you think? i got to think about it futher but i can envision that this could be feasible ... hans On Sep 2, 2011, at 2:36 PM, Hannu Krosing wrote: > On Fri, 2011-09-02 at 14:01 +020

Re: [HACKERS] help with plug-in function for additional (partition/shard) visibility checks

2011-09-02 Thread PostgreSQL - Hans-Jürgen Schönig
hello … i have been thinking about this issue for quite a while ... given your idea i am not sure how this can work at all. consider: begin; insert 1 insert 2 commit assume this ends up in the same node, now you split it into two … 1 and 2 will have exactly the sa

Re: [HACKERS] Caching Python modules

2011-08-17 Thread PostgreSQL - Hans-Jürgen Schönig
On Aug 17, 2011, at 2:19 PM, Jan Urbański wrote: > On 17/08/11 14:09, PostgreSQL - Hans-Jürgen Schönig wrote: >> CREATE OR REPLACE FUNCTION textprocess.add_to_corpus(lang text, t text) >> RETURNS float4 AS $$ >> >>from SecondCorpus import SecondCorpus &g

[HACKERS] Caching Python modules

2011-08-17 Thread PostgreSQL - Hans-Jürgen Schönig
hello … i have just fallen over a nasty problem (maybe missing feature) with PL/Pythonu … consider: -- add a document to the corpus CREATE OR REPLACE FUNCTION textprocess.add_to_corpus(lang text, t text) RETURNS float4 AS $$ from SecondCorpus import SecondCorpus from SecondDocu

Re: [HACKERS] synchronized snapshots

2011-08-15 Thread PostgreSQL - Hans-Jürgen Schönig
On Aug 15, 2011, at 9:40 AM, Simon Riggs wrote: > On Mon, Aug 15, 2011 at 2:31 AM, Joachim Wieland wrote: > >> In short, this is how it works: >> >> SELECT pg_export_snapshot(); >> pg_export_snapshot >> >> 03A1-1 >> (1 row) >> >> >> (and then in a different session

Re: [HACKERS] index-only scans

2011-08-12 Thread PostgreSQL - Hans-Jürgen Schönig
On Aug 12, 2011, at 10:03 PM, Heikki Linnakangas wrote: > On 11.08.2011 23:06, Robert Haas wrote: >> Comments, testing, review appreciated... > > I would've expected this to use an index-only scan: > > postgres=# CREATE TABLE foo AS SELECT generate_series(1,10) AS id; > SELECT 10 > postg

Re: [HACKERS] Will switchover still need a checkpoint in 9.1 SR Hot Standby

2011-08-07 Thread PostgreSQL - Hans-Jürgen Schönig
On Aug 7, 2011, at 11:01 AM, Simon Riggs wrote: > On Sun, Aug 7, 2011 at 8:57 AM, Hannu Krosing wrote: >> In 9.0 (as in earlier versions) a former standby host has to do a full >> checkpoint before becoming available as an independent database instance >> in either switchover or failover scenari

Re: [HACKERS] SYNONYMS (again)

2011-06-23 Thread PostgreSQL - Hans-Jürgen Schönig
On Jun 23, 2011, at 12:52 AM, Alvaro Herrera wrote: > Excerpts from Joshua D. Drake's message of mié jun 22 15:37:17 -0400 2011: >> Per: >> >> http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php >> >> It seems we did come up with a use case in the procpid discussion. The >> abili

Re: [HACKERS] SYNONYMS (again)

2011-06-23 Thread PostgreSQL - Hans-Jürgen Schönig
On Jun 23, 2011, at 12:52 AM, Alvaro Herrera wrote: > Excerpts from Joshua D. Drake's message of mié jun 22 15:37:17 -0400 2011: >> Per: >> >> http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php >> >> It seems we did come up with a use case in the procpid discussion. The >> abili

Re: [HACKERS] bad posix_fadvise support causes initdb to exit ungracefully

2011-06-15 Thread PostgreSQL - Hans-Jürgen Schönig
hello ... 2.4? we know that some versions of 2.4 cause problems due to broken posix_fadvise. if i remember correctly we built some configure magic into PostgreSQL to check for this bug. what does this check do? many thanks, hans On Jun 15, 2011, at 6:12 PM, Merlin Mo

Re: [HACKERS] workaround for expensive KNN?

2011-04-11 Thread PostgreSQL - Hans-Jürgen Schönig
price has a problem :(. "iphone" can be a 20 cents bag or a sticker or a 900 euro thing signed by whoever ... so, words and the sort-number / price are not related in anyway. price is in this case no way to narrow down the problem (e.g. evaluate first or so). many thanks,

Re: [HACKERS] workaround for expensive KNN?

2011-04-08 Thread PostgreSQL - Hans-Jürgen Schönig
hello ... i got that one ... "idx_product_t_product_titleprice" gist (to_tsvector('german'::regconfig, title), int_price) so, i have a combined index on text + number. to me the plan seems fine ... it looks like a prober KNN traversal. the difference between my plan and your plan seems to b

[HACKERS] workaround for expensive KNN?

2011-04-08 Thread PostgreSQL - Hans-Jürgen Schönig
hello all ... given oleg's posting before i also wanted to fire up some KNN related question. let us consider a simple example. i got some million lines and i want all rows matching a tsquery sorted by price. i did some tests: test=# explain (analyze true, buffers true, costs true) SELECT id FRO

Re: [HACKERS] k-neighbourhood search in databases

2011-04-08 Thread PostgreSQL - Hans-Jürgen Schönig
hello ... i have put some research into that some time ago and as far as i have seen there is a 99% chance that no other database can do it the way we do it. it seems nobody comes even close to it (especially not in the flexibility-arena). oracle: disgusting workaround ... http://www.orafaq.com

Re: [HACKERS] WIP: cross column correlation ...

2011-02-26 Thread PostgreSQL - Hans-Jürgen Schönig
>>> >> >> Still, having more data a user can probe would be nice. >> >> I wonder why everyone avoids Microsoft's approach to the subject. >> Apparently, they go in the 'auto-tune as much as possible' direction. >> And tests we did a while ago, involving asking team from Microsoft and a >> team

Re: [HACKERS] WIP: cross column correlation ...

2011-02-23 Thread PostgreSQL - Hans-Jürgen Schönig
On Feb 24, 2011, at 2:09 AM, Josh Berkus wrote: > >> Personally, I think the first thing we ought to do is add a real, bona >> fide planner hint to override the selectivity calculation manually, >> maybe something like this: >> >> WHERE (x < 5 AND y = 1) SELECTIVITY (0.1); >> >> Then, having pr

Re: [HACKERS] WIP: cross column correlation ...

2011-02-23 Thread PostgreSQL - Hans-Jürgen Schönig
On Feb 23, 2011, at 3:46 PM, Robert Haas wrote: > On Wed, Feb 23, 2011 at 12:50 AM, Nathan Boley wrote: >>> Personally, I think the first thing we ought to do is add a real, bona >>> fide planner hint to override the selectivity calculation manually, >>> maybe something like this: >>> >>> WHERE

Re: [HACKERS] WIP: cross column correlation ...

2011-02-23 Thread PostgreSQL - Hans-Jürgen Schönig
>>> >> >> cheapest and easiest solution if you run into this: add "fake" functions >> which the planner cannot estimate properly. >> use OR to artificially prop up estimates or use AND to artificially lower >> them. there is actually no need to redesign the schema to get around it but >> it is

Re: [HACKERS] WIP: cross column correlation ...

2011-02-23 Thread PostgreSQL - Hans-Jürgen Schönig
> Those are real problems, but I still want it. The last time I hit > this problem I spent two days redesigning my schema and adding > triggers all over the place to make things work. If I had been > dealing with a 30TB database instead of a 300MB database I would have > been royally up a creek.

Re: [HACKERS] WIP: cross column correlation ...

2011-02-23 Thread PostgreSQL - Hans-Jürgen Schönig
On Feb 23, 2011, at 2:58 AM, Robert Haas wrote: > 2011/2/22 PostgreSQL - Hans-Jürgen Schönig : >> how does it work? we try to find suitable statistics for an arbitrary length >> list of conditions so that the planner can use it directly rather than >> multiplying all th

[HACKERS] WIP: cross column correlation ...

2011-02-22 Thread PostgreSQL - Hans-Jürgen Schönig
hello everbody, we have spent some time in finally attacking cross column correlation. as this is an issue which keeps bugging us for a couple of applications (some years). this is a WIP patch which can do: special cross column correlation specific syntax: CREAT

[HACKERS] SELECT ... WHERE fti_query ORDER BY numeric_col LIMIT x - problem

2011-02-04 Thread PostgreSQL - Hans-Jürgen Schönig
I have toyed around with KNN a little and I am pretty impressed when it comes to the results we have seen in the GIS world. Given the infrastructure we have at the moment I wonder if KNN can help to speedup queries like that: SELECT ... WHERE fti_query ORDER BY numeric_col LIMIT x The use case

Re: [HACKERS] ugly locking corner cases ...

2010-10-04 Thread Hans-Jürgen Schönig
On Oct 4, 2010, at 1:23 PM, Heikki Linnakangas wrote: > On 04.10.2010 14:02, Hans-Jürgen Schönig wrote: >> it seems we have found a fairly nasty problem. >> imagine a long transaction which piles up XX.XXX of locks (count on >> pg_locks) inside the same transaction by doing

[HACKERS] ugly locking corner cases ...

2010-10-04 Thread Hans-Jürgen Schönig
hello all ... it seems we have found a fairly nasty problem. imagine a long transaction which piles up XX.XXX of locks (count on pg_locks) inside the same transaction by doing some tasty savepoints, with hold cursors and so on. in this case we see that a normal count issued in a second database

Re: [HACKERS] Parallel Query Execution Project

2010-09-28 Thread Hans-Jürgen Schönig
On Sep 28, 2010, at 10:15 AM, Markus Wanner wrote: > Hi, > > On 09/28/2010 07:24 AM, Li Jie wrote: >> I'm interested in this parallel project, >> http://wiki.postgresql.org/wiki/Parallel_Query_Execution >> >> But I can't find any discussion and current progress in the website, it >> seems to sto

Re: [HACKERS] Path question

2010-09-23 Thread Hans-Jürgen Schönig
On Sep 23, 2010, at 3:29 PM, Robert Haas wrote: > On Tue, Sep 21, 2010 at 12:29 AM, David Fetter wrote: >> On Mon, Sep 20, 2010 at 10:57:00PM -0400, Robert Haas wrote: >>> 2010/9/3 Hans-Jürgen Schönig : >>>> On Sep 2, 2010, at 1:20 AM, Robert Haas wrote: >>&g

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-08 Thread Hans-Jürgen Schönig
On Sep 8, 2010, at 4:57 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> Neat. Have you checked what effect this has on memory consumption? >> >> Also, don't forget to add it to >> https://commitfest.postgresql.org/action/commitfest_view/open > > Would be good to have

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-08 Thread Hans-Jürgen Schönig
function call goes down to basically zero. i hope this is something which is useful to some folks out there. many thanks, hans canon-pathkeys-as-rbtree-3-ctxdiff.patch Description: Binary data On Sep 8, 2010, at 4:18 PM, Stephen Frost wrote: > * Hans-Jürgen Schö

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-08 Thread Hans-Jürgen Schönig
Sep 8, 2010, at 3:54 PM, Robert Haas wrote: > On Tue, Sep 7, 2010 at 2:14 PM, Boszormenyi Zoltan wrote: >> Hi, >> >> Robert Haas írta: >>> 2010/9/3 PostgreSQL - Hans-Jürgen Schönig : >>> >>>> i tried this one with 5000 unindexed tables (just one

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 3, 2010, at 4:40 PM, Tom Lane wrote: > =?iso-8859-1?Q?PostgreSQL_-_Hans-J=FCrgen_Sch=F6nig?= > writes: >> imagine a system with, say, 1000 partitions (heavily indexed) or so. the >> time taken by the planner is already fairly heavy in this case. > > As the fine manual points out, the c

Re: [HACKERS] Path question

2010-09-03 Thread Hans-Jürgen Schönig
On Sep 2, 2010, at 1:20 AM, Robert Haas wrote: > On Sep 1, 2010, at 10:21 AM, Greg Stark wrote: >> For what it's worth I disagree with Tom. I think this is a situation >> where we need *both* types of solution. Ideally we will be able to use >> a plain Append node for cases where we know the rel

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 3, 2010, at 2:04 PM, Stephen Frost wrote: > * PostgreSQL - Hans-Jürgen Schönig (postg...@cybertec.at) wrote: >> did anybody think of a solution to this problem. >> or more precisely: can there be a solution to this problem? > > Please post to the correct list (-per

[HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread PostgreSQL - Hans-Jürgen Schönig
hello everybody, we came across an issue which turned out to be more serious than previously expected. imagine a system with, say, 1000 partitions (heavily indexed) or so. the time taken by the planner is already fairly heavy in this case. i tried this one with 5000 unindexed tables (just one c

Re: [HACKERS] Path question

2010-09-01 Thread PostgreSQL - Hans-Jürgen Schönig
hello tom, yeah, we have followed quite a lot of discussion as well ... and yes, no patches. as far as this problem is concerned: we are working on a patch and did some prototyping inside the planner already (attached). the code we have is pretty limited atm (such as checking for a sort clause

Re: [HACKERS] Path question

2010-09-01 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 1, 2010, at 4:10 PM, Tom Lane wrote: > Boszormenyi Zoltan writes: >> we are experimenting with modifying table partitioning >> so the ORDER BY clause can be pushed down to >> child nodes on the grounds that: > > This is really premature, and anything you do along those lines now will > pr

Re: [HACKERS] How to construct an exact plan

2010-08-31 Thread PostgreSQL - Hans-Jürgen Schönig
hello ... here is the ultimate revelation of planner hints in postgres ... let us praise oleg and teodor for solving a practical problem for practical people ... http://www.sai.msu.su/~megera/wiki/plantuner try this one ... it is excellent and definitely helpful for many many people out there.

Re: [HACKERS] "micro bucket sort" ...

2010-08-11 Thread PostgreSQL - Hans-Jürgen Schönig
as tom pointed out - this is not possible. there is no limit 20 in my case - i just used it to indicate that limiting does not make the index scan possible which it does in some other cases. the partial sort thing simon pointed out is what is needed at this point. many thanks,

[HACKERS] "micro bucket sort" ...

2010-08-11 Thread Hans-Jürgen Schönig
hello all ... i am bugged with a small issue which is basically like this ... test=# create table t_test as select x, x % 5 as y from generate_series(1, 100) AS x; SELECT test=# create index idx_a on t_test (x) ; CREATE INDEX test=# ANALYZE ; ANALYZE test=# explain analyze select * from

Re: [HACKERS] non-overlapping, consecutive partitions

2010-07-29 Thread Hans-Jürgen Schönig
to solve it no matter what it takes. many thanks, hans On Jul 26, 2010, at 1:14 AM, Robert Haas wrote: > On Sun, Jul 25, 2010 at 6:40 PM, Greg Stark wrote: >> 2010/7/25 Robert Haas : >>> 2010/7/25 PostgreSQL - Hans-Jürgen Schönig : >>>>

Re: [HACKERS] non-overlapping, consecutive partitions

2010-07-25 Thread PostgreSQL - Hans-Jürgen Schönig
On Jul 25, 2010, at 11:56 AM, Martijn van Oosterhout wrote: > On Fri, Jul 23, 2010 at 10:04:00PM +0200, Hans-Jürgen Schönig wrote: >> create table foo ( x date ); >> create table foo_2010 () INHERITS (foo) >> create table foo_2009 () INHERITS (foo) >>

[HACKERS] non-overlapping, consecutive partitions

2010-07-23 Thread Hans-Jürgen Schönig
hello everybody, i have just come across some issue which has been bugging me for a while. consider: SELECT * FROM foo ORDER BY bar; if we have an index on bar, we can nicely optimize away the sort step by consulting the index - a btree will return sorted output. under normal circumstan

Re: [HACKERS] SHOW TABLES

2010-07-16 Thread Hans-Jürgen Schönig
On Jul 15, 2010, at 6:43 PM, Magnus Hagander wrote: > On Thu, Jul 15, 2010 at 18:35, Simon Riggs wrote: >> On Thu, 2010-07-15 at 17:38 +0200, Magnus Hagander wrote: >> >>> Is there an actual common use-case for having these commands available >>> for *non-psql* interfaces? >> >> There are many

Re: [HACKERS] SHOW TABLES

2010-07-15 Thread Hans-Jürgen Schönig
On Jul 15, 2010, at 6:20 PM, Thom Brown wrote: > On 15 July 2010 17:16, Marc G. Fournier wrote: >> On Thu, 15 Jul 2010, Thom Brown wrote: >> >>> On 15 July 2010 17:07, Marc G. Fournier wrote: On Thu, 15 Jul 2010, Thom Brown wrote: > If it's only a psql problem, why impleme

Re: [HACKERS] SHOW TABLES

2010-07-15 Thread Hans-Jürgen Schönig
On Jul 15, 2010, at 5:20 PM, Simon Riggs wrote: > On Thu, 2010-07-15 at 11:05 -0400, Tom Lane wrote: >> Simon Riggs writes: >>> The biggest turn off that most people experience when using PostgreSQL >>> is that psql does not support memorable commands. >> >>> I would like to implement the follow

Re: [HACKERS] cross column correlation revisted

2010-07-15 Thread Hans-Jürgen Schönig
hello ... a view is already nice but i think it is still too narrow. the problem is: you don't want a view for every potential join. in addition to that - ideally there is not much left of a view when it comes to checking for costs. so, i think, this is not the kind of approach leading to total

Re: [HACKERS] cross column correlation revisted

2010-07-14 Thread PostgreSQL - Hans-Jürgen Schönig
hello ... look at the syntax i posted in more detail: >> ALTER TABLE x SET CORRELATION STATISTICS FOR (x.id = y.id AND x.id2 = >> y.id2) > it says X and Y ... the selectivity of joins are what i am most interested in. cross correlation of columns within the same table are just a byprod

Re: [HACKERS] cross column correlation revisted

2010-07-14 Thread PostgreSQL - Hans-Jürgen Schönig
a command here. many thanks, hans On Jul 14, 2010, at 4:01 PM, Tom Lane wrote: > Heikki Linnakangas writes: >> On 14/07/10 13:12, PostgreSQL - Hans-Jürgen Schönig wrote: >>> maybe somehow like this ... >>> ALTER TABLE x SET CORRELATION STATISTICS

Re: [HACKERS] cross column correlation revisted

2010-07-14 Thread PostgreSQL - Hans-Jürgen Schönig
On Jul 14, 2010, at 12:40 PM, Heikki Linnakangas wrote: > On 14/07/10 13:12, PostgreSQL - Hans-Jürgen Schönig wrote: >> hello everybody, >> >> we are currently facing some serious issues with cross correlation issue. >> consider: 10% of all people have breast cance

[HACKERS] cross column correlation revisted

2010-07-14 Thread PostgreSQL - Hans-Jürgen Schönig
hello everybody, we are currently facing some serious issues with cross correlation issue. consider: 10% of all people have breast cancer. we have 2 genders (50:50). if i select all the men with breast cancer, i will get basically nobody - the planner will overestimate the output. this is the com

Re: [HACKERS] ECPG FETCH readahead

2010-06-24 Thread PostgreSQL - Hans-Jürgen Schönig
On Jun 24, 2010, at 2:13 PM, Michael Meskes wrote: >> I think, yes, it does make sense. Because we are talking >> about porting a whole lot of COBOL applications. > > COBOL??? > yes, COBOL :). it is much more common than people think. it is not the first COBOL request for PostgreSQL hitting my

[HACKERS] one more index for pg_tablespace?

2010-02-15 Thread Hans-Jürgen Schönig
hello ... i have come an interesting corner case this morning and i am not sure if it is worth treating this as a bug or as just "bad luck". imagine creating a directory along with a tablespace ... hans-jurgen-schonigs-macbook:html hs$ mkdir /tmp/x hans-jurgen-schonigs-macbook:html hs$ psql te

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Hans-Jürgen Schönig
Just a side note: in addition to its use for partial replication, this might have potential for performance-prioritizing databases or tablespaces. hello ... this is an absolutely non-starter. the WAL is designed to be "hyper ordered" and hyper critical. once you fuck up order you will

Re: [HACKERS] draft RFC: concept for partial, wal-based replication

2009-11-30 Thread Hans-Jürgen Schönig
On Nov 30, 2009, at 10:32 AM, Stefan Kaltenbrunner wrote: Andres Freund wrote: On Monday 30 November 2009 03:57:11 Itagaki Takahiro wrote: Boszormenyi Zoltan wrote: we tried to discuss on a lower level what should be needed for a partial replication based on streaming replication. We need t

Re: [HACKERS] CommitFest 2009-09, two weeks on

2009-11-13 Thread Hans-Jürgen Schönig
On Nov 13, 2009, at 8:06 AM, Michael Meskes wrote: On Thu, Nov 12, 2009 at 03:07:27PM -0500, Robert Haas wrote: If you want to submit patches in a series like this one, they need to be considered standalone, I think. The Linux kernel devs work differently than us here. Zoltan broke them

Re: [HACKERS] SELECT ... FOR UPDATE [WAIT integer | NOWAIT] for 8.5

2009-05-11 Thread Hans-Jürgen Schönig
hello tom ... the reason for SELECT FOR UPDATE is very simple: this is the typical lock obtained by basically every business application if written properly (updating a product, whatever). the problem with NOWAIT basically is that if a small transaction holds a a lock for a subsecond, you wil

Re: [HACKERS] SYNONYMs revisited

2009-03-04 Thread Hans-Jürgen Schönig
Joshua Tolley wrote: Way back in this thread[1] one of the arguments against allowing some version of CREATE SYNONYM was that we couldn't create a synonym for an object in a remote database. Will the SQL/MED work make this sort of thing a possibility? I realize since it's not standard anyway, the

Re: [HACKERS] Hadoop backend?

2009-02-24 Thread Hans-Jürgen Schönig
why not just stream it in via set-returning functions and make sure that we can mark a set returning function as "STREAMABLE" or so (to prevent joins, whatever). is it the easiest way to get it right and it helps in many other cases. i think that the storage manager is definitely the wrong pla

Re: [HACKERS] Hadoop backend?

2009-02-22 Thread Hans-Jürgen Schönig
hi ... i think the easiest way to do this is to simply add a mechanism to functions which allows a function to "stream" data through. it would basically mean losing join support as you cannot "read data again" in a way which is good enough good enough for joining with the function providing

Re: [HACKERS] Multi CPU Queries - Feedback and/or suggestions wanted!

2008-10-30 Thread Hans-Jürgen Schönig
Bruce Momjian wrote: Greg Stark wrote: I couldn't get async I/O to work on Linux. That is it "worked" but performed the same as reading one block at a time. On solaris the situation is reversed. In what way is fadvise a kludge? I think he is saying AIO gives us more flexibility, b

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Hans-Jürgen Schönig
On Sep 22, 2008, at 12:02 PM, Simon Riggs wrote: On Mon, 2008-09-22 at 04:57 -0400, Greg Smith wrote: -As Greg Stark suggested, the larger the spindle count the larger the speedup, and the larger the prefetch size that might make sense. His suggestion to model the user GUC as "effective_spi

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Hans-Jürgen Schönig
On Sep 22, 2008, at 9:46 AM, Simon Riggs wrote: On Sun, 2008-09-21 at 12:05 -0400, Tom Lane wrote: ... and it goes on to point out how to force immediate space reclamation if you need that. These statements apply independently of whether any particular value is toasted or not. The reas

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Hans-Jürgen Schönig
*snip* Judging from that, the toasted table cleanup may be part of ALTER TABLE DROP COLUMN. That would only help if you were dropping the last potentially- toastable column of a table. And implementing it would require introducing weird corner cases into the tuple toaster, because it m

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-17 Thread Hans-Jürgen Schönig
this entire thing is not about cartesian products at all. it is about kicking out "expensive" queries before they even start to eat up tons of CPU. imagine a user asking for "give me all phone call in the US within the past 10 years". you could kill the guy instantly because you know that

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-03 Thread Hans-Jürgen Schönig
hello ... I still support it. Regrettably, many SQL developers introduce product joins and other unintentional errors. Why let problem queries through? i think the killer is that we don't have to wait until the query dies with a statement_timeout. it is ways more elegant to kill things be

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-02 Thread Hans-Jürgen Schönig
On Aug 2, 2008, at 8:38 PM, Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Hans-Jürgen Schönig wrote: i introduced a GUC called statement_cost_limit which can be used to error out if a statement is expected to be too expensive. You clearly have far more faith in th

[HACKERS] Mini improvement: statement_cost_limit

2008-08-02 Thread Hans-Jürgen Schönig
hello ... i picked up csaba nagy's idea and implemented a very simple yet very useful extension. i introduced a GUC called statement_cost_limit which can be used to error out if a statement is expected to be too expensive. the advantage over statement_timeout is that we are actually able to

[HACKERS] Bug with UTF-8 character

2006-05-25 Thread Hans-Jürgen Schönig
good morning, I got a bug request for the following unicode character in PostgreSQL 8.1.4: 0xedaeb8 ERROR: invalid byte sequence for encoding "UTF8": 0xedaeb8 This one seemed to work properly in PostgreSQL 8.0.3. I think the following code in postgreSQL 814 has a bug in it. File: postgresq

Re: [HACKERS] Tru64/Alpha problems

2006-04-07 Thread Hans-Jürgen Schönig
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: I'd like to know some settings that we can use that will get Tru64 cleanly through the buildfarm set. If noone offers any, I propose that we revert the getaddrinfo() test in configure and use our own on Tru64 until they do. I have

Re: [HACKERS] Proposal for SYNONYMS

2006-03-09 Thread Hans-Jürgen Schönig
Jonah H. Harris wrote: This email is a preliminary design for the implementation of synonyms in PostgreSQL. Comments and suggestions are welcomed. BACKGROUND Synonyms are database objects which can be used in place of their referenced object in SELECT, INSERT, UPDATE, and DELETE SQL state

[HACKERS] status of concurrent VACUUM patch ...

2005-12-20 Thread Hans-Jürgen Schönig
i was just wondering about the status of hannu's concurrent vacuum patch. are there any plans to integrate this or are there still improvements which have to be made? many thanks, hans-juergen schoenig ---(end of broadcast)--- TIP 3: H

Re: [HACKERS] request for enhancement of protocol

2005-11-19 Thread Hans-Jürgen Schönig
i think if the protocol is enhanced again we should also consider adding protocol level support for RESET CONNECTION. i have sent this patch some time ago but i think it is not worth to do the remaining protocol level changes (sql level support is finished) if this is the only change on the pr

[HACKERS] comments on prepared transactions ...

2005-10-06 Thread Hans-Jürgen Schönig
i had to deal with oracle in the past couple of days (*mega sigh*) i have seen a very interesting feature which would make sense for PostgreSQL users. currently we have: test=# \h PREPARE TRANSACTION Command: PREPARE TRANSACTION Description: prepare the current transaction for two-phase co

Re: [HACKERS] postgresql clustering

2005-09-30 Thread Hans-Jürgen Schönig
Luke Lonergan wrote: Dan, On 9/29/05 3:23 PM, "Daniel Duvall" <[EMAIL PROTECTED]> wrote: What about clustered filesystems? At first blush I would think the overhead of something like GFS might kill performance. Could one potentially achieve a fail-over config using multiple nodes with GFS,

Re: R: [HACKERS] feature proposal ...

2005-09-21 Thread Hans-Jürgen Schönig
Joshua D. Drake wrote: Hans-Jürgen Schönig wrote: no because a new is not a heap ... Why not use a function with a temporary table? That way you can pass a table parameter that is the temporary table with a select statement that you can populate the temp table with. Sincerely, Joshua D

Re: R: [HACKERS] feature proposal ...

2005-09-21 Thread Hans-Jürgen Schönig
originale- Da: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] conto di Hans-Jürgen Schönig Inviato: mercoledì 21 settembre 2005 15.30 A: pgsql-hackers@postgresql.org; [EMAIL PROTECTED] Oggetto: [HACKERS] feature proposal ... hackers, currently we have to hack tons of export scripts for various cust

Re: [HACKERS] feature proposal ...

2005-09-21 Thread Hans-Jürgen Schönig
Rod Taylor wrote: the problem is: COPY can write data returned by a SELECT statement to a file. our idea is to implement precisely that. example: COPY TO file_name USING some_select_statement; I have run into plenty of cases where I wanted to dump part of a structure and this could be used

[HACKERS] feature proposal ...

2005-09-21 Thread Hans-Jürgen Schönig
hackers, currently we have to hack tons of export scripts for various customers. the problem is: if tables can be exported straight forward COPY will give you all you need but when data has to be transformed while exporting things start becoming a bit more complex. usually people want to have

[HACKERS] DISTINCT vs. GROUP BY

2005-09-19 Thread Hans-Jürgen Schönig
I was wondering whether it is possible to teach the planner to handle DISTINCT in a more efficient way: em=# explain select distinct lastname from import.testtest; QUERY PLAN Uniq

Re: [HACKERS] random system table corruption ...

2005-09-11 Thread Hans-Jürgen Schönig
Alvaro Herrera wrote: On Sun, Sep 11, 2005 at 01:12:34PM +0200, Hans-Jürgen Schönig wrote: in the past we have faced a couple of problems with corrupted system tables. this seems to be a version independent problem which occurs on hackers' from time to time. i have checked a broken file

  1   2   3   >