Hi, Dawid,
Dawid Kuroczko wrote:
The hybrid approach means: sort as much as you can without spilling to
disk, then aggregate and store aggregate state variables in safe place
(like a tree above), get more tuples from the table, sort them, update
aggregate state variables, lather, rince,
Hi, Tom,
Tom Lane wrote:
Jim C. Nasby [EMAIL PROTECTED] writes:
The specific concern I have is large result sets, like 10s or 100s of MB
(or more). We just added support for not buffering those in psql, so it
seems like a step backwards to have the backend now buffering it (unless
I'm
Hi
I may have a workable idea on a way to add caching to Postgres without
disturbing the MVCC functionality.
Caching, as I've been reading can provide an amazing and sometimes
almost unbelievable performance boost to a database based application,
especially for data that is rarely modified.
On Sun, Oct 15, 2006 at 03:41:25AM -0700, Anon Mous wrote:
However, the problem is surmountable and has been figured out by Oracle,
although I don't know how they did it:
http://www.oracle.com/technology/products/ias/joc/index.html
I'm pretty sure this is application-side caching. The
On 10/15/06, Anon Mous [EMAIL PROTECTED] wrote:
Would it be possible to combine a special memcache implementation of
memcache with a Postgresql interface wrapper?
have you seen
http://people.freebsd.org/~seanc/pgmemcache/
merlin
---(end of
But the same code worked when I inserted around 2500 images in the database. After that it started crashing. So , I don't think it's because of error in the code. Can u suggest some other possible reasons and also why is it crashing at call to memcpy(). I have also checked the memory allocations ,
Did you follow Tom's suggestion of trying with a postgres configured with
--enable-cassert ?
cheers
andrew
Ashish Goel wrote:
But the same code worked when I inserted around 2500 images in the
database. After that it started crashing. So , I don't think it's because
of error in the code.
On Sun, Oct 15, 2006 at 08:31:36PM +0530, Merlin Moncure wrote:
On 10/15/06, Anon Mous [EMAIL PROTECTED] wrote:
Would it be possible to combine a special memcache implementation of
memcache with a Postgresql interface wrapper?
have you seen
http://people.freebsd.org/~seanc/pgmemcache/
[EMAIL PROTECTED] wrote:
Using memcache, I've had problems with consistency brought right to
the front. Both of these have failed me:
1) When updating a PostgreSQL record, I invalidate the memcache record.
If another process comes along in parallel before I commit, notices
Alvaro Herrera wrote:
[EMAIL PROTECTED] wrote:
Using memcache, I've had problems with consistency brought right to
the front. Both of these have failed me:
1) When updating a PostgreSQL record, I invalidate the memcache
record.
If another process comes along in parallel before I
Merlin Moncure wrote:
On 10/15/06, Anon Mous [EMAIL PROTECTED] wrote:
Would it be possible to combine a special memcache implementation of
memcache with a Postgresql interface wrapper?
have you seen
http://people.freebsd.org/~seanc/pgmemcache/
merlin
Now you got me thinkin ;-P
Just
But the same code worked when I inserted around 2500 images in the database. After that it started crashing. So , I don't think it's because of error in the code. Can u suggest some other possible reasons and also why is it crashing at call to memcpy().Tom Lane [EMAIL PROTECTED] wrote: Ashish Goel
Postgre8.1 doesn't seem to support asynchronous I/O. Has its design
been thought off already?
To tried doing with a simple example:
For a Index Nest loop join:
Fetch the outer tuples in an array, and then send all the
corresponding inner-tuple fetch requests asynchronously. Hence while
the IO is
[EMAIL PROTECTED] wrote:
Using memcache, I've had problems with consistency brought right to
the front. Both of these have failed me:
1) When updating a PostgreSQL record, I invalidate the memcache record.
If another process comes along in parallel before I commit, notices
On Sat, Oct 14, 2006 at 07:42:18PM -0400, Tom Lane wrote:
On investigation, the problem seems to lie with ParseDateTime(), which
is being quite finicky about what it will believe is a timezone name.
In particular it won't accept digits as part of the name. That can
probably be fixed but I'm
On Sun, Oct 15, 2006 at 04:16:07AM +0530, Raja Agrawal wrote:
Postgre8.1 doesn't seem to support asynchronous I/O. Has its design
been thought off already?
Sure, I even implemented it once. Didn't get any faster. At that point
I realised that my kernel didn't actually support async I/O, and the
Martijn,
On 10/15/06 10:56 AM, Martijn van Oosterhout kleptog@svana.org wrote:
Have enough systems actually got to the point of actually supporting
async I/O that it's worth implementing?
I think there are enough high end applications / systems that need it at
this point.
The killer use-case
On Sun, 2006-10-15 at 19:56 +0200, Martijn van Oosterhout wrote:
Sure, I even implemented it once. Didn't get any faster.
Did you just do something akin to s/read/aio_read/ etc., or something
more ambitious? I think that really taking advantage of the ability to
have multiple I/O requests
On Sun, Oct 15, 2006 at 06:43:43PM +0100, Lexington Luthor wrote:
[EMAIL PROTECTED] wrote:
Using memcache, I've had problems with consistency brought right to
the front. Both of these have failed me:
1) When updating a PostgreSQL record, I invalidate the memcache record.
If another
On Mon, Oct 16, 2006 at 03:08:39AM +0930, Shane Ambler wrote:
Just throwing some ideas around -
What if we could do something like
CREATE TABLESPACE myramcache LOCATION MEMORY(2GB);
CREATE TABLE mycache (
id as integer, data as varchar(50))
USING TABLESPACE myramcache;
INSERT INTO mycache
I wrote:
It looks to me like basically everywhere in plperl.c that does newSVpv()
should follow it with
#if PERL_BCDVERSION = 0x5006000L
if (GetDatabaseEncoding() == PG_UTF8)
SvUTF8_on(sv);
#endif
Experimentation proved that this was insufficient to fix Vitali's
On Sun, Oct 15, 2006 at 06:43:43PM +0100, Lexington Luthor wrote:
With a bit of careful planning (and a few SELECT FOR UPDATE queries to
prevent deadlock), having perfect consistency and correct caching is
possible.
I didn't respond directly to this claim of yours.
SELECT FOR UPDATE is only
On Sun, Oct 15, 2006 at 02:26:12PM -0400, Neil Conway wrote:
On Sun, 2006-10-15 at 19:56 +0200, Martijn van Oosterhout wrote:
Sure, I even implemented it once. Didn't get any faster.
Did you just do something akin to s/read/aio_read/ etc., or something
more ambitious? I think that really
[EMAIL PROTECTED] wrote:
On Mon, Oct 16, 2006 at 03:08:39AM +0930, Shane Ambler wrote:
You could setup a table in memory to contain known popular data, you
could also use this to create a temporary table in memory to speed up
multiple intermediate calculations without touching disks.
I'm
Martijn van Oosterhout kleptog@svana.org writes:
On Sat, Oct 14, 2006 at 07:42:18PM -0400, Tom Lane wrote:
On investigation, the problem seems to lie with ParseDateTime(), which
is being quite finicky about what it will believe is a timezone name.
Last time I wrote some code with the zic
Shane Ambler [EMAIL PROTECTED] writes:
[EMAIL PROTECTED] wrote:
None of this avoids the cost of query planning, or query execution.
No but you can avoid costly disk access and still have the postgres
level of integrity and integration that memcached doesn't offer.
If you're just trying to
On Sat, 2006-10-14 at 14:55 -0400, Tom Lane wrote:
Marc Munro [EMAIL PROTECTED] writes:
The attached patch provides add-ins with the means to register for
shared memory and LWLocks.
I finally got around to reviewing this patch, and realized that it's got
a pretty fundamental design flaw:
Currently, the FreeBSD ports make the following change when building
python:
--- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004
+++ src/pl/plpython/MakefileTue Dec 28 23:32:16 2004
@@ -9,7 +9,7 @@
# shared library. Since there is no official way to determine this
# (at least
On Mon, Oct 16, 2006 at 05:14:59AM +0930, Shane Ambler wrote:
[EMAIL PROTECTED] wrote:
On Mon, Oct 16, 2006 at 03:08:39AM +0930, Shane Ambler wrote:
You could setup a table in memory to contain known popular data, you
could also use this to create a temporary table in memory to speed up
Marc Munro [EMAIL PROTECTED] writes:
I am content that what you suggest is the right way to go. I will work
on a new patch immediately, unless you would prefer to do this yourself.
I've already got some of the relevant changes made in my local copy,
so I might as well just go ahead and finish
Jim C. Nasby wrote:
Currently, the FreeBSD ports make the following change when building
python:
--- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004
+++ src/pl/plpython/MakefileTue Dec 28 23:32:16 2004
@@ -9,7 +9,7 @@
# shared library. Since there is no official way to
On Sun, Oct 15, 2006 at 10:39:49PM +0200, Peter Eisentraut wrote:
Jim C. Nasby wrote:
Currently, the FreeBSD ports make the following change when building
python:
--- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004
+++ src/pl/plpython/MakefileTue Dec 28 23:32:16 2004
[EMAIL PROTECTED] wrote:
As a thought experiment, I'm not seeing the benefit. I think if you
could prove a benefit, then any proof you provided could be used to
improve the already existing caching layers, and would apply equally
to read-only or read-write pages. For example, why not be able to
Mark,
Using memcache, I've had problems with consistency brought right to
the front. Both of these have failed me:
1) When updating a PostgreSQL record, I invalidate the memcache record.
If another process comes along in parallel before I commit, notices
that the memcache
Tom Lane wrote:
I wrote:
It looks to me like basically everywhere in plperl.c that does newSVpv()
should follow it with
#if PERL_BCDVERSION = 0x5006000L
if (GetDatabaseEncoding() == PG_UTF8)
SvUTF8_on(sv);
#endif
Experimentation proved that this was
Andrew Dunstan [EMAIL PROTECTED] writes:
I am also wondering, now that it's been raised, if we need to issue a use
utf8; in the startup code, so that literals in the code get the right
encoding.
Good question. I took care to ensure that the code strings passed to
Perl are marked as UTF8;
On Sun, Oct 15, 2006 at 04:50:15PM -0500, Andrew Dunstan wrote:
Tom Lane wrote:
I wrote:
It looks to me like basically everywhere in plperl.c that does
newSVpv() should follow it with
#if PERL_BCDVERSION = 0x5006000L
if (GetDatabaseEncoding() == PG_UTF8)
Jim C. Nasby [EMAIL PROTECTED] writes:
Take a look at the failure output...
ERROR: could not load library
/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so: dlopen
(/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so) failed:
On Sun, Oct 15, 2006 at 02:39:36PM -0700, Josh Berkus wrote:
Using memcache, I've had problems with consistency brought right to
the front. Both of these have failed me:
1) When updating a PostgreSQL record, I invalidate the memcache record.
If another process comes along in
On Mon, Oct 16, 2006 at 07:00:20AM +0930, Shane Ambler wrote:
[EMAIL PROTECTED] wrote:
As a thought experiment, I'm not seeing the benefit. I think if you
could prove a benefit, then any proof you provided could be used to
improve the already existing caching layers, and would apply equally
On Sun, 15 Oct 2006, [EMAIL PROTECTED] wrote:
On Sun, Oct 15, 2006 at 08:31:36PM +0530, Merlin Moncure wrote:
On 10/15/06, Anon Mous [EMAIL PROTECTED] wrote:
Would it be possible to combine a special memcache implementation of
memcache with a Postgresql interface wrapper?
have you seen
On 15 Oct 2006, at 19:55, [EMAIL PROTECTED] wrote:
On Mon, Oct 16, 2006 at 07:00:20AM +0930, Shane Ambler wrote:
[EMAIL PROTECTED] wrote:
As a thought experiment, I'm not seeing the benefit. I think if you
could prove a benefit, then any proof you provided could be used to
improve the
I set up the following experiment:
CREATE DOMAIN m_or_p AS char CHECK (VALUE = 'm' OR VALUE = 'p');
CREATE TABLE test_domain (
fkey integer not null,
kinteger not null,
x1 integer not null,
x2 integer,
mp m_or_p not null
);
CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k,
Jeremy Drake [EMAIL PROTECTED] writes:
CREATE TABLE test_domain (
fkey integer not null,
kinteger not null,
x1 integer not null,
x2 integer,
mp m_or_p not null
);
CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k, x1, x2) WHERE mp = 'm';
CREATE INDEX
On Sun, Oct 15, 2006 at 06:19:12PM -0400, Tom Lane wrote:
Jim C. Nasby [EMAIL PROTECTED] writes:
Take a look at the failure output...
ERROR: could not load library
/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so: dlopen
On Sun, 15 Oct 2006, Tom Lane wrote:
Jeremy Drake [EMAIL PROTECTED] writes:
CREATE TABLE test_domain (
fkey integer not null,
kinteger not null,
x1 integer not null,
x2 integer,
mp m_or_p not null
);
CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k, x1, x2)
Jim C. Nasby wrote:
On Sun, Oct 15, 2006 at 06:19:12PM -0400, Tom Lane wrote:
Jim C. Nasby [EMAIL PROTECTED] writes:
Take a look at the failure output...
ERROR: could not load library
/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so: dlopen
47 matches
Mail list logo
