the table at the end.
Yes, I thought we discussed doing this for empty tables -- it would be, per
our tests, a +10% to +30% boost to COPY.
But there was some problem the patch?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
, that
won't help for minor versions.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6: explain analyze is your friend
/ora2pg) into submission. In the end I'm hoping that the move
from Oracle will be made easier for others.
I'm happy to work with you on ora2pg, as long as we can use Perl. Joe
Conway has some useful oracle-table-bulkloading stuff I can probably talk
him out of.
--
--Josh
Josh Berkus
Aglio
of a sudden I think we had this discussion
already? I for sure remember the fresh-pages trick from some other
thread.)
Yes, and that's what shot the proposal down before. But I don't think we
devoted sufficient discussion to the new table case.
--
--Josh
Josh Berkus
Aglio Database Solutions
access for frozen tables.
That would be a *huge* gain, especially with bitmaps. I think we've
discussed this before,though.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0
to newbies. I have seen *far* too many abuses of UUIDs in really
bad database design. People who use them should be experienced enough to
know what they're doing.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
.
For my part, I generally push implementing the UUID concept in a better way
that keeps server, table, and surrogate keys atomic (and thus more useful
and easier to debug).
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
. If there are space/overhead considerations, you can put them
into a reversable hash. I've never understood the complex measures which
application developers take to create universal IDs.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
might
be important in some contexts.
Well, IMHO, if the unguessability of UUIDs is an important part of your spec,
you have some design problems. But it's not my job to lock up the
foot-guns, just to make sure they come with warnings ...
--
Josh Berkus
Aglio Database Solutions
San Francisco
its machine
identifier?
One of the differences between an add-in and core code is support for all
PostgreSQL platforms.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner
or transmission.
I'm also a little baffled to come up with any real application where making
an id number for most tables unguessable would provide any kind of real
protection not far better provided by other means. For your users
table, sure, but that's a very special case.
--
--Josh
Josh
address.
Wanna start a pgFoundry project so that someone *else* can do the Windows
version, if they want?
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6: explain analyze is your friend
deal
with users who change the length via ALTER TABLE?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6: explain analyze is your friend
Folks,
Help on the Configurator is actively solicited. I really think this is a
better solution for this problem.
http://www.pgfoundry.org/projects/configurator
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP
behind.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
files here:
http://pgfoundry.org/docman/index.php?group_id=141
I've already added a doc category Spinlock Testing.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner
to reviewing
documenting it.
Jonathan's piece is actually an excellent article on how to implement
mattviews in user-space. Mostly, it just needs to be linked from somewhere
in the PostgreSQL.org pages.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end
%)
benefits in some heavy-contention environments, at least in OSDL tests.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan
.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
implementation of a cost-based optimizer. You can always
read the code ;-)
Mark Kirkwood put together this nice paper on planner statistics:
http://www.powerpostgresql.com/PlanStats
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
a.a between 5 and 6;
So, is this a real bug in constraints or does the problem lie somewhere
else? Is it fixable?
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ
experience with HT, and the reason why many
software vendors recommend disabling it. Not sure about NUMA.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http
the operator after USING to
be part of an operator class?
Hmmm ... would this prevent the hackish workaround for case-insensitive sort?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner
.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
any number of Postgres - SQL questions answered there.
--Josh
--
__Aglio Database Solutions___
Josh BerkusConsultant
josh@agliodbs.comwww.agliodbs.com
Ph: 415-752-2500Fax: 415-752-2387
2166 Hayes Suite 200San Francisco, CA
to mention that as well.
Planning, yes. Have started, no. It's a major feature implementation if
we want them to be at all worthwhile; I'd like users to be able to create
custom windowing aggregates, for that matter.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
machine for testing. I have, like, 160 performance tests backed up
which are never going to get run before we release 8.1. I'm going to be
hitting up some major PostgreSQL sponsors for hardware donations, any help
is welcome.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
Jonah,
Seems like this was to be somewhat expected. Was there any
stats/diagnostics included in the patch to show the effectiveness of
PCTFREE?
Lots, look up the tests on OSDL, per link.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end
it either. Anyone want to try it
on some other test cases?
Right, this doesn't kill PCTFree, what it does is fail to make a case for
it.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our
Tom,
Or, as you say, we could take the viewpoint that there are commercial
companies willing to take on the burden of supporting back releases, and
the development community ought not spend its limited resources on doing
that. I'm hesitant to push that idea very hard myself, because it would
Steve,
The only crystal ball involved is the assumption that if bizgres has
Neat Stuff(tm) that would be of widespread use in it's development
tree at that point then the odds are good that it, or something
functionally equivalent to it, will appear in the 8.2 development
tree.
It certainly
Ilia,
Maybe new constraint_exclusion staff could help to exclude non-matching
tables from inheritance query ?
Yes, that's exactly what it's for. Your testing is welcome. Download 8.1
and try it today.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
; currently for us external sort is a
*CPU-bound* operation, half of which is value comparisons. (oprofiles
available if anyone cares)
So we need to look, instead, at algorithms which make better use of
work_mem to lower CPU activity, possibly even at the expense of I/O.
--Josh Berkus
Ilia,
Well, Simon is still writing the CE docs. In the meantime:
http://archives.postgresql.org/pgsql-hackers/2005-07/msg00461.php
--josh
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose
on the overall index creation speed.
--Josh Berkus
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
the merge.
What's the timeframe for 8.2?
Too far out to tell yet. Probably 9mo to 1 year, that's been our history.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please
Jeff,
I would just run it under the profiler and see how many times
beginmerge() is called.
Hmm, I'm not seeing it at all in the oprofile results on a 100million-row
sort.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
Ron,
Hmmm.
60GB/5400secs= 11MBps. That's ssllooww. So the first
problem is evidently our physical layout and/or HD IO layer
sucks.
Actually, it's much worse than that, because the sort is only dealing
with one column. As I said, monitoring the iostat our top speed was
2.2mb/s.
--Josh
. that
will give us incremental improvements. But what we need now is a 5-10x
improvement and that's somewhere in the algorithms or the code.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6: explain
more than that either.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
as the patch is finished. Always useful to
gear up the old TPC-H.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
? Or raw FS reads? It's not the same thing.
Also, we're talking *write speed* here, not read speed.
I also find *your* claim suspicious, since there's no way XFS is 300% faster
than ext3 for the *general* case.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end
.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
the 24th disk,
whereas ext3 stops scaling after 16 disks. For writes both XFS and ext3
top out around 8 disks, but in this case XFS tops out at 500MB/sec while
ext3 can't break 350MB/sec.
That would explain it. I seldom get more than 6 disks (and 2 channels) to
test with.
--
--Josh
Josh
Andreas,
pg relys on the OS readahead (== larger block IO) to do efficient IO.
Basically the pg scan performance should match a dd if=file of=/dev/null
bs=8k,
unless CPU bound.
FWIW, we could improve performance by creating larger write blocks when
appropriate, particularly on Unixes like
, but it is not able to
be triggered separately from schema and data. Any reason why this wouldn't
be a good(tm) idea?
It would be an *excellent* idea, along with options to dump specific
functions, and both specific and all views/types/operators. Go for it.
--
Josh Berkus
Aglio Database Solutions
San Francisco
from scratch
send clear_notices();
Hopefully people get the idea?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Presskit: http://pgfoundry.org/docman/view.php/147/90/presskit.xhtml
(note that not all links will work right now)
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ
Ilia,
It is bitmap-OR on multiple index(PK) lookups.
Describing it doesn't help. We need an *actual* EXPLAIN ANALYZE.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Kevin,
I'm looking at trying to fix some clear flaws in costing which cause
of our real-world queries to choose sub-optimal plans under PostgreSQL.
It's clear that there needs to be a tool to analyze the accuracy of
costing for a variety of queries, both to direct any efforts to fix
problems
certainly not a valid leap second.
It's also consistent with how several other RDBMSes do things (SQL Server,
MySQL), and several programming languages.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2
detail, I assume we should take this
off-list.
Well, once you get going we'll use the testperf-general list, which doesn't
get much traffic these days. pgFoundry also supports bug tracking, task
management, and document sharing, you should check it out.
--Josh
--
Josh Berkus
Aglio Database
Jari,
Comparison of different SQL implementations
http://troels.arvin.dk/db/rdbms/ by Troels Arvin [EMAIL PROTECTED]
Should be pretty up to date. Troels often pops into IRC to see if we have any
news for him.
--
Josh Berkus
Aglio Database Solutions
San Francisco
.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
random_page_cost as it is; how are
they going to handle having 5 different parameters?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http
of the tool and the reliability of
its statistics.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining
.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6: explain analyze is your friend
Satoshi,
And I want to get statistic info through system views, like pg_statio_*.
I don't think anyone disagrees with that. It's just a little too late to
get in for 8.1.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
fix the problem.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
an enable_* or
other query cost option at runtime. Not sure how to capture this,
though.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
-programmed object information
for the objects we use.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command
, and this is the first time I can recall someone complaining about
this comparison behavior. So it's obviously not a widespread issue.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our
Tom,
SuSE Linux 9.3 Professional, 2.6.11.4-20a-default,
gcc (GCC) 3.3.5, Athlon64
--with-perl --with-integer-datetimes --with-pgport=5801
--prefix=/usr/local/pg81
==
All 98 tests passed.
==
--
Josh Berkus
Aglio Database Solutions
San Francisco
is about 4:1.
All of this goes to uphold Tom's general assertion that the default of 4 is
more or less correct but the calculation in which we're using that number is
not.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
: if we put this off for another
release, the situation's not going to get any better. Eventually, for some
release, we need to break something, because we can't fix the past.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
Tom,
FreeBSD 5.4 Opteron 64 SMP pass.
--Josh
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
Folks,
Looks like someone left their test settings in postgresql.conf.sample in
the beta4 release:
-Line 224
silent_mode = true
#silent_mode = false # DO NOT USE without syslog or
redirect_stderr
-
--
--Josh
Josh Berkus
Aglio Database Solutions
San
Tom,
Don't see it in my copy, nor in cvsweb.
Sorry. This looks like an artifact of the FreeBSD ports install combining
in weird ways with a CVS install of beta4. Sorry for the bogus report.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end
list?
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can
must
write a record or row variable as the expression.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Folks,
Are foreign keys on temp tables not allowed just because nobody requested
them, or because they're hard to do?
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 5: don't forget to increase your
in the session where the query is executed? If
not, then I don't really find this a solution. In that case, it would be
better to supply a script that allows users to find add_missing_from
dependant views so they can fix them.
--
Josh Berkus
Aglio Database Solutions
San Francisco
be nice
to emit a warning, as well, if that's reasonably easy to do.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Folks,
Thanks, all! Now, if only I could remember who asked me the question ...
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe
, and it
currently requires the username/password to be hard coded into the
source code.
http://pgfoundry.org/projects/dbi-link/
http://pgfoundry.org/projects/dblink-tds/
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6
once it was proved to be feasable.
Peter?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's
. There's no real way to get
around it.
--Josh
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining
: INSERT 1
session2: INSERT 1 ERROR
Get the picture? The only way to avoid a race condition is to be able to
do predicate locking, that is to lock the table against any data write
matching that predicate.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
to take on the task of getting it into production shape.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Heikki,
4. Don't increment xid counter, just use a single xid for all
transactions. Otherwise we might reach the xid wrap-around point, and we
can't vacuum. It seems safe to me, am I missing anything?
Well, it eliminates transaction isolation. Completely.
--
Josh Berkus
Aglio Database
Guys,
I'm not going to get the university research up before American Thanksgiving.
Sorry. Look for it next week.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6: explain analyze is your friend
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 6: explain analyze is your friend
.
Will have to develop better tests ...
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
applications. The two tests will not be the same.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
10,000 old orders to an
archive table.
Oh, also we need to:
3) Run the test for 3+ hours after scaling up, and turn on autovacuum.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: subscribe and unsubscribe
, I started setting up a 200GB DBT3 database on one of OSDL's machines.
You're welcome to it, I don't see myself completeting those tests before the
holidays. Want login?
--Josh
--
__Aglio Database Solutions___
Josh BerkusConsultant
[EMAIL PROTECTED
I've seen cover them.
Can someone help me out?
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere
and
eliminate it as a GUC.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
vacuum_delay setting? Given
that the performance drops seem to correspond to wait times, I think this
could mean a big difference.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 9: the planner will ignore your desire
is updating or adding 100% of the rows in the orders table
every 6-8 minutes?Does that seem right?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
next summer,
The O'Reilly OSCON Team
--
Josh Berkus
PostgreSQL Core Team
[EMAIL PROTECTED]
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
, such as
referring and referenced. Other suggestions are welcome.
So, feedback before I start writing SQL?
Oh, also what file are the system views defined in?
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast
it
for the press releases, there's no reason why the translators can use it for
all translations.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ
Jim,
Just to be clear, I'm not suggesting renaming anything in any of the
existing pg_catalog objects. I'm suggesting creating a new, easier to
use set of views that would sit on top of pg_catalog.
I have no objection to using easier to read names for the system views.
(This is the
the point in prodiving a dual list -- that is,
a list of OIDs in addition to the list of names provided in the columns of
each view. The idea of these views is to keep the users *away* from
technical details like OIDs, which can and will change with the advancing
versions of PostgreSQL.
--
Josh
be to write shell
functions for those that accept fully-qualified object names.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
names programmatically a problem?
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
.
--
Josh Berkus
Aglio Database Solutions
San Francisco
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
401 - 500 of 4746 matches
Mail list logo