At 04:49 PM 8/25/2005, Chris Browne wrote:
[EMAIL PROTECTED] (Ron) writes:
> At 03:45 PM 8/25/2005, Josh Berkus wrote:
>> > Ask me sometime about my replacement for GNU sort. Â It uses the
>> > same sorting algorithm, but it's an order of magnitude faster due
>> > to better I/O strategy. Â Someday, in my infinite spare time, I
>> > hope to demonstrate that kind of improvement with a patch to pg.
>>Since we desperately need some improvements in sort performance, I
>>do hope you follow up on this.
> I'll generalize that. IMO we desperately need any and all
> improvements in IO performance. Even more so than we need
> improvements in sorting or sorting IO performance.
That's frankly a step backwards. Feel free to "specialise" that instead.
We can agree to disagree, I'm cool with that.
I'm well aware that a Systems Approach to SW
Architecture is not always popular in the Open
Source world. Nonetheless, my POV is that if we
want to be taken seriously and beat "the big
boys", we have to do everything smarter and
faster, as well as cheaper, than they do. You
are not likely to be able to do that consistently
without using some of the "icky" stuff one is
required to study as part of formal training in
the Comp Sci and SW Engineering fields.
A patch that improves some specific aspect of
performance is a thousand times better than any
sort of "desperate desire for any and
all improvements in I/O performance."
minor twisting of my words: substituting "desire"
for "need". The need is provable. Just put "the
big 5" (SQL Server, Oracle, DB2, mySQL, and
PostgreSQL) into some realistic benches to see that.
Major twisting of my words: the apparent
implication by you that I don't appreciate
improvements in the IO behavior of specific
things like sorting as much as I'd appreciate
more "general" IO performance
improvements. Performance optimization is best
done as an iterative improvement process that
starts with measuring where the need is greatest,
then improving that greatest need by the most you
can, then repeating the whole cycle. _Every_
improvement in such a process is a specific
improvement, even if the improvement is a
decision to re-architect the entire product to
solve the current biggest issue. Improving
sorting IO is cool. OTOH, if pg's biggest IO
problems are elsewhere, then the amount of
overall benefit we will get from improving
sorting IO is going to be minimized until we
improve the bigger problem(s). Amdahl's Law.
The "specialized patch" is also pointedly better
in that a *confidently submitted* patch is
likely to be way better than any sort of
"desperate clutching at whatever may come to hand."
Another distortion of my statement and POV. I
never suggested nor implied any sort of
"desperate clutching...". We have _measurable_
IO issues that need to be addressed in order for
pg to be a better competitor in the
marketplace. Just as we do with sorting performance.
Far too often, I see people trying to address
performance problems via the "desperate
clutching at whatever seems near to hand," and that
generally turns out very badly as a particular
result of the whole "desperate clutching" part.
If you can get a sort improvement submitted, that's a concrete improvement...
As I said, I'm all in favor of concrete,
measurable improvement. I do not think I ever
stated I was in favor of anything else.
You evidently are mildly ranting because you've
seen some examples of poor SW Engineering
Discipline/Practice by people with perhaps
inadequate skills for the issues they were trying
to address. We all have. "90% of everything is
Jreck (eg of too low a quality)."
OTOH, I do not think I've given you any reason to
think I lack such Clue, nor do I think my post was advocating such thrashing.
My post was intended to say that we need an
Overall Systems Approach to pg optimization
rather than just applying what compiler writer's
call "peephole optimizations" to pg. No more, no less.
I apologize if I somehow misled you,
---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not