Re: [GENERAL] A query planner that learns

2006-10-17 Thread AgentM
On Oct 16, 2006, at 16:17 , Madison Kelly wrote: Alvaro Herrera wrote: Jochem van Dieten wrote: Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Madison Kelly
AgentM wrote: Alvaro's advice is sound. If the patent holder can prove that a developer looked at a patent (for example, from an email in a mailing list archive) and the project proceeded with the implementation regardless, malice can been shown and damages can be substantially higher. You're

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Brian Mathis
On 10/17/06, Madison Kelly [EMAIL PROTECTED] wrote: AgentM wrote: Alvaro's advice is sound. If the patent holder can prove that a developer looked at a patent (for example, from an email in a mailing list archive) and the project proceeded with the implementation regardless, malice can been shown

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Madison Kelly
Brian Mathis wrote: I also am NAL, but I know enough about the patent system (in the US) to know that ignorance *IS* a defense. If you are ignorant of the patent, you only have to pay the damages. If you knew about the patent and did it anyway, you have to pay *triple* damages. Ignorance

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Brian Mathis
On 10/17/06, Madison Kelly [EMAIL PROTECTED] wrote: Brian Mathis wrote: I also am NAL, but I know enough about the patent system (in the US) to know that ignorance *IS* a defense.If you are ignorant of the patent, you only have to pay the damages.If you knew about the patent and did it anyway,

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Scott Ribe
Colour me funny, but wouldn't staying out of the courts in the first place not be the best option? Yes, however some people feel that given the way the patent office is spewing huge quantities of patents, many on old well-known techniques, and the the absurd difficulty of reading patent claims,

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Alvaro Herrera
Madison Kelly wrote: Brian Mathis wrote: I also am NAL, but I know enough about the patent system (in the US) to know that ignorance *IS* a defense. If you are ignorant of the patent, you only have to pay the damages. If you knew about the patent and did it anyway, you have to pay

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Madison Kelly
Alvaro Herrera wrote: Yeah. I invite you to do all the extra (useless) development work required. But please do not charge other people with it. Whoever investigates patents and lets pgsql-hackers know about them, is charging the Postgres community with that work. We sure don't need it. As

Re: [GENERAL] A query planner that learns

2006-10-17 Thread AgentM
On Oct 17, 2006, at 10:46 , Madison Kelly wrote: Brian Mathis wrote: I also am NAL, but I know enough about the patent system (in the US) to know that ignorance *IS* a defense. If you are ignorant of the patent, you only have to pay the damages. If you knew about the patent and did it

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Rafal Pietrak
On Tue, 2006-10-17 at 10:24 -0400, Madison Kelly wrote: Nor am I a lawyer, but I still hold that hoping ignorance will be a decent defense is very, very risky. In the end I am not a pgSQL developer so it isn't in my hands either way. If I may. The hoping, ignorance will save you line of

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Alban Hertroys
Jim C. Nasby wrote: On Thu, Oct 12, 2006 at 05:39:20PM -0500, Scott Marlowe wrote: It may well be that by first looking at the data collected from problems queries, the solution for how to adjust the planner becomes more obvious. Yeah, that would be useful to have. The problem I see is

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Jochem van Dieten
Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns from its mistakes over time is far more desirable. Is it

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Alvaro Herrera
Jochem van Dieten wrote: Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns from its mistakes over time is

Re: [GENERAL] A query planner that learns

2006-10-16 Thread John D. Burger
Jochem van Dieten wrote: I think you might want to check US Patent 6,763,359 before you start writing any code. http://tinyurl.com/yzjdve - John D. Burger MITRE ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ?

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Madison Kelly
Alvaro Herrera wrote: Jochem van Dieten wrote: Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns from its

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Alvaro Herrera
Madison Kelly wrote: Alvaro Herrera wrote: Jochem van Dieten wrote: Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Madison Kelly
Alvaro Herrera wrote: Hasn't IBM release a pile of it's patents for use (or at least stated they won't sue) to OSS projects? If so, is this patent covered by that amnesty? This is useless as a policy, because we have plenty of companies basing their proprietary code on PostgreSQL, which

Re: [GENERAL] A query planner that learns

2006-10-14 Thread Guy Rouillier
Jim C. Nasby wrote: Just recording the query plan and actual vs estimated rowcounts would be a good start, though. And useful to DBA's, provided you had some means to query against it. If the DBMS stores the incoming query (or some parsed version of it) and identifies frequency of usage

Re: [GENERAL] A query planner that learns

2006-10-14 Thread Ron Johnson
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/13/06 10:47, John D. Burger wrote: Erik Jones wrote: [snip] But with both approaches, the planner is just using the static statistics gathered by ANALYZE to estimate the cost of each candidate plan, and these statistics are based on

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Erik Jones
Forgive me if I'm way off here as I'm not all that familiar with the internals of postgres, but isn't this what the genetic query optimizer discussed the one of the manual's appendixes is supposed to do. Or, is that more about using the the known costs of atomic operations that make up a

Re: [GENERAL] A query planner that learns

2006-10-13 Thread John D. Burger
Erik Jones wrote: Forgive me if I'm way off here as I'm not all that familiar with the internals of postgres, but isn't this what the genetic query optimizer discussed the one of the manual's appendixes is supposed to do. No - it's not an optimizer in that sense. When there are a small

Re: [GENERAL] A query planner that learns

2006-10-13 Thread AgentM
On Oct 13, 2006, at 11:47 , John D. Burger wrote: Erik Jones wrote: Forgive me if I'm way off here as I'm not all that familiar with the internals of postgres, but isn't this what the genetic query optimizer discussed the one of the manual's appendixes is supposed to do. No - it's not

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 05:39:20PM -0500, Scott Marlowe wrote: It seems to me the first logical step would be having the ability to flip a switch and when the postmaster hits a slow query, it saves both the query that ran long, as well as the output of explain or explain analyze or

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 11:53:15AM -0400, AgentM wrote: One simple first step would be to run an ANALYZE whenever a sequential scan is executed. Is there a reason not to do this? It Yes. You want a seqscan on a small (couple pages) table, and ANALYZE has a very high overhead on some

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Scott Marlowe
On Fri, 2006-10-13 at 12:48, Jim C. Nasby wrote: On Thu, Oct 12, 2006 at 05:39:20PM -0500, Scott Marlowe wrote: It seems to me the first logical step would be having the ability to flip a switch and when the postmaster hits a slow query, it saves both the query that ran long, as well

[GENERAL] A query planner that learns

2006-10-12 Thread Scott Marlowe
While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns from its mistakes over time is far more desirable. Is it reasonable or possible for

Re: [GENERAL] A query planner that learns

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 03:31:50PM -0500, Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns from its mistakes

Re: [GENERAL] A query planner that learns

2006-10-12 Thread Scott Marlowe
On Thu, 2006-10-12 at 17:14, Jim C. Nasby wrote: On Thu, Oct 12, 2006 at 03:31:50PM -0500, Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep