Re: [HACKERS] Ready for beta2?

2007-10-20 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes:
> We have had very few beta1 issues.  I am thinking we should release
> beta2 next week and perhaps accelerate beta and consider a final release
> in November rather than December.  Because of the length of our feature
> freeze it is possible we are not going to have as many beta bugs.

We need to deal with the open questions about tsearch API before beta2.
Since we've already forced initdb for beta2, there's no cost to
additional system-catalog changes now, but that window will clang shut
again when beta2 goes out.

I'm disappointed that Oleg and Teodor haven't commented on those issues
yet ...

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] 8.3 GSS Issues

2007-10-20 Thread Bruce Momjian

Sorry, wrong email.  Nothing applied.

---

Henry B. Hotz wrote:
> I know I haven't been very active for a while here, but I just got to  
> testing the October 3 version a bit prior to getting back to the Java  
> GSS client stuff I promised.  There seem to be some funny things there.
> 
> The only serious issue is that the server doesn't require the realm  
> name to match.  I haven't looked at how that broke yet, but I know I  
> was careful of that point in my original patches because it's always  
> been wrong in the Kerberos 5 auth method.
> 
> If I set up a server I might conceivably get connections from:
> 
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
>  to how NASA set up AD>
> 
> Now the only two of those that *might* be the same person are the  
> first two, and that's only if the Stanford person has a grant to work  
> on a JPL project and got put in our infrastructure as an affiliate,  
> *and* the username wasn't already taken.
> 
> It appears that you can just put a complete (realm-included) name  
> into postgres, so that's obviously the way to support gssapi  
> connections from non-default realms.
> 
> In short this is a security hole.  IMO it should be fixed prior to  
> release.
> 
> -
> 
> I notice there are hba options for gss and sspi both.  Why?
> 
> Is there some windows-only functionality it enables?  Shouldn't we be  
> using Microsoft's advertised GSSAPI/SSPI compatibility?  If you build  
> on Windows then I'm sure you want to link the SSPI libraries rather  
> than require installation of a separate package, but that shouldn't  
> change the functionality or the wire protocol AFAIK.  In other words  
> I would expect this to be a build-time option.
> 
> -
> 
> At the risk of diluting my message:  I still think it's a mistake to  
> call it gss instead of something like gss-noprot.  I believe this  
> will cause misunderstandings in the future when we get the security  
> layer of gssapi implemented.
> 
> -
> 
> There's no way to specify the gssapi library to use.  I have three on  
> my main development Sun:  MIT, Sun, and Heimdal.  I might have more  
> than one version of one of those three at some times.  Of course  
> there's no way to specify which kerberos 5 library or openssl library  
> you want either, so consider this a feature request for future  
> development.
> 
> 
> The opinions expressed in this message are mine,
> not those of Caltech, JPL, NASA, or the US Government.
> [EMAIL PROTECTED], or [EMAIL PROTECTED]
> 
> 
> 
> ---(end of broadcast)---
> TIP 3: Have you checked our extensive FAQ?
> 
>http://www.postgresql.org/docs/faq

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] 8.3 GSS Issues

2007-10-20 Thread Bruce Momjian

Patch applied.  Thanks.

---


Henry B. Hotz wrote:
> I know I haven't been very active for a while here, but I just got to  
> testing the October 3 version a bit prior to getting back to the Java  
> GSS client stuff I promised.  There seem to be some funny things there.
> 
> The only serious issue is that the server doesn't require the realm  
> name to match.  I haven't looked at how that broke yet, but I know I  
> was careful of that point in my original patches because it's always  
> been wrong in the Kerberos 5 auth method.
> 
> If I set up a server I might conceivably get connections from:
> 
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
> [EMAIL PROTECTED]
>  to how NASA set up AD>
> 
> Now the only two of those that *might* be the same person are the  
> first two, and that's only if the Stanford person has a grant to work  
> on a JPL project and got put in our infrastructure as an affiliate,  
> *and* the username wasn't already taken.
> 
> It appears that you can just put a complete (realm-included) name  
> into postgres, so that's obviously the way to support gssapi  
> connections from non-default realms.
> 
> In short this is a security hole.  IMO it should be fixed prior to  
> release.
> 
> -
> 
> I notice there are hba options for gss and sspi both.  Why?
> 
> Is there some windows-only functionality it enables?  Shouldn't we be  
> using Microsoft's advertised GSSAPI/SSPI compatibility?  If you build  
> on Windows then I'm sure you want to link the SSPI libraries rather  
> than require installation of a separate package, but that shouldn't  
> change the functionality or the wire protocol AFAIK.  In other words  
> I would expect this to be a build-time option.
> 
> -
> 
> At the risk of diluting my message:  I still think it's a mistake to  
> call it gss instead of something like gss-noprot.  I believe this  
> will cause misunderstandings in the future when we get the security  
> layer of gssapi implemented.
> 
> -
> 
> There's no way to specify the gssapi library to use.  I have three on  
> my main development Sun:  MIT, Sun, and Heimdal.  I might have more  
> than one version of one of those three at some times.  Of course  
> there's no way to specify which kerberos 5 library or openssl library  
> you want either, so consider this a feature request for future  
> development.
> 
> 
> The opinions expressed in this message are mine,
> not those of Caltech, JPL, NASA, or the US Government.
> [EMAIL PROTECTED], or [EMAIL PROTECTED]
> 
> 
> 
> ---(end of broadcast)---
> TIP 3: Have you checked our extensive FAQ?
> 
>http://www.postgresql.org/docs/faq

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Ready for beta2?

2007-10-20 Thread Albert Cervera i Areny
A Diumenge 21 Octubre 2007, Bruce Momjian va escriure:
> We have had very few beta1 issues.  I am thinking we should release
> beta2 next week and perhaps accelerate beta and consider a final release
> in November rather than December.  Because of the length of our feature
> freeze it is possible we are not going to have as many beta bugs.

I want to send a patch for the contrib modules by late sunday (europe here ;). 
I'm doing some cleanup and organizing it a bit better and I think somebody 
expected these docs to be in the second beta...

-- 
Albert Cervera i Areny
http://www.NaN-tic.com

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


[HACKERS] Feature freeze

2007-10-20 Thread Bruce Momjian
Many community members were disappointed that feature freeze took so
long, but based on the number of features added to 8.3 our feature
freeze duration was similar to previous releases.

I think our only big mistake was setting expectation that this would be
a short feature freeze.  We can be proud of the 8.3 release.

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


[HACKERS] Ready for beta2?

2007-10-20 Thread Bruce Momjian
We have had very few beta1 issues.  I am thinking we should release
beta2 next week and perhaps accelerate beta and consider a final release
in November rather than December.  Because of the length of our feature
freeze it is possible we are not going to have as many beta bugs.

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 6: explain analyze is your friend


[HACKERS] MVCC, undo log, and HOT

2007-10-20 Thread Bruce Momjian
Those who have been with the community from long ago might remember
discussion about implementing a undo log.  The big advantage of this is
that it allows UPDATE to _replace_ rows and limits the amount of cleanup
required for UPDATEs.

I am hoping that with HOT we will no longer have any need to even
consider undo.

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


[HACKERS] Poorly designed tsearch NOTICEs

2007-10-20 Thread Tom Lane
regression=# SELECT plainto_tsquery('the any'); 
NOTICE:  query contains only stopword(s) or doesn't contain lexeme(s), ignored
 plainto_tsquery 
-
 
(1 row)

regression=# select ''::tsquery;
NOTICE:  tsearch query doesn't contain lexeme(s): ""
 tsquery 
-
 
(1 row)

IMHO, it's really bad design to have this sort of NOTICE emitted by
tsquery input.  Even if an application uses numnode() or querytree() or
something similar to detect bogus queries, it's going to have its logs
cluttered with these notices.

I could see having the @@ operator emit the notice if the query is
actually used for searching --- though I'm not quite sure how to get it
to come out only once per query ... maybe we could put it into the index
consistent() functions somehow?

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] NULL and plpgsql rows

2007-10-20 Thread Sibte Abbas
On 10/2/06, Tom Lane <[EMAIL PROTECTED]> wrote:
> "Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> > However, the test right above that means that we'll fail if the user
> > tries something like "row_variable := NULL;":
>
> The patch you seem to have in mind would allow
> row_variable := int_variable;
> to succeed if the int_variable chanced to contain NULL, which is surely
> not very desirable.
>
> The real issue here is that the bare NULL has type UNKNOWN and we're not
> making any effort to cast it.  I'm not sure whether it'd work to simply
> apply exec_cast_value --- that looks like it's only meant to handle
> scalars, where in general you'd need something close to
> ExecEvalConvertRowtype().
>
> > Of course, setting a row variable to null is a lot more useful if we can
> > actually test for it after the fact, and I'm not really sure how to make
> > that happen.
>
> Doesn't IS NULL work (as of CVS HEAD)?
>

Is there a specific reason why we keep the tuple descriptor of an
unassigned record type to NULL?

Surely we don't know what tuple descriptor it will actually contain,
however, maybe we can have "special" tuple descriptors for un-assigned
record types.

For example, if for NULL/unassigned record type we create a tuple
descriptor of "VOID" type, and then initialize its corresponding (one
column) row to null, we 'll have the  IS NULL check working on
unassigned or NULL record types as well.

regards,
-- 
Sibte Abbas

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-20 Thread Luke Lonergan
Hi Hannu,

On 10/14/07 12:58 AM, "Hannu Krosing" <[EMAIL PROTECTED]> wrote:

> What has happened in reality, is that the speed difference between CPU,
> RAM and disk speeds has _increased_ tremendously

Yes.

> which makes it even
> more important to _decrease_ the size of stored data if you want good
> performance

Or bring the cpu processing closer to the data it's using (or both).

By default, the trend you mention first will continue in an unending way -
the consequence is that the "distance" between a processor and it's target
data will continue to increase ad-infinitum.

By contrast, you can only decrease the data volume so much - so in the end
you'll be left with the same problem - the data needs to be closer to the
processing.  This is the essence of parallel / shared nothing architecture.

Note that we've done this at Greenplum.  We're also implementing a DSM-like
capability and are investigating a couple of different hybrid row / column
store approaches.

Bitmap index with index-only access does provide nearly all of the
advantages of a column store from a speed standpoint BTW.  Even though
Vertica is touting speed advantages - our parallel engine plus bitmap index
will crush them in benchmarks when they show up with real code.

Meanwhile they're moving on to new ideas - I kid you not "Horizontica" is
Dr. Stonebraker's new idea :-)

So - bottom line - some ideas from column store make sense, but it's not a
cure-all.
 
> There is also a MonetDB/X100 project, which tries to make MonetOD
> order(s) of magnitude faster by doing in-page compression in order to
> get even more performance, see:

Actually, the majority of the points made by the MonetDB team involve
decreasing the abstractions in the processing path to improve the IPC
(instructions per clock) efficiency of the executor.

We are also planning to do this by operating on data in vectors of projected
rows in the executor, which will increase the IPC by reducing I-cache misses
and improving D-cache locality.  Tight loops will make a much bigger
difference when long runs of data are the target operands.

- Luke 



---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Including Snapshot Info with Indexes

2007-10-20 Thread Martijn van Oosterhout
On Sat, Oct 20, 2007 at 09:24:07AM +0530, Gokulakannan Somasundaram wrote:
> Hi,
>   I think i have a initial Implementation. It has some bugs and i am working
> on fixing it. But to show the advantages, I want to show the number of
> Logical I/Os on the screen. In order to show that, i tried enabling the
> log_statement option in PostgreSQL.conf. But it shows only the physical
> reads. What i wanted was a Logical reads count( No. of  ReadBuffer calls,
> which is stored in ReadBufferCount variable). So i have added this stats to
> the bufmgr.c(function is BufferUsage, i suppose) to show Logical Reads and
> Physical Reads. Is this a acceptable change?

I'm not sure if the number of logical reads is really a useful
measurement. I can imagine there are places that deliberatly read the
block "logically" a few times but drop the pin in between to allow
others access. This will skew your results as in actual usage only the
first is likely to generate a real I/O.

If your problem is cache it seems to me you should test with a table
larger than your shared buffers and perhaps even larger than your total
memory, since this is the case we're actually interested in.

Have a ncie day,
-- 
Martijn van Oosterhout   <[EMAIL PROTECTED]>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to 
> litigate.


signature.asc
Description: Digital signature


Re: [HACKERS] Debugger

2007-10-20 Thread Heikki Linnakangas
Pedro Belmino wrote:
> 4. Tried to attach the id of the process to the debugger and gave the error:
> "Unable to open socket file: target process not responding or HotSpot VM not
> loaded"

You're trying to use a Java debugger to attach to a non-Java program
(Postgres). That's not going to work. Use a tool like gdb or ddd instead.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Strange error dropping foreign key

2007-10-20 Thread Magnus Hagander
> Ühel kenal päeval, R, 2007-10-19 kell 15:29, kirjutas Magnus Hagander:
> > On Thu, Oct 18, 2007 at 10:48:02AM -0400, Tom Lane wrote:
> > > "Magnus Hagander" <[EMAIL PROTECTED]> writes:
> > > > db=# alter table isi.items_stat drop constraint items_stat_item_id_fkey;
> > > > ERROR:  "items_pkey" is an index
> > > 
> > > Context please?
> > 
> > Yeah, I had that one coming, didn't it... That's what I get for trying to
> > type it up on my phone on the way home.
> > 
> > Anyway. I've asked for a dump of the db to get all the details, but 
> > basically:
> > CREATE TABLE items (
> >id int not null primary key,
> >  
> > );
> > CREATE TABLE items_stat (
> >item_id int not null references items.id,
> >  
> > );
> > 
> > 
> > One thing that might be related - items is a Slony slave table. items_stat
> > is *not* in the Slony set. It's not *supposed* to have a foreign key to the
> > items table, which is why I'm trying to drop it.
> 
> Slony does strange stuff to FK-s and other constraints, at least in 1.X
> versions (like changing the relation type from index to
> ) and thus you are not supposed to do DDL directly
> on slaves (or even masters). 

I'm doing DDL on a table that is NOT replicated. Are you saying I shouldn't 
even have unreplicated tables in the same database?

Now I shouldn't be referencing the table from an unreplicated one, but that's 
what I'm trying to fix!

> You should use EXECUTE SCRIPT slonik command to do DDL, so that drop
> index happens while original state is temporaryly restored)

1) I'm dropping a foreign key, not an index. The problem is that somehow that 
gets translated into trying to drop the primary key of the _other_ table.
2) The constraint doesn't exist on the master, only the slave, so I don't 
beleive EXECUTE SCRIPT will work, no?

> Actually this should belongs to Slony list ;)

I'm not entirely convinced it's a Slony problem. Actually when I first posted I 
didn't even think it might be slony related.

/Magnus

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly