Re: [HACKERS] PATCH: Compiling PostgreSQL using ActiveState Python 3.2

[Ashesh Vashi]

On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane  wrote:

> Peter Eisentraut  writes:
> > On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote:
> >> It's not immediately apparent to me why we should think that
> >> get_python_lib is less trustworthy than LIBPL; but if someone
> >> can make that case, I don't have any objection to this part of
> >> the patch.
>
> > The issue, at least for me, is that the file isn't necessarily called
> > 'config' anymore.  I have
> > /usr/lib/python3.2/config-3.2mu
>
One of the reason, I say that - we do have hard-coded values for the config
directory.
Hence, I used the LIBPL.

>
> Ah, I see.
>
> > LIBPL exists at least as far back as Python 2.2, so its use should be
> > safe.
>
> Yeah, that part of the patch seems sane then.
>
> > Yes, because get_config_vars('LDVERSION') doesn't exist in that version.
> > In theory, it would return '2.7', so everything would fit back together,
> > but LDVERSION doesn't exist before 3.2.
>
Oops - sorry...
I did not know about it..

>
> Could we have the code use 'LDVERSION' if it gets a nonempty result,
> and otherwise fall back to the current scheme?  But I guess first we
> need some details as to why the current scheme isn't sufficient.
>
Please find the attached patch as you suggested.

Reason:
- As per my findings, ActiveState Python 3.2 does not provide shared
libraries along with it.
 (Though - I am not sure about the earlier version of ActiveState Python)
 We can confirm the same using the following command:
 ${PYTHON} -c "import distutils.sysconfig,string;
print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))"
Which returns in this case '0'.

And, getting values for the 'python_ldlibrary' and 'python_so'  are
'libpython3.2m.a' and '.cpython-32m.so' respectively.
So, the condition - *x"${python_ldlibrary}" != x"${ldlibrary}"* is always
failing, and switching it back to link the old way.

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company



*http://www.linkedin.com/in/asheshvashi*


>
>regards, tom lane
>


pg9.1beta3_python_v2.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] FATAL: ReleaseSavepoint: unexpected state STARTED

[Marcin Mańk]

On Wed, Aug 17, 2011 at 11:30 PM, Tom Lane  wrote:
> =?UTF-8?B?TWFyY2luIE1hxYRr?=  writes:
>>  psql -c 'release q; prepare q(int) as select 1'
>> FATAL:  ReleaseSavepoint: unexpected state STARTED
>
> Can't get terribly excited about that, seeing that the statement is
> surely going to draw an error and abort processing the rest of the
> command string in any case ...

Oh, I thought FATAL was disconnectiong other sessions. Yeah, that was
not bad enough.

Here is a better one:

psql postgres -c 'savepoint q; select 1'
FATAL:  DefineSavepoint: unexpected state STARTED
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.

Greetings
Marcin Mańk

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Compiling PostgreSQL using ActiveState Python 3.2

[Ashesh Vashi]

Please ignore the previous patch.
Please find the updated patch.

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company



*http://www.linkedin.com/in/asheshvashi*



On Thu, Aug 18, 2011 at 12:57 PM, Ashesh Vashi <
ashesh.va...@enterprisedb.com> wrote:

>  On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane  wrote:
>
>> Peter Eisentraut  writes:
>> > On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote:
>> >> It's not immediately apparent to me why we should think that
>> >> get_python_lib is less trustworthy than LIBPL; but if someone
>> >> can make that case, I don't have any objection to this part of
>> >> the patch.
>>
>> > The issue, at least for me, is that the file isn't necessarily called
>> > 'config' anymore.  I have
>> > /usr/lib/python3.2/config-3.2mu
>>
> One of the reason, I say that - we do have hard-coded values for the config
> directory.
> Hence, I used the LIBPL.
>
>>
>> Ah, I see.
>>
>> > LIBPL exists at least as far back as Python 2.2, so its use should be
>> > safe.
>>
>> Yeah, that part of the patch seems sane then.
>>
>> > Yes, because get_config_vars('LDVERSION') doesn't exist in that version.
>> > In theory, it would return '2.7', so everything would fit back together,
>> > but LDVERSION doesn't exist before 3.2.
>>
> Oops - sorry...
> I did not know about it..
>
>>
>> Could we have the code use 'LDVERSION' if it gets a nonempty result,
>> and otherwise fall back to the current scheme?  But I guess first we
>> need some details as to why the current scheme isn't sufficient.
>>
> Please find the attached patch as you suggested.
>
> Reason:
> - As per my findings, ActiveState Python 3.2 does not provide shared
> libraries along with it.
>  (Though - I am not sure about the earlier version of ActiveState Python)
>  We can confirm the same using the following command:
>  ${PYTHON} -c "import distutils.sysconfig,string;
> print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))"
> Which returns in this case '0'.
>
> And, getting values for the 'python_ldlibrary' and 'python_so'  are
> 'libpython3.2m.a' and '.cpython-32m.so' respectively.
> So, the condition - *x"${python_ldlibrary}" != x"${ldlibrary}"* is always
> failing, and switching it back to link the old way.
>
> --
>
> Thanks & Regards,
>
> Ashesh Vashi
> EnterpriseDB INDIA: Enterprise PostgreSQL 
> Company
>
>
>
> *http://www.linkedin.com/in/asheshvashi*
>
>
>>
>>regards, tom lane
>>
>
>


pg9.1beta3_python_v3.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Dimitri Fontaine]

Magnus Hagander  writes:
> Also, unrelated to that, wouldn't this information be interesting for
> non-autovacuum queries as well?

I was about to say that I would like to see it for normal queries too,
but I guess we already have it:

=> explain (analyze, buffers, costs off)
   select * from pg_attribute a join pg_class c on a.attrelid = c.oid;  


   QUERY PLAN   

 Hash Join (actual time=0.569..4.255 rows=2158 loops=1)
   Hash Cond: (a.attrelid = c.oid)
   Buffers: shared hit=48
   ->  Seq Scan on pg_attribute a (actual time=0.008..0.462 rows=2158 loops=1)
 Buffers: shared hit=40
   ->  Hash (actual time=0.541..0.541 rows=282 loops=1)
 Buckets: 1024  Batches: 1  Memory Usage: 54kB
 Buffers: shared hit=8
 ->  Seq Scan on pg_class c (actual time=0.010..0.269 rows=282 loops=1)
   Buffers: shared hit=8
 Total runtime: 4.551 ms
(11 rows)


Also, from where I sit the ps title update for normal queries is about
useless, as I see loads of IDLE postgresql backends in top that are
consuming 20% and more CPU time.  The refresh rate is way to low to be
useful, and having the title it updated more frequently would probably
consume enough CPU that it would defeat its purpose (going from
instrumenting to slowing down enough that you can see what's happening
is not where I'd want to go).


Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Euler Taveira de Oliveira]

Em 18-08-2011 03:39, Magnus Hagander escreveu:

Also, unrelated to that, wouldn't this information be interesting for
non-autovacuum queries as well?

Yes, it would. AFAICS, the patch will display that message in process titles. 
However, analyze code also uses the vacuum_delay_point(). How do you handle it?


It would be another patch... autovacuum has an option to display summarized 
information but vacuum don't. Isn't it time to be symmetrical here?



--
  Euler Taveira de Oliveira - Timbira   http://www.timbira.com.br/
  PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] vacuum rusage fix

[Euler Taveira de Oliveira]

Hi,

While looking at Greg's patch I spotted that resource usage code for vacuum is 
initialized even if we won't use it. Attached is a small patch that moves it 
to the right place (this code mimics do_analyze_rel function).



--
  Euler Taveira de Oliveira - Timbira   http://www.timbira.com.br/
  PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento
diff --git a/src/backend/commands/vacuumlazy.c b/src/backend/commands/vacuumlazy.c
index c5bf32e..b5547c5 100644
*** a/src/backend/commands/vacuumlazy.c
--- b/src/backend/commands/vacuumlazy.c
*** lazy_vacuum_rel(Relation onerel, VacuumS
*** 155,165 
  	bool		scan_all;
  	TransactionId freezeTableLimit;
  
- 	pg_rusage_init(&ru0);
- 
  	/* measure elapsed time iff autovacuum logging requires it */
! 	if (IsAutoVacuumWorkerProcess() && Log_autovacuum_min_duration > 0)
! 		starttime = GetCurrentTimestamp();
  
  	if (vacstmt->options & VACOPT_VERBOSE)
  		elevel = INFO;
--- 155,167 
  	bool		scan_all;
  	TransactionId freezeTableLimit;
  
  	/* measure elapsed time iff autovacuum logging requires it */
! 	if (IsAutoVacuumWorkerProcess() && Log_autovacuum_min_duration >= 0)
! 	{
! 		pg_rusage_init(&ru0);
! 		if (Log_autovacuum_min_duration > 0)
! 			starttime = GetCurrentTimestamp();
! 	}
  
  	if (vacstmt->options & VACOPT_VERBOSE)
  		elevel = INFO;

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] plpython crash

[Jan Urbański]

On 17/08/11 23:10, Tom Lane wrote:
> =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?=  writes:
>> Here are two patches that fix two separate bugs that you found
>> simultaneously. Because they're actually separate issues, it turned out
>> fixing them was a bit more tricky than I expected (fixing one was
>> unmasking the other one etc).
> 
> Applied with one non-cosmetic change: I got rid of the test on
> TransactionIdIsValid(arg->typrel_xmin) in PLy_input_tuple_funcs,
> as well as where you'd copied that logic in PLy_output_tuple_funcs.
> AFAICS skipping the update on the xmin/tid, if we're coming through
> there a second time, would be simply wrong.

Thanks!

The way things are set up now I think you never go through
PLy_input_tuple_funcs twice, unless the cache is determined to be
invalid and then you recreate the function from scratch.

But of course it's better to be safe than sorry and even if I'm right
and it was never executed twice, any refactoring effort might have
broken it easily.

Cheers,
Jan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Compiling PostgreSQL using ActiveState Python 3.2

[Peter Eisentraut]

On tor, 2011-08-18 at 14:51 +0530, Ashesh Vashi wrote:
> Please ignore the previous patch.
> Please find the updated patch.

Committed more or less like that.

In passing I also fixed the build with Python 3 on Windows, which
apparently never worked before.  But I suppose you have been referring
to the ActiveState Linux build all along.

> 
> --
> 
> Thanks & Regards,
> 
> Ashesh Vashi
> EnterpriseDB INDIA: Enterprise PostgreSQL Company
> 
> 
> 
> *http://www.linkedin.com/in/asheshvashi*
> 
> 
> 
> On Thu, Aug 18, 2011 at 12:57 PM, Ashesh Vashi <
> ashesh.va...@enterprisedb.com> wrote:
> 
> >  On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane  wrote:
> >
> >> Peter Eisentraut  writes:
> >> > On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote:
> >> >> It's not immediately apparent to me why we should think that
> >> >> get_python_lib is less trustworthy than LIBPL; but if someone
> >> >> can make that case, I don't have any objection to this part of
> >> >> the patch.
> >>
> >> > The issue, at least for me, is that the file isn't necessarily called
> >> > 'config' anymore.  I have
> >> > /usr/lib/python3.2/config-3.2mu
> >>
> > One of the reason, I say that - we do have hard-coded values for the config
> > directory.
> > Hence, I used the LIBPL.
> >
> >>
> >> Ah, I see.
> >>
> >> > LIBPL exists at least as far back as Python 2.2, so its use should be
> >> > safe.
> >>
> >> Yeah, that part of the patch seems sane then.
> >>
> >> > Yes, because get_config_vars('LDVERSION') doesn't exist in that version.
> >> > In theory, it would return '2.7', so everything would fit back together,
> >> > but LDVERSION doesn't exist before 3.2.
> >>
> > Oops - sorry...
> > I did not know about it..
> >
> >>
> >> Could we have the code use 'LDVERSION' if it gets a nonempty result,
> >> and otherwise fall back to the current scheme?  But I guess first we
> >> need some details as to why the current scheme isn't sufficient.
> >>
> > Please find the attached patch as you suggested.
> >
> > Reason:
> > - As per my findings, ActiveState Python 3.2 does not provide shared
> > libraries along with it.
> >  (Though - I am not sure about the earlier version of ActiveState Python)
> >  We can confirm the same using the following command:
> >  ${PYTHON} -c "import distutils.sysconfig,string;
> > print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))"
> > Which returns in this case '0'.
> >
> > And, getting values for the 'python_ldlibrary' and 'python_so'  are
> > 'libpython3.2m.a' and '.cpython-32m.so' respectively.
> > So, the condition - *x"${python_ldlibrary}" != x"${ldlibrary}"* is always
> > failing, and switching it back to link the old way.
> >
> > --
> >
> > Thanks & Regards,
> >
> > Ashesh Vashi
> > EnterpriseDB INDIA: Enterprise PostgreSQL 
> > Company
> >
> >
> >
> > *http://www.linkedin.com/in/asheshvashi*
> >
> >
> >>
> >>regards, tom lane
> >>
> >
> >




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Compiling PostgreSQL using ActiveState Python 3.2

[Ashesh Vashi]

On Thu, Aug 18, 2011 at 5:25 PM, Peter Eisentraut  wrote:

> On tor, 2011-08-18 at 14:51 +0530, Ashesh Vashi wrote:
> > Please ignore the previous patch.
> > Please find the updated patch.
>
> Committed more or less like that.
>
Thanks

>
> In passing I also fixed the build with Python 3 on Windows, which
> apparently never worked before.  But I suppose you have been referring
> to the ActiveState Linux build all along.
>
Yes.

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company



*http://www.linkedin.com/in/asheshvashi*


>
> >
> > --
> >
> > Thanks & Regards,
> >
> > Ashesh Vashi
> > EnterpriseDB INDIA: Enterprise PostgreSQL Company<
> http://www.enterprisedb.com>
> >
> >
> >
> > *http://www.linkedin.com/in/asheshvashi*<
> http://www.linkedin.com/in/asheshvashi>
> >
> >
> >
> > On Thu, Aug 18, 2011 at 12:57 PM, Ashesh Vashi <
> > ashesh.va...@enterprisedb.com> wrote:
> >
> > >  On Thu, Aug 18, 2011 at 1:25 AM, Tom Lane  wrote:
> > >
> > >> Peter Eisentraut  writes:
> > >> > On ons, 2011-08-17 at 13:20 -0400, Tom Lane wrote:
> > >> >> It's not immediately apparent to me why we should think that
> > >> >> get_python_lib is less trustworthy than LIBPL; but if someone
> > >> >> can make that case, I don't have any objection to this part of
> > >> >> the patch.
> > >>
> > >> > The issue, at least for me, is that the file isn't necessarily
> called
> > >> > 'config' anymore.  I have
> > >> > /usr/lib/python3.2/config-3.2mu
> > >>
> > > One of the reason, I say that - we do have hard-coded values for the
> config
> > > directory.
> > > Hence, I used the LIBPL.
> > >
> > >>
> > >> Ah, I see.
> > >>
> > >> > LIBPL exists at least as far back as Python 2.2, so its use should
> be
> > >> > safe.
> > >>
> > >> Yeah, that part of the patch seems sane then.
> > >>
> > >> > Yes, because get_config_vars('LDVERSION') doesn't exist in that
> version.
> > >> > In theory, it would return '2.7', so everything would fit back
> together,
> > >> > but LDVERSION doesn't exist before 3.2.
> > >>
> > > Oops - sorry...
> > > I did not know about it..
> > >
> > >>
> > >> Could we have the code use 'LDVERSION' if it gets a nonempty result,
> > >> and otherwise fall back to the current scheme?  But I guess first we
> > >> need some details as to why the current scheme isn't sufficient.
> > >>
> > > Please find the attached patch as you suggested.
> > >
> > > Reason:
> > > - As per my findings, ActiveState Python 3.2 does not provide shared
> > > libraries along with it.
> > >  (Though - I am not sure about the earlier version of ActiveState
> Python)
> > >  We can confirm the same using the following command:
> > >  ${PYTHON} -c "import distutils.sysconfig,string;
> > > print(distutils.sysconfig.get_config_vars('Py_ENABLE_SHARED'))"
> > > Which returns in this case '0'.
> > >
> > > And, getting values for the 'python_ldlibrary' and 'python_so'  are
> > > 'libpython3.2m.a' and '.cpython-32m.so' respectively.
> > > So, the condition - *x"${python_ldlibrary}" != x"${ldlibrary}"* is
> always
> > > failing, and switching it back to link the old way.
> > >
> > > --
> > >
> > > Thanks & Regards,
> > >
> > > Ashesh Vashi
> > > EnterpriseDB INDIA: Enterprise PostgreSQL Company<
> http://www.enterprisedb.com/>
> > >
> > >
> > >
> > > *http://www.linkedin.com/in/asheshvashi*<
> http://www.linkedin.com/in/asheshvashi>
> > >
> > >
> > >>
> > >>regards, tom lane
> > >>
> > >
> > >
>
>
>
>

Re: [HACKERS] Displaying accumulated autovacuum cost

[Tom Lane]

Magnus Hagander  writes:
> On Thu, Aug 18, 2011 at 03:23, Greg Smith  wrote:
>> On 08/17/2011 07:42 PM, Euler Taveira de Oliveira wrote:
>>> I don't like exposing this information only on title processes.

>> I tend to build the simplest possible thing that is useful enough to work.

> By only putting it in the ps display, you exclude all the users who
> don't have an easy way to look at that information. The big group
> there is Windows, but it's not necessarily easy on all other platforms
> as well, afaik.

Yeah.  Also, process title updates are friggin expensive on some
platforms --- so much so, that we have a GUC setting to disable them.
So I don't think we should use that technique at all.  Put the info
into the stats collector instead (not "also").

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI 2PC coverage

[Heikki Linnakangas]

On 07.07.2011 18:43, Kevin Grittner wrote:

Heikki Linnakangas  wrote:

On 05.07.2011 20:06, Kevin Grittner wrote:



There's two expected output files for this, one for
max_prepared_xacts=0 and another for the "normal" case. The
max_prepared_xacts=0 case isn't very interesting, since all the
PREPARE TRANSACTION commands fail. I think we should adjust the
test harness to not run these tests at all if
max_prepared_xacts=0. It would be better to skip the test and
print out a notice pointing out that it was not run, it'll just
give a false sense of security to run the test and report success,
when it didn't test anything useful.

That alone cuts the size of the expected output to about 1 MB.


OK.  I'll work on this tonight unless Dan jumps in to claim it.


Committed. I removed the second expected output file, and marked the 
prepared-transactions tests in the schedule as "ignore" instead. That 
way if max_prepared_transactions=0, you get a notice that the test case 
failed, but pg_regress still returns 0 as exit status.



That's much better, although it's still a lot of weight just for
expected output. However, it compresses extremely well, to about
16 KB, so this isn't an issue for the size of distribution
tarballs and such, only for git checkouts and on-disk size of
extracted tarballs. I think that would be acceptable, although we
could easily cut it a bit further if we want to. For example
leaving out the word "step" from all the lines of executed test
steps would cut it by about 80 KB.


That seems simple enough.  Any other ideas?


I think it's good enough as it is. I did change the lexer slightly, to 
trim whitespace from the beginning and end of SQL blocks. This cuts the 
size of expected output a bit, and makes it look nicer anyway.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] vacuum rusage fix

[Robert Haas]

On Thu, Aug 18, 2011 at 7:02 AM, Euler Taveira de Oliveira
 wrote:
> While looking at Greg's patch I spotted that resource usage code for vacuum
> is initialized even if we won't use it. Attached is a small patch that moves
> it to the right place (this code mimics do_analyze_rel function).

Looks reasonable.  Committed.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI 2PC coverage

[Tom Lane]

Heikki Linnakangas  writes:
> On 07.07.2011 18:43, Kevin Grittner wrote:
>> OK.  I'll work on this tonight unless Dan jumps in to claim it.

> Committed. I removed the second expected output file, and marked the 
> prepared-transactions tests in the schedule as "ignore" instead. That 
> way if max_prepared_transactions=0, you get a notice that the test case 
> failed, but pg_regress still returns 0 as exit status.

Why did you only commit on 9.1 branch?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI 2PC coverage

[Heikki Linnakangas]

On 18.08.2011 17:07, Tom Lane wrote:

Heikki Linnakangas  writes:

On 07.07.2011 18:43, Kevin Grittner wrote:

OK.  I'll work on this tonight unless Dan jumps in to claim it.



Committed. I removed the second expected output file, and marked the
prepared-transactions tests in the schedule as "ignore" instead. That
way if max_prepared_transactions=0, you get a notice that the test case
failed, but pg_regress still returns 0 as exit status.


Why did you only commit on 9.1 branch?


I did "git push origin master REL9_1_STABLE", and didn't notice that it 
failed on master. Rebased and pushed, thanks!


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Robert Haas]

On Wed, Aug 17, 2011 at 9:23 PM, Greg Smith  wrote:
> On 08/17/2011 07:42 PM, Euler Taveira de Oliveira wrote:
>>
>> I don't like exposing this information only on title processes. It would
>> be difficult for client apps (for example, PGAdmin) to track this kind of
>> information and it is restricted to local access. I'm not objecting to
>> display this information in process title; I'm just saying that that
>> information should be exposed in  functions (say
>> pg_stat_get_vacuum_[hit|miss|dirty]) too.
>
> I tend to build the simplest possible thing that is useful enough to work.
>  The data is getting stored and shown now, where it wasn't before.  If it's
> possible to expose that in additional ways later too, great.  The big step
> up for this information is to go from "unobtainable" to "obtainable".  I'd
> prefer not to add a quest for "easily obtainable" to the requirements until
> that big jump is made, for fear it will cause nothing to get delivered.

Perhaps a reasonable way to break up the patch would be:

- Part 1: Gather the information and display it in the
log_autovacuum_min_duration output.
- Part 2: Add the ability to see the information incrementally (via
some mechanism yet to be agreed upon).

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Change format of FDW options used in \d* commands

[Robert Haas]

2011/8/18 Shigeru Hanada :
> Thanks for the review.
>
> (2011/08/13 3:59), Robert Haas wrote:
>> IMHO, the new format should put parentheses around the options list.
>
> Agreed.  Revised version of patch has been attached.  This version puts
> parentheses around FDW option only when it was not NULL.

ISTM you should do quote_ident() on the option names.  For example, after:

create foreign data wrapper dummy;
create server s1 foreign data wrapper dummy;
create foreign table ft1 (a int) server s1 options ("(" ')');

With your patch, I get:

rhaas=# \d+ ft1
 Foreign table "public.ft1"
 Column |  Type   | Modifiers | FDW Options | Storage | Description
+-+---+-+-+-
 a  | integer |   | | plain   |
Server: s1
FDW Options: (( ')')
Has OIDs: no

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Peter Eisentraut]

On tor, 2011-08-18 at 08:39 +0200, Magnus Hagander wrote:
> Also, unrelated to that, wouldn't this information be interesting for
> non-autovacuum queries as well?

Last year we were discussing some details on progress reporting, and
some people suggested that instead of printing a single percentage, we
should let each type of activity print out whatever metrics it has that
would allow an experienced DBA to track the progress.  Effectively, this
is what this patch is trying to do.

So how about adding a column to pg_stat_activity, progress_metrics or
something like that, and add that information there.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Robert Haas]

On Thu, Aug 18, 2011 at 10:54 AM, Peter Eisentraut  wrote:
> On tor, 2011-08-18 at 08:39 +0200, Magnus Hagander wrote:
>> Also, unrelated to that, wouldn't this information be interesting for
>> non-autovacuum queries as well?
>
> Last year we were discussing some details on progress reporting, and
> some people suggested that instead of printing a single percentage, we
> should let each type of activity print out whatever metrics it has that
> would allow an experienced DBA to track the progress.  Effectively, this
> is what this patch is trying to do.
>
> So how about adding a column to pg_stat_activity, progress_metrics or
> something like that, and add that information there.

That might be a good way to go.  I don't think we want something like
pg_stat_all_tables for this, because it seems that Greg's use case is
to be able to see how a *particular* autovacuum process is doing
*while it's running*, not to look at aggregate statistics over time.
Putting it in pg_stat_activity would be good for that.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Magnus Hagander]

On Thu, Aug 18, 2011 at 17:13, Robert Haas  wrote:
> On Thu, Aug 18, 2011 at 10:54 AM, Peter Eisentraut  wrote:
>> On tor, 2011-08-18 at 08:39 +0200, Magnus Hagander wrote:
>>> Also, unrelated to that, wouldn't this information be interesting for
>>> non-autovacuum queries as well?
>>
>> Last year we were discussing some details on progress reporting, and
>> some people suggested that instead of printing a single percentage, we
>> should let each type of activity print out whatever metrics it has that
>> would allow an experienced DBA to track the progress.  Effectively, this
>> is what this patch is trying to do.
>>
>> So how about adding a column to pg_stat_activity, progress_metrics or
>> something like that, and add that information there.
>
> That might be a good way to go.  I don't think we want something like
> pg_stat_all_tables for this, because it seems that Greg's use case is
> to be able to see how a *particular* autovacuum process is doing
> *while it's running*, not to look at aggregate statistics over time.
> Putting it in pg_stat_activity would be good for that.

It's also good to have it broken down into multiple columns, and not
just a freetext column in the view - if tools should be able to parse
it as well.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Robert Haas]

On Thu, Aug 18, 2011 at 11:14 AM, Magnus Hagander  wrote:
> On Thu, Aug 18, 2011 at 17:13, Robert Haas  wrote:
>> On Thu, Aug 18, 2011 at 10:54 AM, Peter Eisentraut  wrote:
>>> On tor, 2011-08-18 at 08:39 +0200, Magnus Hagander wrote:
 Also, unrelated to that, wouldn't this information be interesting for
 non-autovacuum queries as well?
>>>
>>> Last year we were discussing some details on progress reporting, and
>>> some people suggested that instead of printing a single percentage, we
>>> should let each type of activity print out whatever metrics it has that
>>> would allow an experienced DBA to track the progress.  Effectively, this
>>> is what this patch is trying to do.
>>>
>>> So how about adding a column to pg_stat_activity, progress_metrics or
>>> something like that, and add that information there.
>>
>> That might be a good way to go.  I don't think we want something like
>> pg_stat_all_tables for this, because it seems that Greg's use case is
>> to be able to see how a *particular* autovacuum process is doing
>> *while it's running*, not to look at aggregate statistics over time.
>> Putting it in pg_stat_activity would be good for that.
>
> It's also good to have it broken down into multiple columns, and not
> just a freetext column in the view - if tools should be able to parse
> it as well.

True.  We could have a separate system view that only shows the status
of currently-running vacuum proceses.  That wouldn't bother me either.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Magnus Hagander]

On Thu, Aug 18, 2011 at 17:23, Robert Haas  wrote:
> On Thu, Aug 18, 2011 at 11:14 AM, Magnus Hagander  wrote:
>> On Thu, Aug 18, 2011 at 17:13, Robert Haas  wrote:
>>> On Thu, Aug 18, 2011 at 10:54 AM, Peter Eisentraut  wrote:
 On tor, 2011-08-18 at 08:39 +0200, Magnus Hagander wrote:
> Also, unrelated to that, wouldn't this information be interesting for
> non-autovacuum queries as well?

 Last year we were discussing some details on progress reporting, and
 some people suggested that instead of printing a single percentage, we
 should let each type of activity print out whatever metrics it has that
 would allow an experienced DBA to track the progress.  Effectively, this
 is what this patch is trying to do.

 So how about adding a column to pg_stat_activity, progress_metrics or
 something like that, and add that information there.
>>>
>>> That might be a good way to go.  I don't think we want something like
>>> pg_stat_all_tables for this, because it seems that Greg's use case is
>>> to be able to see how a *particular* autovacuum process is doing
>>> *while it's running*, not to look at aggregate statistics over time.
>>> Putting it in pg_stat_activity would be good for that.
>>
>> It's also good to have it broken down into multiple columns, and not
>> just a freetext column in the view - if tools should be able to parse
>> it as well.
>
> True.  We could have a separate system view that only shows the status
> of currently-running vacuum proceses.  That wouldn't bother me either.

That's what I'd like to have. We could also have aggregate counters on
the table/database level of course.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Greg Smith]

On 08/18/2011 10:12 AM, Robert Haas wrote:

Perhaps a reasonable way to break up the patch would be:
- Part 1: Gather the information and display it in the
log_autovacuum_min_duration output.
- Part 2: Add the ability to see the information incrementally (via
some mechanism yet to be agreed upon).
   


My reaction to all the suggestions for redesign is just that: pull out 
the part that does the incremental updates altogether, improve the part 
that dumps the info into the logs, and resubmit without any incremental 
progress for now.  This is much more valuable to me if the first commit 
that hits is something I can backport trivially.  I'm seeing enough 
production servers running into this problem right now on earlier 
versions to be worried about that, and the log dump at the end would be 
a huge help even if that was all they got.  I'm going to add directly 
computing the write MB/s figure from the dirty data written too, since 
that ends up being the thing that I keep deriving by hand anyway.


--
Greg Smith   2ndQuadrant USg...@2ndquadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Greg Smith]

On 08/18/2011 10:54 AM, Peter Eisentraut wrote:

So how about adding a column to pg_stat_activity, progress_metrics or
something like that, and add that information there.
   


Adding a field here (I'd go for the simpler "progress") and updating it 
regularly would be a reasonable way to go here.  This data doesn't 
really need to go into the traditional statistics infrastructure to be 
useful.  I didn't start there because I was already getting pushback on 
overloading the stats collector with constantly updated metrics last 
time I did something in this area.  I wasn't going to try and argue why 
it was worth it in this case, just like I'm not going to argue about the 
complaint over the command string overhead being too high--just going to 
not do that instead.  If the bikeshed I built doesn't look fancy enough 
to hold the bike I put in there, I'm not going to build a better one 
right now--I'll just put a cheaper bike in there instead.


I was hoping to eventually take the useful summary bits at the end, the 
totals, and save those into statistics somewhere each time a VACUUM of 
either sort finishes.  It would fit with the information shown in 
pg_stat_tables, but that's obviously getting too wide.  Breaking out a 
pg_stat_autovacuum view that contains all the relevant bits currently 
shown in that view, plus these 3 new fields, would be a reasonable start.


--
Greg Smith   2ndQuadrant USg...@2ndquadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Magnus Hagander]

On Thu, Aug 18, 2011 at 17:54, Greg Smith  wrote:
> On 08/18/2011 10:54 AM, Peter Eisentraut wrote:
>>
>> So how about adding a column to pg_stat_activity, progress_metrics or
>> something like that, and add that information there.
>>
>
> Adding a field here (I'd go for the simpler "progress") and updating it
> regularly would be a reasonable way to go here.  This data doesn't really
> need to go into the traditional statistics infrastructure to be useful.  I
> didn't start there because I was already getting pushback on overloading the
> stats collector with constantly updated metrics last time I did something in
> this area.  I wasn't going to try and argue why it was worth it in this
> case, just like I'm not going to argue about the complaint over the command
> string overhead being too high--just going to not do that instead.  If the
> bikeshed I built doesn't look fancy enough to hold the bike I put in there,
> I'm not going to build a better one right now--I'll just put a cheaper bike
> in there instead.

The "current values per-backend" thing can go in shared memory. The
reason the per table ones can't is obviously that they go away when
the backend disconnects..


> I was hoping to eventually take the useful summary bits at the end, the
> totals, and save those into statistics somewhere each time a VACUUM of
> either sort finishes.  It would fit with the information shown in
> pg_stat_tables, but that's obviously getting too wide.  Breaking out a
> pg_stat_autovacuum view that contains all the relevant bits currently shown
> in that view, plus these 3 new fields, would be a reasonable start.

That depends on what you mea nby too wide. If it's intended to be
consumed with "SELECT *" or not...


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

[Robert Haas]

On Thu, Jul 21, 2011 at 5:29 AM, Kohei Kaigai  wrote:
> The attached patch is revised userspace-avc patch.
>
> List of updates:
> - The GUC of sepgsql.avc_threshold was removed.
> - "char *ucontext" of avc_cache was replaced by "bool tcontext_is_valid".
> - Comments added onto static variables
> - Comments of sepgsql_avc_unlabeled() was revised.
> - Comments of sepgsql_avc_compute() was simplified.
> - Comments of sepgsql_avc_check_perms_label() also mention about
>  permissive domain, that performs similar to system's permissive mode.
> - selinux_status_close() become invoked on on_proc_exit() hook.

I tried to give this a test drive today but got stuck.  I got sepgsql
compiled OK, but look what happens when I try to start the server:

[rhaas@f15selinux ~]$ postgres
FATAL:  could not load library
"/home/rhaas/project/lib/postgresql/sepgsql.so":
/home/rhaas/project/lib/postgresql/sepgsql.so: undefined symbol:
getpeercon_raw

This is Fedora 15, with all available updates applied.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Robert Haas]

On Thu, Aug 18, 2011 at 11:41 AM, Greg Smith  wrote:
> On 08/18/2011 10:12 AM, Robert Haas wrote:
>> Perhaps a reasonable way to break up the patch would be:
>> - Part 1: Gather the information and display it in the
>> log_autovacuum_min_duration output.
>> - Part 2: Add the ability to see the information incrementally (via
>> some mechanism yet to be agreed upon).
>
> My reaction to all the suggestions for redesign is just that: pull out the
> part that does the incremental updates altogether, improve the part that
> dumps the info into the logs, and resubmit without any incremental progress
> for now.  This is much more valuable to me if the first commit that hits is
> something I can backport trivially.  I'm seeing enough production servers
> running into this problem right now on earlier versions to be worried about
> that, and the log dump at the end would be a huge help even if that was all
> they got.  I'm going to add directly computing the write MB/s figure from
> the dirty data written too, since that ends up being the thing that I keep
> deriving by hand anyway.

By the way, since I forgot to say it earlier, I think it's great that
you are working on some of this instrumentation stuff, so +1 for the
basic concept here.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

[Robert Haas]

On Thu, Aug 18, 2011 at 12:46 PM, Robert Haas  wrote:
> On Thu, Jul 21, 2011 at 5:29 AM, Kohei Kaigai  
> wrote:
>> The attached patch is revised userspace-avc patch.
>>
>> List of updates:
>> - The GUC of sepgsql.avc_threshold was removed.
>> - "char *ucontext" of avc_cache was replaced by "bool tcontext_is_valid".
>> - Comments added onto static variables
>> - Comments of sepgsql_avc_unlabeled() was revised.
>> - Comments of sepgsql_avc_compute() was simplified.
>> - Comments of sepgsql_avc_check_perms_label() also mention about
>>  permissive domain, that performs similar to system's permissive mode.
>> - selinux_status_close() become invoked on on_proc_exit() hook.
>
> I tried to give this a test drive today but got stuck.  I got sepgsql
> compiled OK, but look what happens when I try to start the server:
>
> [rhaas@f15selinux ~]$ postgres
> FATAL:  could not load library
> "/home/rhaas/project/lib/postgresql/sepgsql.so":
> /home/rhaas/project/lib/postgresql/sepgsql.so: undefined symbol:
> getpeercon_raw
>
> This is Fedora 15, with all available updates applied.

Oh.  Apparently, this is what happens when you try to build sepgsql
without passing --with-selinux to configure.

That's lame.  I think we need to patch contrib/sepgsql so that it
fails to build in that case, rather than building and then not
working.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

[Robert Haas]

On Thu, Aug 18, 2011 at 12:52 PM, Robert Haas  wrote:
> On Thu, Aug 18, 2011 at 12:46 PM, Robert Haas  wrote:
>> On Thu, Jul 21, 2011 at 5:29 AM, Kohei Kaigai  
>> wrote:
>>> The attached patch is revised userspace-avc patch.
>>>
>>> List of updates:
>>> - The GUC of sepgsql.avc_threshold was removed.
>>> - "char *ucontext" of avc_cache was replaced by "bool tcontext_is_valid".
>>> - Comments added onto static variables
>>> - Comments of sepgsql_avc_unlabeled() was revised.
>>> - Comments of sepgsql_avc_compute() was simplified.
>>> - Comments of sepgsql_avc_check_perms_label() also mention about
>>>  permissive domain, that performs similar to system's permissive mode.
>>> - selinux_status_close() become invoked on on_proc_exit() hook.
>>
>> I tried to give this a test drive today but got stuck.  I got sepgsql
>> compiled OK, but look what happens when I try to start the server:
>>
>> [rhaas@f15selinux ~]$ postgres
>> FATAL:  could not load library
>> "/home/rhaas/project/lib/postgresql/sepgsql.so":
>> /home/rhaas/project/lib/postgresql/sepgsql.so: undefined symbol:
>> getpeercon_raw
>>
>> This is Fedora 15, with all available updates applied.
>
> Oh.  Apparently, this is what happens when you try to build sepgsql
> without passing --with-selinux to configure.
>
> That's lame.  I think we need to patch contrib/sepgsql so that it
> fails to build in that case, rather than building and then not
> working.

Also, I get these warnings:

/etc/selinux/targeted/contexts/sepgsql_contexts:  line 33 has invalid
object type db_blobs
/etc/selinux/targeted/contexts/sepgsql_contexts:  line 36 has invalid
object type db_language
/etc/selinux/targeted/contexts/sepgsql_contexts:  line 37 has invalid
object type db_language
/etc/selinux/targeted/contexts/sepgsql_contexts:  line 38 has invalid
object type db_language
/etc/selinux/targeted/contexts/sepgsql_contexts:  line 39 has invalid
object type db_language
/etc/selinux/targeted/contexts/sepgsql_contexts:  line 40 has invalid
object type db_language
 1: sepgsql_restorecon = "t"(typeid = 16, len = 1, typmod = -1, 
byval = t)

The first is mentioned in the latest documentation, but the rest are not.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

[Kohei Kaigai]

> That's lame.  I think we need to patch contrib/sepgsql so that it
> fails to build in that case, rather than building and then not
> working.
> 
It might be the following fix, but I have no idea to generate an error when 
$(with_selinux) != "yes" on makefile.

diff --git a/contrib/sepgsql/Makefile b/contrib/sepgsql/Makefile
index 7f997ee..fec4f1a 100644
--- a/contrib/sepgsql/Makefile
+++ b/contrib/sepgsql/Makefile
@@ -19,6 +19,12 @@ include $(top_builddir)/src/Makefile.global
 include $(top_srcdir)/contrib/contrib-global.mk
 endif

+ifneq ($(with_selinux),yes)
+##
+## Error generation
+##
+endif
+
 SHLIB_LINK += $(filter -lselinux, $(LIBS))
 REGRESS_OPTS += --launcher $(top_builddir)/contrib/sepgsql/launcher

--
NEC Europe Ltd, SAP Global Competence Center
KaiGai Kohei 


> -Original Message-
> From: Robert Haas [mailto:robertmh...@gmail.com]
> Sent: 18. August 2011 17:53
> To: Kohei Kaigai
> Cc: Yeb Havinga; PgHacker; Kohei KaiGai
> Subject: Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache
> 
> On Thu, Aug 18, 2011 at 12:46 PM, Robert Haas  wrote:
> > On Thu, Jul 21, 2011 at 5:29 AM, Kohei Kaigai  
> > wrote:
> >> The attached patch is revised userspace-avc patch.
> >>
> >> List of updates:
> >> - The GUC of sepgsql.avc_threshold was removed.
> >> - "char *ucontext" of avc_cache was replaced by "bool tcontext_is_valid".
> >> - Comments added onto static variables
> >> - Comments of sepgsql_avc_unlabeled() was revised.
> >> - Comments of sepgsql_avc_compute() was simplified.
> >> - Comments of sepgsql_avc_check_perms_label() also mention about
> >>  permissive domain, that performs similar to system's permissive mode.
> >> - selinux_status_close() become invoked on on_proc_exit() hook.
> >
> > I tried to give this a test drive today but got stuck.  I got sepgsql
> > compiled OK, but look what happens when I try to start the server:
> >
> > [rhaas@f15selinux ~]$ postgres
> > FATAL:  could not load library
> > "/home/rhaas/project/lib/postgresql/sepgsql.so":
> > /home/rhaas/project/lib/postgresql/sepgsql.so: undefined symbol:
> > getpeercon_raw
> >
> > This is Fedora 15, with all available updates applied.
> 
> Oh.  Apparently, this is what happens when you try to build sepgsql
> without passing --with-selinux to configure.
> 
> That's lame.  I think we need to patch contrib/sepgsql so that it
> fails to build in that case, rather than building and then not
> working.
> 
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
> 
> 
>  Click
> https://www.mailcontrol.com/sr/i+p!jARD6rnTndxI!oX7Uu+NqWBeKfvsxHen8ElqAIKK2vDQ5PIqETvu3D1VdIOLM1
> BV3YJKcc+1yubdBaCdqw==  to report this email as spam.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

[Robert Haas]

On Thu, Aug 18, 2011 at 1:00 PM, Robert Haas  wrote:
> [more problems]

OK, I'm giving up for now.  I hit two more snags:

1. chkselinuxenv uses "which", and a Fedora 15 minimal install doesn't
include that.  I fixed that by installing "which", but maybe we ought
to be looking for a way to eliminate that dependency, like testing for
the commands you need by running them with --help, or something like
that.

2. restorecon doesn't correctly set the permissions for me on
~/project/bin/psql.  I get:

[rhaas@f15selinux sepgsql]$ ls -Z ~/project/bin/psql
-rwxr-xr-x. rhaas rhaas unconfined_u:object_r:user_home_t:s0
/home/rhaas/project/bin/psql

Now I can fix that by applying bin_t manually, as suggested in the
documentation.  However, that just moves the failure to library load
time.  regression.diffs has multiple copies of this error message:

/home/rhaas/project/bin/psql: error while loading shared libraries:
libpq.so.5: failed to map segment from shared object: Permission
denied

Help!

Thanks,

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Full GUID support

[David E. Wheeler]

On Jul 14, 2011, at 9:53 AM, David E. Wheeler wrote:

>> (2011/07/14 3:49), Thomas Lotterer wrote:
>>> Thanks for the hint.
>>> Our ftp daemon is dumping core.
>>> We are debugging ...
> 
> Ah, good news, thanks.
> 
> Where should I report stuff like this in the future? I sent a message about 
> this to r...@engelschall.com on May 15th and also a Twitter DM but didn't 
> hear back…

Hi Thomas,

Did you ever get your FTP server fixed? I'm still not able to download uuid:

curl -O ftp://ftp.ossp.org/pkg/lib/uuid/uuid-1.6.2.tar.gz
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:--  0:00:02 --:--:-- 0
curl: (7) couldn't connect to host

Thanks,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

[Kohei Kaigai]

> OK, I'm giving up for now.  I hit two more snags:
> 
> 1. chkselinuxenv uses "which", and a Fedora 15 minimal install doesn't
> include that.  I fixed that by installing "which", but maybe we ought
> to be looking for a way to eliminate that dependency, like testing for
> the commands you need by running them with --help, or something like
> that.
> 
Oops, I thought "which" is a part of coreutils.

I'll try to update chkselinuxenv to print a help message when necessary 
commands are not installed.

> 2. restorecon doesn't correctly set the permissions for me on
> ~/project/bin/psql.  I get:
> 
> [rhaas@f15selinux sepgsql]$ ls -Z ~/project/bin/psql
> -rwxr-xr-x. rhaas rhaas unconfined_u:object_r:user_home_t:s0
> /home/rhaas/project/bin/psql
> 
> Now I can fix that by applying bin_t manually, as suggested in the
> documentation.  However, that just moves the failure to library load
> time.  regression.diffs has multiple copies of this error message:
> 
> /home/rhaas/project/bin/psql: error while loading shared libraries:
> libpq.so.5: failed to map segment from shared object: Permission
> denied
>
I guess it tries to mmap(2) libpq.so.5 (labeled as user_home_t) with executable 
mode.
The regression test switches domain of psql command on its execution from 
"unconfined_t" to "sepgsql_regtest_user_t", however, I didn't allow this domain 
to mmap(2) files in user's home directory with executable mode.
It may need to revise the security policy of regression test to support 
installation onto home directory.

As a quick avoidance, how about --prefix=/usr/local/sepgsql instead?

Thanks,
--
NEC Europe Ltd, SAP Global Competence Center
KaiGai Kohei 


> -Original Message-
> From: Robert Haas [mailto:robertmh...@gmail.com]
> Sent: 18. August 2011 18:22
> To: Kohei Kaigai
> Cc: Yeb Havinga; PgHacker; Kohei KaiGai
> Subject: Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache
> 
> On Thu, Aug 18, 2011 at 1:00 PM, Robert Haas  wrote:
> > [more problems]
> 
> OK, I'm giving up for now.  I hit two more snags:
> 
> 1. chkselinuxenv uses "which", and a Fedora 15 minimal install doesn't
> include that.  I fixed that by installing "which", but maybe we ought
> to be looking for a way to eliminate that dependency, like testing for
> the commands you need by running them with --help, or something like
> that.
> 
> 2. restorecon doesn't correctly set the permissions for me on
> ~/project/bin/psql.  I get:
> 
> [rhaas@f15selinux sepgsql]$ ls -Z ~/project/bin/psql
> -rwxr-xr-x. rhaas rhaas unconfined_u:object_r:user_home_t:s0
> /home/rhaas/project/bin/psql
> 
> Now I can fix that by applying bin_t manually, as suggested in the
> documentation.  However, that just moves the failure to library load
> time.  regression.diffs has multiple copies of this error message:
> 
> /home/rhaas/project/bin/psql: error while loading shared libraries:
> libpq.so.5: failed to map segment from shared object: Permission
> denied
> 
> Help!
> 
> Thanks,
> 
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
> 
> 
>  Click
> https://www.mailcontrol.com/sr/g7UEZIfD10rTndxI!oX7Unz1!gA0DCbilsfI53CIRke!PbNpuk4RnjmGfZ8cEe1DM1
> BV3YJKcc9jEfBJ2k7YZA==  to report this email as spam.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

[Robert Haas]

On Thu, Aug 18, 2011 at 1:17 PM, Kohei Kaigai  wrote:
>> That's lame.  I think we need to patch contrib/sepgsql so that it
>> fails to build in that case, rather than building and then not
>> working.
>>
> It might be the following fix, but I have no idea to generate an error when 
> $(with_selinux) != "yes" on makefile.

Actually, as I look at this more, I think this build system is
completely mis-designed.  Given that you want to build sepgsql,
selinux is not an optional feature.  So the stuff in
contrib/sepgsql/Makefile that is intended to link against libselinux
only if --with-selinux was specified at configure time is nonsense.
We should just ALWAYS try to link against libselinux, and if it's not
there, then at least it'll fail right away at compile time instead of
appearing to compile OK but producing an so that then fails to load at
runtime.

The only actual legitimate purpose of --with-selinux is to allow
contrib/Makefile to decide whether, when someone tries to build "all
the contrib modules", we should try to build sepgsql too.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Displaying accumulated autovacuum cost

[Euler Taveira de Oliveira]

Em 18-08-2011 12:54, Greg Smith escreveu:

I was hoping to eventually take the useful summary bits at the end, the
totals, and save those into statistics somewhere each time a VACUUM of
either sort finishes. It would fit with the information shown in
pg_stat_tables, but that's obviously getting too wide. Breaking out a
pg_stat_autovacuum view that contains all the relevant bits currently
shown in that view, plus these 3 new fields, would be a reasonable start.

IMHO the useful summary bits belongs to log. If you want to add it to stats 
collector go for it. But if you go to the latter road, it is recommended to 
move some fields (time-related fields) from pg_stat_*_tables to this new view 
(pg_stat_maintenance?). I don't know how generic you want to go but have in 
mind I would like to cover automatic and manual maintenance commands.


Besides that another view will cover the maintenance activity. This new view 
could contain at least datname, schemaname, relname, command_start, 
command_schedule, operation, progress (?), procpid, and current_command. The 
name has to be generic to cover all maintenance commands (perhaps 
pg_maintenance_activity).



--
  Euler Taveira de Oliveira - Timbira   http://www.timbira.com.br/
  PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] the big picture for index-only scans

[Robert Haas]

On Tue, May 10, 2011 at 8:19 PM, Bruce Momjian  wrote:
> Robert Haas wrote:
>> >> Any thoughts welcome. ?Incidentally, if anyone else feels like working
>> >> on this, feel free to let me know and I'm happy to step away, from all
>> >> of it or from whatever part someone else wants to tackle. ?I'm mostly
>> >> working on this because it's something that I think we really need to
>> >> get done, more than having a burning desire to be the one who does it.
>> >
>> > Indexonly scans are welcome!
>> > I believe I can help on 3 and 4, but (really) not sure for 1 and 2.
>>
>> Well, I have code for #1, and just need reviews, and #2 shouldn't be
>> that hard, and with luck I'll twist Bruce's arm into doing it (*waves
>> to Bruce*).  So #3 and #4 are the next thing to tackle.  Any thoughts
>> on what/how you'd like to contribute there?
>
> I am happy to have pg_upgrade skip upgrading visibility map files --- it
> already has code to conditionally process them because they only exist
> in >= 8.4:
>
>        /* fsm/vm files added in PG 8.4 */
>        if (GET_MAJOR_VERSION(old_cluster.major_version) >= 804)
>        {
>            /*
>             * Copy/link any fsm and vm files, if they exist
>             */
>
> Just give the word and it will be done.

I hereby give the word.  :-)

Specifically, we need to skip copying vm files (only) if coming from a
version prior to this commit:

http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=e16954f3d27fa8e16c379ff6623ae18d6250a39c

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] mb_regress.sh gripes

[Josh Kupershmidt]

Hi all,

A few gripes about mb_regress.sh:
 1. No exit code is specified, so even if there are differences
between results/ and expected/ the script will still return 0.

 2. The 'dropdb' command is used to wipe out the "utf8" database
before the run. This generates an error message like:
  dropdb: database removal failed: ERROR:  database "utf8" does not exist

the first time you run the script. IMO it would be less startling to
just print a NOTICE here.

 3. No error checking for whether createdb succeeds.

The attached patch fixes these problems.

Josh


mb_regress_gripes.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] "make -j4 world" falls over

[Tom Lane]

I ran into $SUBJECT whilst doing trial RPM packaging of 9.1.  The problem
is that make starts building contrib modules before errcodes.h has been
made, leading to failures such as

In file included from ../../src/include/postgres.h:48:0,
 from auth_delay.c:12:
../../src/include/utils/elog.h:69:28: fatal error: utils/errcodes.h: No such 
file or directory
compilation terminated.
In file included from ../../src/include/postgres.h:48:0,
 from adminpack.c:15:
../../src/include/utils/elog.h:69:28: fatal error: utils/errcodes.h: No such 
file or directory
compilation terminated.

I can work around this for the moment, but I think it's a "must fix"
before release.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers