Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 04/03/2014 12:01 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: For OSX we'd construct the list via File::Find to recurse through the directories. So, something like this: Thanks for the tips. The attached patch against buildfarm 4.11 seems to work, cf http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedarydt=2014-04-03%2003%3A33%3A16stg=typedefs I tried it on current OSX (10.9.2) as well as dromedary's 10.6.8. It does not work on prairiedog (10.4.11) --- the debug info format seems to be different that far back. Can't say that's worth worrying about. Thanks, applied. BTW, after looking a bit more closely at what this added to the typedefs list, I realize that this mechanism also collects typedefs that appear in files that are compiled but never installed, for example src/timezone/zic and the ecpg regression tests. This seems like a Good Thing, since certainly pgindent is going to hit the source files for those programs too. I wonder if we ought to switch over to scanning the .o files on all platforms? Sure, if someone wants to work out what's involved. Do the MSVC tools have some gadget for extracting symbols from .o files in a similar way? It would be nice to get them into the mix too. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 04/02/2014 12:25 AM, Andrew Dunstan wrote: On 04/01/2014 09:22 PM, Andrew Dunstan wrote: On 04/01/2014 08:53 PM, Tom Lane wrote: The current typedefs list seems to be lacking any Windows-only typedefs. Noticed while trying to pgindent postmaster.c. Hmm. odd. will check. It's apparently causing the buildfarm to crash, which is why I must have disabled it. I'll chase that down tomorrow. OK, we're back: http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=frogmouthdt=2014-04-02%2016%3A08%3A12stg=typedefs BTW, three animals are currently trying to contribute typedefs but aren't in fact contributing anything: okapi, dromedary and prairiedog. See http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1 I can't really help much on these as my Gentoo facilities are non-existent, and my OSX facilities are not much better. I do recall trying to find a way to get typedefs on OSX a few years ago, without success. cheers andrew cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
Andrew Dunstan and...@dunslane.net writes: BTW, three animals are currently trying to contribute typedefs but aren't in fact contributing anything: okapi, dromedary and prairiedog. See http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1 Man, that's a short list. I wonder if we need to encourage more people to do this. I can't really help much on these as my Gentoo facilities are non-existent, and my OSX facilities are not much better. I do recall trying to find a way to get typedefs on OSX a few years ago, without success. I poked around a bit, and so far as I can tell, OS X does not store debug symbol tables in executables. It looks like gdb goes to the .o files when it wants debug info. What's in the .o files is good ol' DWARF (at least in reasonably recent OS X releases), so it's not any harder to pull out the typedef names than it is on Linux. The problem is that you gotta iterate over all the .o files in the build tree rather than the installed executables. I looked at fixing find_typedefs but it seems like it's pretty fixated on the latter approach; any thoughts on how to revise it? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 04/02/2014 08:43 PM, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
BTW, three animals are currently trying to contribute typedefs but
aren't in fact contributing anything: okapi, dromedary and prairiedog.
See http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1
Man, that's a short list. I wonder if we need to encourage more people
to do this.
I can't really help much on these as my Gentoo facilities are
non-existent, and my OSX facilities are not much better. I do recall
trying to find a way to get typedefs on OSX a few years ago, without
success.
I poked around a bit, and so far as I can tell, OS X does not store debug
symbol tables in executables. It looks like gdb goes to the .o files when
it wants debug info. What's in the .o files is good ol' DWARF (at least
in reasonably recent OS X releases), so it's not any harder to pull out
the typedef names than it is on Linux. The problem is that you gotta
iterate over all the .o files in the build tree rather than the installed
executables. I looked at fixing find_typedefs but it seems like it's
pretty fixated on the latter approach; any thoughts on how to revise it?
Well, the reason it's that way is that that's the way it was done before
the buildfarm took over the task. But it's not holy writ. Doing
something else would be a SMOC.
Essentially, I think the part that would need to change is this:
foreach my $bin (
glob($installdir/bin/*),
glob($installdir/lib/*),
glob($installdir/lib/postgresql/*)
)
For OSX we'd construct the list via File::Find to recurse through the
directories.
So, something like this:
my $using_osx = [some test for OSX];
my @testfiles;
my $obj_wanted = sub {
/^.*\.o\z/s
push(@testfiles, $File::Find::name);
};
if ($using_osx)
{
File::Find::find($obj_wanted,$pgsql);
}
else
{
@testfiles = (
glob($installdir/bin/*),
glob($installdir/lib/*),
glob($installdir/lib/postgresql/*);
}
foreach my $bin (@testfiles)
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 2 Apr 2014, at 5:43 PM, Tom Lane wrote: I poked around a bit, and so far as I can tell, OS X does not store debug symbol tables in executables. It looks like gdb goes to the .o files when it wants debug info. What's in the .o files is good ol' DWARF (at least in reasonably recent OS X releases), so it's not any harder to pull out the typedef names than it is on Linux. The problem is that you gotta iterate over all the .o files in the build tree rather than the installed executables. I looked at fixing find_typedefs but it seems like it's pretty fixated on the latter approach; any thoughts on how to revise it? The Apple development tools gather the debug information during the final link stage (the one that produces the executable or shared object) using dsymutil, which simply iterates over all of the .o files and links the debug info into a separate object, foo.dSYM. Apple's gdb and lldb then find the relevant .dSYM file using a per-build UUID embedded in the executable/library/debug symbol file. The dSYM file is a normal object file which has the DWARF sections but not the usual text/data sections, so if it can be generated/found, it should be possible to dump the DWARF data from it and look for typedefs that way. (I'm pretty sure that if you were to run dsymutil and then merge the resulting object file's sections into the executable/shlib, you'd get a perfectly functional and debuggable result without having to look for or cart around the extra dSYM file--- I haven't tried this though.) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
Wim Lewis w...@omnigroup.com writes: On 2 Apr 2014, at 5:43 PM, Tom Lane wrote: I poked around a bit, and so far as I can tell, OS X does not store debug symbol tables in executables. The Apple development tools gather the debug information during the final link stage (the one that produces the executable or shared object) using dsymutil, which simply iterates over all of the .o files and links the debug info into a separate object, foo.dSYM. Apple's gdb and lldb then find the relevant .dSYM file using a per-build UUID embedded in the executable/library/debug symbol file. Ah. I've forgotten the details, but I'm pretty sure that we have deliberately arranged our build process so that the .dSYM files don't get built during link steps. Debugging seems to work all right anyway, at least if the build tree is available, so I think Apple's gdb is able to work from the symbol tables in the .o files. While perhaps that approach should be rethought, I'm disinclined to mess with it just for the benefit of find_typedefs. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
Andrew Dunstan and...@dunslane.net writes:
For OSX we'd construct the list via File::Find to recurse through the
directories.
So, something like this:
Thanks for the tips. The attached patch against buildfarm 4.11 seems
to work, cf
http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedarydt=2014-04-03%2003%3A33%3A16stg=typedefs
I tried it on current OSX (10.9.2) as well as dromedary's 10.6.8.
It does not work on prairiedog (10.4.11) --- the debug info format
seems to be different that far back. Can't say that's worth worrying
about.
regards, tom lane
*** build-farm-4.11/run_build.pl~ Fri Jun 14 09:05:52 2013
--- build-farm-4.11/run_build.pl Wed Apr 2 23:31:27 2014
***
*** 1506,1520
my $objdump = $host || 'objdump';
my @err = `$objdump -W 21`;
my @readelferr = `readelf -w 21`;
my %syms;
my @dumpout;
my @flds;
! foreach my $bin (
! glob($installdir/bin/*),
! glob($installdir/lib/*),
! glob($installdir/lib/postgresql/*)
! )
{
next if $bin =~ m!bin/(ipcclean|pltcl_)!;
next unless -f $bin;
--- 1506,1536
my $objdump = $host || 'objdump';
my @err = `$objdump -W 21`;
my @readelferr = `readelf -w 21`;
+ my $using_osx = (`uname` eq Darwin\n);
+ my @testfiles;
my %syms;
my @dumpout;
my @flds;
! if ($using_osx)
! {
! # On OS X, we need to examine the .o files
! my $obj_wanted = sub {
! /^.*\.o\z/s push(@testfiles, $File::Find::name);
! };
!
! File::Find::find($obj_wanted,$pgsql);
! }
! else
! {
! # Elsewhere, look at the installed executables and shared libraries
! @testfiles = (
! glob($installdir/bin/*),
! glob($installdir/lib/*),
! glob($installdir/lib/postgresql/*)
! );
! }
! foreach my $bin (@testfiles)
{
next if $bin =~ m!bin/(ipcclean|pltcl_)!;
next unless -f $bin;
***
*** 1534,1540
}
elsif ( @readelferr 10 )
{
-
# FreeBSD, similar output to Linux
@dumpout =
`readelf -w $bin 2/dev/null | egrep -A3 DW_TAG_typedef 2/dev/null`;
--- 1550,1555
***
*** 1546,1551
--- 1561,1579
$syms{$flds[-1]} =1;
}
}
+ elsif ($using_osx)
+ {
+ @dumpout =
+ `dwarfdump $bin 2/dev/null | egrep -A2 TAG_typedef 2/dev/null`;
+ foreach (@dumpout)
+ {
+ @flds = split;
+ next unless (@flds == 3);
+ next unless ($flds[0] eq AT_name();
+ next unless ($flds[1] =~ m/^(.*)$/);
+ $syms{$1} =1;
+ }
+ }
else
{
@dumpout = `$objdump --stabs $bin 2/dev/null`;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
I wrote: Thanks for the tips. The attached patch against buildfarm 4.11 seems to work, cf http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedarydt=2014-04-03%2003%3A33%3A16stg=typedefs BTW, after looking a bit more closely at what this added to the typedefs list, I realize that this mechanism also collects typedefs that appear in files that are compiled but never installed, for example src/timezone/zic and the ecpg regression tests. This seems like a Good Thing, since certainly pgindent is going to hit the source files for those programs too. I wonder if we ought to switch over to scanning the .o files on all platforms? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] It seems no Windows buildfarm members are running find_typedefs
The current typedefs list seems to be lacking any Windows-only typedefs. Noticed while trying to pgindent postmaster.c. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 04/01/2014 08:53 PM, Tom Lane wrote: The current typedefs list seems to be lacking any Windows-only typedefs. Noticed while trying to pgindent postmaster.c. Hmm. odd. will check. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It seems no Windows buildfarm members are running find_typedefs
On 04/01/2014 09:22 PM, Andrew Dunstan wrote: On 04/01/2014 08:53 PM, Tom Lane wrote: The current typedefs list seems to be lacking any Windows-only typedefs. Noticed while trying to pgindent postmaster.c. Hmm. odd. will check. It's apparently causing the buildfarm to crash, which is why I must have disabled it. I'll chase that down tomorrow. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
