stas 2002/12/09 08:29:58
Modified: src/docs/2.0/devel Changes.pod config.cfg
Added: src/docs/2.0/devel/core apache_integration.pod
coding_style.pod explained.pod
src/docs/2.0/devel/debug c.pod perl.pod
src/docs/2.0/devel/debug/code .debug-modperl-init
.debug-modperl-register .debug-modperl-xs
src/docs/2.0/devel/performance size_matters.pod
Removed: src/docs/2.0/devel/core_explained core_explained.pod
src/docs/2.0/devel/debug_c debug_c.pod
src/docs/2.0/devel/debug_c/code .debug-modperl-init
.debug-modperl-register .debug-modperl-xs
src/docs/2.0/devel/debug_perl debug_perl.pod
src/docs/2.0/devel/modperl_style modperl_style.pod
src/docs/2.0/devel/perf_sizeof perf_sizeof.pod
Log:
as the number of documents here grows, we need a better grouping. so this
commit just changes some dir/file names, no content change.
Revision Changes Path
1.3 +5 -0 modperl-docs/src/docs/2.0/devel/Changes.pod
Index: Changes.pod
===================================================================
RCS file: /home/cvs/modperl-docs/src/docs/2.0/devel/Changes.pod,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -r1.2 -r1.3
--- Changes.pod 31 Jul 2002 14:16:48 -0000 1.2
+++ Changes.pod 9 Dec 2002 16:29:57 -0000 1.3
@@ -9,6 +9,11 @@
The most recent changes are listed first.
+=head1 ???
+
+* Reshuffled the dir/file structure to make more sense, as new docs
+ are getting added.
+
=head1 Wed Mar 20 21:20:20 SGT 2002
* docs::general::testing::testing:
1.12 +11 -6 modperl-docs/src/docs/2.0/devel/config.cfg
Index: config.cfg
===================================================================
RCS file: /home/cvs/modperl-docs/src/docs/2.0/devel/config.cfg,v
retrieving revision 1.11
retrieving revision 1.12
diff -u -r1.11 -r1.12
--- config.cfg 13 Aug 2002 11:46:21 -0000 1.11
+++ config.cfg 9 Dec 2002 16:29:57 -0000 1.12
@@ -10,16 +10,21 @@
group => 'mod_perl 2.0 Core Development',
chapters => [qw(
- core_explained/core_explained.pod
- modperl_style/modperl_style.pod
- perf_sizeof/perf_sizeof.pod
+ core/explained.pod
+ core/coding_style.pod
+ performance/size_matters.pod
)],
group => '3rd party modules Development with mod_perl 2.0',
chapters => [qw(
porting/porting.pod
- debug_perl/debug_perl.pod
- debug_c/debug_c.pod
+ )],
+
+ group => 'Debugging',
+ chapters => [qw(
+ porting/porting.pod
+ debug/perl.pod
+ debug/c.pod
)],
group => 'Help',
@@ -30,7 +35,7 @@
changes => 'Changes.pod',
copy_glob => [qw(
- debug_c/code
+ debug/code
)],
1.1
modperl-docs/src/docs/2.0/devel/core/apache_integration.pod
Index: apache_integration.pod
===================================================================
=head1 NAME
mod_perl internals: Apache 2.0 integration
=head1 Description
This document should help to understand the initialization, request
processing and shutdown process of the mod_perl module. This knowledge
is essential for a less-painful debugging experience. It should also
help to know where a new code should be added when a new feature is
added.
Make sure to read also: L<Debugging mod_perl C
Internals|docs::2.0::devel::debug::c>.
=head1 Startup
Apache starts itself and immediately restart itself. The following
sections discuss what happens to mod_perl during this period.
=head2 The Link Between mod_perl and httpd
I<mod_perl.c> defines a special structure:
module AP_MODULE_DECLARE_DATA perl_module = {
STANDARD20_MODULE_STUFF,
modperl_config_dir_create, /* dir config creater */
modperl_config_dir_merge, /* dir merger --- default is to override */
modperl_config_srv_create, /* server config */
modperl_config_srv_merge, /* merge server config */
modperl_cmds, /* table of config file commands */
modperl_register_hooks, /* register hooks */
};
Apache uses this structure to hook mod_perl in, and it specifies six
custom callbacks which Apache will call at various stages that will be
explained later.
C<STANDARD20_MODULE_STUFF> is a standard macro defined in
I<httpd-2.0/include/http_config.h>. Currently its main use if for
attaching Apache version magic numbers, so the previously compiled
module won't be attempted to be used with newer Apache versions, whose
API may have changed.
C<modperl_cmds> is a struct, that defines the mod_perl configuration
directives and the callbacks to be invoked for each of these.
=head1 Configuration Tree Building
At the C<ap_read_config> stage the configuration file is parsed and
stored in a parsed configuration tree is created. Some sections are
stored unmodified in the parsed configuration tree to be processed
after the C<pre_config> hooks were run. Other sections are processed
right away (e.g., the C<Include> directive includes extra
configuration and has to include it as soon as it was seen) and they
may or may not add a subtree to the configuration tree.
C<ap_build_config> feeds the configuration file lines from to
C<ap_build_config_sub>, which tokenizes the input, and uses the first
token as a potential directive (command). It then calls
C<ap_find_command_in_modules()> to find a module that has registered
that command (remember mod_perl has registered the directives in the
C<modperl_cmds> C<command_rec> array, which was passed to
C<ap_add_module> inside the C<perl_module> struct?). If that command
is found and it has the C<EXEC_ON_READ> flag set in its
I<req_override> field, the callback for that command is
invoked. Depending on the command, it may perform some action and
return (e.g., C<User foo>), or it may continue reading from the
configuration file and recursively execute other nested commands till
it's done (e.g., C<E<lt>Location ...E<gt>>). If the command is found
but the C<EXEC_ON_READ> flag is not set or the command is not found,
the current node gets added to the configuration tree and will be
processed during the C<ap_process_config_tree()> stage, after the
C<pre_config> stage will be over.
If the command needs to be executed at this stage as it was just
explained, C<execute_now()> invokes the corresponding callback with
C<invoke_cmd>.
Since C<LoadModule> directive has the C<EXEC_ON_READ> flag set, that
directive is executed as soon as it's seen and the modules its
supposed to load get loaded right away.
For mod_perl loaded as a DSO object, this is when mod_perl starts its
game.
=head2 Enabling the mod_perl Module and Installing its Callbacks
mod_perl can be loaded as a DSO object at startup time, or be
prelinked at compile time.
For statically linked mod_perl, Apache enables mod_perl by calling
C<ap_add_module()>, which happens during the
C<ap_setup_prelinked_modules()> stage. The latter is happening before
the configuration file is parsed.
When mod_perl is loaded as DSO:
<IfModule !mod_perl.c>
LoadModule perl_module "modules/mod_perl.so"
</IfModule>
mod_dso's C<load_module> first loads the shared mod_perl object, and
then immediately calls C<ap_add_loaded_module()> which calls
C<ap_add_module()> to enable mod_perl.
C<ap_add_module()> adds the C<perl_module> structure to the top of
chained module list and calls C<ap_register_hooks()> which calls the
C<modperl_register_hooks()> callback. This is the very first mod_perl
hook that's called by Apache.
C<modperl_register_hooks()> registers all the hooks that it wants to
be called by Apache when the appropriate time comes. That includes
configuration hooks, filter, connection and http protocol hooks. From
now on most of the relationship between httpd and mod_perl is done via
these hooks. Remember that in addition to these hooks, there are four
hooks that were registered with C<ap_add_module()>, and there are:
C<modperl_config_srv_create>, C<modperl_config_srv_merge>,
C<modperl_config_dir_create> and C<modperl_config_dir_merge>.
Finally after the hooks were registered,
C<ap_single_module_configure()> (called from mod_dso's C<load_module>
in case of DSO) runs the configuration process for the module. First
it calls the C<modperl_config_srv_create> callback for the main
server, followed by the C<modperl_config_dir_create> callback to
create a directory structure for the main server. Notice that it
passes C<NULL> for the directory path, since we at the very top level.
If you need to do something as early as possible at mod_perl's
startup, the C<modperl_register_hooks()> is the right place to do
that. For example we add a C<MODPERL2> define to the
C<ap_server_config_defines> here:
*(char **)apr_array_push(ap_server_config_defines) =
apr_pstrdup(p, "MODPERL2");
so the following code will work under mod_perl 2.0 enabled Apache
without explicitly passing C<-DMODPERL2> at the server startup:
<IfDefine MODPERL2>
# 2.0 configuration
PerlSwitches -wT
</IfDefine>
This section, of course, will see the define only if inserted after
the C<LoadModule perl_module ...>, because that's when
C<modperl_register_hooks> is called.
One inconvenience with using that hook, is that the server object is
not among its arguments, so if you need to access that object, the
next earliest function is C<modperl_config_srv_create()>. However
remember that it'll be called once for the main server and one more
time for each virtual host, that has something to do with mod_perl. So
if you need to invoke it only for the main server, you can use a
C<s-E<gt>is_virtual> conditional. For example we need to enable the
debug tracing as early as possible, but we need the server object in
order to do that, so we perform this setting in
C<modperl_config_srv_create()>:
if (!s->is_virtual) {
modperl_trace_level_set(s, NULL);
}
=head1 The pre_config stage
After Apache processes its command line arguments, creates various
pools and reads the configuration file in, it runs the registered
I<pre_config> hooks by calling C<ap_run_pre_config()>. That's when
C<modperl_hook_pre_config> is called. And it does nothing.
=head2 Configuration Tree Processing
C<ap_process_config_tree> calls C<ap_walk_config>, which scans through
all directives in the parsed configuration tree, and executes each one
by calling C<ap_walk_config_sub>. This is a recursive process with
many twists.
Similar to C<ap_build_config_sub> for each command (directive) in the
configuration tree, it calls C<ap_find_command_in_modules> to find a
module that registered that command. If the command is not found the
server dies. Otherwise the callback for that command is invoked with
C<invoke_cmd>, after fetching the current directory configuration:
invoke_cmd(cmd, parms, dir_config, current->args);
The C<invoke_cmd> command is the one that invokes mod_perl's
directives callbacks, which reside in I<modperl_cmd.c>. C<invoke_cmd>
knows how the arguments should be passed to the callbacks, based on
the information in the C<modperl_cmds> array that we have just
mentioned.
Notice that before C<invoke_cmd> is invoked,
C<ap_set_config_vectors()> is called which sets the current server and
section configuration objects for the module in which the directive
has been found. If these objects were't created yet, it calls the
registered callbacks as C<create_dir_config> and
C<create_server_config>, which are C<modperl_config_dir_create> and
C<modperl_config_srv_create> for the mod_perl module. (If you write
your L<custom module in Perl|docs::2.0::user::config::custom>, these
correspond to the C<DIR_CREATE> and C<SERVER_CREATE> Perl
subroutines.)
The command callback won't be invoked if it has the C<EXEC_ON_READ>
flag set, because it was already invoked earlier when the
configuration tree was parsed. C<ap_set_config_vectors()> is called in
any case, because it wasn't called during the C<ap_build_config>.
So we have C<modperl_config_srv_create> and
C<modperl_config_dir_create> both called once for the main server (at
the end of processing the C<LoadModule perl_module ...> directive),
and one more time for each virtual host in which at least one mod_perl
directive is encountered. In addition C<modperl_config_dir_create> is
called for every section and subsection that includes mod_perl
directives (META: or inherits from such a section even though
specifies no mod_perl directives in it?).
=head1 Request Processing
META: need to write
=head1 Shutdown
META: need to write
=head1 Maintainers
Maintainer is the person(s) you should contact with updates,
corrections and patches.
=over
=item *
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=back
=head1 Authors
=over
=item *
=back
Only the major authors are listed above. For contributors see the
Changes file.
=cut
1.1 modperl-docs/src/docs/2.0/devel/core/coding_style.pod
Index: coding_style.pod
===================================================================
=head1 NAME
mod_perl Coding Style Guide
=head1 Description
This document explains the coding style used in the core mod_perl
development and which should be followed by all core developers.
=head1 Coding Style Guide
We try hard to code mod_perl using an identical style. Because
everyone in the team should be able to read and understand the code as
quickly and easily as possible. Some will have to adjust their habits
for the benefit of all.
=over 4
=item * C code
mod_perl's C code follows the Apache style guide:
http://dev.apache.org/styleguide.html
=item * XS code
C code inside XS modules also follows the Apache style guide.
=item * Perl code
mod_perl's Perl code also follows the Apache style guide, in terms of
indentation, braces, etc. Style issues not covered by Apache style of
guide should be looked up in the I<perlstyle> manpage.
=back
Here are the rough guidelines with more stress on the Perl coding
style.
=over 4
=item Indentation and Tabs
Do use 4 characters indentation.
Do NOT use tabs.
Here is how to setup your editor to do the right thing:
=over
=item * x?emacs: cperl-mode
.xemacs/custom.el:
------------------
(custom-set-variables
'(cperl-indent-level 4)
'(cperl-continued-statement-offset 4)
'(cperl-tab-always-indent t)
'(indent-tabs-mode nil)
)
=item * vim
.vimrc:
-------
set expandtab " replaces any tab keypress with the appropriate number of
spaces
set tabstop=4 " sets tabs to 4 spaces
=back
=item Block Braces
Do use a style similar to K&R style, not the same. The following
example is the best guide:
Do:
sub foo {
my($self, $cond, $baz, $taz) = @_;
if ($cond) {
bar();
}
else {
$self->foo("one", 2, "...");
}
return $self;
}
Don't:
sub foo
{
my ($self,$bar,$baz,$taz)[EMAIL PROTECTED];
if( $cond )
{
&bar();
} else { $self->foo ("one",2,"..."); }
return $self;
}
=item Lists and Arrays
Whenever you create a list or an array, always add a comma after the
last item. The reason for doing this is that it's highly probable that
new items will be appended to the end of the list in the future. If
the comma is missing and this isn't noticed, there will be an error.
Do:
my @list = (
"item1",
"item2",
"item3",
);
Don't:
my @list = (
"item1",
"item2",
"item3"
);
=item Last Statement in the Block
The same goes for C<;> in the last statement of the block. Almost
always add it even if it's not required, so when you add a new
statement you don't have to remember to add C<;> on a previous line.
Do:0
sub foo {
statement1;
statement2;
statement3;
}
Don't
sub foo {
statement1;
statement2;
statement3
}
=back
=head1 Function and Variable Prefixes Convention
=over 4
=item modperl_
The prefix for mod_perl C API functions.
=item MP_
The prefix for mod_perl C macros.
=item mpxs_
The prefix for mod_perl XS utility functions.
=item mp_xs_
The prefix for mod_perl I<generated> XS utility functions.
=item MPXS_
The prefix for mod_perl XSUBs with an XS() prototype.
=back
=head1 Coding Guidelines
The following are the Perl coding guidelines:
=head2 Global Variables
=over 4
=item avoid globals in general
=item avoid $&, $', $`
See C<Devel::SawAmpersand>'s I<README> that explains the evilness.
Under mod_perl everybody suffers when one is seen anywhere since the
interpreter is never shutdown.
=back
=head2 Modules
=over 4
=item Exporting/Importing
Avoid too much exporting/importing (glob aliases eat up memory)
When you do wish to import from a module try to use an explicit list or
tag whenever possible, e.g.:
use POSIX qw(strftime);
When you do not wish to import from a module, always use an empty list
to avoid any import, e.g.:
use IO::File ();
(explain how to use Apache::Status to find imported/exported
functions)
=back
=head2 Methods
=over 4
=item indirect vs direct method calls
Avoid indirect method calls, e.g.
Do:
CGI::Cookie->new
Don't:
new CGI::Cookie
=back
=head2 Inheritance
=over 4
=item Avoid inheriting from certain modules
Exporter.
To avoid inheriting B<AutoLoader::AUTOLOAD>
Do:
*import = \&Exporter::import;
Don't:
@MyClass::ISA = qw(Exporter);
=back
=head2 Symbol tables
=over 4
=item %main::
stay away from C<main::> to avoid namespace clashes
=back
=head2 Use of $_ in loops
Avoid using C<$_> in loops unless it's a short loop and you don't call
any subs from within the loop. If the loop started as short and then
started to grow make sure to remove the use of C<$_>:
Do:
for my $idx (1..100) {
....more than few lines...
foo($idx);
....
}
Don't:
for (1..100) {
....more than a few statements...
foo();
....
}
Because foo() might change C<$_> if foo()'s author didn't localize C<$_>.
This is OK:
for (1..100) {
.... a few statements with no subs called
# do something with $_
....
}
=head1 Maintainers
Maintainer is the person(s) you should contact with updates,
corrections and patches.
Stas Bekman E<lt>stas *at* stason.orgE<gt>
=head1 Authors
=over
=item *
Doug MacEachernE<lt>dougm (at) covalent.netE<gt>
=item *
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=back
Only the major authors are listed above. For contributors see the
Changes file.
=cut
1.1 modperl-docs/src/docs/2.0/devel/core/explained.pod
Index: explained.pod
===================================================================
=head1 NAME
mod_perl 2.0 Source Code Explained
=head1 Description
This document explains how to navigate the mod_perl source code,
modify and rebuild the existing code and most important: how to add
new functionality.
=head1 Project's Filesystem Layout
In its pristine state the project is comprised of the following
directories and files residing at the root directory of the project:
Apache-Test/ - test kit for mod_perl and Apache::* modules
ModPerl-Registry/ - ModPerl::Registry sub-project
build/ - utilities used during project build
docs/ - documentation
lib/ - Perl modules
src/ - C code that builds libmodperl.so
t/ - mod_perl tests
todo/ - things to be done
util/ - useful utilities for developers
xs/ - source xs code and maps
Changes - Changes file
LICENSE - ASF LICENSE document
Makefile.PL - generates all the needed Makefiles
After building the project, the following root directories and files
get generated:
Makefile - Makefile
WrapXS/ - autogenerated XS code
blib/ - ready to install version of the package
=head1 Directory src
=head2 Directory src/modules/perl/
The directory I<src/modules/perl> includes the C source files needed
to build the I<libmodperl> library.
Notice that several files in this directory are autogenerated during
the I<perl Makefile> stage.
When adding new source files to this directory you should add their
names to the C<@c_src_names> variable in I<lib/ModPerl/Code.pm>, so
they will be picked up by the autogenerated I<Makefile>.
=head1 Directory xs/
Apache/ - Apache specific XS code
APR/ - APR specific XS code
ModPerl/ - ModPerl specific XS code
maps/ -
tables/ -
Makefile.PL -
modperl_xs_sv_convert.h -
modperl_xs_typedefs.h -
modperl_xs_util.h -
typemap -
=head2 xs/Apache, xs/APR and xs/ModPerl
The I<xs/Apache>, I<xs/APR> and I<xs/ModPerl> directories include I<.h> files
which
have C and XS code in them. They all have the I<.h> extension because
they are always C<#include-d>, never compiled into their own object
file. and only the file that C<#include-s> an I<.h> file from these
directories should be able to see what's in there. Anything else
belongs in a I<src/modules/perl/foo.c> public API.
=head2 xs/maps
The I<xs/maps> directory includes mapping files which describe how
Apache Perl API should be constructed and various XS typemapping.
These files get modified whenever:
=over
=item *
a new function is added or the API of the existing one is modified.
=item *
a new struct is added or the existing one is modified
=item *
a new C datatype or Perl typemap is added or an existing one is
modified.
=back
The execution of:
% make source_scan
or:
% perl build/source_scan.pl
converts these map files into their Perl table representation in the
I<xs/tables/current/> directory. This Perl representation is then used
during C<perl Makefile.PL> to generate the XS code in the I<./WrapXS/>
directory by the xs_generate() function. This XS code is combined of
the Apache API Perl glue and mod_perl specific extensions.
NOTE: source_scan requires C::Scan 0.75, which at the moment is
unreleased, there is a working copy here:
http://perl.apache.org/~dougm/Scan.pm
If you need to skip certain unwanted C defines from being picked by
the source scanning you can add them to the array
C<$Apache::ParseSource::defines_unwanted> in
I<lib/Apache/ParseSource.pm>.
Notice that I<source_scan> target is normally not run during the
project build process, since the source scanning is not stable yet,
therefore everytime the map files change, C<make source_scan> should
be run manually and the updated files ending up in the
I<xs/tables/current/> directory should be committed to the cvs
repository.
The I<source_scan> make target is actually to run
I<build/source_scan.pl>, which can be run directly without needing to
create I<Makefile> first.
There are three different types of map files in the I<xs/maps/>
directory:
=over
=item * Functions Mapping
apache_functions.map
modperl_functions.map
apr_functions.map
=item * Structures Mapping
apache_structures.map
apr_structures.map
=item * Types Mapping
apache_types.map
apr_types.map
modperl_types.map
=back
The following sections describe the syntax of the files in each group
=head3 Functions Mapping
The functions mapping file is comprised of groups of function
definitions. Each group starts with a header similar to XS syntax:
MODULE=... PACKAGE=... PREFIX=... BOOT=... ISA=...
where:
=over
=item * C<MODULE>
the module name where the functions should be put. e.g. C<MODULE
Apache::Connection> will place the functions into
I<WrapXS/Apache/Connection.{pm,xs}>.
=item * C<PACKAGE>
the package name functions belong to, defaults to C<MODULE>. The
value of I<guess> indicates that package name should be guessed based
on first argument found that maps to a Perl class. If the value is
not defined and the function's name starts with I<ap_> the C<Apache>
package will be used, if it starts with I<apr_> then the C<APR>
package is used.
=item * C<PREFIX>
prefix string to be stripped from the function name. If not specified
it defaults to C<PACKAGE>, converted to C name convention, e.g.
C<APR::Base64> makes the prefix: I<apr_base64_>. If the converted
prefix does not match, defaults to I<ap_> or I<apr_>.
=item * C<BOOT>
The C<BOOT> directive tells the XS generator, whether to add the boot
function to the autogenerated XS file or not. If the value of C<BOOT>
is not true or it's simply not declared, the boot function won't be
added.
If the value is true, a boot function will be added to the XS file.
Note, that this function is not declared in the map file.
The boot function name must be constructed from three parts:
'mpxs_' . MODULE . '_BOOT'
where C<MODULE> is the one declared with C<MODULE=> in the map file.
For example if we want to have an XS boot function for a class
C<APR::IO>, we create this function in I<xs/APR/IO/APR__IO.h>:
static void mpxs_APR__IO_BOOT(pTHX)
{
/* boot code here */
}
and now we add the C<BOOT=1> declaration to the
I<xs/maps/modperl_functions.map> file:
MODULE=APR::IO PACKAGE=APR::IO BOOT=1
Notice that the C<PACKAGE=> declaration is a must.
When I<make xs_generate> is run (after running I<make source_scan>),
it autogenerates I<Wrap/APR/IO/IO.xs> and amongst other things will
include:
BOOT:
mpxs_APR__IO_BOOT(aTHXo);
=item * C<ISA>
META: complete
=back
Every function definition is declared on a separate line (use C<\> if
the line is too long), using the following format:
C function name | Dispatch function name | Argspec | Perl alias
where:
=over
=item * C function name
The name of the real C function.
Function names that do not begin with C</^\w/> are skipped. For
details see: C<%ModPerl::MapUtil::disabled_map>.
The return type can be specified before the C function name. It
defaults to I<return_type> in C<{Apache,ModPerl}::FunctionTable>.
META: DEFINE nuances
=item * Dispatch function name
Dispatch function name defaults to C function name. If the dispatch
name is just a prefix (I<mpxs_>, I<MPXS_>) the C function name is
appended to it.
See the explanation about function naming and arguments passing.
=item * Argspec
The argspec defaults to arguments in
C<{Apache,ModPerl}::FunctionTable>. Argument types can be specified
to override those in the C<FunctionTable>. Default values can be
specified, e.g. C<arg=default_value>. Argspec of C<...> indicates
I<passthru>, calling the function with C<(aTHX_ I32 items, SP **sp, SV
**MARK)>.
=item * Perl alias
the Perl alias will be created in the current C<PACKAGE>.
=back
=head3 Structures Mapping
META: complete
=head3 Types Mapping
META: complete
=head3 Modifying Maps
As explained in the beginning of this section, whenever the map file
is modified you need first to run:
% make source_scan
Next check that the conversion to Perl tables is properly done by
verifying the resulting corresponding file in
I<xs/tables/current>. For example I<xs/maps/modperl_functions.map> is
converted into I<xs/tables/current/ModPerl/FunctionTable.pm>.
If you want to do a visual check on how XS code will be generated, run:
% make xs_generate
and verify that the autogenerated XS code under the directory
I<./WrapXS> is correct. Otherwise build the project normally:
% perl Makefile.PL ...
=head2 XS generation process
As mentioned before XS code is generated in the I<WrapXS> directory
either during C<perl Makefile.PL> via xs_generate() if
C<MP_GENERATE_XS=1> is used (which is the default) or explicitly via:
% make xs_generate
In addition it creates a number of files in the I<xs/> directory:
modperl_xs_sv_convert.h
modperl_xs_typedefs.h
=head1 Gluing Existing APIs
If you have an API that you simply want to provide the Perl interface
without writing any code...
META: complete
WrapXS allows you to adjust some arguments and supply default values
for function arguments without writing any code
META: complete
C<MPXS_> functions are final C<XSUBs> and always accept:
aTHX_ I32 items, SP **sp, SV **MARK
as their arguments. Whereas C<mpxs_> functions are either intermediate
thin wrappers for the existing C functions or functions that do
something by themselves. C<MPXS_> functions also can be used for
writing thin wrappers for C macros.
=head1 Adding Wrappers for existing APIs and Creating New APIs
In certain cases the existing APIs need to be adjusted. There are a
few reasons for doing this.
First, is to make the given C API more Perlish. For example C
functions cannot return more than one value, and the pass by reference
technique is used. This is not Perlish. Perl has no problem returning
a list of value, and passing by reference is used only when an array
or a hash in addition to any other variables need to be passes or
returned from the function. Therefore we may want to adjust the C API
to return a list rather than passing a reference to a return value,
which is not intuitive for Perl programmers.
Second, is to adjust the functionality, i.e. we still use the C API
but may want to adjust its arguments before calling the original
function, or do something with return values. And of course optionally
adding some new code.
Third, is to create completely new APIs. It's quite possible that we
need more functionality built on top of the existing API. In that case
we simply create new APIs.
The following sections discuss various techniques for retrieving
function arguments and returning values to the caller. They range from
using usual C argument passing and returning to more complex Perl
arguments' stack manipulation. Once you know how to retrieve the
arguments in various situations and how to put the return values on
the stack, the rest is usually normal C programming potentially
involving using Perl APIs.
Let's look at various ways we can declare functions and what options
various declarions provide to us:
=head2 Functions Returning a Single Value (or Nothing)
If its know deterministically what the function returns and there is
only a single return value (or nothing is returned == I<void>), we are
on the C playground and we don't need to manipulate the returning
stack. However if the function may return a single value or nothing
at all, depending on the inputs and the code, we have to manually
manipulate the stack and therefore this section doesn't apply.
Let's look at various requirements and implement these using simple
examples. The following testing code exercises the interfaces we are
about to develop, so refer to this code to see how the functions are
invoked from Perl and what is returned:
file:t/response/TestApache/coredemo.pm
----------------------------------------
package TestApache::coredemo;
use strict;
use warnings FATAL => 'all';
use Apache::Const -compile => 'OK';
use Apache::Test;
use Apache::TestUtil;
use Apache::CoreDemo;
sub handler {
my $r = shift;
plan $r, tests => 7;
my $a = 7;
my $b = 3;
my ($add, $subst);
$add = Apache::CoreDemo::print($a, $b);
t_debug "print";
ok !$add;
$add = Apache::CoreDemo::add($a, $b);
ok t_cmp($a + $b, $add, "add");
$add = Apache::CoreDemo::add_sv($a, $b);
ok t_cmp($a + $b, $add, "add: return sv");
$add = Apache::CoreDemo::add_sv_sv($a, $b);
ok t_cmp($a + $b, $add, "add: pass/return svs");
($add, $subst) = @{ Apache::CoreDemo::add_subst($a, $b) };
ok t_cmp($a + $b, $add, "add_subst: add");
ok t_cmp($a - $b, $subst, "add_subst: subst");
$subst = Apache::CoreDemo::subst_sp($a, $b);
ok t_cmp($a - $b, $subst, "subst via SP");
Apache::OK;
}
1;
The first case is the simplest: pass two integer arguments, print
these to the STDERR stream and return nothing:
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
static MP_INLINE
void mpxs_Apache__CoreDemo_print(int a, int b)
{
fprintf(stderr, "%d, %d\n", a, b);
}
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
mpxs_Apache__CoreDemo_print
Now let's say that the I<b> argument is optional and in case it wasn't
provided, we want to use a default value, e.g. 0. In that case we
don't need to change the code, but simply adjust the map file to be:
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
mpxs_Apache__CoreDemo_print | | a, b=0
In the previous example, we didn't list the arguments in the map file
since they were automatically retrieved from the source code. In this
example we tell WrapXS to assign a value of C<0> to the argument b, if
it wasn't supplied by the caller. All the arguments must be listed and
in the same order as they are defined in the function.
You may add an extra test that test teh default value assignment:
$add = Apache::CoreDemo::add($a);
ok t_cmp($a + 0, $add, "add (b=0 default)");
The second case: pass two integer arguments and return their sum:
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
static MP_INLINE
int mpxs_Apache__CoreDemo_add(int a, int b)
{
return a + b;
}
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
mpxs_Apache__CoreDemo_add
The third case is similar to the previous one, but we return the sum
as as a Perl scalar. Though in C we say SV*, in the Perl space we will
get a normal scalar:
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
static MP_INLINE
SV *mpxs_Apache__CoreDemo_add_sv(pTHX_ int a, int b)
{
return newSViv(a + b);
}
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
mpxs_Apache__CoreDemo_add_sv
In the second example the XSUB function was converting the returned
I<int> value to a Perl scalar behind the scenes. In this example we
return the scalar ourselves. This is of course to demonstrate that you
can return a Perl scalar, which can be a reference to a complex Perl
datastructure, which we will see in the fifth example.
The forth case demonstrates that you can pass Perl variables to your
functions without needing XSUB to do the conversion. In all previous
examples XSUB was automatically converting Perl scalars in the
argument list to the corresponding C variables, using the typemap
definitions.
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
static MP_INLINE
SV *mpxs_Apache__CoreDemo_add_sv_sv(pTHX_ SV *a_sv, SV *b_sv)
{
int a = (int)SvIV(a_sv);
int b = (int)SvIV(b_sv);
return newSViv(a + b);
}
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
mpxs_Apache__CoreDemo_add_sv_sv
So this example is the same simple case of addition, though we
manually convert the Perl variables to C variables, perform the
addition operation, convert the result to a Perl Scalar of kind I<IV>
(Integer Value) and return it directly to the caller.
In case where more than one value needs to be returned, we can still
implement this without directly manipulating the stack before a
function returns. The fifth case demonstrates a function that returns
the result of addition and substruction operations on its arguments:
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
static MP_INLINE
SV *mpxs_Apache__CoreDemo_add_subst(pTHX_ int a, int b)
{
AV *av = newAV();
av_push(av, newSViv(a + b));
av_push(av, newSViv(a - b));
return newRV_noinc((SV*)av);
}
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
mpxs_Apache__CoreDemo_add_subst
If you look at the corresponding testing code:
($add, $subst) = @{ Apache::CoreDemo::add_subst($a, $b) };
ok t_cmp($a + $b, $add, "add_subst: add");
ok t_cmp($a - $b, $subst, "add_subst: subst");
you can see that this technique comes at a price of needing to
dereference the return value to turn it into a list. The actual code
is very similar to the C<Apache::CoreDemo::add_sv> function which
was doing only the addition operation and returning a Perl
scalar. Here we perform the addition and the substraction operation
and push the two results into a previously created I<AV*> data
structure, which represents an array. Since only the I<SV>
datastructures are allowed to be put on stack, we take a reference
I<RV> (which is of an I<SV> kind) to the existing I<AV> and return it.
The sixth case demonstrates a situation where the number of arguments
or their types may vary and aren't known at compile time. Though
notice that we still know that we are returning at compile time (zero
or one arguments), I<int> in this example:
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
static MP_INLINE
int mpxs_Apache__CoreDemo_subst_sp(pTHX_ I32 items, SV **MARK, SV **SP)
{
int a, b;
if (items != 2) {
Perl_croak(aTHX_ "usage: ...");
}
a = mp_xs_sv2_int(*MARK);
b = mp_xs_sv2_int(*(MARK+1));
return a - b;
}
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
mpxs_Apache__CoreDemo_subst_sp | | ...
In the map file we use a special token C<...> which tells the XSUB
constructor to pass C<items>, C<MARK> and C<SP> arguments to the
function. The macro C<MARK> points to the first argument passed by the
caller in the Perl namespace. For example to access the second
argument to retrieve the value of C<b> we use C<*(MARK+1)>, which if
you remember represented as an I<SV> variable, which nees to be
converted to the corresponding C type.
In this example we use the macro I<mp_xs_sv2_int>, automatically
generated based on the data from the I<xs/typemap> and
I<xs/maps/*_types.map> files, and placed into the
I<xs/modperl_xs_sv_convert.h> file. In the case of I<int> C type the
macro is:
#define mp_xs_sv2_int(sv) (int)SvIV(sv)
which simply converts the I<SV> variable on the stack and generates an
I<int> value.
While in this example you have an access to the stack, you cannot
manipulate the return values, because the XSUB wrapper expects a
single return value of type I<int>, so even if you put something on
the stack it will be ignored.
=head2 Functions Returning Variable Number of Values
We saw earlier that if we want to return an array one of the ways to
go is to return a reference to an array as a single return value,
which fits the C paradigm. So we simply declare the return value as
C<SV*>.
This section talks about cases where it's unknown at compile time how
many return values will be or it's known that there will be more than
one return value--something that C cannot handle via its return
mechanism.
Let's rewrite the function C<mpxs_Apache__CoreDemo_add_subst> from
the earlier section to return two results instead of a reference to a
list:
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
static XS(MPXS_Apache__CoreDemo_add_subst_sp)
{
dXSARGS;
int a, b;
if (items != 2) {
Perl_croak(aTHX_ "usage: Apache::CoreDemo::add_subst_sp($a, $b)");
}
a = mp_xs_sv2_int(ST(0));
b = mp_xs_sv2_int(ST(1));
SP -= items;
if (GIMME == G_ARRAY) {
EXTEND(sp, 2);
PUSHs(sv_2mortal(newSViv(a + b)));
PUSHs(sv_2mortal(newSViv(a - b)));
}
else {
XPUSHs(sv_2mortal(newSViv(a + b)));
}
PUTBACK;
}
Before explaining the function here is the prototype we add to the map
file:
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
DEFINE_add_subst_sp | MPXS_Apache__CoreDemo_add_subst_sp | ...
The C<mpxs_> functions declare in the third column the arguments that
they expect to receive (and optionally the default values). The
C<MPXS> functions are the real C<XSUBs> and therefore they always
accept:
aTHX_ I32 items, SP **sp, SV **MARK
as their arguments. Thefore it doesn't matter what is placed in this
column when the C<MPXS_> function is declared. Usually for
documentation the Perl side arguments are listed. For example you can
say:
DEFINE_add_subst_sp | MPXS_Apache__CoreDemo_add_subst_sp | x, y
In this function we manually manipulate the stack to retrieve the
arguments passed on the Perl side and put the results back onto the
stack. Therefore the first thing we do is to initialize a few special
variables using the C<dXSARGS> macro defined in I<XSUB.h>, which in
fact calls a bunch of other macros. These variables help to manipulate
the stack. C<dSP> is one of these macros and it declares and initial
izes a local copy of the Perl stack pointer C<sp> which . This local
copy should always be accessed as C<SP>.
We retrieve the original function arguments using the C<ST()>
macros. C<ST(0)> and C<ST(1)> point to the first and the second
argument on the stack, respectively. But first we check that we have
exactly two arguments on the stack, and if not we abort the
function. The C<items> variable is the function argument.
Once we have retrieved all the arguments from the stack we set the
local stack pointer C<SP> to point to the bottom of the stack (like
there are no items on the stack):
SP -= items;
Now we can do whatever processing is needed and put the results back
on the stack. In our example we return the results of addition and
substraction operations if the function is called in the list
context. In the scalar context the function returns only the result of
the addition operation. We use the C<GIMME> macro which tells us the
context.
In the list context we make sure that we have two spare slots on the
stack since we are going to push two items, and then we push them
using the C<PUSHs> macro:
EXTEND(sp, 2);
PUSHs(sv_2mortal(newSViv(a + b)));
PUSHs(sv_2mortal(newSViv(a - b)));
Alternatively we could use:
XPUSHs(sv_2mortal(newSViv(a + b)));
XPUSHs(sv_2mortal(newSViv(a - b)));
The C<XPUSHs> macro eI<X>tends the stack before pushing the item into
it if needed. If we plan to push more than a single item onto the
stack, it's more efficient to extend the stack in one call.
In the scalar context we push only one item, so here we use the
C<XPUSHs> macro:
XPUSHs(sv_2mortal(newSViv(a + b)));
The last command we call is:
PUTBACK;
which makes the local stack pointer global. This is a must call if the
state of the stack was changed when the function is about to
return. The stack changes if something was popped from or pushed to
it, or both and changed the number of items on the stack.
In our example we don't need to call C<PUTBACK> if the function is
called in the list context. Because in this case we return two
variables, the same as two function arguments, the count didn't
change. Though in the scalar context we push onto the stack only one
argument, so the function won't return what is expected. The simplest
way to avoid errors here is to always call C<PUTBACK> when the stack
is changed.
For more information refer to the I<perlcall> manpage which explains
the stack manipulation process in great details.
Finally we test the function in the list and scalar contexts:
file:t/response/TestApache/coredemo.pm
----------------------------------------
...
my $a = 7;
my $b = 3;
my ($add, $subst);
# list context
($add, $subst) = Apache::CoreDemo::add_subst_sp($a, $b);
ok t_cmp($a + $b, $add, "add_subst_sp list context: add");
ok t_cmp($a - $b, $subst, "add_subst_sp list context: subst");
# scalar context
$add = Apache::CoreDemo::add_subst_sp($a, $b);
ok t_cmp($a + $b, $add, "add_subs_spt scalar context: add");
...
=head2 Wrappers Functions for C Macros
Let's say you have a C macro which you want to provide a Perl
interface for. For example let's take a simple macro which performs
the power of function:
file:xs/Apache/CoreDemo/Apache__CoreDemo.h
----------------------------------------------
#define mpxs_Apache__CoreDemo_power(x, y) pow(x, y)
To create the XS glue code we use the following entry in the map file:
file:xs/maps/modperl_functions.map
----------------------------------
MODULE=Apache::CoreDemo
double:DEFINE_power | | double:x, double:y
This works very similar to the C<MPXS_Apache__CoreDemo_add_subst_sp>
function presented earlier. But since this is a macro the XS wrapper
needs to know the types of the arguments and the return type, so these
are added. The return type is added just before the function name and
separated from it by the colon (C<:>), the argument types are
specified in the third column. The type is always separated from the
name of the variable by the colon (C<:>).
And of course finally we need to test that the function works in Perl:
file:t/response/TestApache/coredemo.pm
----------------------------------------
...
my $a = 7;
my $b = 3;
my $power = Apache::CoreDemo::power($a, $b);
ok t_cmp($a ** $b, $power, "power macro");
...
=head1 Wrappers for modperl_, apr_ and ap_ APIs
If you already have a C function whose name starts from I<modperl_>,
I<apr_> or I<ap_> and you want to do something before calling the real
C function, you can write a XS wrapper using the same method as in the
L<MPXS_Apache__CoreDemo_add_subst_sp
|/Functions_Returning_Variable_Number_of_Values>. The only difference
is that it'll be clearly seen in the map file that this is a wrapper
for an existing C API.
Let's say that we have an existing C function apr_power(), this is how
we declare its wrapper:
file:xs/maps/apr_functions.map
----------------------------------
MODULE=APR::Foo
apr_power | MPXS_ | x, y
The first column specifies the existing function's name, the second
tells that the XS wrapper will use the C<MPXS_> prefix, which means
that the wrapper must be called C<MPXS_apr_power>. The third column
specifies the argument names, but for C<MPXS_> no matter what you
specify there the C<...> will be passed:
aTHX_ I32 items, SP **sp, SV **MARK
so you can leave that column empty, but here we use C<x> and C<y> to
remind us that these two arguments are passed from Perl.
If the forth column is empty this function will be called
C<APR::Foo::power> in the Perl namespace. But you can use that column
to give a different Perl name, e.g with:
apr_power | MPXS_ | x, y | pow
This function will be available from Perl as C<APR::Foo::pow>.
Similarly you can write a C<MPXS_modperl_power> wrapper for a
C<modperl_power()> function but here you have to explicitly give the
Perl function's name in the forth column:
file:xs/maps/apr_functions.map
----------------------------------
MODULE=Apache::CoreDemo
modperl_power | MPXS_ | x, y | mypower
and the Perl function will be called C<Apache::CoreDemo::mypower>.
The C<MPXS_> wrapper's implementation is similar to
L<MPXS_Apache__CoreDemo_add_subst_sp
|/Functions_Returning_Variable_Number_of_Values>.
=head1 MP_INLINE vs C Macros vs Normal Functions
To make the code maintainable and reusable functions and macros are
used in when programming in C (and other languages :).
When function is marked as I<inlined> it's merely a hint to the
compiler to replace the call to a function with the code inside this
function (i.e. inlined). Not every function can be inlined. Some
typical reasons why inlining is sometimes not done include:
=over
=item *
the function calls itself, that is, is recursive
=item *
the function contains loops such as C<for(;;)> or C<while()>
=item *
the function size is too large
=back
Most of the advantage of inline functions comes from avoiding the
overhead of calling an actual function. Such overhead includes saving
registers, setting up stack frames, etc. But with large functions the
overhead becomes less important.
Use the C<MP_INLINE> keyword in the declaration of the functions that
are to be inlined. The functions should be inlined when:
=over
=item *
Only ever called once (the I<wrappers> that are called from I<.xs>
files), no matter what the size of code is.
=item *
Short bodies of code called in a I<hot> code (like
I<modperl_env_hv_store>, which is called many times inside of a loop),
where it is cleaner to see the code in function form rather than macro
with lots of C<\>'s. Remember that an inline function takes much more
space than a normal functions if called from many places in the code.
=back
Of course C macros are a bit faster then inlined functions, since
there is not even I<short jump> to be made, the code is literally
copied into the place it's called from. However using macros comes at
a price:
=over
=item *
Also unlike macros, in functions argument types are checked, and
necessary conversions are performed correctly. With macros it's
possible that weird things will happen if the caller has passed
arguments of the wrong type when calling a macro.
=item *
One should be careful to pass only absolute values as I<"arguments">
to macros. Consider a macro that returns an absolute value of the
passed argument:
#define ABS(v) ( (v) >= 0 ? (v) : -(v) )
In our example if you happen to pass a function it will be called
twice:
abs_val = ABS(f());
Since it'll be extended as:
abs_val = f() >= 0 ? f() : -f();
You cannot do simple operation like increment--in our example it will
be called twice:
abs_val = ABS(i++);
Because it becomes:
abs_val = i++ >= 0 ? i++ : -i++;
=item *
It's dangerous to use the if() condition without enclosing the code in
C<{}>, since the macro may be called from inside another if-else
condition, which may cause the else part called if the if() part from
the macro fails.
But we always use C<{}> for the code inside the if-else condition, so
it's not a problem here.
=item *
A multi-line macro can cause problems if someone uses the macro in a
context that demands a single statement.
while (foo) MYMACRO(bar);
But again, we always enclose any code in conditional with C<{}>, so
it's not a problem for us.
=item *
Inline functions present a problem for debuggers and profilers,
because the function is expanded at the point of call and loses its
identity. This makes the debugging process a nightmare.
A compiler will typically have some option available to disable
inlining.
=back
In all other cases use normal functions.
=head1 Adding New Interfaces
=head2 Adding Typemaps for new C Data Types
Sometimes when a new interface is added it may include C data types
for which we don't have corresponding XS typemaps yet. In such a case,
the first thing to do is to provide the required typemaps.
Let's add a prototype for the I<typedef struct scoreboard> data type
defined in I<httpd-2.0/include/scoreboard.h>.
First we include the relevant header files in
I<src/modules/perl/modperl_apache_includes.h>:
#include "scoreboard.h"
If you want to specify your own type and don't have a header file for
it (e.g. if you extend some existing datatype within mod_perl) you may
add the I<typedef> to I<src/modules/perl/modperl_types.h>.
After deciding that C<Apache::Scoreboard> is the Perl class will be
used for manipulating C I<scoreboard> data structures, we map the
I<scoreboard> data structure to the C<Apache::Scoreboard>
class. Therefore we add to I<xs/maps/apache_types.map>:
struct scoreboard | Apache::Scoreboard
Since we want the I<scoreboard> data structure to be an opaque object
on the perl side, we simply let mod_perl use the default C<T_PTROBJ>
typemap. After running C<make xs_generate> you can check the assigned
typemap in the autogenerated I<WrapXS/typemap> file.
If you need to do some special handling while converting from C to
Perl and back, you need to add the conversion functions to the
I<xs/typemap> file. For example the C<Apache::RequestRec> objects need
special handling, so you can see the special C<INPUT> and C<OUTPUT>
typemappings for the corresponding C<T_APACHEOBJ> object type.
Now we run C<make xs_generate> and find the following definitions in
the autogenerated files:
file:xs/modperl_xs_typedefs.h
-----------------------------
typedef scoreboard * Apache__Scoreboard;
file:xs/modperl_xs_sv_convert.h
-------------------------------
#define mp_xs_sv2_Apache__Scoreboard(sv) \
((SvROK(sv) && (SvTYPE(SvRV(sv)) == SVt_PVMG)) \
|| (Perl_croak(aTHX_ "argument is not a blessed reference \
(expecting an Apache::Scoreboard derived object)"),0) ? \
(scoreboard *)SvIV((SV*)SvRV(sv)) : (scoreboard *)NULL)
#define mp_xs_Apache__Scoreboard_2obj(ptr) \
sv_setref_pv(sv_newmortal(), "Apache::Scoreboard", (void*)ptr)
The file I<xs/modperl_xs_typedefs.h> declares the typemapping from C
to Perl and equivalent to the C<TYPEMAP> section of the XS's
I<typemap> file. The second file I<xs/modperl_xs_sv_convert.h>
generates two macros. The first macro is used to convert from Perl to
C datatype and equivalent to the I<typemap> file's C<INPUT>
section. The second macro is used to convert from C to Perl datatype
and equivalent to the I<typemap>'s C<OUTPUT> section.
Now proceed on adding the glue code for the new interface.
=head2 Importing Constants and Enums into Perl API
To I<import> httpd and APR constants and enums into Perl API, edit
I<lib/Apache/ParseSource.pm>. To add a new type of C<DEFINE> constants
adjust the C<%defines_wanted> variable, for C<enums> modify
C<%enums_wanted>.
For example to import all C<DEFINE>s starting with C<APR_FLOCK_> add:
my %defines_wanted = (
...
APR => {
...
flock => [qw{APR_FLOCK_}],
...
},
);
When deciding which constants are to be exported, the regular
expression will be used, so in our example all matches
C</^APR_FLOCK_/> will be imported into the Perl API.
For example to import an I<read_type_e> C<enum> for APR, add:
my %enums_wanted = (
APR => { map { $_, 1 } qw(apr_read_type) },
);
Notice that I<_e> part at the end of the enum name has gone.
After adding/modifying the datastructures make sure to run C<make
source_scan> or C<perl build/source_scan.pl> and verify that the
wanted constant or enum were picked by the source scanning
process. Simply grep I<xs/tables/current> for the wanted string. For
example after adding I<apr_read_type_e> enum we can check:
% more xs/tables/current/Apache/ConstantsTable.pm
...
'read_type' => [
'APR_BLOCK_READ',
'APR_NONBLOCK_READ'
],
Of course the newly added constant or enum's typemap should be
declared in the appropriate I<xs/maps/*_types.map> files, so the XS
conversion of arguments will be performed correctly. For example
I<apr_read_type> is an APR enum so it's declared in
I<xs/maps/apr_types.map>:
apr_read_type | IV
C<IV> is used as a typemap, Since enum is just an integer. In more
complex cases the typemap can be different. (META: examples)
=head1 Maintainers
Maintainer is the person(s) you should contact with updates,
corrections and patches.
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=head1 Authors
=over
=item *
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=back
Only the major authors are listed above. For contributors see the
Changes file.
=cut
1.1 modperl-docs/src/docs/2.0/devel/debug/c.pod
Index: c.pod
===================================================================
=head1 NAME
Debugging mod_perl C Internals
=head1 Description
This document explains how to debug C code under mod_perl, including
mod_perl core itself.
=head1 Debug notes
META: needs more organization
=head2 Setting gdb breakpoints with mod_perl built as DSO
If mod_perl is built as a DSO module, you cannot set the breakpoint in
the mod_perl source files when the I<httpd> program gets loaded into
the debugger. The reason is simple: At this moment I<httpd> has no
idea about mod_perl module yet. After the configuration file is
processed and the mod_perl DSO module is loaded then the breakpoints
in the source of mod_perl itself can be set.
The trick is to break at I<apr_dso_load>, let it load
I<libmodperl.so>, then you can set breakpoints anywhere in the modperl
code:
% gdb httpd
(gdb) b apr_dso_load
(gdb) run -DONE_PROCESS
[New Thread 1024 (LWP 1600)]
[Switching to Thread 1024 (LWP 1600)]
Breakpoint 1, apr_dso_load (res_handle=0xbfffb48c, path=0x811adcc
"/home/stas/apache.org/modperl-perlmodule/src/modules/perl/libmodperl.so",
pool=0x80e1a3c) at dso.c:138
141 void *os_handle = dlopen(path, RTLD_NOW | RTLD_GLOBAL);
(gdb) finish
...
Value returned is $1 = 0
(gdb) b modperl_hook_init
(gdb) continue
This example shows how to set a breakpoint at I<modperl_hook_init>.
To automate things you can put those in the I<.gdb-jump-to-init> file:
b apr_dso_load
run -DONE_PROCESS -d `pwd`/t -f `pwd`/t/conf/httpd.conf
finish
b modperl_hook_init
continue
and then start the debugger with:
% gdb /home/stas/httpd-2.0/bin/httpd -command \
`pwd`/t/.gdb-jump-to-init
=head2 Starting the Server Fast under gdb
When the server is started under gdb, it first loads the symbol tables
of the dynamic libraries that it sees going to be used. Some versions
of gdb may take ages to complete this task, which makes the debugging
very irritating if you have to restart the server all the time and it
doesn't happen immediately.
The trick is to set the C<auto-solib-add> flag to 0:
set auto-solib-add 0
as early as possible in I<~/.gdbinit> file.
With this setting in effect, you can load only the needed dynamic
libraries with I<sharedlibrary> command. Remember that in order to set
a breakpoint and step through the code inside a certain dynamic
library you have to load it first. For example consider this gdb
commands file:
.gdb-commands
------------
file ~/httpd/prefork/bin/httpd
handle SIGPIPE pass
handle SIGPIPE nostop
set auto-solib-add 0
b ap_run_pre_config
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf \
-DONE_PROCESS -DAPACHE2 -DPERL_USEITHREADS
sharedlibrary modperl
b modperl_hook_init
# start: modperl_hook_init
continue
# restart: ap_run_pre_config
continue
# restart: modperl_hook_init
continue
b apr_poll
continue
# load APR/PerlIO/PerlIO.so
sharedlibrary PerlIO
b PerlIOAPR_open
which can be used as:
% gdb -command=.gdb-commands
This script stops in I<modperl_hook_init()>, so you can step through
the mod_perl startup. We had to use the I<ap_run_pre_config> so we can
load the I<libmodperl.so> library as explained earlier. Since httpd
restarts on the start, we have to I<continue> until we hit
I<modperl_hook_init> second time, where we can set the breakpoint at
I<apr_poll>, the very point where httpd polls for new request and run
again I<continue> so it'll stop at I<apr_poll>. This particular script
passes over modperl_hook_init(), since we run the C<continue> command
a few times to reach the I<apr_poll> breakpoint. See the L<Precooked
gdb Startup Scripts|/Precooked_gdb_Startup_Scripts> section for
standalone script examples.
When gdb stops at the function I<apr_poll> it's a time to start the
client, that will issue a request that will exercise the server
execution path we want to debug. For example to debug the
implementation of C<APR::Pool> we may run:
% t/TEST -run apr/pool
which will trigger the run of a handler in
I<t/response/TestAPR/pool.pm> which in turn tests the C<APR::Pool>
code.
But before that if we want to debug the server response we need to set
breakpoints in the libraries we want to debug. For example if we want
to debug the function C<PerlIOAPR_open> which resides in
I<APR/PerlIO/PerlIO.so> we first load it and then we can set a
breakpoint in it. Notice that gdb may not be able to load a library if
it wasn't referenced by any of the code. In this case we have to load
this library at the server startup. In our example we load:
PerlModule APR::PerlIO
in I<httpd.conf>. To check which libraries' symbol tables can be
loaded in gdb, run (when the server has been started):
gdb> info sharedlibrary
which also shows which libraries are loaded already.
Also notice that you don't have to type the full path of the library
when trying to load them, even a partial name will suffice. In our
commands file example we have used C<sharedlibrary modperl> instead of
saying C<sharedlibrary libmodperl.so>.
If you want to set breakpoints and step through the code in the Perl
and APR core libraries you should load their appropriate libraries:
gdb> sharedlibrary libperl
gdb> sharedlibrary libapr
gdb> sharedlibrary libaprutil
Setting I<auto-solib-add> to 0 makes the debugging process unusual,
since originally gdb was loading the dynamic libraries automatically,
whereas now it doesn't. This is the price one has to pay to get the
debugger starting the program very fast. Hopefully the future versions
of gdb will improve.
Just remember that if you try to I<step-in> and debugger doesn't do
anything, that means that the library the function is located in
wasn't loaded. The solution is to create a commands file as explained
in the beginning of this section and craft the startup script the way
you need to avoid extra typing and mistakes when repeating the same
debugging process again and again.
Another important thing is that whenever you want to be able to see
the source code for the code you are stepping through, the library or
the executable you are in must have the debug symbols present. That
means that the code has to be compiled with I<-g> option for the gcc
compiler. For example if I want to set a breakpoint in /lib/libc.so, I
can do that by loading:
gdb> sharedlibrary /lib/libc.so
But most likely that this library has the debug symbols stripped off,
so while gdb will be able to break at the breakpoint set inside this
library, you won't be able to step through the code. In order to do
so, recompile the library to add the debug symbols.
If debug code in response handler you usually start the client after
the server was started, when doing this a lot you may find it annoying
to need to wait before the client can be started. Therefore you can
use a few tricks to do it in one command. If the server starts fast
you can use sleep():
% ddd -command=.debug-modperl-init & ; \
sleep 2 ; t/TEST -verbose -run apr/pool
or the C<Apache::Test> framework's C<-ping=block> option:
% ddd -command=.debug-modperl-init & ; \
t/TEST -verbose -run -ping=block apr/pool
which will block till the server starts responding, and only then will
try to run the test.
=head2 Precooked gdb Startup Scripts
Here are a few startup scripts you can use with gdb to accomplish one
of the common debugging tasks. To execute the startup script, simply run:
% gdb -command=.debug-script-filename
They can be run under gdb and any of the gdb front-ends. For example
to run the scripts under C<ddd> substitute C<gdb> with C<ddd>:
% ddd -command=.debug-script-filename
=over
=item * Debugging mod_perl Initialization
The F<code/.debug-modperl-init> startup script breaks at the
C<modperl_hook_init()> function, which is useful for debugging code at
the modperl's initialization phase.
=item * Debugging mod_perl's Hooks Registeration With httpd
Similar to the previous startup script, the
F<code/.debug-modperl-register> startup script breaks at the
C<modperl_register_hooks()>, which is the very first hook called in
the mod_perl land. Therefore use this one if you need to start
debugging at an even earlier entry point into mod_perl.
Refer to the notes inside the script to adjust it for a specific
I<httpd.conf> file.
=item * Debugging mod_perl XS Extensions
The F<code/.debug-modperl-xs> startup script breaks at the
C<mpxs_Apache__Filter_print()> function implemented in
I<xs/Apache/Filter/Apache__Filter.h>. This is an example of debugging
code in XS Extensions. For this particular example the complete test
case is:
% ddd -command=.debug-modperl-xs & \
t/TEST -v -run -ping=block filter/api
When I<filter/api> test is running it calls
mpxs_Apache__Filter_print() which is when the breakpoint is reached.
=back
=head1 Obtaining Core Files
META: need to review (unfinished)
=head2 ...
=head1 Analyzing Dumped Core Files
When your application dies with the I<Segmentation fault> error (which
generates a C<SIGSEGV> signal) and optionally generates a I<core> file
you can use C<gdb> or a similar debugger to find out what caused the
I<Segmentation fault> (or I<segfault> as we often call it).
=head2 Getting Ready to Debug
In order to debug the I<core> file we may need to recompile Perl and
mod_perl with debugging symbols inside. Usually you have to recompile
only mod_perl, but if the I<core> dump happens in the I<libmodperl.so>
library and you want to see the whole backtrace, you probably want to
recompile Perl as well.
Recompile Perl with I<-DDEBUGGING> during the ./Configure stage (or
even better with I<-Doptimize="-g"> which in addition to adding the
C<-DDEBUGGING> option, adds the I<-g> options which allows you to
debug the Perl interpreter itself).
After recompiling Perl, recompile mod_perl with C<MP_DEBUG=1> during
the I<Makefile.PL> stage.
Building mod_perl with C<PERL_DEBUG=1> will:
=over
=item 1
add `-g' to EXTRA_CFLAGS
=item 1
turn on MP_TRACE (tracing)
=item 1
Set PERL_DESTRUCT_LEVEL=2
=item 1
Link against C<libperld> if -e
$Config{archlibexp}/CORE/libperld$Config{lib_ext}
=back
If you build a static mod_perl, remember that during I<make install>
Apache strips all the debugging symbols. To prevent this you should
use the Apache I<--without-execstrip> C<./configure> option. So if you
configure Apache via mod_perl, you should do:
panic% perl Makefile.PL USE_APACI=1 \
APACI_ARGS='--without-execstrip' [other options]
Alternatively you can copy the unstripped binary manually. For example
we did this to give us an Apache binary called C<httpd_perl> which
contains debugging symbols:
panic# cp httpd-2.x/httpd /home/httpd/httpd_perl/bin/httpd_perl
Now the software is ready for a proper debug.
=head2 Creating a Faulty Package
Next stage is to create a package that aborts abnormally with the
I<Segmentation fault> error. We will write faulty code on purpose,
so you will be able to reproduce the problem and exercise the
debugging technique explained here. Of course in a real case you
will have some real bug to debug, so in that case you may want to
skip this stage of writing a program with a deliberate bug.
We will use the C<Inline.pm> module to embed some code written in C
into our Perl script. The faulty function that we will add is this:
void segv() {
int *p;
p = NULL;
printf("%d", *p); /* cause a segfault */
}
For those of you not familiar with C programming, I<p> is a pointer to
a segment of memory. Setting it to C<NULL> ensures that we try to
read from a segment of memory to which the operating system does not
allow us access, so of course dereferencing C<NULL> pointer causes a
segmentation fault. And that's what we want.
So let's create the C<Bad::Segv> package. The name I<Segv> comes from
the C<SIGSEGV> (segmentation violation signal) that is generated when
the I<Segmentation fault> occurs.
First we create the installation sources:
panic% h2xs -n Bad::Segv -A -O -X
Writing Bad/Segv/Segv.pm
Writing Bad/Segv/Makefile.PL
Writing Bad/Segv/test.pl
Writing Bad/Segv/Changes
Writing Bad/Segv/MANIFEST
Now we modify the I<Segv.pm> file to include the C code. Afterwards
it looks like this:
package Bad::Segv;
use strict;
BEGIN {
$Bad::Segv::VERSION = '0.01';
}
use Inline C => <<'END_OF_C_CODE';
void segv() {
int *p;
p = NULL;
printf("%d", *p); /* cause a segfault */
}
END_OF_C_CODE
1;
Finally we modify I<test.pl>:
use Inline SITE_INSTALL;
BEGIN { $| = 1; print "1..1\n"; }
END {print "not ok 1\n" unless $loaded;}
use Bad::Segv;
$loaded = 1;
print "ok 1\n";
Note that we don't test Bad::Segv::segv() in I<test.pl>, since this
will abort the I<make test> stage abnormally, and we don't want this.
Now we build and install the package:
panic% perl Makefile.PL
panic% make && make test
panic% su
panic# make install
Running I<make test> is essential for C<Inline.pm> to prepare the binary
object for the installation during I<make install>.
META: stopped here!
Now we can test the package:
panic% ulimit -c unlimited
panic% perl -MBad::Segv -e 'Bad::Segv::segv()'
Segmentation fault (core dumped)
panic% ls -l core
-rw------- 1 stas stas 1359872 Feb 6 14:08 core
Indeed, we can see that the I<core> file was dumped, which will be used
to present the debug techniques.
=head2 Getting the core File Dumped
Now let's get the I<core> file dumped from within the mod_perl
server. Sometimes the program aborts abnormally via the SIGSEGV signal
(I<Segmentation Fault>), but no I<core> file is dumped. And without
the I<core> file it's hard to find the cause of the problem, unless
you run the program inside C<gdb> or another debugger in first
place. In order to get the I<core> file, the application has to:
=over
=item *
have the effective UID the same as real UID (the same goes for
GID). Which is the case of mod_perl unless you modify these settings
in the program.
=item *
be running from a directory which at the moment of the
I<Segmentation fault> is writable by the process. Notice that the
program might change its current directory during its run, so it's
possible that the I<core> file will need to be dumped in a different
directory from the one the program was started from. For example when
mod_perl runs an C<Apache::Registry> script it changes its directory
to the one in which the script source is located.
=item *
be started from a shell process with sufficient resource allocations
for the I<core> file to be dumped. You can override the default
setting from within a shell script if the process is not started
manually. In addition you can use C<BSD::Resource> to manipulate the
setting from within the code as well.
You can use C<ulimit> for C<bash> and C<limit> for C<csh> to check and
adjust the resource allocation. For example inside C<bash>, you may
set the core file size to unlimited:
panic% ulimit -c unlimited
or for C<csh>:
panic% limit coredumpsize unlimited
For example you can set an upper limit on the I<core> file size to 8MB
with:
panic% ulimit -c 8388608
So if the core file is bigger than 8MB it will be not created.
=item *
Of course you have to make sure that you have enough disk space to
create a big core file (mod_perl I<core> files tend to be of a few
MB in size).
=back
Note that when you are running the program under a debugger like
C<gdb>, which traps the C<SIGSEGV> signal, the I<core> file will not
be dumped. Instead it allows you to examine the program stack and
other things without having the I<core> file.
So let's write a simple script that uses C<Bad::Segv>:
core_dump.pl
------------
use strict;
use Bad::Segv ();
use Cwd()
my $r = shift;
$r->content_type('text/plain');
my $dir = getcwd;
$r->print("The core should be found at $dir/core\n");
Bad::Segv::segv();
In this script we load the C<Bad::Segv> and C<Cwd> modules. After
that we acquire the request object and send the HTTP header. Now we
come to the real part--we get the current working directory, print out
the location of the I<core> file that we are about to dump and finally
we call Bad::Segv::segv() which dumps the I<core> file.
Before we run the script we make sure that the shell sets the I<core>
file size to be unlimited, start the server in single server mode as a
non-root user and generate a request to the script:
panic% cd /home/httpd/httpd_perl/bin
panic% limit coredumpsize unlimited
panic% ./httpd_perl -X
# issue a request here
Segmentation fault (core dumped)
Our browser prints out:
The core should be found at /home/httpd/perl/core
And indeed the core file appears where we were told it will (remember
that C<Apache::Registry> scripts change their directory to the
location of the script source):
panic% ls -l /home/httpd/perl/core
-rw------- 1 stas httpd 3227648 Feb 7 18:53 /home/httpd/perl/core
As you can see it's a 3MB I<core> file. Notice that mod_perl was
started as user I<stas>, which had write permission for directory
I</home/httpd/perl>.
=head2 Analyzing the core File
First we start C<gdb>:
panic% gdb /home/httpd/httpd_perl/bin/httpd_perl /home/httpd/perl/core
with the location of the mod_perl executable and the core file as the
arguments.
To see the backtrace you run the I<where> or the I<bt> command:
(gdb) where
#0 0x4025ea08 in XS_Apache__Segv_segv ()
from
/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/Bad/Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39/Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39.so
#1 0x40136528 in PL_curcopdb ()
from /usr/lib/perl5/5.6.0/i386-linux/CORE/libperl.so
Well, you can see the last commands, but our perl and mod_perl are
probably without the debug symbols. So we recompile Perl and mod_perl
with debug symbols as explained earlier in this chapter.
Now when we repeat the process of starting the server, issuing a
request and getting the core file, after which we run C<gdb> again
against the executable and the dumped I<core> file.
panic% gdb /home/httpd/httpd_perl/bin/httpd_perl /home/httpd/perl/core
Now we can see the whole backtrace:
(gdb) bt
#0 0x40323a30 in segv () at
Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39.xs:9
#1 0x40323af8 in XS_Apache__Segv_segv (cv=0x85f2b28)
at Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39.xs:24
#2 0x400fcbda in Perl_pp_entersub () at pp_hot.c:2615
#3 0x400f2c56 in Perl_runops_debug () at run.c:53
#4 0x4008b088 in S_call_body (myop=0xbffff788, is_eval=0) at perl.c:1796
#5 0x4008ac4f in perl_call_sv (sv=0x82fc2e4, flags=4) at perl.c:1714
#6 0x807350e in perl_call_handler ()
#7 0x80729cd in perl_run_stacked_handlers ()
#8 0x80701b4 in perl_handler ()
#9 0x809f409 in ap_invoke_handler ()
#10 0x80b3e8f in ap_some_auth_required ()
#11 0x80b3efa in ap_process_request ()
#12 0x80aae60 in ap_child_terminate ()
#13 0x80ab021 in ap_child_terminate ()
#14 0x80ab19c in ap_child_terminate ()
#15 0x80ab80c in ap_child_terminate ()
#16 0x80ac03c in main ()
#17 0x401b8cbe in __libc_start_main () from /lib/libc.so.6
Reading the trace from bottom to top, we can see that it starts with Apache
calls, followed by Perl syscalls. At the top we can see the segv()
call which was the one that caused the Segmentation fault, we can also
see that the faulty code was at line 9 of I<Segv.xs> file (with MD5
signature of the code in the name of the file, because of the way
C<Inline.pm> works). It's a little bit tricky with C<Inline.pm> since
we have never created any I<.xs> files ourselves, (C<Inline.pm> does
it behind the scenes). The solution in this case is to tell
C<Inline.pm> not to cleanup the build directory, so we can see the
created I<.xs> file.
We go back to the directory with the source of C<Bad::Segv> and
force the recompilation, while telling C<Inline.pm> not to cleanup
after the build and to print a lot of other useful info:
panic# cd Bad/Segv
panic# perl -MInline=FORCE,NOCLEAN,INFO Segv.pm
Information about the processing of your Inline C code:
Your module is already compiled. It is located at:
/home/httpd/perl/Bad/Segv/_Inline/lib/auto/Bad/Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39/Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39.so
But the FORCE_BUILD option is set, so your code will be recompiled.
I'll use this build directory:
/home/httpd/perl/Bad/Segv/_Inline/build/Bad/Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39/
and I'll install the executable as:
/home/httpd/perl/Bad/Segv/_Inline/lib/auto/Bad/Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39/Segv_C_0_01_e6b5959d800f515de36a7e7eeab28b39.so
The following Inline C function(s) have been successfully bound to Perl:
void segv()
It tells us that the code was already compiled, but since we have
forced it to recompile we can look at the files after the build. So we
go into the build directory reported by C<Inline.pm> and find the
I<.xs> file there, where on line 9 we indeed find the faulty code:
9: printf("%d",*p); // cause a segfault
Notice that in our example we knew what script has caused the
Segmentation fault. In a real world the chances are that you will find
the I<core> file without any clue to which of handler or script has
triggered it. The special I<curinfo> C<gdb> macro comes to help:
panic% gdb /home/httpd/httpd_perl/bin/httpd_perl /home/httpd/perl/core
(gdb) source mod_perl-x.xx/.gdbinit
(gdb) curinfo
9:/home/httpd/perl/core_dump.pl
We start the C<gdb> debugger as before. I<.gdbinit>, the file with
various useful C<gdb> macros is located in the source tree of
mod_perl. We use the C<gdb> source() function to load these macros,
and when we run the I<curinfo> macro we learn that the core was dumped
when I</home/httpd/perl/core_dump.pl> was executing the code at line
9.
These are the bits of information that are important in order to
reproduce and resolve a problem: the filename and line where the
faulty function was called (the faulty function is Bad::Segv::segv()
in our case) and the actual line where the Segementation fault occured
(the printf("%d",*p) call in XS code). The former is important for
problem reproducing, it's possible that if the same function was
called from a different script the problem won't show up (not the case
in our example, where the using of a value dereferenced from the NULL
pointer will always cause the Segmentation fault).
=head2 Obtaining core Files under Solaris
On Solaris the following method can be used to generate a core file.
=over
=item 1
Use truss(1) as I<root> to stop a process on a segfault:
panic% truss -f -l -t \!all -s \!SIGALRM -S SIGSEGV -p <pid>
or, to monitor all httpd processes (from bash):
panic% for pid in `ps -eaf -o pid,comm | fgrep httpd | cut -d'/' -f1`;
do truss -f -l -t \!all -s \!SIGALRM -S SIGSEGV -p $pid 2>&1 &
done
The used truss(1) options are:
=over
=item *
C<-f> - follow forks.
=item *
C<-l> - (that's an el) includes the thread-id and the pid (the pid is
what we want).
=item *
C<-t> - specifies the syscalls to trace,
=item *
!all - turns off the tracing of syscalls specified by C<-t>
=item *
C<-s> - specifies signals to trace and the C<!SIGALRM> turns off the
numerous alarms Apache creates.
=item *
C<-S> - specifies signals that stop the process.
=item *
C<-p> - is used to specify the pid.
=back
Instead of attaching to the process, you can start it under truss(1):
panic% truss -f -l -t \!all -s \!SIGALRM -S SIGSEGV \
/usr/local/bin/httpd -f httpd.conf 2>&1 &
=item 1
Watch the I<error_log> file for reaped processes, as when they get
SISSEGV signals. When the process is reaped it's stopped but not
killed.
=item 1
Use gcore(1) to get a I<core> of stopped process or attach to it with
gdb(1). For example if the process id is 662:
%panic gcore 662
gcore: core.662 dumped
Now you can load this I<core> file in gdb(1).
=item 1
C<kill -9> the stopped process. Kill the truss(1) processes as well,
if you don't need to trap other segfaults.
=back
Obviously, this isn't great to be doing on a production system since
truss(1) stops the process after it dumps core and prevents Apache
from reaping it. So, you could hit the clients/threads limit if you
segfault a lot.
=head1 Expanding C Macros
Perl, mod_perl and httpd C code makes an extensive use of C macros,
which sometimes use many other macros in their definitions, so it
becomes quite a task to figure out how to figure out what a certain
macro expands to, especially when the macro expands to different
values in differnt environments. Luckily there are ways to automate
the expansion process.
=head2 Expanding C Macros with C<make>
The mod_perl I<Makefile>'s include a rule for macro expansions which
you can find by looking for the C<c.i.> rule. To expand all macros in
a certain C file, you should run C<make filename.c>, which will create
I<filename.i> with all macros expanded in it. For example to create
I<apr_perlio.i> with all macros used in I<apr_perlio.c>:
% cd modperl-2.0/xs/APR/PerlIO
% make apr_perlio.i
the I<apr_perlio.i> file now lists all the macros:
% less apr_perlio.i
# 1 "apr_perlio.c"
# 1 "<built-in>"
#define __VERSION__ "3.1.1 (Mandrake Linux 8.3 3.1.1-0.4mdk)"
...
=head2 Expanding C Macros with C<gdb>
With gcc-3.1 or higher and gdb-5.2-dev or higher you can expand macros
in gdb, when you step through the code. e.g.:
(gdb) macro expand pTHX_
expands to: PerlInterpreter *my_perl __attribute__((unused)),
(gdb) macro expand PL_dirty
expands to: (*Perl_Tdirty_ptr(my_perl))
For each library that you want to use this feature with you have to
compile it with:
CFLAGS="-gdwarf-2 -g3"
or whatever is appropriate for your system, refer to the gcc manpage
for more info.
To compile perl with this debug feature, pass C<-Doptimize='-gdwarf-2
-g3'> to C<./Configure>. For Apache run:
CFLAGS="-gdwarf-2 -g3" ./configure [...]
for mod_perl you don't have to do anything, as it'll pick the
C<$Config{optimize}> Perl flags automatically, if Perl is compiled
with C<-DDEBUGGING> (which is implied on most systems, if you use
C<-Doptimize='-g'> or similar.)
Notice that this will make your libraries B<huge>! e.g. on Linux 2.4
Perl 5.8.0's normal I<libperl.so> is about 0.8MB on linux, compiled
with C<-Doptimize='-g'> about 2.7MB and with C<-Doptimize='-gdwarf-2
-g3'> 12.5MB. C<httpd> is also becomes about 10 times bigger with this
feature enabled. I<mod_perl.so> instead of 0.2k becomes 11MB. You get
the idea. Of course since you may want this only during the
development/debugging, that shouldn't be a problem.
The complete details are at:
http://sources.redhat.com/gdb/current/onlinedocs/gdb_10.html#SEC69
=head1 Maintainers
Maintainer is the person(s) you should contact with updates,
corrections and patches.
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=head1 Authors
=over
=item *
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=back
Only the major authors are listed above. For contributors see the
Changes file.
=cut
1.1 modperl-docs/src/docs/2.0/devel/debug/perl.pod
Index: perl.pod
===================================================================
=head1 NAME
Debugging mod_perl Perl Internals
=head1 Description
This document explains how to debug Perl code under mod_perl.
=head1 Maintainers
Maintainer is the person(s) you should contact with updates,
corrections and patches.
=over
=item *
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=back
=head1 Authors
=over
=item *
Stas Bekman E<lt>stas (at) stason.orgE<gt>
=back
Only the major authors are listed above. For contributors see the
Changes file.
=cut
1.1
modperl-docs/src/docs/2.0/devel/debug/code/.debug-modperl-init
Index: .debug-modperl-init
===================================================================
# This gdb startup script breaks at the modperl_hook_init() function,
# which is useful for debug things at the modperl init phase.
#
# Invoke as:
# gdb -command=.debug-modperl-init
#
# see ADJUST notes for things that may need to be adjusted
# ADJUST: the path to the httpd executable if needed
file ~/httpd/worker/bin/httpd
handle SIGPIPE nostop
handle SIGPIPE pass
set auto-solib-add 0
define myrun
tbreak main
break ap_run_pre_config
# ADJUST: the httpd.conf file's path if needed
# ADJUST: add -DPERL_USEITHREADS to debug threaded mpms
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf -DONE_PROCESS -DAPACHE2
continue
end
define modperl_init
sharedlibrary modperl
b modperl_hook_init
continue
end
define sharedap
sharedlibrary apr
sharedlibrary aprutil
#sharedlibrary mod_ssl.so
continue
end
define sharedperl
sharedlibrary libperl
end
# start the server and run till modperl_hook_init on start
myrun
modperl_init
# ADJUST: uncomment to reach modperl_hook_init on restart
#continue
#continue
# ADJUST: uncomment if you need to step through the code in apr libs
#sharedap
# ADJUST: uncomment if you need to step through the code in perlib
#sharedperl
1.1
modperl-docs/src/docs/2.0/devel/debug/code/.debug-modperl-register
Index: .debug-modperl-register
===================================================================
# This gdb startup script allows to break at the very first invocation
# of mod_perl initialization, just after it was loaded. When the
# perl_module is loaded, and its pointer struct is added via
# ap_add_module(), the first hook that will be called is
# modperl_register_hooks().
#
# Invoke as:
# gdb -command=.debug-modperl-register
#
# see ADJUST notes for things that may need to be adjusted
define sharedap
sharedlibrary apr
sharedlibrary aprutil
#sharedlibrary mod_ssl.so
end
define sharedperl
sharedlibrary libperl
end
### Run ###
# ADJUST: the path to the httpd executable if needed
file ~/httpd/prefork/bin/httpd
handle SIGPIPE nostop
handle SIGPIPE pass
set auto-solib-add 0
tbreak main
# assuming that mod_dso is compiled in
b load_module
# ADJUST: the httpd.conf file's path if needed
# ADJUST: add -DPERL_USEITHREADS to debug threaded mpms
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf \
-DONE_PROCESS -DNO_DETATCH -DAPACHE2
# skip over 'tbreak main'
continue
# In order to set the breakpoint in mod_perl.so, we need to get to
# the point where it's loaded.
#
# With static mod_perl, the bp can be set right away
#
# With DSO mod_perl, mod_dso's load_module() loads the mod_perl.so
# object and it immediately calls ap_add_module(), which calls
# modperl_register_hooks(). So if we want to bp at the latter, we need
# to stop at load_module(), set the 'bp modperl_register_hooks' and
# then continue.
# Assuming that 'LoadModule perl_module' is the first LoadModule
# directive in httpd.conf, you need just one 'continue' after
# 'ap_add_module'. If it's not the first one, you need to add as many
# 'continue' commands as the number of 'LoadModule foo' before
# perl_module, but before setting the 'ap_add_module' bp.
#
# If mod_perl is compiled statically, everything is already preloaded,
# so you can set modperl_* the breakpoints right away
b ap_add_module
continue
sharedlibrary modperl
b modperl_register_hooks
continue
#b modperl_hook_init
#b modperl_config_srv_create
#b modperl_startup
#b modperl_init_vhost
#b modperl_dir_config
#b modperl_cmd_load_module
#modperl_config_apply_PerlModule
# ADJUST: uncomment if you need to step through the code in apr libs
#sharedap
# ADJUST: uncomment if you need to step through the code in perlib
#sharedperl
1.1
modperl-docs/src/docs/2.0/devel/debug/code/.debug-modperl-xs
Index: .debug-modperl-xs
===================================================================
# This gdb startup script breaks at the mpxs_Apache__Filter_print()
# function from the XS code, as an example how you can debug the code
# in XS extensions.
#
# Invoke as:
# gdb -command=.debug-modperl-xs
# and then run:
# t/TEST -v -run -ping=block filter/api
#
# see ADJUST notes for things that may need to be adjusted
# ADJUST: the path to the httpd executable if needed
file /home/stas/httpd/worker/bin/httpd
handle SIGPIPE nostop
handle SIGPIPE pass
set auto-solib-add 0
define myrun
tbreak main
break ap_run_pre_config
# ADJUST: the httpd.conf file's path if needed
# ADJUST: add -DPERL_USEITHREADS to debug threaded mpms
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf \
-DONE_PROCESS -DNO_DETATCH -DAPACHE2
continue
end
define sharedap
sharedlibrary apr
sharedlibrary aprutil
#sharedlibrary mod_ssl.so
continue
end
define sharedperl
sharedlibrary libperl
end
define gopoll
b apr_poll
continue
continue
end
define mybp
# load Apache/Filter.so
sharedlibrary Filter
b mpxs_Apache__Filter_print
# no longer needed and they just make debugging harder under threads
disable 2
disable 3
continue
end
myrun
gopoll
mybp
# ADJUST: uncomment if you need to step through the code in apr libs
#sharedap
# ADJUST: uncomment if you need to step through the code in perlib
#sharedperl
1.1
modperl-docs/src/docs/2.0/devel/performance/size_matters.pod
Index: size_matters.pod
===================================================================
=head1 NAME
Measure sizeof() of Perl's C Structures
=head1 Description
This document describes the I<sizeof> various structures, as
determined by I<util/sizeof.pl>. These measurements are mainly for
research purposes into making Perl things smaller, or rather, how to
use less Perl things.
=head1 Perl Structures
Structures diagrams are courtesy gdb (print pretty) and a bit of hand
crafting.
=over 4
=item CV - 229 minimum, 254 minimum w/ symbol table entry
cv = {
sv_any = { // XPVCV *
xpv_pv = 0x0, // char *
xpv_cur = 0, // STRLEN
xpv_len = 0, // STRLEN
xof_off = 0 , // IV
xnv_nv = 0, // NV
xmg_magic = 0x0, // MAGIC *
xmg_stash = 0x0, // HV *
xcv_stash = 0x0, // HV *
xcv_start = 0x0, // OP *
xcv_root = 0x0, // OP *
xcv_xsub = 0x0, // void (*)(register PerlInterpreter *, CV *)
xcv_xsubany = { // ANY
any_ptr = 0x0,
any_i32 = 0,
any_iv = 0,
any_long = 0,
any_dptr = 0,
any_dxptr = 0
},
xcv_gv = { // GV *
sv_any = { // void *
xpv_pv = 0x0, // char *
xpv_cur = 0, // STRLEN
xpv_len = 0, // STRLEN
xiv_iv = 0, // IV
xnv_nv = 0, // NV
xmg_magic = { // MAGIC *
mg_moremagic = 0x0, // MAGIC *
mg_virtual = 0x0, // MGVTBL *
mg_private = 0, // U16
mg_type = 0, // char
mg_flags = 0, // U8
mg_obj = 0x0, // SV *
mg_ptr = 0x0, // char *
mg_len = 0, // I32
},
xmg_stash = 0x0, // HV *
xgv_gp = { // GP *
gp_sv = { // SV *
sv_any = 0x0, // void *
sv_refcnt = 0, // U32
sv_flags = 0 // U32
},
gp_refcnt = 0, // U32
gp_io = 0x0, // struct io *
gp_form = 0x0, // CV *
gp_av = 0x0, // AV *
gp_hv = 0x0, // HV *
gp_egv = 0x0, // GV *
gp_cv = 0x0, // CV *
gp_cvgen = 0, // U32
gp_flags = 0, // U32
gp_line = 0, // line_t
gp_file = 0x0, // char *
},
xgv_name = 0x0, // char *
xgv_namelen = 0, // STRLEN
xgv_stash = 0x0, // void *
xgv_flags = 0, // U8
},
sv_refcnt = 0, // U32
sv_flags = 0, // U32
},
xcv_file = 0x0, // char *
xcv_depth = 0, // long
xcv_padlist = 0x0, // AV *
xcv_outside = 0x0, // CV *
xcv_flags = 0, // cv_flags_t
}
sv_refcnt = 0, // U32
sv_flags = 0, // U32
};
In addition to the minimum bytes:
=over 4
=item name of the subroutine: GvNAMELEN(CvGV(cv))+1
=item symbol table entry: HvENTRY (25 + GvNAMELEN(CvGV(cv))+1)
=item minimum sizeof(AV) * 3: xcv_padlist if !CvXSUB(cv)
=item CvROOT(cv) optree
=back
=item HV - 60 minmum
hv = {
sv_any = { // SV *
xhv_array = 0x0, // char *
xhv_fill = 0, // STRLEN
xhv_max = 0, // STRLEN
xhv_keys = 0, // IV
xnv_nv = 0, // NV
xmg_magic = 0x0, // MAGIC *
xmg_stash = 0x0, // HV *
xhv_riter = 0, // I32
xhv_eiter = 0x0, // HE *
xhv_pmroot = 0x0, // PMOP *
xhv_name = 0x0 // char *
},
sv_refcnt = 0, // U32
sv_flags = 0, // U32
};
Each entry adds C<sizeof(HvENTRY)>, minimum of 7 (initial C<xhv_max>).
Note that keys of the same value share C<sizeof(HEK)>, across all
hashes.
=item HvENTRY - 25 + HeKLEN+1
sizeof(HE *) + sizeof(HE) + sizeof(HEK)
=item HE - 12
he = {
hent_next = 0x0, // HE *
hent_hek = 0x0, // HEK *
hent_val = 0x0 // SV *
};
=item HEK - 9 + hek_len
hek = {
hek_hash = 0, // U32
hek_len = 0, // I32
hek_key = 0, // char
};
=item AV - 53
av = {
sv_any = { // SV *
xav_array = 0x0, // char *
xav_fill = 0, // size_t
xav_max = 0, // size_t
xof_off = 0, // IV
xnv_nv = 0, // NV
xmg_magic = 0x0, // MAGIC *
xmg_stash = 0x0, // HV *
xav_alloc = 0x0, // SV **
xav_arylen = 0x0, // SV *
xav_flags = 0, // U8
},
sv_refcnt = 0, // U32
sv_flags = 0 // U32
};
In addition to the minimum bytes:
=over 4
=item AvFILL(av) * sizeof(SV *)
=back
=back
=head1 SEE ALSO
perlguts(3), B::Size(3),
http://gisle.aas.no/perl/illguts/
=head1 Maintainers
Maintainer is the person(s) you should contact with updates,
corrections and patches.
=over
=item *
Doug MacEachern E<lt>dougm (at) covalent.netE<gt>
=back
=head1 Authors
=over
=item *
Doug MacEachern E<lt>dougm (at) covalent.netE<gt>
=back
=cut
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
