Author: REHSACK
Date: Wed Dec 19 10:40:11 2012
New Revision: 15524
Modified:
dbi/branches/sqlengine/lib/DBD/File/Developers.pod
dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine.pm
dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine/Developers.pod
Log:
applying changes made to the driver infrastructure to documentation
Modified: dbi/branches/sqlengine/lib/DBD/File/Developers.pod
==============================================================================
--- dbi/branches/sqlengine/lib/DBD/File/Developers.pod (original)
+++ dbi/branches/sqlengine/lib/DBD/File/Developers.pod Wed Dec 19 10:40:11 2012
@@ -357,6 +357,120 @@
=back
+=head2 DBD::File::TableSource::FileSystem
+
+Provides data sources and table information on database driver and database
+handle level.
+
+ package DBD::File::TableSource::FileSystem;
+
+ sub data_sources ($;$)
+ {
+ my ( $class, $drh, $attrs ) = @_;
+ ...
+ }
+
+ sub avail_tables
+ {
+ my ( $class, $drh ) = @_;
+ ...
+ }
+
+The C<data_sources> method is called when the user invokes any of the
+following:
+
+ @ary = DBI->data_sources($driver);
+ @ary = DBI->data_sources($driver, \%attr);
+
+ @ary = $dbh->data_sources();
+ @ary = $dbh->data_sources(\%attr);
+
+The C<avail_tables> method is called when the user invokes any of the
+following:
+
+ @names = $dbh->tables( $catalog, $schema, $table, $type );
+
+ $sth = $dbh->table_info( $catalog, $schema, $table, $type );
+ $sth = $dbh->table_info( $catalog, $schema, $table, $type, \%attr );
+
+ $dbh->func( "list_tables" );
+
+Everytime where an C<\%attr> argument can be specified, this C<\%attr>
+object's C<sql_table_source> attribute is preferred over the C<$dbh>
+attribute or the driver default.
+
+=head2 DBD::File::DataSource::Stream
+
+ package DBD::File::DataSource::Stream;
+
+ @DBD::File::DataSource::Stream::ISA = 'DBI::DBD::SqlEngine::DataSource';
+
+ sub complete_table_name {
+ my ($self, $meta, $file, $respect_case) = @_;
+ ...
+ }
+
+Unsets all meta attributes identifying a file: C<f_fqfn>, C<f_fqbn> and
+C<f_fqln>. The table name is set according to C<$respect_case> and
+C<< $meta->{sql_identifier_case} >> (SQL_IC_LOWER, SQL_IC_UPPER).
+
+ sub apply_encoding {
+ my ($self, $meta, $fn) = @_;
+ ...
+ }
+
+Applies the encoding from I<meta information> (C<< $meta->{f_encoding} >>)
+to the file handled opened in C<open_data>.
+
+ sub open_data {
+ my ($self, $meta, $attrs, $flags) = @_;
+ ...
+ }
+
+Opens (C<dup(2)>) the file handle provided in C<< $meta->{f_file} >>.
+
+ sub can_flock { ... }
+
+Returns whether C<flock(2)> is available or not (avoids retesting in
+subclasses).
+
+=head2 DBD::File::DataSource::File
+
+ package DBD::File::DataSource::File;
+
+ sub complete_table_name ($$;$)
+ {
+ my ( $self, $meta, $table, $respect_case ) = @_;
+ ...
+ }
+
+The method C<complete_table_name> tries to map a filename to the associated
+table name. It is called with a partially filled meta structure for the
+resulting table containing at least the following attributes:
+C<< f_ext >>, C<< f_dir >>, C<< f_lockfile >> and C<< sql_identifier_case >>.
+
+If a file/table map can be found then this method sets the C<< f_fqfn
+>>, C<< f_fqbn >>, C<< f_fqln >> and C<< table_name >> attributes in
+the meta structure. If a map cannot be found the table name will be
+undef.
+
+ package DBD::File::DataSource::File;
+
+ sub open_data ($)
+ {
+ my ( $self, $meta, $attrs, $flags ) = @_;
+ ...
+ }
+
+Depending on the attributes set in the table's meta data, the
+following steps are performed. Unless C<< f_dontopen >> is set to a
+true value, C<< f_fqfn >> must contain the full qualified file name
+for the table to work on (file2table ensures this). The encoding in
+C<< f_encoding >> is applied if set and the file is opened. If
+C<<f_fqln >> (full qualified lock name) is set, this file is opened,
+too. Depending on the value in C<< f_lock >>, the appropriate lock is
+set on the opened data file or lock file.
+
=head2 DBD::File::Statement
Derives from DBI::SQL::Nano::Statement to provide following method:
@@ -396,18 +510,6 @@
=over 4
-=item file2table
-
-This method tries to map a filename to the associated table
-name. It is called with a partially filled meta structure for the
-resulting table containing at least the following attributes:
-C<< f_ext >>, C<< f_dir >>, C<< f_lockfile >> and C<< sql_identifier_case >>.
-
-If a file/table map can be found then this method sets the C<< f_fqfn
->>, C<< f_fqbn >>, C<< f_fqln >> and C<< table_name >> attributes in
-the meta structure. If a map cannot be found the table name will be
-undef.
-
=item bootstrap_table_meta
Initializes a table meta structure. Can be safely overridden in a
@@ -415,9 +517,8 @@
of the overridden method.
It copies the following attributes from the database into the table meta data
-C<< f_dir >>, C<< f_ext >>, C<< f_encoding >>, C<< f_lock >>, C<< f_schema >>,
-C<< f_lockfile >> and C<< sql_identifier_case >> and makes them sticky to the
-table.
+C<< f_dir >>, C<< f_ext >>, C<< f_encoding >>, C<< f_lock >>, C<< f_schema >>
+and C<< f_lockfile >> and makes them sticky to the table.
This method should be called before you attempt to map between file
name and table name to ensure the correct directory, extension etc. are
Modified: dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine.pm
==============================================================================
--- dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine.pm (original)
+++ dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine.pm Wed Dec 19 10:40:11 2012
@@ -1827,6 +1827,15 @@
it's strongly recommended to set this flag before any statement is
executed (best place is connect attribute hash).
+=head4 sql_engine_in_gofer
+
+This value has a true value in case of this driver is operated via
+L<DBD::Gofer>. The impact of being operated via Gofer is a read-only
+driver (not read-only databases!), so you cannot modify any attributes
+later - neither any table settings. B<But> you won't get an error in
+cases you modify table attributes, so please carefully watch
+C<sql_engine_in_gofer>.
+
=head4 sql_meta
Private data area which contains information about the tables this
@@ -1850,13 +1859,122 @@
Controls the class which will be used for fetching available tables.
-XXX See L</TableSource>
+See L</DBI::DBD::SqlEngine::TableSource> for details.
=head4 sql_data_source
Contains the class name to be used for opening tables.
-XXX See L</DataSource>
+See L</DBI::DBD::SqlEngine::DataSource> for details.
+
+=head2 Extensibility
+
+=head3 DBI::DBD::SqlEngine::TableSource
+
+Provides data sources and table information on database driver and database
+handle level.
+
+ package DBI::DBD::SqlEngine::TableSource;
+
+ sub data_sources ($;$)
+ {
+ my ( $class, $drh, $attrs ) = @_;
+ ...
+ }
+
+ sub avail_tables
+ {
+ my ( $class, $drh ) = @_;
+ ...
+ }
+
+The C<data_sources> method is called when the user invokes any of the
+following:
+
+ @ary = DBI->data_sources($driver);
+ @ary = DBI->data_sources($driver, \%attr);
+
+ @ary = $dbh->data_sources();
+ @ary = $dbh->data_sources(\%attr);
+
+The C<avail_tables> method is called when the user invokes any of the
+following:
+
+ @names = $dbh->tables( $catalog, $schema, $table, $type );
+
+ $sth = $dbh->table_info( $catalog, $schema, $table, $type );
+ $sth = $dbh->table_info( $catalog, $schema, $table, $type, \%attr );
+
+ $dbh->func( "list_tables" );
+
+Everytime where an C<\%attr> argument can be specified, this C<\%attr>
+object's C<sql_table_source> attribute is preferred over the C<$dbh>
+attribute or the driver default, eg.
+
+ @ary = DBI->data_sources("dbi:CSV:", {
+ f_dir => "/your/csv/tables",
+ # note: this class doesn't comes with DBI
+ sql_table_source => "DBD::File::Archive::Tar::TableSource",
+ # scan tarballs instead of directories
+ });
+
+When you're going to implement such a DBD::File::Archive::Tar::TableSource
+class, remember to add correct attributes (including C<sql_table_source>
+and C<sql_data_source>) to the returned DSN's.
+
+=head3 DBI::DBD::SqlEngine::DataSource
+
+Provides base functionality for dealing with tables. It is primarily
+designed for allowing transparent access to files on disk or already
+opened (file-)streams (eg. for DBD::CSV).
+
+Derived classes shall be restricted to similar functionality, too (eg.
+opening streams from an archive, transparently compress/uncompress
+log files before parsing them,
+
+ package DBI::DBD::SqlEngine::DataSource;
+
+ sub complete_table_name ($$;$)
+ {
+ my ( $self, $meta, $table, $respect_case ) = @_;
+ ...
+ }
+
+The method C<complete_table_name> is called when first setting up the
+I<meta information> for a table:
+
+ "SELECT user.id, user.name, user.shell FROM user WHERE ..."
+
+results in opening the table C<user>. First step of the table open
+process is completing the name. Let's imagine you're having a L<DBD::CSV>
+handle with following settings:
+
+ $dbh->{sql_identifier_case} = SQL_IC_LOWER;
+ $dbh->{f_ext} = '.lst';
+ $dbh->{f_dir} = '/data/web/adrmgr';
+
+Those settings will result in looking for files matching
+C<[Uu][Ss][Ee][Rr](\.lst)?$> in C</data/web/adrmgr/>. The scanning of the
+directory C</data/web/adrmgr/> and the pattern match check will be done
+in C<DBD::File::DataSource::File> by the C<complete_table_name> method.
+
+If you intend to provide other sources of data streams than files, in
+addition to provide an appropriate C<complete_table_name> method, a method
+to open the resource is required:
+
+ package DBI::DBD::SqlEngine::DataSource;
+
+ sub open_data ($)
+ {
+ my ( $self, $meta, $attrs, $flags ) = @_;
+ ...
+ }
+
+After the method C<open_data> has been run successfully, the table's meta
+information are in a state which allowes the table's data accessor methods
+will be able to fetch/store row information. Implementation details heavily
+depends on the table implementation, whereby the most famous is surely
+L<DBD::File/DBD::File::Table|DBD::File::Table>.
=head1 SUPPORT
Modified: dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine/Developers.pod
==============================================================================
--- dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine/Developers.pod (original)
+++ dbi/branches/sqlengine/lib/DBI/DBD/SqlEngine/Developers.pod Wed Dec 19
10:40:11 2012
@@ -194,7 +194,7 @@
This package defines the database methods, which are called via the DBI
database handle C<< $dbh >>.
-Methods provided by DBI::DBD::SqlEngine:
+=head3 Methods provided by C<< DBI::DBD::SqlEngine::db >>:
=over 4
@@ -290,12 +290,18 @@
=item init_default_attributes
This method is called after the database handle is instantiated to
-initialize the default attributes.
+initialize the default attributes. It expects one argument: C<$phase>.
+If C<$phase> is not given, C<connect> of C<DBI::DBD::SqlEngine::dr>
+expects this is an old-fashioned driver which isn't capable of multi-phased
+initialization.
C<< DBI::DBD::SqlEngine::db::init_default_attributes >> initializes the
attributes C<sql_identifier_case>, C<sql_quoted_identifier_case>,
-C<sql_handler>, C<sql_engine_version>, C<sql_nano_version> and
-C<sql_statement_version> when L<SQL::Statement> is available.
+C<sql_handler>, C<sql_init_order>, C<sql_meta>, C<sql_engine_version>,
+C<sql_nano_version> and C<sql_statement_version> when L<SQL::Statement>
+is available.
+
+It sets C<sql_init_order> to the given C<$phase>.
When the derived implementor class provides the attribute to validate
attributes (e.g. C<< $dbh->{dbm_valid_attrs} = {...}; >>) or the attribute
@@ -364,14 +370,105 @@
=item commit
Warns about a useless call (if warnings enabled) and returns.
-DBI::DBD::SqlEngine is typically a driver which commits every action instantly
when
-executed.
+DBI::DBD::SqlEngine is typically a driver which commits every action
+instantly when executed.
=item rollback
Warns about a useless call (if warnings enabled) and returns.
-DBI::DBD::SqlEngine is typically a driver which commits every action instantly
when
-executed.
+DBI::DBD::SqlEngine is typically a driver which commits every action
+instantly when executed.
+
+=back
+
+=head3 Attributes used by C<< DBI::DBD::SqlEngine::db >>:
+
+This section describes attributes which are important to developers of DBI
+DataBase Drivers derived from C<DBI::DBD::SqlEngine>.
+
+=over 4
+
+=item sql_init_order
+
+This attribute contains a hash with priorities as key and an array
+containing the C<$dbh> attributes to be initialized during before/after
+other attributes.
+
+C<DBI::DBD::SqlEngine> initializes following attributes:
+
+ $dbh->{sql_init_order} = {
+ 0 => [qw( Profile RaiseError PrintError AutoCommit )],
+ 90 => [ "sql_meta", $dbh->{$drv_pfx_meta} ? $dbh->{$drv_pfx_meta} : () ]
+ }
+
+The default priority of not listed attribute keys is C<50>. It is well
+known that a lot of attributes needed to be set before some table settings
+are initialized. For example, for L<DBD::DBM>, when using
+
+ my $dbh = DBI->connect( "dbi:DBM:", undef, undef, {
+ f_dir => "/path/to/dbm/databases",
+ dbm_type => "BerkeleyDB",
+ dbm_mldbm => "JSON", # use MLDBM::Serializer::JSON
+ dbm_tables => {
+ quick => {
+ dbm_type => "GDBM_File",
+ dbm_MLDBM => "FreezeThaw"
+ }
+ }
+ });
+
+This defines a known table C<quick> which uses the L<GDBM_File> backend and
+L<FreezeThaw> as serializer instead of the overall default L<BerkeleyDB> and
+L<JSON>. B<But> all files containing the table data have to be searched in
+C<< $dbh->{f_dir} >>, which requires C<< $dbh->{f_dir} >> must be initialized
+before C<< $dbh->{sql_meta}->{quick} >> is initialized by
+C<bootstrap_table_meta> method of L</DBI::DBD::SqlEngine::Table> to get
+C<< $dbh->{sql_meta}->{quick}->{f_dir} >> being initialized properly.
+
+=item sql_init_phase
+
+This attribute is only set during the initialization steps of the DBI
+DataBase Driver. It contains the value of the currently run initialization
+phase. Currently supported phases are I<phase 0> and I<phase 1>. This
+attribute is set in C<init_default_attributes> and removed in C<init_done>.
+
+=item sql_engine_in_gofer
+
+This value has a true value in case of this driver is operated via
+L<DBD::Gofer>. The impact of being operated via Gofer is a read-only
+driver (not read-only databases!), so you cannot modify any attributes
+later - neither any table settings. B<But> you won't get an error in
+cases you modify table attributes, so please carefully watch
+C<sql_engine_in_gofer>.
+
+=item sql_table_source
+
+Names a class which is responsible for delivering I<data sources> and
+I<available tables> (DataBase Driver related). I<data sources> here
+refers to L<DBI/data_sources>, not C<sql_data_source>.
+
+See L</DBI::DBD::SqlEngine::TableSource> for details.
+
+=item sql_data_source
+
+Name a class which is responsible for handling table resources open
+and completing table names requested via SQL statements.
+
+See L</DBI::DBD::SqlEngine::DataSource> for details.
+
+=item sql_dialect
+
+Controls the dialect understood by SQL::Parser. Possible values (delivery
+state of SQL::Statement):
+
+ * ANSI
+ * CSV
+ * AnyData
+
+Defaults to "CSV". Because an SQL::Parser is instantiated only once and
+SQL::Parser doesn't allow to modify the dialect once instantiated,
+it's strongly recommended to set this flag before any statement is
+executed (best place is connect attribute hash).
=back
@@ -430,6 +527,102 @@
=back
+=head2 DBI::DBD::SqlEngine::TableSource
+
+Provides data sources and table information on database driver and database
+handle level.
+
+ package DBI::DBD::SqlEngine::TableSource;
+
+ sub data_sources ($;$)
+ {
+ my ( $class, $drh, $attrs ) = @_;
+ ...
+ }
+
+ sub avail_tables
+ {
+ my ( $class, $drh ) = @_;
+ ...
+ }
+
+The C<data_sources> method is called when the user invokes any of the
+following:
+
+ @ary = DBI->data_sources($driver);
+ @ary = DBI->data_sources($driver, \%attr);
+
+ @ary = $dbh->data_sources();
+ @ary = $dbh->data_sources(\%attr);
+
+The C<avail_tables> method is called when the user invokes any of the
+following:
+
+ @names = $dbh->tables( $catalog, $schema, $table, $type );
+
+ $sth = $dbh->table_info( $catalog, $schema, $table, $type );
+ $sth = $dbh->table_info( $catalog, $schema, $table, $type, \%attr );
+
+ $dbh->func( "list_tables" );
+
+Everytime where an C<\%attr> argument can be specified, this C<\%attr>
+object's C<sql_table_source> attribute is preferred over the C<$dbh>
+attribute or the driver default.
+
+=head2 DBI::DBD::SqlEngine::DataSource
+
+Provides base functionality for dealing with tables. It is primarily
+designed for allowing transparent access to files on disk or already
+opened (file-)streams (eg. for DBD::CSV).
+
+Derived classes shall be restricted to similar functionality, too (eg.
+opening streams from an archive, transparently compress/uncompress
+log files before parsing them,
+
+ package DBI::DBD::SqlEngine::DataSource;
+
+ sub complete_table_name ($$;$)
+ {
+ my ( $self, $meta, $table, $respect_case ) = @_;
+ ...
+ }
+
+The method C<complete_table_name> is called when first setting up the
+I<meta information> for a table:
+
+ "SELECT user.id, user.name, user.shell FROM user WHERE ..."
+
+results in opening the table C<user>. First step of the table open
+process is completing the name. Let's imagine you're having a L<DBD::CSV>
+handle with following settings:
+
+ $dbh->{sql_identifier_case} = SQL_IC_LOWER;
+ $dbh->{f_ext} = '.lst';
+ $dbh->{f_dir} = '/data/web/adrmgr';
+
+Those settings will result in looking for files matching
+C<[Uu][Ss][Ee][Rr](\.lst)?$> in C</data/web/adrmgr/>. The scanning of the
+directory C</data/web/adrmgr/> and the pattern match check will be done
+in C<DBD::File::DataSource::File> by the C<complete_table_name> method.
+
+If you intend to provide other sources of data streams than files, in
+addition to provide an appropriate C<complete_table_name> method, a method
+to open the resource is required:
+
+ package DBI::DBD::SqlEngine::DataSource;
+
+ sub open_data ($)
+ {
+ my ( $self, $meta, $attrs, $flags ) = @_;
+ ...
+ }
+
+After the method C<open_data> has been run successfully, the table's meta
+information are in a state which allowes the table's data accessor methods
+will be able to fetch/store row information. Implementation details heavily
+depends on the table implementation, whereby the most famous is surely
+L<DBD::File/DBD::File::Table|DBD::File::Table>.
+
=head2 DBI::DBD::SqlEngine::Statement
Derives from DBI::SQL::Nano::Statement for unified naming when deriving
@@ -438,13 +631,112 @@
=head2 DBI::DBD::SqlEngine::Table
Derives from DBI::SQL::Nano::Table for unified naming when deriving
-new drivers. No additional feature is provided from here.
+new drivers.
You should consult the documentation of C<< SQL::Eval::Table >> (see
L<SQL::Eval>) to get more information about the abstract methods of the
table's base class you have to override and a description of the table
meta information expected by the SQL engines.
+=over 4
+
+=item bootstrap_table_meta
+
+Initializes a table meta structure. Can be safely overridden in a
+derived class, as long as the C<< SUPER >> method is called at the end
+of the overridden method.
+
+It copies the following attributes from the database into the table meta data
+C<< $dbh->{ReadOnly} >> into C<< $meta->{readonly} >>, C<sql_identifier_case>
+and C<sql_data_source> and makes them sticky to the table.
+
+This method should be called before you attempt to map between file
+name and table name to ensure the correct directory, extension etc. are
+used.
+
+=item init_table_meta
+
+Initializes more attributes of the table meta data - usually more
+expensive ones (e.g. those which require class instantiations) - when
+the file name and the table name could mapped.
+
+=item get_table_meta
+
+Returns the table meta data. If there are none for the required table,
+a new one is initialized. When after bootstrapping a new I<table_meta>
+and L</DBI::DBD::SqlEngine::DataSource|completing the table name> a
+mapping can be established between an existing I<table_meta> and the
+new bootstrapped one, the already existing is used and a mapping
+shortcut between the recent used table name and the already known
+table name is hold in C<< $dbh->{sql_meta_map} >>. When it fails,
+nothing is returned. On success, the name of the table and the meta data
+structure is returned.
+
+=item get_table_meta_attr
+
+Returns a single attribute from the table meta data. If the attribute
+name appears in C<%compat_map>, the attribute name is updated from
+there.
+
+=item set_table_meta_attr
+
+Sets a single attribute in the table meta data. If the attribute
+name appears in C<%compat_map>, the attribute name is updated from
+there.
+
+=item table_meta_attr_changed
+
+Called when an attribute of the meta data is modified.
+
+If the modified attribute requires to reset a calculated attribute, the
+calculated attribute is reset (deleted from meta data structure) and
+the I<initialized> flag is removed, too. The decision is made based on
+C<%register_reset_on_modify>.
+
+=item register_reset_on_modify
+
+Allows C<set_table_meta_attr> to reset meta attributes when special
+attributes are modified. For DBD::File, modifying one of C<f_file>, C<f_dir>,
+C<f_ext> or C<f_lockfile> will reset C<f_fqfn>. DBD::DBM extends the
+list for C<dbm_type> and C<dbm_mldbm> to reset the value of C<dbm_tietype>.
+
+If your DBD has calculated values in the meta data area, then call
+C<register_reset_on_modify>:
+
+ my %reset_on_modify = ( "xxx_foo" => "xxx_bar" );
+ __PACKAGE__->register_reset_on_modify( \%reset_on_modify );
+
+=item register_compat_map
+
+Allows C<get_table_meta_attr> and C<set_table_meta_attr> to update the
+attribute name to the current favored one:
+
+ # from DBD::DBM
+ my %compat_map = ( "dbm_ext" => "f_ext" );
+ __PACKAGE__->register_compat_map( \%compat_map );
+
+=item open_data
+
+Called to open the table's data storage. This is silently forwarded
+to C<< $meta->{sql_data_source}->open_data() >>.
+
+After this is done, a derived class might add more steps in an overridden
+C<< open_file >> method.
+
+=item new
+
+Instantiates the table. This is done in 3 steps:
+
+ 1. get the table meta data
+ 2. open the data file
+ 3. bless the table data structure using inherited constructor new
+
+It is not recommended to override the constructor of the table class.
+Find a reasonable place to add you extensions in one of the above four
+methods.
+
+=back
+
=head1 AUTHOR
The module DBI::DBD::SqlEngine is currently maintained by
