Author: timbo Date: Tue Nov 9 14:07:44 2004 New Revision: 572 Modified: dbi/trunk/Roadmap.pod Log: Re-fmt and a few tweaks to the roadmap
Modified: dbi/trunk/Roadmap.pod ============================================================================== --- dbi/trunk/Roadmap.pod (original) +++ dbi/trunk/Roadmap.pod Tue Nov 9 14:07:44 2004 @@ -7,8 +7,8 @@ =head2 SYNOPSIS -This document gives a high level overview of the future of -the Perl DBI module. +This document gives a high level overview of the future of the Perl +DBI module. The DBI module is the standard database interface for Perl applications. It is used worldwide in countless applications, in every kind of @@ -20,15 +20,16 @@ and load balancing, batch statements, Unicode, database portability, and more. -Addressing these issues together, in coordinated way, will help ensure -maximum future functionality with minimal disruptive (incompatible) -upgrades. +Addressing these issues together, in coordinated way, will help +ensure maximum future functionality with minimal disruptive +(incompatible) upgrades. =head2 SCOPE -Broad categories of changes are outlined here along with some rationale, -but implementation details and minor planned enhancements are omitted. -More details can be found in: L<http://svn.perl.org/modules/dbi/trunk/ToDo> +Broad categories of changes are outlined here along with some +rationale, but implementation details and minor planned enhancements +are omitted. More details can be found in: +L<http://svn.perl.org/modules/dbi/trunk/ToDo> =head1 CHANGES AND ENHANCEMENTS @@ -37,16 +38,18 @@ =head2 Performance -The DBI has always treated performance as a priority. Some parts of the -implementation, however, remain unoptimized, especially in relation to threads. +The DBI has always treated performance as a priority. Some parts +of the implementation, however, remain unoptimized, especially in +relation to threads. * When the DBI is used with a Perl built with thread support enabled (such as for Apache mod_perl 2, and some common Linux distributions) -it runs significantly slower. There are two reasons for this and both -can be fixed but require non-trivial changes to both the DBI and drivers. +it runs significantly slower. There are two reasons for this and +both can be fixed but require non-trivial changes to both the DBI +and drivers. -* Connection pooling in a threaded application, such as mod_perl, is -difficult because DBI handles cannot be passed between threads. +* Connection pooling in a threaded application, such as mod_perl, +is difficult because DBI handles cannot be passed between threads. An alternative mechanism for passing connections between threads has been defined, and an experimental connection pool module implemented using it, but development has stalled. @@ -55,16 +58,16 @@ Moving most of this to C will speed up handle creation significantly. * The popular fetchrow_hashref() method is many times slower than -fetchrow_arrayref(). It has to get the names of the columns, then create and -load a new hash each time. A $h->{FetchHashReuse} attribute would allow the -same hash to be reused each time making fetchrow_hashref() about the same speed -as fetchrow_arrayref(). - -* Support for asynchronous (non-blocking) DBI method calls would enable -applications to continue processing in parallel with database activity. -This is also relevant for GUI and other event-driven applications. -The DBI needs to define a standard interface for this so drivers can -implement it in a portable way, where possible. +fetchrow_arrayref(). It has to get the names of the columns, then +create and load a new hash each time. A $h->{FetchHashReuse} attribute +would allow the same hash to be reused each time making fetchrow_hashref() +about the same speed as fetchrow_arrayref(). + +* Support for asynchronous (non-blocking) DBI method calls would +enable applications to continue processing in parallel with database +activity. This is also relevant for GUI and other event-driven +applications. The DBI needs to define a standard interface for +this so drivers can implement it in a portable way, where possible. These changes would significantly enhance the performance of the DBI and many applications which use the DBI. @@ -72,48 +75,52 @@ =head2 Testing -The DBI has a test suite. Every driver has a test suite. Each is limited in -its scope. The driver test suite is testing for behavior that the driver -author I<thinks> the DBI specifies, but may be subtly incorrect. These test -suites are poorly maintained because the return on investment for a single -driver is too low to provide sufficient incentive. +The DBI has a test suite. Every driver has a test suite. Each is +limited in its scope. The driver test suite is testing for behavior +that the driver author I<thinks> the DBI specifies, but may be +subtly incorrect. These test suites are poorly maintained because +the return on investment for a single driver is too low to provide +sufficient incentive. A common test suite that can be reused by all the drivers is needed. It would: * Improve the quality of the DBI and drivers. -* Ensure all drivers conform to the DBI specification. -Easing the porting of applications between databases, and the implementation of -database independent modules layered over the DBI. +* Ensure all drivers conform to the DBI specification. Easing the +porting of applications between databases, and the implementation +of database independent modules layered over the DBI. -* Improve the DBI specification by clarifying unclear issues in order to -implement test cases. +* Improve the DBI specification by clarifying unclear issues in +order to implement test cases. -* Encourage expansion of the test suite as driver authors and others will be -motivated by the greater benefits of their contributions. +* Encourage expansion of the test suite as driver authors and others +will be motivated by the greater benefits of their contributions. -* Detect and record optional functionality that a driver has not yet implemented. +* Detect and record optional functionality that a driver has not +yet implemented. * Improve the testing of DBI subclassing, DBI::PurePerl and the various "transparent" drivers, such as DBD::Proxy and DBD::Multiplex, by automatically running the test suite through them. -These changes would improve the quality of all applications using the DBI. +These changes would improve the quality of all applications using +the DBI. =head2 High Availability and Load Balancing -* The DBD::Multiplex driver provides a framework to enable a wide range of -dynamic functionality, including support for high-availability, load-balancing, -caching, and access to distributed data. It is currently being rewritten to -greatly increase its flexibility and has potential to be a very powerful tool, -but development has stalled. +* The DBD::Multiplex driver provides a framework to enable a wide +range of dynamic functionality, including support for high-availability, +failover, load-balancing, caching, and access to distributed data. +It is currently being rewritten to greatly increase its flexibility +and has potential to be a very powerful tool, but development has +stalled. * The DBD::Proxy module is complex and relatively inefficient because -it's trying to be a complete proxy for most DBI method calls. For many -applications a simpler proxy architecture that operates with a single -round-trip to the server would be sufficient and preferable. +it's trying to be a complete proxy for most DBI method calls. For +many applications a simpler proxy architecture that operates with +a single round-trip to the server would be sufficient and preferable. New proxy client and server classes are needed, which could be subclassed to support specific client to server transport mechanisms @@ -122,23 +129,25 @@ servers. * The DBI currently offers no support for distributed transactions. -The most useful elements of the standard XA distributed transaction interface -standard could be included in the DBI specification. Drivers for databases -which support distributed transactions could then be extended to support it. +The most useful elements of the standard XA distributed transaction +interface standard could be included in the DBI specification. +Drivers for databases which support distributed transactions could +then be extended to support it. -These changes would enable new kinds of DBI applications for critical environments. +These changes would enable new kinds of DBI applications for critical +environments. =head2 Unicode -Use of Unicode with the DBI is growing rapidly. The DBI should do more -to help drivers support Unicode and help applications work with drivers -that don't yet support Unicode directly. +Use of Unicode with the DBI is growing rapidly. The DBI should do +more to help drivers support Unicode and help applications work +with drivers that don't yet support Unicode directly. * Define expected behavior for fetching data and binding parameters. -* Provide interfaces to support Unicode issues for XS and pure Perl drivers -and applications. +* Provide interfaces to support Unicode issues for XS and pure Perl +drivers and applications. * Provide functions for applications to help diagnose inconsistencies between byte string contents and setting of the SvUTF8 flag. @@ -149,8 +158,9 @@ =head2 Batch Statements -Batch statements are a sequence of SQL statements, or a stored procedure -containing a sequence of SQL statements, which can be executed as a whole. +Batch statements are a sequence of SQL statements, or a stored +procedure containing a sequence of SQL statements, which can be +executed as a whole. Currently the DBI has no standard interface for dealing with multiple results from batch statements. After considerable discussion, an @@ -176,14 +186,15 @@ * Currently is it not possible to discover all the child statement handles that belong to a database handle (or all database handles that belong to a driver handle). This makes certain tasks more -difficult, especially some debugging scenarios. A cache of -weak references to child handles would solve the problem without -creating reference loops. - -* It is often useful to know which handle attributes have been changed -since the handle was created (e.g., in mod_perl where a handle needs -to be reset or cloned). This will become more important as developers -start exploring use of the newly added $h1->swap_inner_handle($h2) method. +difficult, especially some debugging scenarios. A cache of weak +references to child handles would solve the problem without creating +reference loops. + +* It is often useful to know which handle attributes have been +changed since the handle was created (e.g., in mod_perl where a +handle needs to be reset or cloned). This will become more important +as developers start exploring use of the newly added +$h1->swap_inner_handle($h2) method. These changes would simplify and improve the stability of many advanced uses of the DBI. @@ -212,8 +223,8 @@ for example, to modify the behavior of a handle independent of any subclassing in use. -These changes would enable cleaner and more powerful integration between -applications, layered modules, and the DBI. +These changes would enable cleaner and more powerful integration +between applications, layered modules, and the DBI. =head2 Debugability @@ -230,7 +241,7 @@ * Integration with the Perl debugger would make it simpler to perform actions on a per-handle basis (such as breakpoint on execute, breakpoint on error). - + These changes would enable more rapid application development and fault finding. @@ -241,26 +252,26 @@ dialects. This is the main hurdle limiting database portability for DBI applications. -The goal is I<not> to fully parse the SQL and rewrite it in a different -dialect. That's well beyond the scope of the DBI and should be -left to layered modules. A simple token rewriting mechanism +The goal is I<not> to fully parse the SQL and rewrite it in a +different dialect. That's well beyond the scope of the DBI and +should be left to layered modules. A simple token rewriting mechanism for five comment styles, two quoting styles, four placeholder styles, plus the ODBC "{foo ...}" escape syntax, is sufficient to significantly raise the level of SQL portability. * Another problem area is date/time formatting. Since version 1.41 -the DBI has defined a way to express that dates should be fetched in SQL -standard date format (YYYY-MM-DD). This is one example of the more general -case where bind_col() needs to be called with particular attributes on all -columns of a particular type. - -A mechanism is needed whereby an application can specify default bind_col() -attributes to be applied automatically for each column type. With a single step, -all DATE type columns, for example, can be set to be returned in the standard -format. +the DBI has defined a way to express that dates should be fetched +in SQL standard date format (YYYY-MM-DD). This is one example of +the more general case where bind_col() needs to be called with +particular attributes on all columns of a particular type. + +A mechanism is needed whereby an application can specify default +bind_col() attributes to be applied automatically for each column +type. With a single step, all DATE type columns, for example, can +be set to be returned in the standard format. -These changes would enable greater database portability for applications -and greater functionality for layered modules. +These changes would enable greater database portability for +applications and greater functionality for layered modules. =head2 Intellectual Property @@ -271,6 +282,9 @@ * Establish a procedure for vetting future contributions for any intellectual property issues. +These changes are important for companies taking a formal approach +to assessing their risks in using Open Source software. + =head2 Other Enhancements @@ -282,32 +296,36 @@ =head2 Parrot and Perl 6 The current DBI implementation in C code is unlikely to run on Perl 6. -Perl 6 will target the Parrot virtual machine and so the internal architecture -will be radically different from Perl 5. +Perl 6 will target the Parrot virtual machine and so the internal +architecture will be radically different from Perl 5. -One of the goals of the Parrot project is to be a platform for many dynamic -languages (including Python, PHP, Ruby, etc) and to enable those languages to -reuse each others modules. A database interface for Parrot is also a database -interface for any and all languages that run on Parrot. - -The Perl DBI would make an excellent base for a Parrot database interface -because it has more functionality, and is more mature and extensible, -than the database interfaces of the other dynamic languages. +One of the goals of the Parrot project is to be a platform for many +dynamic languages (including Python, PHP, Ruby, etc) and to enable +those languages to reuse each others modules. A database interface +for Parrot is also a database interface for any and all languages +that run on Parrot. + +The Perl DBI would make an excellent base for a Parrot database +interface because it has more functionality, and is more mature and +extensible, than the database interfaces of the other dynamic +languages. I plan to better define the API between the DBI and the drivers and -use that API as the primary API for the 'raw' Parrot database interface. -This project is known a Parrot DBDI (for "DataBase Driver Interface"). -The announcement can be read in -L<http://groups.google.com/[EMAIL PROTECTED]>. - -The bulk of the work will be translating the DBI C and Perl base class -code into Parrot PIR, or a suitable language that generates PIR. -The project stalled, due to Parrot not having key functionality at the time. -That has been resolved but the project has not yet restarted. - -Each language targeting Parrot would implement their own small 'thin' -language-specific method dispatcher (a "Perl6 DBI", "Python DBI", -"PHP DBI" etc) layered over the common Parrot DBDI interface and drivers. +use that API as the primary API for the 'raw' Parrot database +interface. This project is known a Parrot DBDI (for "DataBase +Driver Interface"). The announcement can be read in +<http://groups.google.com/[EMAIL PROTECTED]> + +The bulk of the work will be translating the DBI C and Perl base +class code into Parrot PIR, or a suitable language that generates +PIR. The project stalled, due to Parrot not having key functionality +at the time. That has been resolved but the project has not yet +restarted. + +Each language targeting Parrot would implement their own small +'thin' language-specific method dispatcher (a "Perl6 DBI", "Python +DBI", "PHP DBI" etc) layered over the common Parrot DBDI interface +and drivers. The major benefit of the DBDI project is that a much wider community of developers share the same database drivers. There would be more @@ -330,8 +348,8 @@ The first priority is to make all the infrastructure changes that impact drivers and make an alpha release available that driver authors can target. As far as possible, the changes will be implemented -in a way that enables driver authors use the same code base for DBI -v1 and DBI v2. +in a way that enables driver authors use the same code base for DBI v1 +and DBI v2. The main changes required by driver authors are: @@ -359,15 +377,14 @@ See L<http://dbi.perl.org/contributing> for I<how you can help>. -If your company has benefited from the DBI then please consider -if the company could make donation to The Perl Foundation -DBI Development fund at L<http://dbi.perl.org/donate> to secure -future development. - -Alternatively, if your company would benefit from a specific new DBI -feature, please consider sponsoring its development through my -consulting company, Data Plan Services. Work is performed rapidly on -a fixed-price payment-on-delivery basis. Contact me for details. +If your company has benefited from the DBI, please consider if +it could make a donation to The Perl Foundation "DBI Development" +fund at L<http://dbi.perl.org/donate> to secure future development. + +Alternatively, if your company would benefit from a specific new +DBI feature, please consider sponsoring its development through my +consulting company, Data Plan Services. Work is performed rapidly +on a fixed-price payment-on-delivery basis. Contact me for details. Using such targeted financing allows you to contribute to DBI development and rapidly get something specific and directly valuable
