Jorge Gonzalez wrote:
Hi,
I'm developing an app with Catalyst 5.80024, on perl 5.8.8, 32 bits;
DBIx::Class models.
> I experienced very bad performance: index pages loading in 30 seconds or
more, and Catalyst reporting that the request took 40s and the like.
Completely unbearable.
After installing Devel::NYTProf and profiling my app, NYTProf showed
that about 45% (!!!!) of the time was spent in the sub
Class::C3::_calculate_method_dispatch_table. Knowing that there's a XS
module for speeding the C3 module, I install Class::C3::XS which
Class::C3 loads and uses if available.
> - Has anyone stepped over this performance problems before?
- Has DBIx::Class scalability problems beyond a given number of tables?
- My perl (5.8.8) is a bit old (comes with Centos 5), could this issues
be solved in later versions of perl? I know C3 is integrated in basee
perl in latest versions...
- What if I need an app to handle >100 tables?
Attached is a test run and all my module versions so you can compare with
your setup. 2 seconds for 1000 (albeit simple) result classes is a good
thing in my book (and that's without C::C3::XS). Your hypothetical use
case of 100 tables takes less than 100ms, so it's not very interesting.
Please try to find out what in your setup is different from what I have
below, causing you to experience such unacceptable slowdowns.
Cheers
rabbit@Thesaurus:~/devel/dbic/dbgit$ grep VERSION lib/DBIx/Class.pm
use vars qw($VERSION);
$VERSION = '0.08123';
$VERSION = eval $VERSION if $VERSION =~ /_/; # numify for warning-free dev
releases
rabbit@Thesaurus:~/devel/dbic/dbgit$ perl -v
This is perl, v5.8.8 built for i686-linux-thread-multi
Copyright 1987-2006, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
rabbit@Thesaurus:~/devel/dbic/dbgit$ prove -l t/39load_namespaces_stress.t -v
t/39load_namespaces_stress.t ..
ok 1 - Start with no sources
# 1298931981.42485: load_namespaces start
# 1298931983.28045: load_namespaces finished
ok 2 - All sources attached
1..2
ok
All tests successful.
Files=1, Tests=2, 4 wallclock secs ( 0.03 usr 0.01 sys + 3.93 cusr 0.08
csys = 4.05 CPU)
Result: PASS
rabbit@Thesaurus:~/devel/dbic/dbgit$ perl -d:TraceUse -Ilib
t/39load_namespaces_stress.t -v
ok 1 - Start with no sources
# 1298932023.15427: load_namespaces start
# 1298932025.34957: load_namespaces finished
ok 2 - All sources attached
1..2
Modules used from t/39load_namespaces_stress.t:
1. strict 1.03, t/39load_namespaces_stress.t line 1 [main]
2. warnings 1.05, t/39load_namespaces_stress.t line 2 [main]
3. Carp 1.04, warnings.pm line 134
4. Exporter 5.63, Carp.pm line 193
9. Exporter::Heavy 5.63, Exporter.pm line 19
5. Test::More 0.96, t/39load_namespaces_stress.t line 3 [main]
6. Test::Builder::Module 0.96, Test/More.pm line 23
7. Test::Builder 0.96, Test/Builder/Module.pm line 5
8. Config, Test/Builder.pm line 19
10. Time::HiRes 1.86, t/39load_namespaces_stress.t line 4 [main]
11. vars 1.01, Time/HiRes.pm line 4
12. warnings::register 1.01, vars.pm line 7
13. DynaLoader 1.05, Time/HiRes.pm line 7
14. AutoLoader 5.60, DynaLoader.pm line 22
15. lib 0.5565, t/39load_namespaces_stress.t line 6 [main]
40. DBICTest::Schema, DBICTest.pm line 7
16. Path::Class 0.21, DBICTest/AuthorCheck.pm line 7
17. Path::Class::File 0.21, Path/Class.pm line 10
18. Path::Class::Dir 0.21, Path/Class/File.pm line 6
19. base 2.07, Path/Class/Dir.pm line 8
20. Path::Class::Entity 0.21, base.pm line 82 (eval 5)
21. File::Spec 3.33, Path/Class/Entity.pm line 6
22. File::Spec::Unix 3.33, File/Spec.pm line 22
23. File::stat 1.00, Path/Class/Entity.pm line 7
24. Class::Struct 0.63, File/stat.pm line 27
25. Cwd 3.33, Path/Class/Entity.pm line 8
26. XSLoader 0.10, Cwd.pm line 247
27. overload 1.04, Path/Class/Entity.pm line 15
41. DBIx::Class::Schema -1, set by base.pm, base.pm line 82 (eval 14)
42. DBIx::Class::Exception, DBIx/Class/Schema.pm line 6
43. Carp::Clan 6.04, DBIx/Class/Exception.pm line 6
44. Scalar::Util 1.23, DBIx/Class/Exception.pm line 7
45. List::Util 1.23, Scalar/Util.pm line 12
46. Try::Tiny 0.07, DBIx/Class/Exception.pm line 8
47. namespace::clean 0.20, DBIx/Class/Exception.pm line 9
48. Sub::Name 0.05, namespace/clean.pm line 14
49. Sub::Identify 0.04, namespace/clean.pm line 15
50. Package::Stash 0.23, namespace/clean.pm line 16
51. Package::Stash::XS 0.20, Package/Stash.pm line 24 (eval
19)
52. Package::DeprecationManager 0.10, Package/Stash.pm line
56
53. List::MoreUtils 0.22, Package/DeprecationManager.pm
line 10
54. Params::Util 1.01, Package/DeprecationManager.pm line
11
55. Sub::Install 0.925, Package/DeprecationManager.pm line
12
56. B::Hooks::EndOfScope 0.09, namespace/clean.pm line 17
57. Variable::Magic 0.44, B/Hooks/EndOfScope.pm line 14
58. Sub::Exporter 0.982, B/Hooks/EndOfScope.pm line 19
59. Data::OptList 0.106, Sub/Exporter.pm line 7
60. Module::Find 0.10, DBIx/Class/Schema.pm line 12
61. File::Find 1.10, Module/Find.pm line 8
62. Storable 2.15, DBIx/Class/Schema.pm line 13
63. Log::Agent, Storable.pm line 33 (eval 32) (FAILED)
79. locale 1.00, DBIx/Class/Schema.pm line 175
64. DBIx::Class 0.08123, base.pm line 82 (eval 33)
65. MRO::Compat 0.11, DBIx/Class.pm line 6
66. Class::C3 0.23, MRO/Compat.pm line 27
67. Class::C3::XS, Class/C3.pm line 18 (eval 34) (FAILED)
68. Algorithm::C3 0.08, Class/C3.pm line 25
69. Class::C3::next, Class/C3.pm line 26
70. DBIx::Class::Optional::Dependencies, DBIx/Class.pm line 9
78. DBIx::Class::StartupCheck, DBIx/Class.pm line 13
71. DBIx::Class::Componentised -1, set by base.pm, base.pm line 82
(eval 35)
72. Class::C3::Componentised 1.0006, base.pm line 82 (eval 36)
73. Class::Inspector 1.24, Class/C3/Componentised.pm line 46
74. utf8 1.06, Class/Inspector.pm line 58 (eval 37)
80. DBICTest::Schema::Artist -1, set by base.pm,
Class/C3/Componentised.pm line 135
83. DBIx::Class::Relationship, Class/C3/Componentised.pm line 135
84. DBIx::Class::Relationship::Helpers, Class/C3/Componentised.pm
line 135
85. DBIx::Class::Relationship::HasMany, Class/C3/Componentised.pm
line 135
86. DBIx::Class::Relationship::HasOne, Class/C3/Componentised.pm
line 135
87. DBIx::Class::Relationship::BelongsTo,
Class/C3/Componentised.pm line 135
88. DBIx::Class::Relationship::ManyToMany,
Class/C3/Componentised.pm line 135
89. DBIx::Class::Relationship::Accessor, Class/C3/Componentised.pm
line 135
90. DBIx::Class::Relationship::CascadeActions,
Class/C3/Componentised.pm line 135
91. DBIx::Class::Relationship::ProxyMethods,
Class/C3/Componentised.pm line 135
92. DBIx::Class::Relationship::Base, Class/C3/Componentised.pm
line 135
93. DBIx::Class::InflateColumn, Class/C3/Componentised.pm line 135
95. DBIx::Class::PK::Auto, Class/C3/Componentised.pm line 135
96. DBIx::Class::PK, Class/C3/Componentised.pm line 135
97. DBIx::Class::ResultSourceProxy::Table,
Class/C3/Componentised.pm line 135
99. DBIx::Class::ResultSource::Table,
DBIx/Class/ResultSourceProxy/Table.pm line 8
100. DBIx::Class::ResultSet -1, set by base.pm,
DBIx/Class/ResultSource/Table.pm line 6
101. Data::Page 2.02, DBIx/Class/ResultSet.pm line 8
105. integer 1.00, Data/Page.pm line 142
106. DBIx::Class::ResultSetColumn, DBIx/Class/ResultSet.pm
line 10
107. DBIx::Class::ResultSourceHandle, DBIx/Class/ResultSet.pm
line 11
108. DBIx::Class::ResultSource, Class/C3/Componentised.pm line 135
110. DBICTest::Schema::CD -1, set by base.pm,
Class/C3/Componentised.pm line 135
111. DBICTest::Schema::Track, Class/C3/Componentised.pm line 135
112. DBIx::Class::InflateColumn::DateTime,
Class/C3/Componentised.pm line 135
113. DBIx::Class::Ordered, Class/C3/Componentised.pm line 135
114. DBICTest::Schema::Lyrics, Class/C3/Componentised.pm line 135
115. DBICTest::Schema::LyricVersion, Class/C3/Componentised.pm line
135
116. DBICTest::Schema::Tag, Class/C3/Componentised.pm line 135
117. DBICTest::Schema::CD_to_Producer, Class/C3/Componentised.pm
line 135
118. DBICTest::Schema::LinerNotes, Class/C3/Componentised.pm line
135
119. DBICTest::Schema::Artwork, Class/C3/Componentised.pm line 135
120. DBICTest::Schema::Image, Class/C3/Componentised.pm line 135
121. DBICTest::Schema::Artwork_to_Artist, Class/C3/Componentised.pm
line 135
122. DBICTest::Schema::TwoKeys, Class/C3/Componentised.pm line 135
123. DBICTest::Schema::OneKey, Class/C3/Componentised.pm line 135
124. DBICTest::Schema::SequenceTest, Class/C3/Componentised.pm line
135
125. DBICTest::Schema::BindType, Class/C3/Componentised.pm line 135
126. DBICTest::Schema::Employee, Class/C3/Componentised.pm line 135
127. DBICTest::Schema::Encoded, Class/C3/Componentised.pm line 135
128. DBICTest::Schema::FileColumn, Class/C3/Componentised.pm line
135
129. File::Temp 0.22, DBICTest/Schema/FileColumn.pm line 7
130. Errno 1.0901, File/Temp.pm line 148
131. constant 1.05, File/Temp.pm line 217
132. Carp::Heavy, File/Temp.pm line 156
133. DBICTest::Schema::Genre, Class/C3/Componentised.pm line 135
134. DBICTest::Schema::Link, Class/C3/Componentised.pm line 135
135. DBICTest::Schema::Bookmark, Class/C3/Componentised.pm line 135
136. DBICTest::Schema::Year2000CDs, Class/C3/Componentised.pm line
135
137. DBIx::Class::ResultSource::View, Class/C3/Componentised.pm
line 135
138. DBICTest::Schema::Year1999CDs, Class/C3/Componentised.pm line
135
139. DBICTest::Schema::CustomSql, Class/C3/Componentised.pm line 135
140. DBICTest::Schema::Money, Class/C3/Componentised.pm line 135
141. DBICTest::Schema::TimestampPrimaryKey,
Class/C3/Componentised.pm line 135
142. DBICTest::Schema::Serialized, Class/C3/Componentised.pm line
135
143. DBICTest::Schema::FourKeys, Class/C3/Componentised.pm line 135
144. DBICTest::Schema::FourKeys_to_TwoKeys,
Class/C3/Componentised.pm line 135
145. DBICTest::Schema::SelfRef, Class/C3/Componentised.pm line 135
146. DBICTest::Schema::SelfRefAlias, Class/C3/Componentised.pm line
135
147. DBICTest::Schema::ArtistUndirectedMap,
Class/C3/Componentised.pm line 135
148. DBICTest::Schema::ArtistSourceName, Class/C3/Componentised.pm
line 135
149. DBICTest::Schema::ArtistSubclass, Class/C3/Componentised.pm
line 135
150. DBICTest::Schema::Producer, Class/C3/Componentised.pm line 135
151. DBICTest::Schema::Dummy, Class/C3/Componentised.pm line 135
152. DBICTest::Schema::TreeLike, Class/C3/Componentised.pm line 135
153. DBICTest::Schema::TwoKeyTreeLike, Class/C3/Componentised.pm
line 135
154. DBICTest::Schema::Event, Class/C3/Componentised.pm line 135
155. DBICTest::Schema::EventTZ, Class/C3/Componentised.pm line 135
156. DBICTest::Schema::NoPrimaryKey, Class/C3/Componentised.pm line
135
157. DBICTest::Schema::Collection, Class/C3/Componentised.pm line
135
158. DBICTest::Schema::CollectionObject, Class/C3/Componentised.pm
line 135
159. DBICTest::Schema::TypedObject, Class/C3/Componentised.pm line
135
160. DBICTest::Schema::Owners, Class/C3/Componentised.pm line 135
161. DBICTest::Schema::BooksInLibrary, Class/C3/Componentised.pm
line 135
162. DBICTest::Schema::ForceForeign, Class/C3/Componentised.pm line
135
75. Class::Accessor::Grouped 0.10002, base.pm line 82 (eval 39)
76. Class::XSAccessor 1.11, Class/Accessor/Grouped.pm line 512
77. Class::XSAccessor::Heavy 1.11, Class/XSAccessor.pm line 6
81. DBICTest::BaseResult -1, set by base.pm, base.pm line 82 (eval
47)
109. DBICTest::BaseResultSet, DBICTest/BaseResult.pm line 8
82. DBIx::Class::Core -1, set by base.pm, base.pm line 82 (eval 48)
94. DBIx::Class::Row -1, set by base.pm, base.pm line 82 (eval 53)
98. DBIx::Class::ResultSourceProxy -1, set by base.pm, base.pm line
82 (eval 54)
102. Class::Accessor::Chained::Fast -1, set by base.pm, base.pm line
82 (eval 58)
103. Class::Accessor::Fast 0.34, base.pm line 82 (eval 59)
104. Class::Accessor 0.34, base.pm line 82 (eval 60)
28. IO::Dir 1.05, Path/Class/Dir.pm line 10
29. Symbol 1.06, IO/Dir.pm line 13
30. IO::File 1.13, IO/Dir.pm line 15
31. SelectSaver 1.01, IO/File.pm line 132
32. IO::Seekable 1.1, IO/File.pm line 133
33. IO::Handle 1.25, IO/Seekable.pm line 101
34. IO 1.22, IO/Handle.pm line 262
35. Fcntl 1.05, IO/Seekable.pm line 104
36. Tie::Hash 1.02, IO/Dir.pm line 17
37. File::Path 2.08, Path/Class/Dir.pm line 11
38. File::Basename 2.74, File/Path.pm line 7
39. re 0.05, File/Basename.pm line 44
Modules used, but not reported:
/home/rabbit/perl5/perlbrew/perls/5.8.8/lib/5.8.8/i686-linux-thread-multi/auto/Storable/autosplit.ix
DBICTest.pm
DBICTest/AuthorCheck.pm
mro.pm
rabbit@Thesaurus:~/devel/dbic/dbgit$ cat t/39load_namespaces_stress.t
use strict;
use warnings;
use Test::More;
use Time::HiRes qw/gettimeofday/;
use lib qw(t/lib);
use DBICTest; # do not remove even though it is not used
our $src_count = 1000;
for (1 .. $src_count) {
eval <<EOM or die $@;
package DBICTest::NS::Stress::Schema::Result::T$_;
use base qw/DBIx::Class::Core/;
__PACKAGE__->table($_);
__PACKAGE__->add_columns (
id => { data_type => 'integer', is_auto_increment => 1 },
data => { data_type => 'varchar', size => 255 },
);
__PACKAGE__->set_primary_key('id');
__PACKAGE__->add_unique_constraint(['data']);
EOM
}
{
package DBICTest::NS::Stress::Schema;
use base qw/DBIx::Class::Schema/;
sub _findallmod {
return $_[1] eq ( __PACKAGE__ . '::Result' )
? ( map { __PACKAGE__ . "::Result::T$_" } 1 .. $::src_count )
: ()
;
}
}
is (DBICTest::NS::Stress::Schema->sources, 0, 'Start with no sources');
note gettimeofday . ":\tload_namespaces start";
DBICTest::NS::Stress::Schema->load_namespaces;
note gettimeofday . ":\tload_namespaces finished";
is (DBICTest::NS::Stress::Schema->sources, $src_count, 'All sources attached');
done_testing;
_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/[email protected]
