Just in case someone is curious, I've replaced ActiveRecord models with
Sequel ones and my slowest example as reported by "rspec -d" got faster
from about 0.5s to 0.2s.
That is awesome! :)
It seems AR is bloated...
As if this wasn't already amazing enough, my models seem much cleaner
with Sequel than with AR.
Cheers,
Rodrigo.
Em 16-04-2012 14:17, Rodrigo Rosenfeld Rosas escreveu:
Em 16-04-2012 09:27, Rodrigo Rosenfeld Rosas escreveu:
Thank you David and Matt for your comments.
Indeed I had considered using FakeWeb but I thought that maybe there
could be an easier way just to mock 'open' directly since I guess it
would be faster to run.
With regards to the work around to overcome the issue with mocking AR
so that I could speed up my tests I think that creating helpers in
the model that will restrict the use of the AR API sounds like a good
balance, but that wouldn't make my specific tests faster.
Here is why: my Model class shouldn't be created or updated by any
other code in my application. So I'd just have to add a new method to
the Model class that would be specific to this action and I'd need to
write a "unit" test for this new method. I quoted "unit" because it
is actually an integration test as it will touch the database or
otherwise I wouldn't be able to upgrade Rails to a new version if I'm
mocking some methods from AR and they change in a future version.
So I'd need to touch the database anyway and my 9 specs would still
take almost a full second to run while this specific controller spec
takes about 0.7s (up to 2s depending on my system load).
Well, actually, I guess RSpec has changed the way it measure spec
runs since I last used it. I'm saying that because it will report me
from 0.7 to 2s when I run "rspec path/to/my_controller_spec.rb", but
if I run "rspec --profile" it will run the full suite about the same
time and this specific controller action will take 0.38s.
Anyway, I don't see how I would create hundreds of specs and expect
them to run in a few seconds and still have a reliable test suite. I
mean, I want to feel safe if I'm upgrading to a newer Rails version
even if it changed its AR API.
The only real improvements I can see for my tests running fasts
without changing anything in the specs is to use something like Hydra
so that I could use the full power of my 6-core processor.
Does RSpec has a built-in threaded option? I guess not because I
don't think it would be much effective on CRuby because it would
probably lock on database access (or maybe not, I don't know how the
pg driver is implemented). For my particular application I suspect it
is not cpu-heavy, but io-expensive instead. So maybe a multi-thread
spec runner could improve its performance when running multiple specs.
Best,
Rodrigo.
After some more investigation I could find out why it is taking so
much time for the spec to run. Even for a simple model, calling a
single "Model.create! name: 'Model name'" will take almost 100ms. So,
here is usually what happens:
root = FactoryGirl.create :model # 2 creates:
# section = Section.create! name: 'Section'
# root = Model.create! section: section, label: 'Label'
post 'action', parent_id: root.id # another Model creation in the
controller alongside with 2 other creations of a has_many association
So, without counting any assertions just for creating all records so
far about 400ms were already spent. There will also be about 2 more
updates and 2 "delete" statements in this spec and that will complete
about 540ms of total spec run time.
Another interesting fact is that for measuring this I've mocked the
user and controller authentication related methods, but if I actually
create the user the time for completing the spec will be about the
same, even if in the logs it is reported to spend about 70ms for
executing the insert in the user's table.
But I don't think it should take so much time for creating/updating
those records. See the output of the logging, for example:
Completed 200 OK in 83ms (Views: 0.6ms | ActiveRecord: 14.9ms)
This is consistent with the individual timing for each query/insert
statement including validations. Even so, the first 'post' method call
in the spec takes about 100ms.
So, my question is, if AR and Views are taking about 16ms what else
could be taking about 70ms?
This is also hard to profile since PostgreSQL will need more time for
the first queries and will be much faster for subsequent ones... This
means that running all specs will take much less time than if I run
each spec separately and compare to the total time.
But I don't really think I'll be able to test this action faster. I
need to check for all created records including associations and
fields set by the before_validation and removed by after_save so that
I can get confidence in the code. And since all of this should only
happen through this action, it wouldn't matter if I moved lots of
those checks to the "unit" test of my Model as the total time would
remain the same. It would only make my specs harder to read with lots
of mocks and stubbed methods in my controller spec.
I mean, that is ok, I don't aim in having single assertions per unit
test. I just need to make sure things won't break in a future
refactoring or requirement changes. And of course I'd like my tests to
run faster, but my current understanding is that I can't get this test
to run faster because it is testing a lot of code. Of course that an
in-memory AR adapter could help me speeding up my tests but using
mocks here won't give me any confidence on my code base.
Usually slow tests are not a problem when I'm working in some
controller as I don't need to run the full suite. But when I change
some model or library it would be great to run the full suite and I'm
a bit worried if that will become super slow some time in the future.
But I think I'll start to worry about this when this future becomes
present because there is no easy replacement that will allow my specs
to run faster. I'll still be able to try Hydra before having to resort
to mocks if things get really unsustainable.
But thank you very much for your input on the subject. I was just
worried if I was doing something fundamentally wrong.
Cheers,
Rodrigo.
_______________________________________________
rspec-users mailing list
rspec-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/rspec-users
_______________________________________________
rspec-users mailing list
rspec-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/rspec-users
