Sorry Jim, I'm working on both Radiant and Refinery projects at the moment, and 
getting a little confused by the "R" words. I meant "does radiant come with 
cucumber scenarios", but I think you got that. 

It's kind of strange testing my Radiant app. When I run rake spec, it runs 
*everything*. Heaps of specs from Radiant itself, for models, controllers, 
helpers, etc. It runs specs for all of the extensions. Interestingly, the 
entire process fails if any of the radiant specs fail, but spec failures in the 
extension suites don't cause the "rake spec" process to return failure as a 
whole. The whole suite takes ages, over five minutes.

When I add extensions, they inevitably monkey patch Radiant, and then it's 
really hard to keep the specs passing. Extensions that do things like reorder 
pages inevitably cause radiant specs to fail (since it expects a different 
order). This is to be expected I imagine. It's sort of odd to run the tests for 
Radiant and all the extensions on one big app. How can they all be expected to 
pass? Since I'm now to the Radiant app I'm working with, I've carefully kept 
them all passing, and it took days. I've now got all the extensions I need 
installed, and a couple of the extensions have failing specs but Radiant 
doesn't, and so the process as a whole passes. 

More sensible would be if only my application itself ran specs using the app 
code. Radiant specs should run without my app, or extensions loaded. Extension 
specs should run with radiant loaded, but not anything else. I shouldn't 
typically run Radiant or extension specs when I'm developing my app, only if 
I'm working on radiant itself, or an extension. I should be able to test 
extensions from within their own repositories without having to put them into 
an app.

Refinery (and I do mean refinery this time) does this by using the rather crude 
approach of putting a dummy rails app into the spec directory of Refinery 
itself. So, it doesn't copy that into the gem, but it is used for running the 
specs. I'm a bit less sure how extension specs work.

It seems to me like poor testing processes are a struggle for Radiant. It's a 
more feature rich CMS than Radiant, with more useful extensions available. 
However, the extensions typically get orphaned, or change without a proper 
upgrade path. My experience of upgrading a refinery 0.8 app, with custom 
extensions, to a 1.0.1 app has led me through a huge number of github forks of 
extensions, and shown some pretty hideous code in some of the extensions. I've 
seen work by two separate consultancies doing Radiant work professionally and 
yet not writing tests, perhaps because it seems too hard.

Anyway, pretty easy to pick on things from the outside. Obviously a huge amount 
of work has gone into things, and I don't want to be too critical. I hope the 
above thoughts are of some constructive use to folks. 

-- 
Craig Ambrose


On Tuesday, 1 May 2012 at 1:10 PM, Jim Gay wrote:

> On Mon, Apr 30, 2012 at 8:28 PM, Craig Ambrose <cr...@craigambrose.com 
> (mailto:cr...@craigambrose.com)> wrote:
> > Hi Jim,
> > 
> > At some point in my experimentation, the error changed to something like a
> > failure to find the feature directory. I was attempting to proceed in very
> > slow and measured steps, but I missed noticing exactly how and when that
> > error change occurred. To fix that later error, I added an empty cucumber
> > directory structure to my base app. Now, when it tries to run the
> > integration tests (by which it seems to mean cucumber scenarios) it finds
> > zero cucumber features and so passes. That means that my "rake spec" is
> > passing again, so it seems good.
> > 
> > Is that what I should be expecting? Does refinery come with any cucumber
> > scenarios of it's own, or is it fine to be passing with zero specs?
> > 
> 
> 
> Ha ha. I don't know if refinery does, but Radiant has cucumber features ;-P
> I don't recall if the task runs the Radiant specs along with your own,
> but I'll try to dig into this sometime this week if I can.
> 
> -Jim
> 
> > 
> > regards,
> > 
> > Craig
> > 
> > --
> > Craig Ambrose
> > 
> > On Tuesday, 1 May 2012 at 8:33 AM, Jim Gay wrote:
> > 
> > Can you paste the full stack trace somewhere?
> > 
> > On Thu, Apr 19, 2012 at 9:36 PM, Craig Ambrose <craigambr...@gmail.com 
> > (mailto:craigambr...@gmail.com)>
> > wrote:
> > 
> > Hi everyone,
> > 
> > I have an app I have upgraded from 0.8.6 to 1.0.1, and all the specs
> > are working fine, except for the integration specs. Running them
> > generates the following error immediately:
> > 
> > no such file to load -- file_not_found_page (MissingSourceFile)
> > /Users/craig/.rvm/gems/ruby-1.8.7-p334@nzffa/gems/activesupport-2.3.14/
> > lib/active_support/dependencies.rb:184:in `require'
> > /Users/craig/.rvm/gems/ruby-1.8.7-p334@nzffa/gems/activesupport-2.3.14/
> > lib/active_support/dependencies.rb:184:in `require'
> > /Users/craig/.rvm/gems/ruby-1.8.7-p334@nzffa/gems/radiant-1.0.1/spec/
> > datasets/file_not_found_dataset.rb:1
> > 
> > This is true if I run "bundle exec rake spec:integration", or if they
> > run as part of a full spec run.
> > 
> > Although my app is an upgrade, I've disabled all extensions and still
> > get this issue.
> > 
> > Line 1 of radiant-1.0.1/spec/datasets/file_not_found_dataset.rb does
> > indeed try to require 'file_not_found_page.rb'. If I require this file
> > from within the console of my app, it works fine. So, possibly this is
> > an issue related to include paths. Potentially this may relate to
> > which exact version of cucumber I'm using. My Gemfile.lock includes:
> > 
> >    cucumber (1.1.9)
> >      builder (>= 2.1.2)
> >      diff-lcs (>= 1.1.2)
> >      gherkin (~> 2.9.0)
> >      json (>= 1.4.6)
> >      term-ansicolor (>= 1.0.6)
> >    cucumber-rails (0.3.2)
> >      cucumber (>= 0.8.0)
> >    database_cleaner (0.6.7)
> > 
> > Does anyone have any leads on this?
> > 
> > cheers,
> > 
> > Craig Ambrose
> > 
> > 
> > 
> > 
> > --
> > Write intention revealing code #=> http://www.clean-ruby.com
> > 
> > Jim Gay
> > Saturn Flyer LLC
> > 571-403-0338
> > 
> 
> 
> 
> 
> -- 
> Write intention revealing code #=> http://www.clean-ruby.com
> 
> Jim Gay
> Saturn Flyer LLC
> 571-403-0338
> 
> 


Reply via email to