Without reading your code line by line.. sometimes I've found I need to do a "heroku restart" to get it to pick up some categories of changes. I don't really have any good info on what circumstances or types of changes require it, but give it a try?
On Mon, Jan 3, 2011 at 2:31 AM, Michael Baldock <[email protected]> wrote: > Dear All, > > Fairly new to rails and Heroku, so could be doing something wacky - do > let me know if you think my code practice is off, even if unrelated to > this error, I'd like to learn! > > I'm using Rails 3.0.0, ruby 1.8.7, and 'sqlite3-ruby', > '1.2.5', :require => 'sqlite3'. > > I've got an application that goes off and scrapes websites for event > information, to do this I'm using: > > nokogiri - for scraping > date - to help with parsing > chronic - to help recognise dates > open-uri - to open the url > > all of these are just gem 'gem-name', so latest unspecified versions. > > The code that does this scraping is in the ApplicationController class > - a bad place for it? > I've just got it working as a background process using delayed_job, I > have a class called EventSearcher with the required 'perform' method. > > My error occurs when I upload to Heroku, if I make changes to the app, > but NOT to ApplicationController, > do $git add . / git commit .... / git push heroku master / heroku > workers 1 / heroku open > then the process runs, appears successful in the logs, but returns no > events > > The error can be solved I've discovered by making small changes to > Application controller, such as adding the line > logger.debug"Changing AppController again" > then doing $git add . / git commit .... / git push heroku master / > heroku workers 1 / heroku open > this time the process runs, appears in logs to take longer, and > returns the scraped info as required. > > Any ideas why this should be?? > > I have tried solving this myself by putting in 'logger.debug" > commands, but these don't show on my heroku logs, any idea how to get > them to show up? > > I've put the code below, for ApplicationController, EventSearcher, and > part of SuggestedEvents it's a bit of a mess I know, but runs fine on > local machine. > > > -------------- Application Controller > > class ApplicationController < ActionController::Base > > require 'date' > require 'nokogiri' > require 'open-uri' > require 'chronic' > > protect_from_forgery > include SessionsHelper > > def authenticate > deny_access unless signed_in? > end > > > def find_all_events > > �...@event_searches = EventSearch.all > > �...@event_searches.each do |es| > > do_search(es) > > end > end > > def find_one_band_events(event_id) > > es = EventSearch.where(:id => event_id)[0] > do_search(es) > end > > def do_search(event_search) > > logger.debug"Changing AppController again" > logger.debug"in do search" > url = event_search.urlOne > > if validate(url) > > logger.debug"Passed Validate URL" > > doc = Nokogiri::HTML(open(url)) > > #search the doc with the saved css search strings > if !(event_search.eventDateCSS.empty?) > startDate = doc.css(event_search.eventDateCSS) else eventDate > = 0 > end > if !(event_search.eventNameCSS.empty?) > eventName = doc.css(event_search.eventNameCSS) else eventName > = 0 > end > if !(event_search.eventLocationCSS.empty?) > location = doc.css(event_search.eventLocationCSS) else > location = 0 > end > if !(event_search.eventTimeCSS.empty?) > time = doc.css(event_search.eventTimeCSS) else time = 0 end > if !(event_search.priceCSS.empty?) > price = doc.css(event_search.priceCSS) else price = 0 end > if !(event_search.descriptionCSS.empty?) > description = doc.css(event_search.descriptionCSS) else > description > = 0 end > i=0 > > if eventDate != 0 > while i < startDate.length # find all events on that page going > through each case of start date > > SuggestedEvent.new do |@savedEvent| > > if eventDate != 0 > �[email protected] = > date_from_string(startDate[i].content) > logger.debug(startDate[i].content) > end > > test = SuggestedEvent.where(:bandName => > event_search.bandName, > > :eventDate => @savedEvent.eventDate) > > if !test[0] > > �[email protected] = event_search.bandName > if eventName != 0 > �[email protected] = > remove_whitespace(eventName[i].content) > end > if time != 0 > �[email protected] = > Chronic.parse(time[i].content) end > if price != 0 > �[email protected] = price[i].content end > if description !=0 > �[email protected] = > remove_whitespace(description[i].content) end > > if location != 0 > savedVenueName = > paramMatcher('Venue','venueName', > remove_whitespace(location[i].content)) > if savedVenueName > �[email protected] = > savedVenueName[0] > �[email protected]_location = > savedVenueName[1] > �[email protected] = > savedVenueName[2] > �[email protected] = > savedVenueName[3] > else > �...@newvenue = Venue.new > �[email protected] = > remove_whitespace(location[i].content) > �[email protected]_venue_name = > @newVenue.venueName > �[email protected] > �[email protected] = > remove_whitespace(location[i].content) > end #of if > else > �[email protected] = 'No saved venue > found' > end #of location != > �[email protected] > end # of if !test > end # of .new do > i += 1 > end # of while > > end #of if eventDate !=0 > > end # of if url passes > > > end # of def > > > # get the date from whatever string gets thrown > def date_from_string(date) > > # should remove st rd nd and th > > firstParse = Chronic.parse(date) > r1 = /[a-zA-Z]/ > > #daY Less than 12, assume chronic wrong for Britain, as long as also > no characters such as December, > #where it would be right > if firstParse.day <= 12 and !r1.match(date) > > #swap month with day > firstParse = firstParse.change(:day => firstParse.month, :month => > firstParse.day) > > end #of if > > return firstParse > > end #of def > > > > # remove whitespace and other characters such as newline > > def remove_whitespace(dirty_name) > > return dirty_name.split(' ').join(" ") > > end > > > #find names in strings from the net > def paramMatcher(modelString, param, cssScrapedString) > > case modelString > when 'Venue' > venues = Venue.all > i=0 > venues.each do |v| > > if v.venueName.scan(cssScrapedString)[0] or > cssScrapedString.scan(v.venueName)[0] > > return [v.venueName, v.city_location, v.longitude, > v.latitude] > end > > end # of .each > end # of case > > return nil > end # of def param Matcher > > def validate(url) > begin > uri = URI.parse(url) > if uri.class != URI::HTTP > return false > end > rescue URI::InvalidURIError > return false > end > return true > end > > > end # of class > > > > ----------------- Event Searcher > > > class EventSearcher < ApplicationController > > attr_accessor :all_events > attr_accessor :one_event_id > > def perform > > logger.debug"in perform" > > if all_events == 1 # searches all band names > > logger.debug"in perform, all_events == 1" > > find_all_events > > end > > if one_event_id != nil #searches just one band name using id > > logger.debug"in perform, one_event != nil" > logger.debug(one_event_id) > find_one_band_events(one_event_id) > > end > > end # Of Perform > > end #of class > > > -------------- Suggested Events > > class SuggestedEventsController < ApplicationController > > #.... other functions removed for brevity.... > > def search_all_events > > logger.debug"Sugg E search all events" > #the instance of the object that gets put on the queue > �...@event_searcher = EventSearcher.new > �...@event_searcher.one_event_id = nil > �...@event_searcher.all_events = 1 > Delayed::Job.enqueue @event_searcher > > flash[:notice] = "Finding All events, this takes a while, refresh > after 10 mins" > redirect_to suggested_events_path > > end > > > end > > > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
