Thanks guys.. I'm trying Capybara with Poltergeist / phantomjs and the hints from https://github.com/teamcapybara/capybara#calling-remote-servers. I'll post my experiences here again.
Martin Am Montag, 19. Juni 2017 16:18:48 UTC+2 schrieb Jason FB: > > > > I think he's scraping someone else's site. > > You obviously can't do this with Ruby alone, as there is no headless web > browser written entirely in Ruby (that's just nonsense) > > If you can get *phantomjs* working on your production site, that's > probably the way to go. Look deep into the internals of Capybara to > understand how it drives phantomjs. With phantomjs, you basically have a > headless web browser and you can use Capybara's DSL to access parts of the > page, including evaluating scripts and parsing the DOM. > > Just keep in mind phantomjs is an actual executable so it needs to be > compiled and built for your production environment explicitly, which might > be a little tricky depending on where your site is. > > But a little birdie told me a few months ago that the phantomjs team has > decided that once Chrome has a headless mode, which I believe is > forthcoming, they plan to abandon phantomjs in favor of Chrome's headless > mode. Not sure if that's really true or when that will happen. > > -Jason > > > > On Jun 18, 2017, at 8:36 AM, Colin Law <[email protected] <javascript:>> > wrote: > > On 18 June 2017 at 12:21, Martin Luy <[email protected] <javascript:>> > wrote: > > I'm extracting content from some websites. Currently I evaluate HTML code > using Nokogiri. But the relevant content is not contained in the responded > body of the HTTP GET request. This is because there is some Javascript code > like $(window).load() or $(document).ready() that will send some Ajax > requests and fill the original HTML code. > > So I'm searching for some library that automatically executes Javascript > code and Ajax requests just like a normal browser. > > > Understood. Don't think I can help I am afraid. Does the site not work > with js disabled in the browser? > > Colin > > > Martin > > Von meinem Samsung Gerät gesendet. > > > -------- Ursprüngliche Nachricht -------- > Von: Colin Law <[email protected] <javascript:>> > Datum: 18.06.17 09:42 (GMT+01:00) > An: "Ruby on Rails: Talk" <[email protected] <javascript:>> > Betreff: Re: [Rails] browser simulator independent of web framework > > On 17 June 2017 at 22:58, Martin L. <[email protected] <javascript:>> > wrote: > > Hi all, > > Is there any browser simulator that fulfills these requirements: > > - gem written in Ruby > - automatically performing Ajax and Javascript code (XSS is not an issue > in > my case) > - independent of the frameworks used by the website (Rails, JavaEE, > ASP.NET <http://asp.net>, > ...) > - only client-side > - no testing > - no browser dependency > > > What do you mean 'no testing'? If not for testing then what is it for? > > Colin > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Ruby on Rails: Talk" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/rubyonrails-talk/H_YImOIzNNo/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected] <javascript:>. > To post to this group, send email to [email protected] > <javascript:>. > To view this discussion on the web visit > > https://groups.google.com/d/msgid/rubyonrails-talk/CAL%3D0gLvqHArcWpbZ5gsfCiPg0EF%3D4kD8QzZbC3KnicN58uAZ8A%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] > <javascript:>. > To view this discussion on the web visit > > https://groups.google.com/d/msgid/rubyonrails-talk/smyckbsskql3om4h9odf01hm.1497784510166%40email.android.com > . > > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] > <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/rubyonrails- > talk/CAL%3D0gLsaS8DzC2o0%3DHkKt_%2BMvvzqutm0mSHUj5jf7CoKvnqJEg% > 40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. > > > ---- > > Jason Fleetwood-Boldt > [email protected] <javascript:> > http://www.jasonfleetwoodboldt.com/writing > > If you'd like to reply by encrypted email you can find my public key on > jasonfleetwoodboldt.com (more about setting GPG: https://gpgtools.org) > > -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/b62e5fa7-9ac0-4d6f-ba7b-f9c5580172f3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
