Hi Alistair, The example is not working for me. When I run it, a chrome session is open but nothing happens there, except that my image gets frozen until I close chrome and then I get this message: "ConnectionTimedOut: Cannot connect to 127.0.0.1:9222". What is the expected behavior? PharoChrome expects the user to have a Google account or be logged in by default to work (that would be a shame for those of us that don't a Google account and still value our privacy).
Thanks, Offray On 14/11/17 11:26, Alistair Grant wrote: > I've committed some fixes to the development branch: > > 1. MacOS hopefully works now (I don't have access to the platform, so > can't test it). > 2. The development version of Beacon is loaded (which is required for > the InMemoryLogger). > 3. The README is a tiny bit better. > 4. Added #extractTables. > > As an example of how historical stock market data can be extracted, > the following retrieves data for the Australian S&P200 index from > yahoo: > > > | rootNode tables historicalData dataFrame | > > rootNode := GoogleChrome get: > 'https://finance.yahoo.com/quote/%5EAXJO/history?p=%5EAXJO'. > tables := rootNode extractTables. > historicalData := (tables sorted: #size ascending) last. > dataFrame := DataFrame fromRows: (historicalData select: [ :each | > each size = 7 ]). > dataFrame asStringTable. > > " > | 1 2 3 4 5 6 > 7 > -----+----------------------------------------------------------------------------- > 1 | Date Open High Low Close* Adj > Close** Volume > 2 | Nov 14, 2017 6,021.80 6,021.80 5,957.10 5,966.00 5,966.00 > - > 3 | Nov 13, 2017 6,029.40 6,029.40 6,010.70 6,021.80 6,021.80 > - > 4 | Nov 10, 2017 6,049.40 6,049.40 6,020.70 6,029.40 6,029.40 > - > etc. > " > > > To load the development version on MacOS or Linux in a 32 bit image: > > "Assuming you don't have OSProcess loaded:" > Metacello new > configuration: 'OSSubprocess'; > repository: 'github://marianopeck/OSSubprocess:master/repository'; > version: #stable; > load. > > Metacello new > baseline: 'Chrome'; > repository: 'github://akgrant43/Pharo-Chrome:development/repository'; > load. > > > Cheers, > Alistair > > > On 12 November 2017 at 20:09, Alistair Grant <akgrant0...@gmail.com> wrote: >> Hi Sean, >> >> Thanks for your feedback! (responses below) >> >> >> On 12 November 2017 at 18:11, Sean P. DeNigris <s...@clipperadams.com> wrote: >>> Alistair Grant wrote >>>> https://github.com/akgrant43/Pharo-Chrome >>> Wow, that was a wild ride! >> Sorry about that. >> >> >>> Lessons learned along the way: >>> 1. On a Mac, to use the snazzy `chrome` terminal command referenced all over >>> the place in the docs, you must first `alias chrome="/Applications/Google\ >>> Chrome.app/Contents/MacOS/Google\ Chrome"` >> I'm an Ubuntu Linux user, however if you look at OSXChromePlatform >> class>>defaultExecutableLocation you can see that is where it should >> be looking for the exe, so the alias shouldn't really be necessary. >> Torsten wrote this, so maybe has more insight. >> >> >>> 2. Chrome must be started with certain flags: `chrome >>> --remote-debugging-port=9222 --disable-gpu` (not sure if the last flag is >>> needed, but `#get:` seemed to hang before using; reference >>> https://developers.google.com/web/updates/2017/04/headless-chrome) >> I've been using this without headless mode. I'll add a headless flag >> that also disables the gpu. >> >> >> >>> 3. Beacon has renamed InMemoryLogger to MemoryLogger >>> 4. I guess Beacon has renamed `#log` to `#emit` >> Sorry about that. I didn't realise that the Pharo-Chrome baseline is >> loading Beacon stable while my install script upgrades it to >> #development. #development is more recent, so I'll update the >> baseline. >> >> >> >>> 5. I had to comment out `chromeProcess sigterm.` because `chromeProcess` was >>> nil and also #sigterm seemed not to be defined anywhere in the image. I'm >>> not sure what the issue is there. >> chromeProcess is set in GoogleChrome>>openURL:. Can you give me a >> small example that demonstrates the problem? >> >> #sigterm is implemented by OSSUnixSubprocess, which is what I >> ultimately use to launch the Chrome process on Ubuntu. >> >> But... this will be broken on Mac at the moment because the current >> method of launching chrome doesn't keep track of the process, so >> doesn't support #sigterm. Do you know if OSSUnixSubprocess works on >> Mac? If it does, I can update the code (but not test it :-(). >> >> >>> Pull request issued for #3 & #4. >> Once I update the baseline this shouldn't be required. >> >> >>> Also, I'm not sure what platforms you >>> support, but you may want to tag the example methods with <gtExample> or >>> similar so that they are runnable from the browser and open an inspector if >>> there is an interesting return value. >> Good idea, I'll do this. >> >> I'm also making a few other changes: >> >> 1. Add an #extractTables method that searches through the page and >> returns an array of rows for each table it finds in the page >> (something that can easily be loaded in to DataFrame using #fromRows:, >> but I don't want to make Pharo-Chrome dependent on DataFrame at the >> moment). Most of the time I use Pharo-Chrome it is extracting data >> from tables. >> >> 2. I don't know of any reliable way to tell when a page has loaded >> since there can always be javascript that periodically updates the >> page. At the moment it waits until the page hasn't changed for a >> configurable amount of time. I'm planning to add a check for specific >> content to determine if the page is considered loaded. >> >> 3. Add some documentation to the readme :-) >> >> >> >>> ----- >>> Cheers, >>> Sean >> I'll let you know when I have a new version available (hopefully in >> the next few days). >> >> >> Thanks again, >> Alistair >
