Re: [CODE4LIB] Free covers from Google
Seb, The last time I built an application using the Google Java API the documentation stated a limit of 1,000 queries per day, which was in 2005. It probably hasn't changed, although I'm not sure. Perhaps a way around that is to cache queries into another file and then first query the cache before sending a query to Google. I guess that the idea is not to have cover images stored locally because of the size of the files, however. Michael - Michael Sutherland Web Services Librarian Montana State University Libraries P.O. Box 173320 Bozeman, MT, USA 59717-3320 Ph: (406) 994-6429 [EMAIL PROTECTED] -Original Message- From: Code for Libraries [mailto:[EMAIL PROTECTED] On Behalf Of Sebastian Hammer Sent: Sunday, March 16, 2008 10:32 AM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Free covers from Google Is there any word on a limit to the number of hits per day on the Google API? I missed it in the docs if it's there, only saw an ominous warning that you might see the service temporarily disabled if you generated an 'unusually high number of hits' during development. --Seb Joe Atzberger wrote: Impressive! As luck would have it, I'm working on the question of book images in Koha this week... --joe atzberger On Sat, Mar 15, 2008 at 3:14 AM, Godmar Back [EMAIL PROTECTED] wrote: Hi Tim, I think this proposal suffers from the same shortcoming as LibraryThing's widgets, which is that only one per page is allowed. Aj better way may be to use spans and classes and keep the JavaScript in a library. I've attached the resulting HTML below; see http://libx.org/gbs/ for a demo. - Godmar --- index.html: !DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01 Transitional//EN html head script src=http://libx.org/gbs/gbsclasses.js; type=text/javascript /script titleSimple Demo for Google Book Classes/title /head body span title=ISBN:0743226720 class=gbs-thumbnail/span span title=ISBN:0061234001 class=gbs-thumbnail/span span title=ISBN:1931798230 class=gbs-thumbnail/span span title=ISBN:0596000278 class=gbs-thumbnail/span span title=0439554934 class=gbs-thumbnail/span span title=OCLC:60348769 class=gbs-thumbnail/span span title=LCCN:2004022563 class=gbs-thumbnail/span /body /html On Sat, Mar 15, 2008 at 2:04 AM, Tim Spalding [EMAIL PROTECTED] wrote: (Apologies for cross-posting) I just posted a simple way to get free book covers into your OPAC. It uses the new Google Book Search API. http://www.librarything.com/thingology/2008/03/free-covers-for-your-libr ary-from.php I think Google has as much cover coverage as anyone. The API is free. Most libraries pay. I'm thinking this is a big deal? We'll probably fancy it up a bit as an add-on to our LibraryThing for Libraries service, but the core idea can be implemented by anyone. I look forward to refinements. Tim -- Check out my library at http://www.librarything.com/profile/timspalding -- Sebastian Hammer, Index Data [EMAIL PROTECTED] www.indexdata.com Ph: (603) 209-6853 Fax: (866) 383-4485
Re: [CODE4LIB] Free covers from Google
It gets its metadata in different ways. the strange thing about this example however, is that in the metadata of this specific book at Google the correct ISBN is shown !! (well I presume ii ts the correct isbn (ISBN 0851993575) However, it can not be found with a query on this ISBN, but it is found on an incorrect isbn:0851992544 Peter -Original Message- From: Code for Libraries [mailto:[EMAIL PROTECTED] On Behalf Of Karen Coyle Sent: dinsdag 18 maart 2008 1:16 To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Free covers from Google I've seen this frequently in publisher data -- especially from small publishers. Their metadata goes out with the same ISBN for every book they publish. I don't know if this gets corrected later (I assume it does to make the ordering process work). My guess is that they send out the first metadata before they've gotten an ISBN assigned to the book, so they just re-use one because the ISBN is required. In the few examples I saw, the Amazon records had the correct ISBN, but the Library of Congress records had the wrong ISBN that was originally (presumably) transmitted. The publishers have an incentive to keep Amazon up to date and correct (and perhaps also the bookstore databases and Books in Print). The real question is: where is Google getting its metadata, and what is it doing with metadata from different sources for the same item? kc Boheemen, Peter van wrote: Hmm, just found an example where Google clearly does something wrong: Look at: http://library.wur.nl/WebQuery/catalog/lang/948125 It shows a cover and links to the wrong book !!! The look up is based on isbn 0851992544, but this book has got isbn 0851993575 and this is also clearly stated in the Google book info !! If you just go to Google book search and type isbn:0851992544 in the search box. You will find 4 titles !! of which the first one is the one I get from the service I use. The second one is the correct one. If you type isbn:0851993575 you even get 6 hits and they all seem to share this ISBN !!! I don't know if it does a lot of these mistakes. I just ran into them and I only request covers if Amazon can not provide them, so occasionally, so I suspect this must happen pretty often or I am just lucky this time (as opposed to any lottery I have ever been in.) Peter Drs. P.J.C. van Boheemen Hoofd Applicatieontwikkeling en beheer - Bibliotheek Wageningen UR Head of Application Development and Management - Wageningen University and Research Library tel. +31 317 48 25 17 http://library.wur.nl http://library.wur.nl/ P Please consider the environment before printing this e-mail -- --- Karen Coyle / Digital Library Consultant [EMAIL PROTECTED] http://www.kcoyle.net ph.: 510-540-7596 skype: kcoylenet fx.: 510-848-3913 mo.: 510-435-8234
Re: [CODE4LIB] Free covers from Google
So the other question is whether metadata gets updated at Google. I have had the experience with non-librarians working on library (and other) bibliographic data that they don't realize that the records get updated at the source and need to be updated in the receiver's database. When Google gets data from libraries or from Amazon, does it process updates from those sources on the records it already has? How is metadata (or is metdata) ever corrected? kc Boheemen, Peter van wrote: It gets its metadata in different ways. the strange thing about this example however, is that in the metadata of this specific book at Google the correct ISBN is shown !! (well I presume ii ts the correct isbn (ISBN 0851993575) However, it can not be found with a query on this ISBN, but it is found on an incorrect isbn:0851992544 Peter -Original Message- From: Code for Libraries [mailto:[EMAIL PROTECTED] On Behalf Of Karen Coyle Sent: dinsdag 18 maart 2008 1:16 To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Free covers from Google I've seen this frequently in publisher data -- especially from small publishers. Their metadata goes out with the same ISBN for every book they publish. I don't know if this gets corrected later (I assume it does to make the ordering process work). My guess is that they send out the first metadata before they've gotten an ISBN assigned to the book, so they just re-use one because the ISBN is required. In the few examples I saw, the Amazon records had the correct ISBN, but the Library of Congress records had the wrong ISBN that was originally (presumably) transmitted. The publishers have an incentive to keep Amazon up to date and correct (and perhaps also the bookstore databases and Books in Print). The real question is: where is Google getting its metadata, and what is it doing with metadata from different sources for the same item? kc Boheemen, Peter van wrote: Hmm, just found an example where Google clearly does something wrong: Look at: http://library.wur.nl/WebQuery/catalog/lang/948125 It shows a cover and links to the wrong book !!! The look up is based on isbn 0851992544, but this book has got isbn 0851993575 and this is also clearly stated in the Google book info !! If you just go to Google book search and type isbn:0851992544 in the search box. You will find 4 titles !! of which the first one is the one I get from the service I use. The second one is the correct one. If you type isbn:0851993575 you even get 6 hits and they all seem to share this ISBN !!! I don't know if it does a lot of these mistakes. I just ran into them and I only request covers if Amazon can not provide them, so occasionally, so I suspect this must happen pretty often or I am just lucky this time (as opposed to any lottery I have ever been in.) Peter Drs. P.J.C. van Boheemen Hoofd Applicatieontwikkeling en beheer - Bibliotheek Wageningen UR Head of Application Development and Management - Wageningen University and Research Library tel. +31 317 48 25 17 http://library.wur.nl http://library.wur.nl/ P Please consider the environment before printing this e-mail -- --- Karen Coyle / Digital Library Consultant [EMAIL PROTECTED] http://www.kcoyle.net ph.: 510-540-7596 skype: kcoylenet fx.: 510-848-3913 mo.: 510-435-8234 -- --- Karen Coyle / Digital Library Consultant [EMAIL PROTECTED] http://www.kcoyle.net ph.: 510-540-7596 skype: kcoylenet fx.: 510-848-3913 mo.: 510-435-8234
Re: [CODE4LIB] Free covers from Google
If you can find a public email address anywhere or comment form, let us know. You can send a response to them here: http://www.google.com/support/librariancenter/bin/request.py The form seems to be for librarians so maybe they'll understand the issue and talk to people who may be able to make a change. Mike Beccaria Systems Librarian Head of Digital Initiatives Paul Smith's College 518.327.6376 [EMAIL PROTECTED]
Re: [CODE4LIB] Free covers from Google
This is a little off-topic, but does anyone know if journal covers are available anywhere, similar to books covers that are provided by Google and Amazon? Thanks --- Bin Zhang, Digital Information Services Librarian Library Systems Information Technology Services California State University, Sacramento 2000 State University Drive, East, Sacramento, CA 95819-6039 +1 916 278-5664 (office); +1 916 278-3891 (fax) [EMAIL PROTECTED] -Original Message- From: Code for Libraries [mailto:[EMAIL PROTECTED] On Behalf Of Michael Beccaria Sent: Tuesday, March 18, 2008 12:57 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Free covers from Google If you can find a public email address anywhere or comment form, let us know. You can send a response to them here: http://www.google.com/support/librariancenter/bin/request.py The form seems to be for librarians so maybe they'll understand the issue and talk to people who may be able to make a change. Mike Beccaria Systems Librarian Head of Digital Initiatives Paul Smith's College 518.327.6376 [EMAIL PROTECTED]
Re: [CODE4LIB] Free covers from Google
As i wrote earlier, I have implemented a link using the Google API in our library catalog. It worked . for a while :) What we notice now is, that Google responds with an error message. It thinks that it has detected spyware or some virus. i see the same effect now when I click on the examples Godmar and Tim created. When I go to Google books directly with my browser now, I get the same message and get the request to enter a non machine readable string and then I can go on. My API calls however, still fail. This has probably got to do with the fact that anybody who is accessing Google from the university campus exposes the same IP adress to Google. This is probably a trigger for Google to respond with this error. Does anybody have any ideas about what to do about this, before I try to get in touch with Google? Peter van Boheemen Wageningen University and Research Library The Netherlands
Re: [CODE4LIB] Free covers from Google
FWIW, realize that this is client-side mashup. Google will see individual requests from individual IP addresses from everybody viewing your page. For each IP address from which it sees requests it'll decide whether to block or not. It'll block if it thinks you're harvesting their data. Wageningen University owns the 137.224/16 network, so I find it doubtful that you're all sharing the same IP address. It's probably just your desktop IP address (or, if you're behind a NAT device, the address used by that device - but that's probably only a small group of computers.) That makes it even more concerning that Google's defenses could be triggered by your development and testing activities. Do complain about it to them. (I doubt they change their logic, but you can try.) I've received the CAPTCHA from Google in the past a few times if I use it as a calculator. Enter more than a dozen or so expressions, and it thinks I'm a computer who needs help from Google to compute simple things such as english-to-metric conversions. I think that's a huge drawback, actually. How does Amazon's image service work? Does it suffer from the same issue? - Godmar On Mon, Mar 17, 2008 at 4:50 AM, Boheemen, Peter van [EMAIL PROTECTED] wrote: As i wrote earlier, I have implemented a link using the Google API in our library catalog. It worked . for a while :) What we notice now is, that Google responds with an error message. It thinks that it has detected spyware or some virus. i see the same effect now when I click on the examples Godmar and Tim created. When I go to Google books directly with my browser now, I get the same message and get the request to enter a non machine readable string and then I can go on. My API calls however, still fail. This has probably got to do with the fact that anybody who is accessing Google from the university campus exposes the same IP adress to Google. This is probably a trigger for Google to respond with this error. Does anybody have any ideas about what to do about this, before I try to get in touch with Google? Peter van Boheemen Wageningen University and Research Library The Netherlands
Re: [CODE4LIB] Free covers from Google
Given this latest information, I'd be rather hesitant to even try using Google's images as our network traffic is all NAT'ed and all student traffic from a campus goes out one ONE NAT address and ALL staff traffic on another (in our case x.x.x.204 and x.x.x.205). We currently use Amazon's images with a link back to them and have no problem with this. Giles W. Riesner Jr. Library Tech Support Library System Manager Community College of Baltimore Co.- Catonsville 800 S. Rolling Road Baltimore, MD 21228 USA Phone: 1-410-455-4245 Email: [EMAIL PROTECTED] -Original Message- From: Code for Libraries [mailto:[EMAIL PROTECTED] On Behalf Of Godmar Back Sent: Monday, March 17, 2008 9:09 AM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] Free covers from Google FWIW, realize that this is client-side mashup. Google will see individual requests from individual IP addresses from everybody viewing your page. For each IP address from which it sees requests it'll decide whether to block or not. It'll block if it thinks you're harvesting their data. Wageningen University owns the 137.224/16 network, so I find it doubtful that you're all sharing the same IP address. It's probably just your desktop IP address (or, if you're behind a NAT device, the address used by that device - but that's probably only a small group of computers.) That makes it even more concerning that Google's defenses could be triggered by your development and testing activities. Do complain about it to them. (I doubt they change their logic, but you can try.) I've received the CAPTCHA from Google in the past a few times if I use it as a calculator. Enter more than a dozen or so expressions, and it thinks I'm a computer who needs help from Google to compute simple things such as english-to-metric conversions. I think that's a huge drawback, actually. How does Amazon's image service work? Does it suffer from the same issue? - Godmar On Mon, Mar 17, 2008 at 4:50 AM, Boheemen, Peter van [EMAIL PROTECTED] wrote: As i wrote earlier, I have implemented a link using the Google API in our library catalog. It worked . for a while :) What we notice now is, that Google responds with an error message. It thinks that it has detected spyware or some virus. i see the same effect now when I click on the examples Godmar and Tim created. When I go to Google books directly with my browser now, I get the same message and get the request to enter a non machine readable string and then I can go on. My API calls however, still fail. This has probably got to do with the fact that anybody who is accessing Google from the university campus exposes the same IP adress to Google. This is probably a trigger for Google to respond with this error. Does anybody have any ideas about what to do about this, before I try to get in touch with Google? Peter van Boheemen Wageningen University and Research Library The Netherlands -- BEGIN-ANTISPAM-VOTING-LINKS -- Teach CanIt if this mail (ID 35790824) is spam: Spam: https://ssl.ccbcmd.edu:7726/canit/b.php?i=35790824m=293b60dcf36bc=s Not spam: https://ssl.ccbcmd.edu:7726/canit/b.php?i=35790824m=293b60dcf36bc=n Forget vote: https://ssl.ccbcmd.edu:7726/canit/b.php?i=35790824m=293b60dcf36bc=f -- END-ANTISPAM-VOTING-LINKS
Re: [CODE4LIB] Free covers from Google
Although I completely agree that server-side queryability is something we should ask from Google, I'd like to follow up on: On Mon, Mar 17, 2008 at 11:06 AM, Jonathan Rochkind [EMAIL PROTECTED] wrote: The architecture of SFX would make it hard to implement Google Books API access as purely client javascript, without losing full integration with SFX on par with other 'services' used by SFX. We will see what happens. Could you elaborate? Do you mean 'hard' or 'impossible'? Meanwhile, I've extended the google book classes (libx.org/gbs ) to provide more flexibility; it now supports these classes: gbs-thumbnail Include an img... embedding the thumbnail image gbs-link-to-preview Wrap span in link to preview at GBS gbs-link-to-info Wrap span in link to info page at GBS gbs-link-to-thumbnail Wrap span in link to thumbnail at GBS gbs-if-noview Keep this span only if GBS reports that book's viewability is 'noview' gbs-if-partial-or-full Keep this span only if GBS reports that book's viewability is at least 'partial' gbs-if-partial Keep this span only if GBS reports that book's viewability is 'partial' gbs-if-full Keep this span only if GBS reports that book's viewability is 'full' gbs-remove-on-failure Remove this span if GBS doesn't return bookInfo for this item - Godmar
Re: [CODE4LIB] Free covers from Google
On Mon, Mar 17, 2008 at 11:13 AM, Tim Spalding [EMAIL PROTECTED] wrote: limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. Aren't we still talking about covers? I see *no* reason to go server-side on that. Browser-side gets you what you want—covers from Google—without the risk they'll shut you down over overuse. But Peter's experience says otherwise, no? His computer was shut down during development - I don't see how Google would tell his use from the use of someone doing research using a library catalog. Especially if NAT is used with a substantial number of users as in Giles's use case. - Godmar
Re: [CODE4LIB] Free covers from Google
Original message Date: Mon, 17 Mar 2008 11:13:58 -0400 From: Tim Spalding [EMAIL PROTECTED] Subject: Re: [CODE4LIB] Free covers from Google To: CODE4LIB@LISTSERV.ND.EDU limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. Aren't we still talking about covers? I see *no* reason to go server-side on that. Browser-side gets you what you want—covers from Google—without the risk they'll shut you down over overuse. I could see one reason to do cover images server-side. Say a library has a new titles list. These books (and hence the images) are going to be the same for quite a while. It might make sense to try to optimize it by downloading said images once and caching it on a local server. Otherwise every time someone hits the new titles list they'll have to wait for Google to respond. Not sure how much of an advantage it really would be to host it server side. Jon Gorman
Re: [CODE4LIB] Free covers from Google
This is what I'm worried about too. The API is _intended_ to be used as a client-side javascript. _Technically_ it's of course possible to use it on the server side too. But I am worried that this will run up against un-advertised Google rate limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. When I emailed Google to ask about this, all I got back was a statement that this is only supported for client side Javascript access. I doubt that users using software with this kind of in-browser Javascript access enabled will run up against any problems with rate limiting. But I think if we try to do it server side, we very well might. Of course, there are many kinds of functions that are difficult or impossible to do solely client side, especially integrating with existing software. So this is a concern. Interestingly, the Ex Libris SFX software demoing integration with Google Books---as far as I can tell makes the API calls server-side! Someone from Ex Libris confirmed to me that they have no special deal or communications with Google. I think they didn't neccesarily realize they might run into Google rate limiting (or even terms of service violation?) issues. It will be interesting to see if they do or not. The architecture of SFX would make it hard to implement Google Books API access as purely client javascript, without losing full integration with SFX on par with other 'services' used by SFX. We will see what happens. Google's _stated_ (to me, in email; I got one reply, but couldn't get them to reply to my followup) reason for only allowing client-side access is that availability may depend on location, and they need to know the end user's location (via IP address) to get accurate availability for that particular end user. This could of course be easily solved if the API URL format allowed the calling client to pass on the end-user's IP address in the URL (client_ip=x.x.x.x or what have you). But I don't believe the Google Books API currently supports this. I suspect that Google may have other business model reasons. Not sure. Jonathan Sebastian Hammer wrote: Is there any word on a limit to the number of hits per day on the Google API? I missed it in the docs if it's there, only saw an ominous warning that you might see the service temporarily disabled if you generated an 'unusually high number of hits' during development. --Seb Joe Atzberger wrote: Impressive! As luck would have it, I'm working on the question of book images in Koha this week... --joe atzberger On Sat, Mar 15, 2008 at 3:14 AM, Godmar Back [EMAIL PROTECTED] wrote: Hi Tim, I think this proposal suffers from the same shortcoming as LibraryThing's widgets, which is that only one per page is allowed. Aj better way may be to use spans and classes and keep the JavaScript in a library. I've attached the resulting HTML below; see http://libx.org/gbs/ for a demo. - Godmar --- index.html: !DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01 Transitional//EN html head script src=http://libx.org/gbs/gbsclasses.js; type=text/javascript /script titleSimple Demo for Google Book Classes/title /head body span title=ISBN:0743226720 class=gbs-thumbnail/span span title=ISBN:0061234001 class=gbs-thumbnail/span span title=ISBN:1931798230 class=gbs-thumbnail/span span title=ISBN:0596000278 class=gbs-thumbnail/span span title=0439554934 class=gbs-thumbnail/span span title=OCLC:60348769 class=gbs-thumbnail/span span title=LCCN:2004022563 class=gbs-thumbnail/span /body /html On Sat, Mar 15, 2008 at 2:04 AM, Tim Spalding [EMAIL PROTECTED] wrote: (Apologies for cross-posting) I just posted a simple way to get free book covers into your OPAC. It uses the new Google Book Search API. http://www.librarything.com/thingology/2008/03/free-covers-for-your-library-from.php I think Google has as much cover coverage as anyone. The API is free. Most libraries pay. I'm thinking this is a big deal? We'll probably fancy it up a bit as an add-on to our LibraryThing for Libraries service, but the core idea can be implemented by anyone. I look forward to refinements. Tim -- Check out my library at http://www.librarything.com/profile/timspalding -- Sebastian Hammer, Index Data [EMAIL PROTECTED] www.indexdata.com Ph: (603) 209-6853 Fax: (866) 383-4485 -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu
Re: [CODE4LIB] Free covers from Google
Well, the SFX architecture has a feature called display logic that let's you on the server side determine how the menu will display based on what services are available. This is more obviously relevant to digitized text availability from Google Books than just cover images. You might want to suppress ILL links if there is digitized text (in fact, you probably wouldn't in that particular case, but that gives you the idea of what things you might want to do. At least my library wouldn't, maybe others with especially small ILL budgets might). Or just give a pre-ILL warning message (are you sure the Google text isn't sufficient?), that might be more realistic. Anyway, you obviously couldn't do this using the existing SFX display logic feature if the Google Books info is only client side. Now, impossible? In the world of software development, few things are actually impossible. You could try to duplicate that feature using only client-side Javascript hiding and showing various DIVs. The SFX HTML currently isn't that clean, it woudl be hard. But you have the capability to customize the SFX HTML however you want to. (And your customizations will likely break with a future SFX release). So nothings impossible, but I wouldn't want to go down that road. Jonathan Godmar Back wrote: Although I completely agree that server-side queryability is something we should ask from Google, I'd like to follow up on: On Mon, Mar 17, 2008 at 11:06 AM, Jonathan Rochkind [EMAIL PROTECTED] wrote: The architecture of SFX would make it hard to implement Google Books API access as purely client javascript, without losing full integration with SFX on par with other 'services' used by SFX. We will see what happens. Could you elaborate? Do you mean 'hard' or 'impossible'? Meanwhile, I've extended the google book classes (libx.org/gbs ) to provide more flexibility; it now supports these classes: gbs-thumbnail Include an img... embedding the thumbnail image gbs-link-to-preview Wrap span in link to preview at GBS gbs-link-to-info Wrap span in link to info page at GBS gbs-link-to-thumbnail Wrap span in link to thumbnail at GBS gbs-if-noview Keep this span only if GBS reports that book's viewability is 'noview' gbs-if-partial-or-full Keep this span only if GBS reports that book's viewability is at least 'partial' gbs-if-partial Keep this span only if GBS reports that book's viewability is 'partial' gbs-if-full Keep this span only if GBS reports that book's viewability is 'full' gbs-remove-on-failure Remove this span if GBS doesn't return bookInfo for this item - Godmar -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu
Re: [CODE4LIB] Free covers from Google
You can, of course, mix the two approaches—get once browser-side, tell your servers what it said and store that for a while. We do something like that with Amazon covers. We don't store the covers, but we store *whether* Amazon has the cover. That way, it can know whether to try Amazon or not, and whether to fall back on another cover. T On 3/17/08, Jonathan Rochkind [EMAIL PROTECTED] wrote: I was thinking of both covers and 'digitized text availability'. But the reason I want to in fact do both server side is because of the architecture I am trying to create here. We have a variety of systems that should use both these services. We, like many people, are trying to move to a more 'service oriented' type architecture, where I have one component of software that does all of these lookups, and then provides the resulting data in a uniform format via a local web service for all my other user-facing interfaces to use. Of course, doing that requires a server-side lookup. But that overall architecture is much preferable (from a code efficiency and maintenance perspective) than having to include customized AJAX for a variety of services (Google Scholar being just one; Scopus is another content-provider that frustratingly provides an Javascript-only API) in a variety of ever-changing user-facing interfaces. DRY and all. Jonathan Tim Spalding wrote: limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. Aren't we still talking about covers? I see *no* reason to go server-side on that. Browser-side gets you what you want—covers from Google—without the risk they'll shut you down over overuse. T -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu -- Check out my library at http://www.librarything.com/profile/timspalding
Re: [CODE4LIB] Free covers from Google
I was thinking of both covers and 'digitized text availability'. But the reason I want to in fact do both server side is because of the architecture I am trying to create here. We have a variety of systems that should use both these services. We, like many people, are trying to move to a more 'service oriented' type architecture, where I have one component of software that does all of these lookups, and then provides the resulting data in a uniform format via a local web service for all my other user-facing interfaces to use. Of course, doing that requires a server-side lookup. But that overall architecture is much preferable (from a code efficiency and maintenance perspective) than having to include customized AJAX for a variety of services (Google Scholar being just one; Scopus is another content-provider that frustratingly provides an Javascript-only API) in a variety of ever-changing user-facing interfaces. DRY and all. Jonathan Tim Spalding wrote: limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. Aren't we still talking about covers? I see *no* reason to go server-side on that. Browser-side gets you what you want—covers from Google—without the risk they'll shut you down over overuse. T -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu
Re: [CODE4LIB] Free covers from Google
My general philosophy is still always to put as _little_ Javascript as possible. Thus my way-too-clever idea to have some javascript which actually sends the Google (or similar) API response back to my server via AJAX for _real_ processing. :) But if you DO want or need to do javascript-heavy stuff, I _highly_ encourage you to take a look at some of the various Javascript client libraries that are out there, like Prototype. Such libraries can provides support for easy 'inheritance' in Javascript (implemented through behind the scenes fakery), as well as ways to see what versions/functions of javascript are supported in the browser. But if you are trying to make things available without client javascript at all... if a given API like Google _requires_ it, well, then there's no way to have Google results used by the interface in a non-javascript client. That's obviously just a syllogism. [It's interesting to know though that, contrary to popular belief, recent versions of JAWS _do_ support javascript. But only some kinds of javascript done in certain ways. Doing javascript that will work for JAWS is yet another layer of complexity, yet another headache. Added headaches and layers of complexity is why I try to minimize my javascript altogether. And JAWS is just one kind of 'accessibilty'. And the rules you have to follow whether you like it or not for 'accessibility' may or may not actually be rationally related to actual accessible use cases, like it or not.] Joe Hourcle wrote: On Mon, 17 Mar 2008, Jonathan Rochkind wrote: Of course, all these things _can_ be done with only client-side javascript API. To my perspective, it is quite a bit more complex to do it that way. And complexity is the enemy of maintainability, especially in my limited staff resources library environment. But perhaps my perspective is not sufficiently 2.0 (or 3.0); I've been a software developer for a long time, but doing client side javascript stuff for less. Maybe I'm wrong. I dunno. To me, it looks quite a bit more complex to try and provide these services with the same level of control and customization in a don't repeat yourself modular way accross all my various display systems that need this functionality, using client side javascript services. Other developers, do you think I'm wrong? I agree, but mostly because I'm concerned with the differing ECMAScript implementations out there (JavaScript, JScript, different versions, etc.), and the lack of the ability to test for the existance of a function ... you can test for what version the client thinks it can run, but that doesn't given you fine control. (and as try/catch wasn't included 'til 1.3, I had to go through elaborate hoops to try to verify that the code would run cleanly) ... then there's the issue that I need to support Section 508 requirements, and our normal procedure is to make sure that whatever features we have don't require client-side support -- they can be _enhanced_ by client-side scripting, but even without scripting turned on, the applications are at least usable. I'm also not a fan of how 'NOSCRIPT' is handled in web browsers. Say for instance, that I actually know which spec I'm coding to, and I have some functionality that requires 1.0, and some that requires 1.3. I'd want the NOSCRIPT to be directly tied to the SCRIPT tag, so I can give appropriate messages when they have scripting completely turned off and both don't load, vs. when they're running a 1.2 spec, and can't load the 1.3 code. http://www.w3.org/TR/html401/interact/scripts.html#h-18.3.1 Now, in the HTML spec, they describe that the original intent was for people to embed other languages ... but they don't ever say how you'd handle the case where the browser loaded javascript, but not tcl. Of course, they've also deprecated the 'language' attribute, so it's more difficult to handle the code based on the version of the language that's supported. ... It may be that I was stung by trying to move over to client-side scripting too early (10 years ago?), and I've used it in small amounts through the years, but I'm currently working on my first 'real' JavaScript application, where it's going to require significant amounts of processing to render output ... but I have no idea how much memory is safe to use, or if my text machines are representative of our user base ... and I've already found that the render time is more than doubled in FireFox compared to Safari or IE. ... okay, this is slowly getting into a rant -- but yes, I agree -- I have less control, more worries about security implications, issues with debugging, etc. I think the only reason I've gone along with the change is because it's giving me a chance to clean up some of the UI code and I can get a better separation between it and the back-end logic. ... oh -- and the inability to inherit code from within JavaScript is a major step backwards -- I have to have whatever HTML file (or whatever generates the HTML)
Re: [CODE4LIB] Free covers from Google
Godmar, It did not shut down during development, yesterday, when I developed it from home. It broke down today, when people started to use it. All university desktop computer have got dynamic 10.*.*.* adresses. The gateway does NAT so they are exposed to google with about three possible IP adresses. Or, if they use SFX, the IP adress of the Open URL resolver, or in the case of citrix, the IP adress of that machine. Anyway, all these approaches suffer from the same problem with Google's policy. Besides all that, I prefer a clean XML interface like Amazon provides above the JSON approach of Google. And I do prefer server side handling, since I got control over what the user is presented and I do not have problems with javascript support of the specific browser that the user is equiped with. Peter Drs. P.J.C. van Boheemen Hoofd Applicatieontwikkeling en beheer - Bibliotheek Wageningen UR Head of Application Development and Management - Wageningen University and Research Library tel. +31 317 48 25 17 http://library.wur.nl http://library.wur.nl/ P Please consider the environment before printing this e-mail Van: Code for Libraries namens Godmar Back Verzonden: ma 17-3-2008 16:21 Aan: CODE4LIB@LISTSERV.ND.EDU Onderwerp: Re: [CODE4LIB] Free covers from Google On Mon, Mar 17, 2008 at 11:13 AM, Tim Spalding [EMAIL PROTECTED] wrote: limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. Aren't we still talking about covers? I see *no* reason to go server-side on that. Browser-side gets you what you want-covers from Google-without the risk they'll shut you down over overuse. But Peter's experience says otherwise, no? His computer was shut down during development - I don't see how Google would tell his use from the use of someone doing research using a library catalog. Especially if NAT is used with a substantial number of users as in Giles's use case. - Godmar
Re: [CODE4LIB] Free covers from Google
Interesting that these problems arise even when using the API as Google intends on the client side. I would encourage people to tell Google about this. If only we knew a way to tell Google about it. If you can find a public email address anywhere or comment form, let us know. And if you are having problems in production even when using the API client side as intended, let Google know. Maybe there will be a miracle and they'll care. Chances are higher here, because it seems like they created this feature in large part for libraries. If libraries are finding it does not work in production... Jonathan Boheemen, Peter van wrote: Godmar, It did not shut down during development, yesterday, when I developed it from home. It broke down today, when people started to use it. All university desktop computer have got dynamic 10.*.*.* adresses. The gateway does NAT so they are exposed to google with about three possible IP adresses. Or, if they use SFX, the IP adress of the Open URL resolver, or in the case of citrix, the IP adress of that machine. Anyway, all these approaches suffer from the same problem with Google's policy. Besides all that, I prefer a clean XML interface like Amazon provides above the JSON approach of Google. And I do prefer server side handling, since I got control over what the user is presented and I do not have problems with javascript support of the specific browser that the user is equiped with. Peter Drs. P.J.C. van Boheemen Hoofd Applicatieontwikkeling en beheer - Bibliotheek Wageningen UR Head of Application Development and Management - Wageningen University and Research Library tel. +31 317 48 25 17 http://library.wur.nl http://library.wur.nl/ P Please consider the environment before printing this e-mail Van: Code for Libraries namens Godmar Back Verzonden: ma 17-3-2008 16:21 Aan: CODE4LIB@LISTSERV.ND.EDU Onderwerp: Re: [CODE4LIB] Free covers from Google On Mon, Mar 17, 2008 at 11:13 AM, Tim Spalding [EMAIL PROTECTED] wrote: limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. Aren't we still talking about covers? I see *no* reason to go server-side on that. Browser-side gets you what you want-covers from Google-without the risk they'll shut you down over overuse. But Peter's experience says otherwise, no? His computer was shut down during development - I don't see how Google would tell his use from the use of someone doing research using a library catalog. Especially if NAT is used with a substantial number of users as in Giles's use case. - Godmar -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu
Re: [CODE4LIB] Free covers from Google
I found a response form on the google books site. It is meant to be for people who want their books published I am afrais, but this was not all to clear. Anyway, it accepted my Google account and I submitted this problem Peter Drs. P.J.C. van Boheemen Hoofd Applicatieontwikkeling en beheer - Bibliotheek Wageningen UR Head of Application Development and Management - Wageningen University and Research Library tel. +31 317 48 25 17 http://library.wur.nl http://library.wur.nl/ P Please consider the environment before printing this e-mail Van: Code for Libraries namens Jonathan Rochkind Verzonden: ma 17-3-2008 21:05 Aan: CODE4LIB@LISTSERV.ND.EDU Onderwerp: Re: [CODE4LIB] Free covers from Google Interesting that these problems arise even when using the API as Google intends on the client side. I would encourage people to tell Google about this. If only we knew a way to tell Google about it. If you can find a public email address anywhere or comment form, let us know. And if you are having problems in production even when using the API client side as intended, let Google know. Maybe there will be a miracle and they'll care. Chances are higher here, because it seems like they created this feature in large part for libraries. If libraries are finding it does not work in production... Jonathan Boheemen, Peter van wrote: Godmar, It did not shut down during development, yesterday, when I developed it from home. It broke down today, when people started to use it. All university desktop computer have got dynamic 10.*.*.* adresses. The gateway does NAT so they are exposed to google with about three possible IP adresses. Or, if they use SFX, the IP adress of the Open URL resolver, or in the case of citrix, the IP adress of that machine. Anyway, all these approaches suffer from the same problem with Google's policy. Besides all that, I prefer a clean XML interface like Amazon provides above the JSON approach of Google. And I do prefer server side handling, since I got control over what the user is presented and I do not have problems with javascript support of the specific browser that the user is equiped with. Peter Drs. P.J.C. van Boheemen Hoofd Applicatieontwikkeling en beheer - Bibliotheek Wageningen UR Head of Application Development and Management - Wageningen University and Research Library tel. +31 317 48 25 17 http://library.wur.nl http://library.wur.nl/ http://library.wur.nl/ P Please consider the environment before printing this e-mail Van: Code for Libraries namens Godmar Back Verzonden: ma 17-3-2008 16:21 Aan: CODE4LIB@LISTSERV.ND.EDU Onderwerp: Re: [CODE4LIB] Free covers from Google On Mon, Mar 17, 2008 at 11:13 AM, Tim Spalding [EMAIL PROTECTED] wrote: limits. I don't think it's a strict hits-per-day, I think it's heuristic software meant to stop exactly what we'd be trying to do, server-side machine-based access. Aren't we still talking about covers? I see *no* reason to go server-side on that. Browser-side gets you what you want-covers from Google-without the risk they'll shut you down over overuse. But Peter's experience says otherwise, no? His computer was shut down during development - I don't see how Google would tell his use from the use of someone doing research using a library catalog. Especially if NAT is used with a substantial number of users as in Giles's use case. - Godmar -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu
Re: [CODE4LIB] Free covers from Google
Hmm, just found an example where Google clearly does something wrong: Look at: http://library.wur.nl/WebQuery/catalog/lang/948125 It shows a cover and links to the wrong book !!! The look up is based on isbn 0851992544, but this book has got isbn 0851993575 and this is also clearly stated in the Google book info !! If you just go to Google book search and type isbn:0851992544 in the search box. You will find 4 titles !! of which the first one is the one I get from the service I use. The second one is the correct one. If you type isbn:0851993575 you even get 6 hits and they all seem to share this ISBN !!! I don't know if it does a lot of these mistakes. I just ran into them and I only request covers if Amazon can not provide them, so occasionally, so I suspect this must happen pretty often or I am just lucky this time (as opposed to any lottery I have ever been in.) Peter Drs. P.J.C. van Boheemen Hoofd Applicatieontwikkeling en beheer - Bibliotheek Wageningen UR Head of Application Development and Management - Wageningen University and Research Library tel. +31 317 48 25 17 http://library.wur.nl http://library.wur.nl/ P Please consider the environment before printing this e-mail
Re: [CODE4LIB] Free covers from Google
At 02:55 PM 3/17/2008, you wrote: My general philosophy is still always to put as _little_ Javascript as possible. Thus my way-too-clever idea to have some javascript which actually sends the Google (or similar) API response back to my server via AJAX for _real_ processing. :) But if you DO want or need to do javascript-heavy stuff, I _highly_ encourage you to take a look at some of the various Javascript client libraries that are out there, like Prototype. I was playing with another one today: jQuery. It's very slick. I've been developing a web application that uses Spring Web MVC and a services oriented architecture and it plugged in rather nicely. John Fereira [EMAIL PROTECTED] Ithaca, NY
Re: [CODE4LIB] Free covers from Google
Impressive! As luck would have it, I'm working on the question of book images in Koha this week... --joe atzberger On Sat, Mar 15, 2008 at 3:14 AM, Godmar Back [EMAIL PROTECTED] wrote: Hi Tim, I think this proposal suffers from the same shortcoming as LibraryThing's widgets, which is that only one per page is allowed. Aj better way may be to use spans and classes and keep the JavaScript in a library. I've attached the resulting HTML below; see http://libx.org/gbs/ for a demo. - Godmar --- index.html: !DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01 Transitional//EN html head script src=http://libx.org/gbs/gbsclasses.js; type=text/javascript /script titleSimple Demo for Google Book Classes/title /head body span title=ISBN:0743226720 class=gbs-thumbnail/span span title=ISBN:0061234001 class=gbs-thumbnail/span span title=ISBN:1931798230 class=gbs-thumbnail/span span title=ISBN:0596000278 class=gbs-thumbnail/span span title=0439554934 class=gbs-thumbnail/span span title=OCLC:60348769 class=gbs-thumbnail/span span title=LCCN:2004022563 class=gbs-thumbnail/span /body /html On Sat, Mar 15, 2008 at 2:04 AM, Tim Spalding [EMAIL PROTECTED] wrote: (Apologies for cross-posting) I just posted a simple way to get free book covers into your OPAC. It uses the new Google Book Search API. http://www.librarything.com/thingology/2008/03/free-covers-for-your-library-from.php I think Google has as much cover coverage as anyone. The API is free. Most libraries pay. I'm thinking this is a big deal? We'll probably fancy it up a bit as an add-on to our LibraryThing for Libraries service, but the core idea can be implemented by anyone. I look forward to refinements. Tim -- Check out my library at http://www.librarything.com/profile/timspalding
