(I moved this thread over to bibdesk-develop because it's becoming a development-oriented thread now, not a user-oriented thread.)
Here's one possible framework for creating importers. Importers are in a new bundle type, say .bdimporter, and new importers can be added by the user in ~/Library/Application Support/BibDesk/ Importers, say. In the Info.plist in that bundle, we include: - a field which indicates the filename of the main script within that bundle - a field which indicates the URLs that this importer knows how to parse (probably a regular expression as Zotero uses) - a field which indicates the type of script, early possibilities being "AppleScript", "shell script", and "Javascript". Rather than aiming at making complicated bridges to various languages, Python, Perl, etc., just let them be shell scripts that are passed a URL and return BibTeX entry/entries. All an importer needs to do is take a URL and return BibTeX entry/ entries. BibDesk will parse the resulting BibTeX and import it; or alternatively it could just load it up in the web group for further action. BibDesk also needs to pull all the regular expressions from the various Info.plist files so that it knows which importer to use for a given URL. About the only non-self-contained thing an importer would have to do is retrieve the URL. An alternative would be that they get passed the downloaded URL as well (or a temporary file on the file system containing the contents of the file), but one can imagine the importer needing to load additional URLs based on the URL that's given. For example, if it's known that www.example.com/article/0343.html is an article and www.example.com/article/0343.bib is a BibTeX file from the publisher that corresponds to the article, then the importer would need to be able to retrieve that second URL. But we could just leave it up to the importer to download any URLs that it needs to. The point is that the interface to an importer need not be complicated. The link that Adam sent about what Alf Eaton was working on related to using Zotero's Javascripts is very interesting, and if an automatic way of using Zotero importers could be developed that would be amazing. I can play around with this over the next little while to see if I can make sense of any of it. Douglas On 2009-Feb-20, at 8:38 AM, Michael McCracken wrote: > On Fri, Feb 20, 2009 at 5:27 AM, Christiaan Hofman > <cmhof...@gmail.com> wrote: >> >> On 20 Feb 2009, at 6:45 AM, Douglas Stebila wrote: >> >>> On 2009-Feb-19, at 1:20 AM, Christiaan Hofman wrote: >>> >>>> Are you aware of the Web Group? >>> >>> >>> The Web Group is nice, but (a) does not fit naturally into many >>> people's workflow, and (b) does not allow for adding importers >>> without >>> altering the BibDesk source code. >>> >>> On 2009-Feb-19, at 9:49 AM, Maxwell, Adam R wrote: >>> >>>> The short answer is no. A few sites are supported in the Web group >>>> that >>>> Christiaan mentioned, but there's no way for users to write plugins >>>> that tie >>>> into that group. >>> >>> It seems to me the natural way to do this right would be to have any >>> easy way of plugins that can be used either in the web group or >>> can be >>> accessed by AppleScript, which then allows for easy scripts to be >>> written for Safari or Firefox or your web browser of choice. >>> >> >> But indeed, BD does not live "inside" your browser. And I don't think >> BD should somehow go inside. The best I could think of would be some >> scripting support to tell BD to import from some URL. >> >> It's not completely clear to me what part of the process you'd want >> BD >> to perform. >> >>>> If you're using Firefox, you might be interested in this: >>>> http://www.mackerron.com/. It uses Zotero's scrapers to push >>>> references to >>>> BibDesk. >>> >>> I was not aware of Citeulike or Zotero. Since a lot of work has >>> been >>> done by people in developing importers for Zotero it would be nice >>> if >>> those importers could be adapted for more generic use outside of the >>> Zotero framework, but, after perusing the code, that seems unlikely. >>> It would be great if there was a standardized importer framework so >>> that importers could be written once and used in lots of programs -- >>> Zotero, BibDesk, etc. -- but that may be beyond the scope of what is >>> reasonable to do at this point. >>> >> >> I don't know too much about it, but the language that these importers >> use is very different from anything BD supports, so it would be very >> difficult / a lot of work to implement it. I never considered it very >> likely. >> >>> If someone more familiar with the BibDesk source code is >>> interested in >>> this and willing to work with me on this, I am willing to put some >>> time into developing this. But I don't want to just go off and >>> write >>> it without knowing that it fits within the vision the core >>> developers >>> have for BibDesk. >>> >>> Douglas >> >> We've had some talk about writing importers in the past. At first >> Mike >> didn't think it was a good idea, because it's more manageable to >> write >> directly in BD code. However I think that was based on the assumption >> of a Obj-C plugin. I think it would be worthwhile to have a plugin >> mechanism for more general (scripting) languages, but only if it is >> well thought through beforehand (otherwise it will be a real PITA to >> work with it in the long run) and well designed. Moreover, I think >> also only if it supports various common languages, I'd say at least >> python. And I think here comes the pain, because communicating with >> any language other than AppleScript is still far from trivial from >> Cocoa. I wouldn't know how to do it. But for sure if someone does and >> wants to com in, I'd be happy to assist form the BD side. > > Integrating Zotero might be getting easier - here is a bridge between > javascript and objc: > http://code.google.com/p/jsxobjc/ > > Also, adding support for python (actually pyobjc) plugins might be > easier than it sounds. > > I understand code talks and I'm just writing emails here, but I > actually think that Applescript is no longer any easier to support > (and may actually be harder) than some more popular languages. > > I'm going to try to spend a little time this weekend putting PyObjC > into BibDesk. We'll see how it goes. > > > -mike > >> Christiaan >> >>> >>>> On 18 Feb 2009, at 6:29 PM, Douglas Stebila wrote: >>>> >>>>> Is there a standard, simple way to write and install methods for >>>>> BibDesk to import citations from Safari/Firefox? >>>>> >>>>> Here's the scenario I'm thinking of. Users are browsing the web >>>>> in >>>>> Safari or Firefox and they come across a paper on a major site >>>>> (arXiv, >>>>> Springer, IEEE, etc.) for which they'd like to import the citation >>>>> into BibDesk. It would be great if they could access an >>>>> AppleScript >>>>> in Safari/Firefox, or switch to BibDesk and click a single button, >>>>> and >>>>> have BibDesk parse the website, pull out the right bibliographic >>>>> data, >>>>> and create a new citation. >>>>> >>>>> Now, obviously having BibDesk parse all the data and figure it out >>>>> automatically is too hard. But one could write parsers for >>>>> individual >>>>> websites that know the idiosyncracies of various websites. >>>>> Indeed, >>>>> this is what a lot of the importer AppleScripts on the BibDesk >>>>> wiki >>>>> do: >>>>> >>>>> http://apps.sourceforge.net/mediawiki/bibdesk/index.php?title=BibDesk_Applescripts >>>>> >>>>> But this results in a lot of duplication of work and work being >>>>> done >>>>> in a non-standard way. I think there should be (and perhaps there >>>>> is >>>>> and I just don't know about it) a standard way of adding importers >>>>> of >>>>> this form into BibDesk at runtime, and moreover that these >>>>> importers >>>>> should be able to be written in scripting languages (AppleScript, >>>>> shell script, etc.) as opposed to having to be written in >>>>> Objective-C >>>>> in the main BibDesk source tree. This would allow users to easily >>>>> add >>>>> importers for their favourite websites without needing to know >>>>> Objective-C. >>>>> >>>>> I've actually written such an extensible framework entirely in >>>>> AppleScript that has importer scripts for websites that I use >>>>> (arXiv, >>>>> IEEE, ACM, Springer), so if you want to see what I mean a >>>>> temporary >>>>> version is available for download at >>>>> http://www.douglas.stebila.ca/files/code/bibdesk/ >>>>> Importers-0.9.2.zip >>>>> I'd be happy to contribute to developing part of this if people >>>>> think >>>>> it should be added to BibDesk, but I'm not a very good Objective-C >>>>> coder. >>>>> >>>>> But I think this is functionality that lots of people will want >>>>> and >>>>> so >>>>> there should be one way to do it, not lots of people writing >>>>> random >>>>> scripts and posting them in random locations. >>>>> >>>>> Regards, >>>>> >>>>> Douglas >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San >>>>> Francisco, CA >>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the >>>>> Enterprise >>>>> -Strategies to boost innovation and cut costs with open source >>>>> participation >>>>> -Receive a $600 discount off the registration fee with the source >>>>> code: SFAD >>>>> http://p.sf.net/sfu/XcvMzF8H >>>>> _______________________________________________ >>>>> Bibdesk-users mailing list >>>>> bibdesk-us...@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/bibdesk-users >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Open Source Business Conference (OSBC), March 24-25, 2009, San >>>> Francisco, CA >>>> -OSBC tackles the biggest issue in open source: Open Sourcing the >>>> Enterprise >>>> -Strategies to boost innovation and cut costs with open source >>>> participation >>>> -Receive a $600 discount off the registration fee with the source >>>> code: SFAD >>>> http://p.sf.net/sfu/XcvMzF8H >>>> _______________________________________________ >>>> Bibdesk-users mailing list >>>> bibdesk-us...@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/bibdesk-users >>> >>> >>> ------------------------------------------------------------------------------ >>> Open Source Business Conference (OSBC), March 24-25, 2009, San >>> Francisco, CA >>> -OSBC tackles the biggest issue in open source: Open Sourcing the >>> Enterprise >>> -Strategies to boost innovation and cut costs with open source >>> participation >>> -Receive a $600 discount off the registration fee with the source >>> code: SFAD >>> http://p.sf.net/sfu/XcvMzF8H >>> _______________________________________________ >>> Bibdesk-users mailing list >>> bibdesk-us...@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/bibdesk-users >> >> >> ------------------------------------------------------------------------------ >> Open Source Business Conference (OSBC), March 24-25, 2009, San >> Francisco, CA >> -OSBC tackles the biggest issue in open source: Open Sourcing the >> Enterprise >> -Strategies to boost innovation and cut costs with open source >> participation >> -Receive a $600 discount off the registration fee with the source >> code: SFAD >> http://p.sf.net/sfu/XcvMzF8H >> _______________________________________________ >> Bibdesk-users mailing list >> bibdesk-us...@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bibdesk-users >> > > > > -- > Michael McCracken > UCSD CSE PhD Candidate > research: http://www.cse.ucsd.edu/~mmccrack/ > misc: http://michael-mccracken.net/wp/ > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San > Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the > Enterprise > -Strategies to boost innovation and cut costs with open source > participation > -Receive a $600 discount off the registration fee with the source > code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Bibdesk-users mailing list > bibdesk-us...@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bibdesk-users ------------------------------------------------------------------------------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H _______________________________________________ Bibdesk-develop mailing list Bibdesk-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bibdesk-develop
