Hi Morten,
I'm attaching a small patch along with two new classes that should go in the
/src/java/net/sf/jabref/imports/ directory. What I basically did was copy
the SPIRESFetcher and SPIRESBibtexFilterReader and to new files,
INPSIREFetcher and INSPIREBibtexFilterReader, and then just modified the
latter to point to inspire. The patch file is just modifications to the
main code to call these classes. Since the original spires fetcher was part
of the main code it seemed to make sense to do the same for inspire rather
than create a plugin.
It seems to work relatively straightforwardly. There are a few hiccups:
- INSPIRE requires a number of entries to return in the query URL (else it
defaults to 25) so I set this to 1000 but I think INSPIRE has some built-in
max which is below this since I never seem to get this many entries.
- If you have an old database with SPIRES entries and you do a new INSPIRE
query for the same records only some of them are recognized as duplicates.
I'm guessing this is because of differences in the INSPIRE and SPIRES bibtex
entries. In particular INPSIRE doesn't keep a slaccitation field and has
different representations of some entries (e.g. titles with latex seem to be
represented differently). I don't see any elegant way to deal with this.
I haven't tested the code very thoroughly but will try to test it further in
day to day use. I think my modifications were pretty contained but I don't
really have a thorough knowledge of how things fit together. I just made
changes by grepping around for the relevant part of the code.
I'll try to eventually implement an XML interface when I have some time.
thanks
On Wed, Oct 5, 2011 at 8:26 AM, Morten Omholt Alver
<[email protected]>wrote:
> On 27 September 2011 23:17, funky dev <[email protected]> wrote:
> > Hi,
> >
> > I've been looking at the jabref code a little with the intention to make
> the
> > following modifications:
> >
> > - Implement Inpsire support (as opposed to spires).
> > - Include more fields from Inspire such as abstract and number of
> citations
> > by switching to an XML format for the data (right now spires returns
> bibtex
> > embedded in some HTML).
> > - Maybe track citations as well.
> >
> > Previously I used Spires.app and would like to try to mimic all its
> > functionality in jabref.
> >
> > So I have a few questions in this direction. First, is someone already
> > working on adding inspire support? Second, does anyone have any resource
> > regarding inspire's support of MARCXML or OAI2 or any other standard?
> The
> > Inspire website seems very light on API documentation. Finally, is there
> a
> > resource to help one get started with the code base?
>
> I'm not aware of anyone else working on Inspire support. If you mean
> getting started with the JabRef code base, there's unfortunately not
> that much to find, but I'd be happy to help out if you have questions.
>
> > I've got a working hack to add inspire support to JabRef (as an
> additional
> > plugin rather than a modification of the spires one). I basically just
> > re-used the spires code and pointed it at inspire...very little
> modification
> > was needed as both spires and inspire outputs bibtex embedded in an html
> > file.
> > There was some functionality I could not test because I'm not sure if its
> > been disabled in JabRef. In particular there seems to be an openSpires
> > action that I didn't bother to port over because I'm not sure it works
> for
> > spires in the current version (it seems to be disabled). It would be
> nice
> > to hear from someone who knows this code to see if these functions are
> still
> > supported.
>
> If I remember correctly, the openSpires action was for either
> completing an entry based on Spires data, or for opening a Spires
> link, and the action didn't quite fit into the JabRef interface, so it
> was disabled. If the action can be integrated in a way that makes
> sense, I don't think there's any problem with it otherwise.
>
> > But the bigger issue is that I'd like to access inspire via MARCXML or
> > endref or some other XML format that returns additional fields like the
> > abstract because I'd like to store them as well. This is a bit more
> > involved and I'm not sure what the most efficient way to do it is.
> JabRef
> > does not seem to have a Filter for any relevant inspire output format
> > besides bibtex and endref. While the later contains the abstract it
> doesn't
> > include the eprint number which is a worse omission.
> > As spires looks like its about to go the way of the dinosaurs I'd be
> happy
> > to provide this inspire patch to the main tree or to anyone who just
> wants
> > to patch their own version.
>
> You may have to make a new filter for it - there are a couple of XML
> filters already (e.g. Medline) that might be useful as a starting
> point.
>
> If you have a plugin that's ready to use, we can publish it on the
> JabRef web page. If the code belongs in the main tree, we should
> consider adding it - if you want to, you could send me a patch to have
> a look at.
>
>
> Morten
>
Index: jabref/src/java/net/sf/jabref/JabRefFrame.java
===================================================================
--- jabref/src/java/net/sf/jabref/JabRefFrame.java (revision 3658)
+++ jabref/src/java/net/sf/jabref/JabRefFrame.java (working copy)
@@ -295,6 +295,14 @@
openSpires = new GeneralAction("openSpires", "Open SPIRES entry",
Globals.lang("Open SPIRES entry"),
prefs.getKey("Open SPIRES entry")),
+ /*
+ * It looks like this wasn't being implemented for spires anyway so we
+ * comment it out for now.
+ *
+ openInspire = new GeneralAction("openInspire", "Open INSPIRE entry",
+ Globals.lang("Open INSPIRE entry"),
+ prefs.getKey("Open INSPIRE entry")),
+ */
dupliCheck = new GeneralAction("dupliCheck", "Find duplicates"),
//strictDupliCheck = new GeneralAction("strictDupliCheck", "Find and remove exact duplicates"),
plainTextImport = new GeneralAction("plainTextImport",
@@ -1515,7 +1523,9 @@
selectAll, copyKey, copyCiteKey, copyKeyAndTitle, editPreamble, editStrings, toggleGroups, toggleSearch,
makeKeyAction, normalSearch,
incrementalSearch, replaceAll, importMenu, exportMenu, fetchCiteSeer,
- openPdf, openUrl, openFile, openSpires, togglePreview, dupliCheck, /*strictDupliCheck,*/ highlightAll,
+ /* openSpires wasn't being supported so no point in supporting
+ * openInspire */
+ openPdf, openUrl, openFile, openSpires, /*openInspire,*/ togglePreview, dupliCheck, /*strictDupliCheck,*/ highlightAll,
highlightAny, newEntryAction, plainTextImport, massSetField,
closeDatabaseAction, switchPreview, integrityCheckAction, autoSetPdf, autoSetPs,
toggleHighlightAny, toggleHighlightAll, databaseProperties, abbreviateIso,
Index: jabref/src/java/net/sf/jabref/BasePanel.java
===================================================================
--- jabref/src/java/net/sf/jabref/BasePanel.java (revision 3658)
+++ jabref/src/java/net/sf/jabref/BasePanel.java (working copy)
@@ -100,6 +100,7 @@
import net.sf.jabref.imports.AppendDatabaseAction;
import net.sf.jabref.imports.BibtexParser;
import net.sf.jabref.imports.SPIRESFetcher;
+import net.sf.jabref.imports.INSPIREFetcher;
import net.sf.jabref.journals.AbbreviateAction;
import net.sf.jabref.journals.UnabbreviateAction;
import net.sf.jabref.labelPattern.LabelPatternUtil;
@@ -1294,6 +1295,35 @@
}
});
+ /*
+ * It looks like this action was not being supported for SPIRES anyway
+ * so we don't bother to implement it.
+ actions.put("openInspire", new BaseAction() {
+ public void action() {
+ BibtexEntry[] bes = mainTable.getSelectedEntries();
+ if ((bes != null) && (bes.length == 1)) {
+ Object link = null;
+ if (bes[0].getField("eprint") != null)
+ link = INSPIREFetcher.constructUrlFromEprint(bes[0].getField("eprint").toString());
+ else if (bes[0].getField("slaccitation") != null)
+ link = INSPIREFetcher.constructUrlFromSlaccitation(bes[0].getField("slaccitation").toString());
+ if (link != null) {
+ //output(Globals.lang("Calling external viewer..."));
+ try {
+ Util.openExternalViewer(metaData(), link.toString(), "url");
+ output(Globals.lang("External viewer called")+".");
+ } catch (IOException ex) {
+ output(Globals.lang("Error") + ": " + ex.getMessage());
+ }
+ }
+ else
+ output(Globals.lang("No url defined")+".");
+ } else
+ output(Globals.lang("No entries or multiple entries selected."));
+ }
+ });
+ */
+
actions.put("replaceAll", new BaseAction() {
public void action() {
Index: jabref/src/java/net/sf/jabref/JabRefPreferences.java
===================================================================
--- jabref/src/java/net/sf/jabref/JabRefPreferences.java (revision 3658)
+++ jabref/src/java/net/sf/jabref/JabRefPreferences.java (working copy)
@@ -950,6 +950,7 @@
defKeyBinds.put("Write XMP", "ctrl F4");
defKeyBinds.put("New file link", "ctrl N");
defKeyBinds.put("Fetch SPIRES", "ctrl F8");
+ defKeyBinds.put("Fetch INSPIRE", "ctrl F2");
defKeyBinds.put("Back", "alt LEFT");
defKeyBinds.put("Forward", "alt RIGHT");
defKeyBinds.put("Import into current database", "ctrl I");
Index: jabref/src/plugins/net.sf.jabref.core/plugin.xml
===================================================================
--- jabref/src/plugins/net.sf.jabref.core/plugin.xml (revision 3658)
+++ jabref/src/plugins/net.sf.jabref.core/plugin.xml (working copy)
@@ -87,6 +87,15 @@
value="Fetch Entries from SLAC SPIRES Database." />
</extension>
+ <extension id="INSPIREFetcher" plugin-id="net.sf.jabref.core"
+ point-id="EntryFetcher">
+ <parameter id="entryFetcher"
+ value="net.sf.jabref.imports.INSPIREFetcher" />
+ <parameter id="name" value="INSPIRE Entry Fetcher" />
+ <parameter id="description"
+ value="Fetch Entries from SLAC INSPIRE Database." />
+ </extension>
+
<extension id="MedlineFetcher" plugin-id="net.sf.jabref.core"
point-id="EntryFetcher">
<parameter id="entryFetcher"
Index: jabref/src/resource/Menu_en.properties
===================================================================
--- jabref/src/resource/Menu_en.properties (revision 3658)
+++ jabref/src/resource/Menu_en.properties (working copy)
@@ -129,6 +129,7 @@
Export_to_external_SQL_database=Export_to_external_SQL_database
Search_JSTOR=Search_JSTOR
Fetch_SPIRES=Fetch_SPIRES
+Fetch_INSPIRE=Fetch_INSPIRE
Fetch_CiteSeer_by_ID=Fetch_CiteSeer_by_ID
Search_Medline=Search_Medline
Import_from_external_SQL_database=Import_from_external_SQL_database
@@ -144,4 +145,4 @@
Set/clear/rename_fields=Set/clear/rename_fields
Search_ScienceDirect=Search_ScienceDirect
-Resolve_duplicate_BibTeX_keys=Resolve_duplicate_BibTeX_keys
\ No newline at end of file
+Resolve_duplicate_BibTeX_keys=Resolve_duplicate_BibTeX_keys
package net.sf.jabref.imports;
import java.io.BufferedReader;
import java.io.FilterReader;
import java.io.IOException;
import java.io.Reader;
/**
*
* Warning -- it is not a generic filter, only read is implemented!
*
* Note: this is just a quick port of the original SPIRESBibtexFilterReader.
*
* @author Fedor Bezrukov
* @author Sheer El-Showk
*
* @version $Id$
*
* TODO: Fix grammar in bibtex entries -- it ma return invalid bibkeys (with space)
*
*/
public class INSPIREBibtexFilterReader extends FilterReader {
protected BufferedReader in;
private String line;
private int pos;
private boolean pre;
INSPIREBibtexFilterReader(Reader _in) {
super(_in);
in = new BufferedReader(_in);
pos=-1;
pre=false;
}
private String readpreLine() throws IOException {
String l;
do {
l=in.readLine();
if (l==null)
return null;
if (l.equals("<pre>")) {
pre = true;
l=in.readLine();
}
if (l.equals("</pre>"))
pre = false;
} while (!pre);
return l;
}
private String fixBibkey(String in) {
if (in== null)
return null;
//System.out.println(in);
if ( in.matches("@Article\\{.*,") ) {
//System.out.println(in.replace(' ','_'));
return in.replace(' ', '_');
} else
return in;
}
public int read() throws IOException {
if ( pos<0 ) {
line=fixBibkey(readpreLine());
pos=0;
if ( line == null )
return -1;
}
if ( pos>=line.length() ) {
pos=-1;
return '\n';
}
return line.charAt(pos++);
}
}
package net.sf.jabref.imports;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import javax.swing.JOptionPane;
import javax.swing.JPanel;
import net.sf.jabref.BibtexDatabase;
import net.sf.jabref.BibtexEntry;
import net.sf.jabref.GUIGlobals;
import net.sf.jabref.Globals;
import net.sf.jabref.OutputPrinter;
/**
*
* This class allows to access the Slac INSPIRE database. It is just a port of
* the original SPIRES Fetcher.
*
* It can either be a GeneralFetcher to pose requests to the database or fetch
* individual entries.
*
* @author Fedor Bezrukov
* @author Sheer El-Showk
*
* @version $Id$
*
*/
public class INSPIREFetcher implements EntryFetcher {
private static String inspireHost = "inspirebeta.net";
public INSPIREFetcher() {
}
/**
* Construct the query URL
*
* NOTE: we truncate at 1000 returned entries but its likely INSPIRE returns
* fewer anyway. This shouldn't be a problem since users should probably do
* more specific searches.
*
* @param key
* The key of the OAI2 entry that the url should poitn to.
*
* @return a String denoting the query URL
*/
public String constructUrl(String key) {
String identifier = "";
try {
identifier = URLEncoder.encode(key, "UTF-8");
} catch (UnsupportedEncodingException e) {
return "";
}
StringBuffer sb = new StringBuffer("http://";).append(inspireHost)
.append("/");
sb.append("/search?ln=en&ln=en&p=find+");
//sb.append("spires/find/hep/www").append("?");
//sb.append("rawcmd=find+");
sb.append(identifier);
//sb.append("&action_search=Search&sf=&so=d&rm=&rg=25&sc=0&of=hx");
sb.append("&action_search=Search&sf=&so=d&rm=&rg=1000&sc=0&of=hx");
//sb.append("&FORMAT=WWWBRIEFBIBTEX&SEQUENCE=");
System.out.print("Inspire URL: " + sb.toString() + "\n");
return sb.toString();
}
/**
* Constructs a INSPIRE query url from slaccitation field
*
* @param slaccitation
* @return query string
*
public static String constructUrlFromSlaccitation(String slaccitation) {
String cmd = "j";
String key = slaccitation.replaceAll("^%%CITATION = ", "").replaceAll(
";%%$", "");
if (key.matches("^\\w*-\\w*[ /].*"))
cmd = "eprint";
try {
key = URLEncoder.encode(key, "UTF-8");
} catch (UnsupportedEncodingException e) {
}
StringBuffer sb = new StringBuffer("http://";).append(inspireHost)
.append("/");
sb.append("spires/find/hep/www").append("?");
sb.append("rawcmd=find+").append(cmd).append("+");
sb.append(key);
return sb.toString();
}
/**
* Construct an INSPIRE query url from eprint field
*
* @param eprint
* @return query string
*
public static String constructUrlFromEprint(String eprint) {
String key = eprint.replaceAll(" [.*]$", "");
try {
key = URLEncoder.encode(key, "UTF-8");
} catch (UnsupportedEncodingException e) {
return "";
}
StringBuffer sb = new StringBuffer("http://";).append(inspireHost)
.append("/");
sb.append("spires/find/hep/www").append("?");
sb.append("rawcmd=find+eprint+");
sb.append(key);
return sb.toString();
}*/
/**
* Import an entry from an OAI2 archive. The BibtexEntry provided has to
* have the field OAI2_IDENTIFIER_FIELD set to the search string.
*
* @param key
* The OAI2 key to fetch from ArXiv.
* @return The imnported BibtexEntry or null if none.
*/
private BibtexDatabase importInspireEntries(String key, OutputPrinter frame) {
String url = constructUrl(key);
try {
HttpURLConnection conn = (HttpURLConnection) (new URL(url)).openConnection();
conn.setRequestProperty("User-Agent", "Jabref");
InputStream inputStream = conn.getInputStream();
INSPIREBibtexFilterReader reader = new INSPIREBibtexFilterReader(
new InputStreamReader(inputStream));
ParserResult pr = BibtexParser.parse(reader);
return pr.getDatabase();
} catch (IOException e) {
frame.showMessage( Globals.lang(
"An Exception ocurred while accessing '%0'", url)
+ "\n\n" + e.toString(), Globals.lang(getKeyName()),
JOptionPane.ERROR_MESSAGE);
} catch (RuntimeException e) {
frame.showMessage( Globals.lang(
"An Error occurred while fetching from INSPIRE source (%0):",
new String[] { url })
+ "\n\n" + e.getMessage(), Globals.lang(getKeyName()),
JOptionPane.ERROR_MESSAGE);
}
return null;
}
// public void addSpiresURL(BibtexEntry entry) {
// String url = "http://"+spiresHost+"/spires/find/hep/www?texkey+";;
// url = url+entry.getCiteKey();
// entry.setField("url", url);
// }
//
// public void addSpiresURLtoDatabase(BibtexDatabase db) {
// Iterator<BibtexEntry> iter = db.getEntries().iterator();
// while (iter.hasNext())
// addSpiresURL(iter.next());
// }
/*
* @see net.sf.jabref.imports.EntryFetcher
*/
public String getHelpPage() {
return "Spires.html";
}
public URL getIcon() {
return GUIGlobals.getIconUrl("www");
}
public String getKeyName() {
return "Fetch INSPIRE";
}
public JPanel getOptionsPanel() {
// we have no additional options
return null;
}
public String getTitle() {
return Globals.menuTitle(getKeyName());
}
/*
* @see net.sf.jabref.gui.ImportInspectionDialog.CallBack
*/
public void cancelled() {
}
public void done(int entriesImported) {
}
public void stopFetching() {
}
/*
* @see java.lang.Runnable
*/
public boolean processQuery(String query, ImportInspector dialog,
OutputPrinter frame) {
try {
frame.setStatus("Fetching entries from Inspire");
/* query the archive and load the results into the BibtexEntry */
BibtexDatabase bd = importInspireEntries(query,frame);
/* addSpiresURLtoDatabase(bd); */
frame.setStatus("Adding fetched entries");
/* add the entry to the inspection dialog */
if (bd.getEntryCount() > 0)
for (BibtexEntry entry : bd.getEntries())
dialog.addEntry(entry);
/* update the dialogs progress bar */
// dialog.setProgress(i + 1, keys.length);
/* inform the inspection dialog, that we're done */
} catch (Exception e) {
frame.showMessage(Globals.lang("Error while fetching from Inspire: ")
+ e.getMessage());
e.printStackTrace();
}
return true;
}
}
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1
_______________________________________________
Jabref-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jabref-users
