----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6409/#review9945 -----------------------------------------------------------
trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java <https://reviews.apache.org/r/6409/#comment21145> +1, I'm for option #1. - Chris Mattmann On Aug. 7, 2012, 12:38 a.m., Ross Laidlaw wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/6409/ > ----------------------------------------------------------- > > (Updated Aug. 7, 2012, 12:38 a.m.) > > > Review request for oodt, Chris Mattmann and Paul Ramirez. > > > Description > ------- > > A NumberFormatException occurs when using the cas-product RSS service to view > file transfers in progress, as detailed by OODT-483. I traced this exception > to two methods in the XmlRpcStructFactory class, getXmlRpcFileTransferStatus > and getFileTransferStatusFromXmlRpc: > > public static Hashtable<String, Object> > getXmlRpcFileTransferStatus(FileTransferStatus status) > { > Hashtable<String, Object> statusHash = new Hashtable<String, Object>(); > statusHash.put("bytesTransferred", > Long.toBinaryString(status.getBytesTransferred())); > statusHash.put("parentProduct", > getXmlRpcProduct(status.getParentProduct())); > statusHash.put("fileRef", getXmlRpcReference(status.getFileRef())); > return statusHash; > } > > public static FileTransferStatus > getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash) > { > FileTransferStatus status = new FileTransferStatus(); > > status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString())); > status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) > statusHash.get("parentProduct"))); > status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>) > statusHash.get("fileRef"))); > return status; > } > > The following line in getXmlRpcFileTransferStatus converts the value returned > by status.getBytesTransferred() to a binary representation in a string and > stores the result in the statusHash Hashtable: > > statusHash.put("bytesTransferred", > Long.toBinaryString(status.getBytesTransferred())); > > > The following line in getFileTransferStatusFromXmlRpc retrieves the value for > key "bytesTransferred" from the supplied statusHash Hashtable, but it does > not assume that the number is in binary format and attempts to convert it > directly to a long to pass to the setBytesTransferred() method for the status > (type FileTransferStatus) object: > > > status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString())); > > > This causes a NumberFormatException for larger files because the parseLong > can't fit the binary representation inside a long. > > > Here are two alternative simple fixes: > > 1) in getXmlRpcFileTransferStatus use 'toString' instead of 'toBinaryString' > to store a decimal representation of the number in the string (in statusHash): > > statusHash.put("bytesTransferred", > Long.toString(status.getBytesTransferred())); > > > 2) in getFileTransferStatusFromXmlRpc use a radix argument for parseLong to > specify that the "bytesTransferred" string (from statusHash) represents a > binary number: > > > status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), > 2)); > > > I have uploaded two patches, one for each of the above suggestions. > > How important is the binary representation? Is it used to prevent loss of > precision? Looking back at the commit history for XmlRpcStructFactory, it > looks like the 'toBinaryString' conversion was introduced as a result of > OODT-54. This was part of a bug fix to prevent loss of precision that > happened when casting from longs to integers and then back again. > > The changes can be seen here (see changes for lines 75 and 86): > http://svn.apache.org/viewvc/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java?r1=1096655&r2=1070942&diff_format=h > > > This addresses bugs OODT-483 and OODT-54. > https://issues.apache.org/jira/browse/OODT-483 > https://issues.apache.org/jira/browse/OODT-54 > > > Diffs > ----- > > > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java > 1365193 > > Diff: https://reviews.apache.org/r/6409/diff/ > > > Testing > ------- > > I have tested both of the patches by using File Manager to ingest a large > file (over 100MB) and then running the following from the command line to > access the RSS output from the RSSProductTransferServlet in the 'cas-product' > (fmprod) web application: > > curl "http://localhost:8080/cas-product-0.5-SNAPSHOT/viewTransfers"; > > An example output is shown below: > > <?xml version="1.0" encoding="UTF-8" standalone="no"?> > <rss xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas"; version="2.0"> > <channel> > <title>File Manager Transfers</title> > <link>http://localhost:8080/cas-product-0.5-SNAPSHOT/viewTransfers</link> > <description>Current Files Being Transferred to the File Manager</description> > <language>en-us</language> > <copyright>Copyright 2010: Apache Software Foundation</copyright> > <pubDate>Mon, 06 Aug 2012 12:07:41 BST</pubDate> > <category>data transfer</category> > <generator>CAS File Manager</generator> > <lastBuildDate>Mon, 06 Aug 2012 12:07:41 BST</lastBuildDate> > <item> > <title>t1.m4v</title> > <description>GenericFile</description> > <link>http://localhost:8080/cas-product-0.5-SNAPSHOT/viewTransfer?ref=file:///usr/local/oodt/cas-filemgr-0.5/test/t1.m4v&size=124699632</link> > <pubDate>Mon, 06 Aug 2012 13:07:32 BST</pubDate> > <source>file manager transfers</source> > <cas:source>file manager transfers</cas:source> > <cas:bytesTransferred>101998592</cas:bytesTransferred> > <cas:totalBytes>124699632</cas:totalBytes> > <cas:percentComplete>0.8179542342193921</cas:percentComplete> > </item> > </channel> > </rss> > > > Thanks, > > Ross Laidlaw > >
