[
https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429154#comment-13429154
]
Chris A. Mattmann commented on OODT-483:
----------------------------------------
Ross, +1 this makes a ton of sense. Feel free to use Review Board
http://reviews.apache.org/ to post up a review and link it to this JIRA issue.
Brian Foster, Ricky, Sheryl, Tom, Paul R. and me and others are all active
consumers of Review Board reviews, so that would be a great thing.
Review Board or not, +1 to commit this patch and this is a critical fix.
Thanks!
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
> Key: OODT-483
> URL: https://issues.apache.org/jira/browse/OODT-483
> Project: OODT
> Issue Type: Bug
> Components: file manager
> Reporter: Ross Laidlaw
> Assignee: Ross Laidlaw
> Priority: Minor
> Labels: gsoc2012
> Fix For: 0.5
>
> Attachments: OODT-483.rlaidlaw.2012-08-06.patch.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the
> progress of the transfer using the 'cas-product' RSS web application
> (RSSProductTransferServlet [1]). This caused a NumberFormatException as
> follows:
> {code}
> java.lang.NumberFormatException: For input string:
> "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files
> approximately 10MB in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to
> set the value of a long variable using a binary representation (i.e. treating
> the binary number as a decimal number). The size of the binary
> representation exceeds the maximum capacity of the long because it is not
> being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory
> [2]:
> {code}
> public static Hashtable<String, Object>
> getXmlRpcFileTransferStatus(FileTransferStatus status)
> {
> Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
>
> statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
> statusHash.put("parentProduct",
> getXmlRpcProduct(status.getParentProduct()));
> statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
> return statusHash;
> }
> public static FileTransferStatus
> getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
> {
> FileTransferStatus status = new FileTransferStatus();
>
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("parentProduct")));
> status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
> return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line
> converts the value returned by 'status.getBytesTransferred()' to a binary
> representation and stores it as a String in the 'statusHash' Hashtable:
> {code}
> statusHash.put("bytesTransferred",
> Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method
> getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:
> {code}
>
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long.
> In the example above, the input string "11100111001101000000000000" converted
> directly to a long (i.e. treated as a decimal) will exceed the maximum size
> of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in
> method getFileTransferStatusFromXmlRpc, as follows:
> {code}
>
> status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(),
> 2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the
> problem. Would this be an acceptable fix? I'll attach a patch to this issue
> for reference.
> An alternative solution would be not to store a binary representation in the
> statusHash Hashtable. But I'm assuming there's a reason why it's converted
> to binary for the hash.
> [1]
> http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2]
> http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
