Richard Platania created SINGA-160:
--------------------------------------
Summary: Reading larger records from HDFS
Key: SINGA-160
URL: https://issues.apache.org/jira/browse/SINGA-160
Project: Singa
Issue Type: Bug
Reporter: Richard Platania
Priority: Minor
The current code for HDFSFile::Next in hdfsfile.cc handles reading data in at
most two attempts. It may take more than two calls to hdfsRead(), depending on
the user's configuration of hdfs-client.xml. For example, if
input.localread.default.buffersize is more than 2x smaller than the data being
read. I've found this to be a common occurrence when reading image mean files
for larger datasets. A solution for this can be to wrap hdfsRead in a loop
until the total amount read reaches the the size of the data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
