[
https://issues.apache.org/jira/browse/SINGA-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548830#comment-15548830
]
ASF subversion and git services commented on SINGA-251:
-------------------------------------------------------
Commit 17ac160258093973602eda883562b39069504fc8 in incubator-singa's branch
refs/heads/master from [~zhaojing]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=17ac160 ]
SINGA-251 - Implement image loader for pysinga
Add ImageBatchIter in data.py, which prefetches images into mini-batches
and pre-processes images using user defined transform function.
The process is done in another process via multiprocessing
fixed a bug in merge layer which should replicate the gradients in the returned
values
To test this feature,
please prepare a train.txt file which consists of multiple lines, where each
line is , e.g.
image1.jpg, 0
image2.jpg, 3
image3.jpg, 1
...
The folder should be like
SINGA ROOT/
train.txt
images/
image1.jpg
image2.jpg
...
Then you can run python python/singa/data.py (under the root dir of singa).
After that, you should see a list of images which are transformed from the
images in images/ folder.
> Implement image loader for pysinga
> ----------------------------------
>
> Key: SINGA-251
> URL: https://issues.apache.org/jira/browse/SINGA-251
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
>
> Image loading and pre-processing typically takes a long time, which would
> become the bottleneck of training when loading time > BP time.
> This ticket would implement an image loader which fetches the images and
> conducts pre-processing using multi-threading via multiprocessing module from
> python.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
