Github user avulanov commented on the pull request:
https://github.com/apache/spark/pull/1290#issuecomment-111332814
@hntd187 This is true, however it seems that "modern" datasets tend to have
more features, so 784 features of mnist might seems too little these days.
Anyway, the basic idea of benchmark is as follows: compare performance of Caffe
and this implementation both in CPU and GPU mode with different numbers of
nodes (workers) for Spark. Performance should be measured in samples/second
processed. Here comes another problem: data formats that are supported by Spark
and Caffe do not intersect. I can convert mnist8m (libsvm) to HDF5 for Caffe,
however it will have different size that means that Caffe will read different
amount of data from disk. Do you have an idea how to handle this problem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
