[
https://issues.apache.org/jira/browse/MXNET-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395518#comment-16395518
]
Chris Olivier commented on MXNET-11:
------------------------------------
I suppose the multiple threads would call the C API. Python is a bit tricky to
get it to do multithreading, so I wouldn't be concerned with a python entry
point at this point.
So above, you launched many processes, loaded a model and ran inference in
parallel?
Or you loaded a model in many threads and ran inference through those?
The main problem right now is that there's not a good way to *share* parameters
between graphs, so that would probably be some large bulk of the work. There's
actually several other use-cases for this, including Keras integration.
> Multithreaded Inference
> -----------------------
>
> Key: MXNET-11
> URL: https://issues.apache.org/jira/browse/MXNET-11
> Project: Apache MXNet
> Issue Type: Epic
> Components: MXNet Engine
> Reporter: Chris Olivier
> Priority: Major
> Labels: inference
>
> Add the ability to do multithreaded inference without using fork() or using
> multiple copies of a given model
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
