[
https://issues.apache.org/jira/browse/PIO-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663040#comment-16663040
]
ASF GitHub Bot commented on PIO-182:
------------------------------------
takezoe commented on a change in pull request #482: [PIO-182] Add async methods
to LEventStore
URL: https://github.com/apache/predictionio/pull/482#discussion_r228002189
##########
File path:
data/src/main/scala/org/apache/predictionio/data/store/LEventStore.scala
##########
@@ -20,15 +20,27 @@ package org.apache.predictionio.data.store
import org.apache.predictionio.data.storage.Storage
import org.apache.predictionio.data.storage.Event
-
import org.joda.time.DateTime
-import scala.concurrent.Await
-import scala.concurrent.ExecutionContext.Implicits.global
+import scala.concurrent.{Await, ExecutionContext, Future}
import scala.concurrent.duration.Duration
/** This object provides a set of operation to access Event Store
- * without going through Spark's parallelization
+ * without going through Spark's parallelization.
+ *
+ * Note that blocking methods of this object uses
+ * `scala.concurrent.ExecutionContext.Implicits.global` internally.
+ * Since this is a thread pool which has a number of threads equal to
available
+ * processors, parallelism is limited up to the number of processors.
+ *
+ * If this limitation become bottleneck of resource usage, you can increase
the
+ * number of threads by declaring following VM options before calling "pio
deploy":
+ *
+ * <pre>
+ * export JAVA_OPTS="$JAVA_OPTS \
+ * -Dscala.concurrent.context.numThreads=1000 \
+ * -Dscala.concurrent.context.maxThreads=1000"
+ * </pre>
Review comment:
Oops, fixed. thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add asynchronous (non-blocking) methods to LEventStore
> ------------------------------------------------------
>
> Key: PIO-182
> URL: https://issues.apache.org/jira/browse/PIO-182
> Project: PredictionIO
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.13.0
> Reporter: Naoki Takezoe
> Assignee: Naoki Takezoe
> Priority: Major
>
> The current implementation of {{LEventStore}} has only synchronous (blocking)
> methods. Since they use {{ExecutionContext.Implicit.global}}, its parallelism
> is limited up to the number of processors. This means engine server's
> parallelism is also limited if we use these methods in prediction logic.
> To solve this problem, {{Future}} version of these methods should be added to
> {{LEventStore}} and also current blocking methods should be modified to take
> {{ExecutionContext}} (as an implicit parameter).
> See also:
> https://lists.apache.org/thread.html/f14e4f8f29410e4585b3d8e9f646b88293a605f4716d3c4d60771854@%3Cuser.predictionio.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
