Arun Mahadevan created SPARK-24136:
--------------------------------------
Summary: MemoryStreamDataReader.next should skip sleeping if
record is available
Key: SPARK-24136
URL: https://issues.apache.org/jira/browse/SPARK-24136
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 2.4.0
Reporter: Arun Mahadevan
Currently the code sleeps 10ms for each invocation of the next even if the
record is available.
{code:java}
override def next(): Boolean = {
current = None
while (current.isEmpty) {
Thread.sleep(10)
current = endpoint.askSync[Option[Row]](
GetRecord(ContinuousMemoryStreamPartitionOffset(partition, currentOffset)))
}{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
