[
https://issues.apache.org/jira/browse/SPARK-13170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136273#comment-15136273
]
Sean Owen commented on SPARK-13170:
-----------------------------------
Yeah, tough one. The problem with the API is that it assumes the implementation
it is given is thread-safe, since it's something that is inherently modified by
the caller and Spark, but does not demand a SynchronizedQueue. Despite the
recommendation to use ConcurrentLinkedQueue, there's no conversion (?) to/from
Scala Queue.
I think the simplest thing is to change the implementation and usages to use a
Queue but synchronize access, and document this. At least, I'll open a PR to
that effect for discussion.
> Investigate replacing SynchronizedQueue as it is deprecated
> -----------------------------------------------------------
>
> Key: SPARK-13170
> URL: https://issues.apache.org/jira/browse/SPARK-13170
> Project: Spark
> Issue Type: Sub-task
> Components: Streaming, Tests
> Reporter: holdenk
> Priority: Trivial
>
> In some of our tests we use SynchronizedQueue to append to the queue after
> creating a queue stream. SynchronizedQueue is deprecated and we should see if
> we can replace it. This is a bit tricky since the queue stream API is public,
> and while it doesn't depend on having a SynchronizedQueue as input
> (thankfully) it does require a Queue. We could possibly change the tests to
> not depend on the SynchronizedQueue or change the QueueStream to also work
> with Iterables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
