[
https://issues.apache.org/jira/browse/BAHIR-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christian Kadner updated BAHIR-8:
---------------------------------
Assignee: Christian Kadner
Affects Version/s: 2.0.0
Issue Type: Task (was: Bug)
The current set of streaming modules in Bahir (_akka, mqtt, zeromq, twitter_)
are using 7 different methods/constructors/fields which are
private/packaged-scoped to {{org.apache.spark.*}}.
{noformat}
core/src/main/scala/org/apache/spark/SparkContext.scala
- private[spark] def clean[F <: AnyRef](f: F, checkSerializable: Boolean =
true): F = { ...
core/src/main/scala/org/apache/spark/util/Utils.scala
- private[spark] object Utils extends Logging { ...
core/src/test/scala/org/apache/spark/SparkFunSuite.scala
- private[spark] abstract class SparkFunSuite { ...
core/src/main/scala/org/apache/spark/Logging.scala
- private[spark] trait Logging { ...
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala
- private[streaming] val sc: SparkContext = { ...
- private[streaming] def withNamedScope[U](name: String)(body: => U): U = { ...
streaming/src/main/scala/org/apache/spark/streaming/dstream/InputDStream.scala
- private[streaming] def name: String = { ...
{noformat}
I suggest to create a _shim_ module named *common* or *common-streaming* to the
Bahir repository that will contains common utils and the necessary shim
classes/objects to make these "private APIs" accessible to the rest of the
classes/objects under the {{org.apache.bahir}} namespace. I am working on a PR
for that.
> Investigate weather we can use org.apache.bahir packages on Apache Spark
> extensions
> -----------------------------------------------------------------------------------
>
> Key: BAHIR-8
> URL: https://issues.apache.org/jira/browse/BAHIR-8
> Project: Bahir
> Issue Type: Task
> Components: Spark Streaming Connectors
> Affects Versions: 2.0.0
> Reporter: Luciano Resende
> Assignee: Christian Kadner
>
> Experiment if we can move the current extensions to org.apache.bahir package.
> We might see issues related to the private[spark], etc restriction that scala
> imposes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
