[
https://issues.apache.org/jira/browse/BAHIR-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015199#comment-16015199
]
ASF GitHub Bot commented on BAHIR-117:
--------------------------------------
Github user lresende commented on a diff in the pull request:
https://github.com/apache/bahir/pull/43#discussion_r117160442
--- Diff:
streaming-twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala
---
@@ -85,10 +85,8 @@ class TwitterReceiver(
}
})
- val query = new FilterQuery
- if (filters.size > 0) {
- query.track(filters.mkString(","))
- newTwitterStream.filter(query)
+ if (query.isDefined) {
+ newTwitterStream.filter(query.get)
} else {
--- End diff --
It looks like the query option was being available as a seq of string,
which then was converted to FilterQuery, and doing so we were hiding the
implementation detail specific interface from the user. What were the
limitations of using that strategy ?
> Expand filtering options for TwitterInputDStream
> ------------------------------------------------
>
> Key: BAHIR-117
> URL: https://issues.apache.org/jira/browse/BAHIR-117
> Project: Bahir
> Issue Type: Improvement
> Components: Spark Streaming Connectors
> Reporter: Clemens Wolff
>
> Currently, the TwitterInputDStream only supports filtering by keywords [1]
> which corresponds to the "track" option in the Twitter API [2]. The Twitter
> API supports many more ways to receive a filtered stream (e.g. get Tweets in
> a particular location [3]). It would be very useful to expose these
> additional filtering options in this library.
> Proposal: add a new public method to TwitterUtils which follows the same
> interface as createStream [4] but which takes a FilterQuery [5] object as
> argument. In this way, we give full filtering flexibility to our users.
> I'm currently working on Project Fortis, a social data analysis platform for
> the United Nations [6]. The extra filtering options would be very useful for
> my project so I'm happy to implement this and create a pull request.
> [1]
> https://github.com/apache/bahir/blob/fd4c35fc9f7ebb57464d231cf5d66e7bc4096a1b/streaming-twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala#L44
> [2] https://dev.twitter.com/streaming/overview/request-parameters#track
> [3] https://dev.twitter.com/streaming/overview/request-parameters#locations
> [4]
> https://github.com/apache/bahir/blob/fd4c35fc9f7ebb57464d231cf5d66e7bc4096a1b/streaming-twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala#L39
> [5] http://twitter4j.org/javadoc/twitter4j/FilterQuery.html
> [6] https://fortis-web.azurewebsites.net/#/site/ocha/
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
