[
https://issues.apache.org/jira/browse/TAJO-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14457080#comment-14457080
]
ASF GitHub Bot commented on TAJO-1480:
--------------------------------------
Github user blrunner commented on the pull request:
https://github.com/apache/tajo/pull/482#issuecomment-89885161
You need to add tajo-storage-kakfa module to the pom.xml file of
tajo-storage module. And I found many dependencies as follows.
```
[INFO]
------------------------------------------------------------------------
[INFO] Building Tajo Kafka Storage 0.11.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloading:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
(3 KB at 2.2 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloading:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
(2 KB at 5.1 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloading:
https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloaded:
https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom (2
KB at 6.1 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloading:
https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
(4 KB at 10.1 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloading:
https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
(2 KB at 6.3 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloading:
https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloaded:
https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
(11 KB at 33.0 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloading:
https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloaded:
https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.3/zkclient-0.3.pom
(3 KB at 9.7 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloading:
https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
(3 KB at 6.8 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/testing/0.88/testing-0.88.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/testing/0.88/testing-0.88.pom
Downloading:
https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.pom
Downloaded:
https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.pom
(2 KB at 4.1 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/airlift/0.88/airlift-0.88.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/airlift/0.88/airlift-0.88.pom
Downloading:
https://repo.maven.apache.org/maven2/io/airlift/airlift/0.88/airlift-0.88.pom
Downloaded:
https://repo.maven.apache.org/maven2/io/airlift/airlift/0.88/airlift-0.88.pom
(10 KB at 21.3 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/airbase/20/airbase-20.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/airbase/20/airbase-20.pom
Downloading:
https://repo.maven.apache.org/maven2/io/airlift/airbase/20/airbase-20.pom
Downloaded:
https://repo.maven.apache.org/maven2/io/airlift/airbase/20/airbase-20.pom (59
KB at 61.5 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloading:
https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloaded:
https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
(764 B at 2.4 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloading:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
(9 KB at 26.4 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloading:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
(25 KB at 52.4 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloading:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
(6 KB at 19.4 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloading:
https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloaded:
https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.pom
(3 KB at 9.7 KB/sec)
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/testing/0.88/testing-0.88.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloading:
https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/testing/0.88/testing-0.88.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloading:
http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloading:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading:
https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading:
https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading:
https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloaded:
https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
(52 KB at 57.5 KB/sec)
Downloading:
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloaded:
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
(25 KB at 81.4 KB/sec)
Downloading:
https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.jar
Downloaded:
https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
(162 KB at 109.2 KB/sec)
Downloading:
https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloaded:
https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.jar
(26 KB at 47.8 KB/sec)
Downloading:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
(317 KB at 163.2 KB/sec)
Downloading:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloaded:
https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
(73 KB at 129.0 KB/sec)
Downloading:
https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
(61 KB at 166.2 KB/sec)
Downloaded:
https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.jar
(64 KB at 120.3 KB/sec)
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
(242 KB at 236.5 KB/sec)
Downloaded:
https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
(3900 KB at 372.1 KB/sec)
Downloaded:
https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
(6960 KB at 537.3 KB/sec)
```
I would like it more simple if you exclude unnecessary dependencies.
> Kafka Consumer for kafka strage.
> --------------------------------
>
> Key: TAJO-1480
> URL: https://issues.apache.org/jira/browse/TAJO-1480
> Project: Tajo
> Issue Type: Sub-task
> Components: storage
> Reporter: YeonSu Han
> Assignee: YeonSu Han
> Labels: kafka_storage
>
> Scanner of kafka storage needs to implement a Kafka Consumer for to fetch
> data from kafka.
> The Kafka Consumer have methods like this,
> - getPartitions(): Get partition id list from specific topic.
> - fetch(): Fetch data from kafka.
> - getReadOffset(): Get offset from specific partition.
> - findLeader(): Find leader broker of specific partition.
> - etc..
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
