[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[ https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951772#comment-15951772 ] ASF GitHub Bot commented on BAHIR-97: - Github user ckadner commented on the issue: https://github.com/apache/bahir/pull/38 @sbcd90 -- thanks for you continuous updates and sorry for the piecemeal review from my end ... I started with your test cases since we were still in the process of fixing our Jenkins build setup. But your test cases are great now :-) Perhaps more important, my first request to you should have been to add a README and examples so users can start using your connector without having to read through too much code ... i.e. please further follow the precedence set by `sql-streaming-mqtt`. **Thank you!** > Akka as a streaming source for SQL Streaming. > - > > Key: BAHIR-97 > URL: https://issues.apache.org/jira/browse/BAHIR-97 > Project: Bahir > Issue Type: New Feature > Components: Spark SQL Data Sources >Affects Versions: Spark-2.1.0 >Reporter: Subhobrata Dey > > Hello, > This issue is created to propose the addition of Akka compatible streaming > source for Spark SQL Streaming. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[
https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951751#comment-15951751
]
ASF GitHub Bot commented on BAHIR-97:
-
Github user ckadner commented on a diff in the pull request:
https://github.com/apache/bahir/pull/38#discussion_r109263328
--- Diff: sql-streaming-akka/src/main/assembly/assembly.xml ---
@@ -0,0 +1,44 @@
+
+
+test-jar-with-dependencies
+
+jar
+
+false
+
+
+
+
${project.build.directory}/scala-${scala.binary.version}/test-classes
+
+
+
+
+
+
+true
+test
+true
+
+org.apache.hadoop:*:jar
+org.apache.zookeeper:*:jar
+org.apache.avro:*:jar
+
+
+
+
+
--- End diff --
add new line
> Akka as a streaming source for SQL Streaming.
> -
>
> Key: BAHIR-97
> URL: https://issues.apache.org/jira/browse/BAHIR-97
> Project: Bahir
> Issue Type: New Feature
> Components: Spark SQL Data Sources
>Affects Versions: Spark-2.1.0
>Reporter: Subhobrata Dey
>
> Hello,
> This issue is created to propose the addition of Akka compatible streaming
> source for Spark SQL Streaming.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[
https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951753#comment-15951753
]
ASF GitHub Bot commented on BAHIR-97:
-
Github user ckadner commented on a diff in the pull request:
https://github.com/apache/bahir/pull/38#discussion_r109263538
--- Diff: sql-streaming-akka/src/test/resources/feeder_actor.conf ---
@@ -0,0 +1,34 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+akka {
+ loglevel = "INFO"
+ actor {
+provider = "akka.remote.RemoteActorRefProvider"
+ }
+ remote {
+enabled-transports = ["akka.remote.netty.tcp"]
+netty.tcp {
+ hostname = "127.0.0.1"
+ port = 0
+}
+log-sent-messages = on
+log-received-messages = on
+ }
+ loggers.0 = "akka.event.slf4j.Slf4jLogger"
+ log-dead-letters-during-shutdown = "off"
+}
--- End diff --
add new line
> Akka as a streaming source for SQL Streaming.
> -
>
> Key: BAHIR-97
> URL: https://issues.apache.org/jira/browse/BAHIR-97
> Project: Bahir
> Issue Type: New Feature
> Components: Spark SQL Data Sources
>Affects Versions: Spark-2.1.0
>Reporter: Subhobrata Dey
>
> Hello,
> This issue is created to propose the addition of Akka compatible streaming
> source for Spark SQL Streaming.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[
https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951752#comment-15951752
]
ASF GitHub Bot commented on BAHIR-97:
-
Github user ckadner commented on a diff in the pull request:
https://github.com/apache/bahir/pull/38#discussion_r109263376
--- Diff: sql-streaming-akka/pom.xml ---
@@ -0,0 +1,120 @@
+
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+org.apache.bahir
+bahir-parent_2.11
+2.2.0-SNAPSHOT
+../pom.xml
+
+
+org.apache.bahir
+spark-sql-streaming-akka_2.11
+
+sql-streaming-akka
+
+jar
+Apache Bahir - Spark SQL Streaming Akka
+http://bahir.apache.org
+
+
+
+org.apache.spark
+spark-tags_${scala.binary.version}
+
+
+org.apache.spark
+spark-sql_${scala.binary.version}
+${spark.version}
+
+
+org.apache.spark
+spark-sql_${scala.binary.version}
+${spark.version}
+test-jar
+test
+
+
+org.apache.spark
+spark-core_${scala.binary.version}
+${spark.version}
+test-jar
+test
+
+
+${akka.group}
+akka-actor_${scala.binary.version}
+${akka.version}
+
+
+${akka.group}
+akka-remote_${scala.binary.version}
+${akka.version}
+
+
+${akka.group}
+akka-slf4j_${scala.binary.version}
+${akka.version}
+
+
+org.rocksdb
+rocksdbjni
+5.1.2
+
+
+
+
+
target/scala-${scala.binary.version}/classes
+
target/scala-${scala.binary.version}/test-classes
+
+
+
+org.apache.maven.plugins
+maven-source-plugin
+
+
+
+
+org.apache.maven.plugins
+maven-assembly-plugin
+
+
+test-jar-with-dependencies
+package
+
+single
+
+
+
+
spark-streaming-akka-test-${project.version}
+
${project.build.directory}/scala-${scala.binary.version}
+false
+
+false
+
+
src/main/assembly/assembly.xml
+
+
+
+
+
+
+
+
--- End diff --
add new line
> Akka as a streaming source for SQL Streaming.
> -
>
> Key: BAHIR-97
> URL: https://issues.apache.org/jira/browse/BAHIR-97
> Project: Bahir
> Issue Type: New Feature
> Components: Spark SQL Data Sources
>Affects Versions: Spark-2.1.0
>Reporter: Subhobrata Dey
>
> Hello,
> This issue is created to propose the addition of Akka compatible streaming
> source for Spark SQL Streaming.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (BAHIR-101) Add Spark SQL datasource for CounchDB/Cloudant
[ https://issues.apache.org/jira/browse/BAHIR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luciano Resende resolved BAHIR-101. --- Resolution: Fixed Assignee: Yang Lei > Add Spark SQL datasource for CounchDB/Cloudant > -- > > Key: BAHIR-101 > URL: https://issues.apache.org/jira/browse/BAHIR-101 > Project: Bahir > Issue Type: Improvement > Components: Spark SQL Data Sources >Reporter: Yang Lei >Assignee: Yang Lei > Fix For: Spark-2.1.1 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-101) Add SparkSQL for Cloudant
[ https://issues.apache.org/jira/browse/BAHIR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951742#comment-15951742 ] ASF subversion and git services commented on BAHIR-101: --- Commit f0d9a84f76cb34a432e1d2db053d2471a8ab2ba4 in bahir's branch refs/heads/master from [~yanglei] [ https://git-wip-us.apache.org/repos/asf?p=bahir.git;h=f0d9a84 ] [BAHIR-101] Spark SQL datasource for CounchDB/Cloudant Initial code supporting CounchDB/Cloudant as an Spark SQL data source. The initial source contains the core connector, examples, and basic documentation on the README. Closes #39. > Add SparkSQL for Cloudant > -- > > Key: BAHIR-101 > URL: https://issues.apache.org/jira/browse/BAHIR-101 > Project: Bahir > Issue Type: Improvement >Reporter: Yang Lei > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-101) Add SparkSQL for Cloudant
[ https://issues.apache.org/jira/browse/BAHIR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951743#comment-15951743 ] ASF GitHub Bot commented on BAHIR-101: -- Github user asfgit closed the pull request at: https://github.com/apache/bahir/pull/39 > Add SparkSQL for Cloudant > -- > > Key: BAHIR-101 > URL: https://issues.apache.org/jira/browse/BAHIR-101 > Project: Bahir > Issue Type: Improvement >Reporter: Yang Lei > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-101) Add SparkSQL for Cloudant
[ https://issues.apache.org/jira/browse/BAHIR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951518#comment-15951518 ] ASF GitHub Bot commented on BAHIR-101: -- Github user yanglei99 commented on the issue: https://github.com/apache/bahir/pull/39 All the 8 examples (3 in python, 5 in scala) are tested successfully. > Add SparkSQL for Cloudant > -- > > Key: BAHIR-101 > URL: https://issues.apache.org/jira/browse/BAHIR-101 > Project: Bahir > Issue Type: Improvement >Reporter: Yang Lei > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-101) Add SparkSQL for Cloudant
[ https://issues.apache.org/jira/browse/BAHIR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951516#comment-15951516 ] ASF GitHub Bot commented on BAHIR-101: -- Github user ApacheBahir commented on the issue: https://github.com/apache/bahir/pull/39 Build successful > Add SparkSQL for Cloudant > -- > > Key: BAHIR-101 > URL: https://issues.apache.org/jira/browse/BAHIR-101 > Project: Bahir > Issue Type: Improvement >Reporter: Yang Lei > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[ https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951504#comment-15951504 ] ASF GitHub Bot commented on BAHIR-97: - Github user ApacheBahir commented on the issue: https://github.com/apache/bahir/pull/38 Build successful > Akka as a streaming source for SQL Streaming. > - > > Key: BAHIR-97 > URL: https://issues.apache.org/jira/browse/BAHIR-97 > Project: Bahir > Issue Type: New Feature > Components: Spark SQL Data Sources >Affects Versions: Spark-2.1.0 >Reporter: Subhobrata Dey > > Hello, > This issue is created to propose the addition of Akka compatible streaming > source for Spark SQL Streaming. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[ https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951505#comment-15951505 ] ASF GitHub Bot commented on BAHIR-97: - Github user ApacheBahir commented on the issue: https://github.com/apache/bahir/pull/38 Refer to this link for build results (access rights to CI server needed): http://169.45.79.58:8080/job/bahir_spark_pr_builder/48/ > Akka as a streaming source for SQL Streaming. > - > > Key: BAHIR-97 > URL: https://issues.apache.org/jira/browse/BAHIR-97 > Project: Bahir > Issue Type: New Feature > Components: Spark SQL Data Sources >Affects Versions: Spark-2.1.0 >Reporter: Subhobrata Dey > > Hello, > This issue is created to propose the addition of Akka compatible streaming > source for Spark SQL Streaming. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[ https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951503#comment-15951503 ] ASF GitHub Bot commented on BAHIR-97: - Github user sbcd90 commented on the issue: https://github.com/apache/bahir/pull/38 Hi @ckadner , I have made the changes you mentioned to remove the remaining verbose. Please have a look. > Akka as a streaming source for SQL Streaming. > - > > Key: BAHIR-97 > URL: https://issues.apache.org/jira/browse/BAHIR-97 > Project: Bahir > Issue Type: New Feature > Components: Spark SQL Data Sources >Affects Versions: Spark-2.1.0 >Reporter: Subhobrata Dey > > Hello, > This issue is created to propose the addition of Akka compatible streaming > source for Spark SQL Streaming. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-97) Akka as a streaming source for SQL Streaming.
[ https://issues.apache.org/jira/browse/BAHIR-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951502#comment-15951502 ] ASF GitHub Bot commented on BAHIR-97: - Github user sbcd90 commented on the issue: https://github.com/apache/bahir/pull/38 retest this please > Akka as a streaming source for SQL Streaming. > - > > Key: BAHIR-97 > URL: https://issues.apache.org/jira/browse/BAHIR-97 > Project: Bahir > Issue Type: New Feature > Components: Spark SQL Data Sources >Affects Versions: Spark-2.1.0 >Reporter: Subhobrata Dey > > Hello, > This issue is created to propose the addition of Akka compatible streaming > source for Spark SQL Streaming. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BAHIR-101) Add SparkSQL for Cloudant
[ https://issues.apache.org/jira/browse/BAHIR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951300#comment-15951300 ] ASF GitHub Bot commented on BAHIR-101: -- Github user lresende commented on the issue: https://github.com/apache/bahir/pull/39 Thank you very much @yanglei99, build is passing, samples and readme looks good... LGTM, we can take care of tests on a new PR. Will merge if there are no more comments > Add SparkSQL for Cloudant > -- > > Key: BAHIR-101 > URL: https://issues.apache.org/jira/browse/BAHIR-101 > Project: Bahir > Issue Type: Improvement >Reporter: Yang Lei > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
