[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-04-03 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809249#comment-16809249
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

asfgit commented on pull request #75: [BAHIR-186] Reduce size of sql-cloudant 
test database
URL: https://github.com/apache/bahir/pull/75
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Assignee: Lukasz Antoniak
>Priority: Major
> Fix For: Spark-2.4.0
>
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-04-03 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808993#comment-16808993
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

emlaver commented on issue #75: [BAHIR-186] Reduce size of sql-cloudant test 
database
URL: https://github.com/apache/bahir/pull/75#issuecomment-479596755
 
 
   @ckadner This issue that you've witnessed is related to the core service and 
not this library.  The Cloudant team is aware of this issue.  We should be OK 
to merge this.  I can add open an issue if needed with additional details.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Assignee: Lukasz Antoniak
>Priority: Major
> Fix For: Spark-2.4.0
>
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-04-03 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808985#comment-16808985
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

emlaver commented on issue #75: [BAHIR-186] Reduce size of sql-cloudant test 
database
URL: https://github.com/apache/bahir/pull/75#issuecomment-479596755
 
 
   @ckadner This issue that you've witnessed is related to the Cloudant core 
service and not this library.  We should be OK to merge this.  I can add open 
an issue if needed with additional details.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Assignee: Lukasz Antoniak
>Priority: Major
> Fix For: Spark-2.4.0
>
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-02-04 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760364#comment-16760364
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

ckadner commented on issue #75: [BAHIR-186] Reduce size of sql-cloudant test 
database
URL: https://github.com/apache/bahir/pull/75#issuecomment-460471642
 
 
   @emlaver -- your changes look good and help speeding up the test execution 👍 
   
   I do have an issue that may predate your changes though. After running `mvn 
clean install` and a few subsequent `mvn test -pl sql-cloudant` I keep running 
into the following test error in `CloudantChangesDFSuite`:
   
   ```
   - load data and verify deleted doc is not in results *** FAILED ***
 org.apache.bahir.cloudant.common.CloudantException: Error retrieving 
_changes feed data from database 'n_flight' with response code 401: 
{"error":"unauthorized","reason":"Name or password is incorrect."}
   ```
   
   When I run only the `CloudantChangesDFSuite` independently I do not get that 
error.
   ``` 
   $ mvn scalatest:test -pl sql-cloudant 
-Dsuites='org.apache.bahir.cloudant.CloudantChangesDFSuite'
   
   Run starting. Expected test count is: 9
   CloudantChangesDFSuite:
   
   Sql-cloudant tests that require Cloudant databases have been enabled by
   the environment variables CLOUDANT_USER and CLOUDANT_PASSWORD.
   
   - load and save data from Cloudant database
   - load and count data from Cloudant search index
   - load data and verify deleted doc is not in results
   - load data and count rows in filtered dataframe
   - save filtered dataframe to database
   - save dataframe to database using createDBOnSave=true option
   - load and count data from view
   - load data from view with MapReduce function
   - load data and verify total count of selector, filter, and view option
   Run completed in 2 minutes, 40 seconds.
   Total number of tests run: 9
   Suites: completed 1, aborted 0
   Tests: succeeded 9, failed 0, canceled 0, ignored 0, pending 0
   All tests passed.
   ```
   
   Maven log (console output) for all unit tests in SQL-Cloudant:
   ```
   $ mvn test -pl sql-cloudant
   
   ...
   [INFO] --- scalatest-maven-plugin:1.0:test (test) @ spark-sql-cloudant_2.11 
---
   Discovery starting.
   
   Sql-cloudant tests that require Cloudant databases have been enabled by
   the environment variables CLOUDANT_USER and CLOUDANT_PASSWORD.

   ...
   
   CloudantChangesDFSuite:
   - load and save data from Cloudant database
   - load and count data from Cloudant search index
   - load data and verify deleted doc is not in results *** FAILED ***
 org.apache.bahir.cloudant.common.CloudantException: Error retrieving 
_changes feed data from database 'n_flight' with response code 401: 
{"error":"unauthorized","reason":"Name or password is incorrect."}
 at org.apache.bahir.cloudant.DefaultSource.create(DefaultSource.scala:162)
 at 
org.apache.bahir.cloudant.DefaultSource.createRelation(DefaultSource.scala:95)
 at 
org.apache.bahir.cloudant.DefaultSource.createRelation(DefaultSource.scala:87)
 at 
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:341)
 at 
org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:239)
 at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:227)
 at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:174)
 at 
org.apache.bahir.cloudant.CloudantChangesDFSuite$$anonfun$8.apply$mcV$sp(CloudantChangesDFSuite.scala:67)
 at 
org.apache.bahir.cloudant.CloudantChangesDFSuite$$anonfun$8.apply(CloudantChangesDFSuite.scala:62)
 at 
org.apache.bahir.cloudant.CloudantChangesDFSuite$$anonfun$8.apply(CloudantChangesDFSuite.scala:62)
 ...
   - load data and count rows in filtered dataframe
   - save filtered dataframe to database
   - save dataframe to database using createDBOnSave=true option
   - load and count data from view
   - load data from view with MapReduce function
   - load data and verify total count of selector, filter, and view option
   
   ...
   
   Run completed in 3 minutes, 34 seconds.
   Total number of tests run: 25
   Suites: completed 6, aborted 0
   Tests: succeeded 24, failed 1, canceled 0, ignored 0, pending 0
   *** 1 TEST FAILED ***
   [INFO] 

   [INFO] BUILD FAILURE
   ```
   
   However, when I take a look at the `sql-cloudant/target/unit-tests.log` I 
don't find any errors logged during the execution of that test case:
   ```
   = TEST OUTPUT FOR org.apache.bahir.cloudant.CloudantChangesDFSuite: 
'load data and verify deleted doc is not in results' =
   
   19/02/04 15:07:11.114 ScalaTest-main-running-CloudantChangesDFSuite INFO 
SharedState: loading hive config file: 
jar:file:/Users/ckadner/.m2/repository/org/apache/spark/spark-sql_2.11/2.3.2/sp

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-02-04 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760354#comment-16760354
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

ckadner commented on issue #75: [BAHIR-186] Reduce size of sql-cloudant test 
database
URL: https://github.com/apache/bahir/pull/75#issuecomment-460471642
 
 
   @emlaver -- your changes look good and help speeding up the test execution 👍 
   
   I do have an issue that predates your changes though. After running `mvn 
clean install` and a few subsequent `mvn test -pl sql-cloudant` I keep running 
into the following test error in `CloudantChangesDFSuite`:
   
   ```
   - load data and verify deleted doc is not in results *** FAILED ***
 org.apache.bahir.cloudant.common.CloudantException: Error retrieving 
_changes feed data from database 'n_flight' with response code 401: 
{"error":"unauthorized","reason":"Name or password is incorrect."}
   ```
   
   When I run only the `CloudantChangesDFSuite` independently I do not get that 
error.
   ``` 
   $ mvn scalatest:test -pl sql-cloudant 
-Dsuites='org.apache.bahir.cloudant.CloudantChangesDFSuite'
   
   Run starting. Expected test count is: 9
   CloudantChangesDFSuite:
   
   Sql-cloudant tests that require Cloudant databases have been enabled by
   the environment variables CLOUDANT_USER and CLOUDANT_PASSWORD.
   
   - load and save data from Cloudant database
   - load and count data from Cloudant search index
   - load data and verify deleted doc is not in results
   - load data and count rows in filtered dataframe
   - save filtered dataframe to database
   - save dataframe to database using createDBOnSave=true option
   - load and count data from view
   - load data from view with MapReduce function
   - load data and verify total count of selector, filter, and view option
   Run completed in 2 minutes, 40 seconds.
   Total number of tests run: 9
   Suites: completed 1, aborted 0
   Tests: succeeded 9, failed 0, canceled 0, ignored 0, pending 0
   All tests passed.
   ```
   
   Maven log (console output) for all unit tests in SQL-Cloudant:
   ```
   $ mvn test -pl sql-cloudant
   
   ...
   [INFO] --- scalatest-maven-plugin:1.0:test (test) @ spark-sql-cloudant_2.11 
---
   Discovery starting.
   
   Sql-cloudant tests that require Cloudant databases have been enabled by
   the environment variables CLOUDANT_USER and CLOUDANT_PASSWORD.

   ...
   
   CloudantChangesDFSuite:
   - load and save data from Cloudant database
   - load and count data from Cloudant search index
   - load data and verify deleted doc is not in results *** FAILED ***
 org.apache.bahir.cloudant.common.CloudantException: Error retrieving 
_changes feed data from database 'n_flight' with response code 401: 
{"error":"unauthorized","reason":"Name or password is incorrect."}
 at org.apache.bahir.cloudant.DefaultSource.create(DefaultSource.scala:162)
 at 
org.apache.bahir.cloudant.DefaultSource.createRelation(DefaultSource.scala:95)
 at 
org.apache.bahir.cloudant.DefaultSource.createRelation(DefaultSource.scala:87)
 at 
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:341)
 at 
org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:239)
 at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:227)
 at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:174)
 at 
org.apache.bahir.cloudant.CloudantChangesDFSuite$$anonfun$8.apply$mcV$sp(CloudantChangesDFSuite.scala:67)
 at 
org.apache.bahir.cloudant.CloudantChangesDFSuite$$anonfun$8.apply(CloudantChangesDFSuite.scala:62)
 at 
org.apache.bahir.cloudant.CloudantChangesDFSuite$$anonfun$8.apply(CloudantChangesDFSuite.scala:62)
 ...
   - load data and count rows in filtered dataframe
   - save filtered dataframe to database
   - save dataframe to database using createDBOnSave=true option
   - load and count data from view
   - load data from view with MapReduce function
   - load data and verify total count of selector, filter, and view option
   
   ...
   
   Run completed in 3 minutes, 34 seconds.
   Total number of tests run: 25
   Suites: completed 6, aborted 0
   Tests: succeeded 24, failed 1, canceled 0, ignored 0, pending 0
   *** 1 TEST FAILED ***
   [INFO] 

   [INFO] BUILD FAILURE
   ```
   
   However, when I take a look at the `sql-cloudant/target/unit-tests.log` I 
don't find any errors logged during the execution of that test case:
   ```
   = TEST OUTPUT FOR org.apache.bahir.cloudant.CloudantChangesDFSuite: 
'load data and verify deleted doc is not in results' =
   
   19/02/04 15:07:11.114 ScalaTest-main-running-CloudantChangesDFSuite INFO 
SharedState: loading hive config file: 
jar:file:/Users/ckadner/.m2/repository/org/apache/spark/spark-sql_2.11/2.3.2/spark

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-02-04 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760190#comment-16760190
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

emlaver commented on issue #75: [BAHIR-186] Reduce size of sql-cloudant test 
database
URL: https://github.com/apache/bahir/pull/75#issuecomment-460401357
 
 
   @ckadner Would you be able to review?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Assignee: Lukasz Antoniak
>Priority: Major
> Fix For: Spark-2.4.0
>
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-01-15 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743209#comment-16743209
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

Github user emlaver commented on the issue:

https://github.com/apache/bahir/pull/75
  
@lresende Ok, rebased in 42d9ec2.


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Assignee: Lukasz Antoniak
>Priority: Major
> Fix For: Spark-2.4.0
>
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2019-01-15 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743179#comment-16743179
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

Github user lresende commented on the issue:

https://github.com/apache/bahir/pull/75
  
@emlaver could you please rebase and force a new build


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Assignee: Lukasz Antoniak
>Priority: Major
> Fix For: Spark-2.4.0
>
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2018-12-11 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718377#comment-16718377
 ] 

ASF subversion and git services commented on BAHIR-186:
---

Commit a73ab48a2dfec866b2ffa0ccf0d2bfeaba6fc782 in bahir's branch 
refs/heads/master from [~lukasz.antoniak]
[ https://git-wip-us.apache.org/repos/asf?p=bahir.git;h=a73ab48 ]

[BAHIR-186] SSL support in MQTT structured streaming

Closes #74


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Priority: Major
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2018-12-11 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718378#comment-16718378
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

Github user asfgit closed the pull request at:

https://github.com/apache/bahir/pull/74


> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Priority: Major
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2018-12-10 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16716032#comment-16716032
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

GitHub user emlaver opened a pull request:

https://github.com/apache/bahir/pull/75

[BAHIR-186] 

_What_
Reduce the number of documents from 1967 to 100 in the n_flight.json test 
file.  This should shorten the time to finish the `sql-cloudant` tests against 
the remote database.

_How_
- Reduced JSON test files to shorten time to complete test cases
- Added missing protocol for spark config

See [BAHIR-187](https://issues.apache.org/jira/browse/BAHIR-187)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/emlaver/bahir reduce-test-db-file-size

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bahir/pull/75.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #75


commit 4f0bab661662adf988c100b2699bda8be6b5ab48
Author: Esteban Laver 
Date:   2018-12-11T02:46:33Z

Reduced JSON test files to shorten time to complete test cases
- Added missing protocol for spark config




> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Priority: Major
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-186) Support SSL connection in MQTT SQL Streaming

2018-12-05 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/BAHIR-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710614#comment-16710614
 ] 

ASF GitHub Bot commented on BAHIR-186:
--

GitHub user lukasz-antoniak opened a pull request:

https://github.com/apache/bahir/pull/74

[BAHIR-186] SSL support in MQTT structured streaming

JIRA ticket: https://issues.apache.org/jira/browse/BAHIR-186.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lukasz-antoniak/bahir BAHIR-186

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bahir/pull/74.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #74


commit 0a53c75d19676158de6e81da9669969f8271f569
Author: Lukasz Antoniak 
Date:   2018-12-05T20:44:28Z

[BAHIR-186] SSL support in MQTT structured streaming




> Support SSL connection in MQTT SQL Streaming
> 
>
> Key: BAHIR-186
> URL: https://issues.apache.org/jira/browse/BAHIR-186
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Lukasz Antoniak
>Priority: Major
>
> Mailing list discussion: 
> https://www.mail-archive.com/user@bahir.apache.org/msg00022.html.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)