ASF GitHub Bot commented on BAHIR-154:

Github user ricellis commented on a diff in the pull request:

    --- Diff: 
sql-cloudant/src/main/scala/org/apache/bahir/cloudant/CloudantConfig.scala ---
    @@ -16,34 +16,127 @@
     package org.apache.bahir.cloudant
    -import java.net.URLEncoder
    +import java.net.{URL, URLEncoder}
    -import play.api.libs.json.{JsArray, JsObject, Json, JsValue}
    +import scala.collection.JavaConverters._
    +import scala.collection.mutable
    +import scala.reflect.io.File
    +import com.cloudant.client.api.{ClientBuilder, CloudantClient, Database}
    +import com.cloudant.client.api.model.SearchResult
    +import com.cloudant.client.api.views._
    +import com.cloudant.http.{Http, HttpConnection}
    +import com.cloudant.http.interceptors.Replay429Interceptor
    +import com.google.gson.{JsonObject, JsonParser}
     import org.apache.bahir.cloudant.common._
    +import org.apache.bahir.cloudant.common.JsonUtil.JsonConverter
     * Only allow one field pushdown now
     * as the filter today does not tell how to link the filters out And v.s. Or
     class CloudantConfig(val protocol: String, val host: String,
    -                     val dbName: String, val indexName: String, val 
viewName: String)
    +                     val dbName: String, val indexPath: String, val 
viewPath: String)
                         (implicit val username: String, val password: String,
                          val partitions: Int, val maxInPartition: Int, val 
minInPartition: Int,
                          val requestTimeout: Long, val bulkSize: Int, val 
schemaSampleSize: Int,
                          val createDBOnSave: Boolean, val endpoint: String,
                          val useQuery: Boolean = false, val queryLimit: Int)
       extends Serializable {
    +  @transient private lazy val client: CloudantClient = ClientBuilder
    +    .url(getClientUrl)
    +    .username(username)
    +    .password(password)
    +    .interceptors(Replay429Interceptor.WITH_DEFAULTS)
    +    .build
    +  @transient private lazy val database: Database = client.database(dbName, 
       lazy val dbUrl: String = {protocol + "://" + host + "/" + dbName}
    +  lazy val designDoc: String = {
    +    if (viewPath != null && viewPath.nonEmpty) {
    +      viewPath.split("/")(1)
    +    } else {
    +    null
    +    }
    +  }
    +  lazy val searchName: String = {
    +    // verify that the index path matches '_design/ddoc/_search/searchname'
    +    if (indexPath != null && indexPath.nonEmpty && 
indexPath.matches("\\w+\\/\\w+\\/\\w+\\/\\w+")) {
    --- End diff --
    I think you could leverage scala's raw interpreter here to make the regex a 
bit less escape-y:
    Also are word characters sufficient here? Aren't there some other 
characters that could be part of a design document id and allowed unencoded in 
a URL path (e.g. maybe `@`?) that would be excluded by this regex? I'd be 
inclined to maybe use something like: `_design\/(^\/)+\/_search\/(^\/)+`

> Refactor sql-cloudant to use Cloudant's java-cloudant features
> --------------------------------------------------------------
>                 Key: BAHIR-154
>                 URL: https://issues.apache.org/jira/browse/BAHIR-154
>             Project: Bahir
>          Issue Type: Improvement
>    Affects Versions: Spark-2.2.0
>            Reporter: Esteban Laver
>            Assignee: Esteban Laver
> Cloudant's java-cloudant library (which is currently used for testing) 
> contains several features that sql-cloudant can benefit from:
> - HTTP 429 backoff
> - View builder API to potentially simplify loading for _all_docs/views
> - Improved exception handling when executing HTTP requests
> - Future support for IAM API key
> Would need to replace current scala HTTP library with OkHttp library, and 
> also replace play-json with GSON library.

This message was sent by Atlassian JIRA

Reply via email to