[jira] [Commented] (SOLR-9963) Add Calcite Avatica handler to Solr
[ https://issues.apache.org/jira/browse/SOLR-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959656#comment-15959656 ] Kevin Risden commented on SOLR-9963: [~elserj] - Just pinging you since you might be somewhat interested in this. Avatica was very easy to hook into Solr and should work with ODBC/JDBC drivers. It would be simple to switch out to protobuf instead of json too. > Add Calcite Avatica handler to Solr > --- > > Key: SOLR-9963 > URL: https://issues.apache.org/jira/browse/SOLR-9963 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Reporter: Kevin Risden >Assignee: Kevin Risden > Attachments: SOLR-9963.patch, SOLR-9963.patch, SOLR-9963.patch, > test_avatica_solr.sh > > > Calcite Avatica has an http endpoint which allows Avatica drivers to connect > to the server. This can be wired in as a handler to Solr. This would allow > Solr to be used by any Avatica JDBC/ODBC driver. This depends on the Calcite > work from SOLR-8593. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9963) Add Calcite Avatica handler to Solr
[
https://issues.apache.org/jira/browse/SOLR-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959653#comment-15959653
]
Kevin Risden commented on SOLR-9963:
[~joel.bernstein] - I had some time to dig into this. The response is formatted
for Avatica JSON as documented here:
https://calcite.apache.org/avatica/docs/json_reference.html
The result is actually streamed back as requested based on if its a single
fetch or multiple fetches. Here is a script I wrote to demonstrate the
streaming of results back.
{code}
./test_avatica_solr.sh "http://localhost:8983/solr/test/avatica; "select * from
test limit 10"
{code}
{code}
#!/usr/bin/env bash
set -u
#set -x
AVATICA=$1
SQL=$2
CONNECTION_ID="conn-$(whoami)-$(date +%s)"
MAX_ROW_COUNT=100
NUM_ROWS=2
OFFSET=0
echo "Open connection"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data
"{\"request\": \"openConnection\",\"connectionId\": \"${CONNECTION_ID}\"}"
# Example of how to set connection properties with info key
#curl -i "$AVATICA" -H "Content-Type: application/json" --data "{\"request\":
\"openConnection\",\"connectionId\": \"${CONNECTION_ID}\",\"info\": {\"zk\":
\"$ZK\",\"lex\": \"MYSQL\"}}"
echo
echo "Create statement"
STATEMENTRSP=$(curl -s "$AVATICA" -H "Content-Type: application/json" --data
"{\"request\": \"createStatement\",\"connectionId\": \"${CONNECTION_ID}\"}")
STATEMENTID=$(echo "$STATEMENTRSP" | jq .statementId)
echo
echo "PrepareAndExecuteRequest"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data
"{\"request\": \"prepareAndExecute\",\"connectionId\":
\"${CONNECTION_ID}\",\"statementId\": $STATEMENTID,\"sql\":
\"$SQL\",\"maxRowCount\": ${MAX_ROW_COUNT}, \"maxRowsInFirstFrame\":
${NUM_ROWS}}"
echo
# Loop through all the results
ISDONE=false
while ! $ISDONE; do
OFFSET=$((OFFSET + NUM_ROWS))
echo "FetchRequest - Offset=$OFFSET"
FETCHRSP=$(curl -s "$AVATICA" -H "Content-Type: application/json" --data
"{\"request\": \"fetch\",\"connectionId\":
\"${CONNECTION_ID}\",\"statementId\": $STATEMENTID,\"offset\":
${OFFSET},\"fetchMaxRowCount\": ${NUM_ROWS}}")
echo "$FETCHRSP"
ISDONE=$(echo "$FETCHRSP" | jq .frame.done)
echo
done
echo "Close statement"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data
"{\"request\": \"closeStatement\",\"connectionId\":
\"${CONNECTION_ID}\",\"statementId\": $STATEMENTID}"
echo
echo "Close connection"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data
"{\"request\": \"closeConnection\",\"connectionId\": \"${CONNECTION_ID}\"}"
echo
{code}
The script is also being attached to this ticket.
> Add Calcite Avatica handler to Solr
> ---
>
> Key: SOLR-9963
> URL: https://issues.apache.org/jira/browse/SOLR-9963
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Parallel SQL
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9963.patch, SOLR-9963.patch, SOLR-9963.patch
>
>
> Calcite Avatica has an http endpoint which allows Avatica drivers to connect
> to the server. This can be wired in as a handler to Solr. This would allow
> Solr to be used by any Avatica JDBC/ODBC driver. This depends on the Calcite
> work from SOLR-8593.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9963) Add Calcite Avatica handler to Solr
[ https://issues.apache.org/jira/browse/SOLR-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869881#comment-15869881 ] Kevin Risden commented on SOLR-9963: The response is raw from the Avatica handler. Right now this is using the JSON response from Avatica and outputting exactly as Avatica server would. If Avatica supports streaming this should too. > Add Calcite Avatica handler to Solr > --- > > Key: SOLR-9963 > URL: https://issues.apache.org/jira/browse/SOLR-9963 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Reporter: Kevin Risden >Assignee: Kevin Risden > Attachments: SOLR-9963.patch, SOLR-9963.patch, SOLR-9963.patch > > > Calcite Avatica has an http endpoint which allows Avatica drivers to connect > to the server. This can be wired in as a handler to Solr. This would allow > Solr to be used by any Avatica JDBC/ODBC driver. This depends on the Calcite > work from SOLR-8593. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9963) Add Calcite Avatica handler to Solr
[ https://issues.apache.org/jira/browse/SOLR-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864911#comment-15864911 ] Joel Bernstein commented on SOLR-9963: -- I was reading through the patch. I'm not sure I fully understand it yet. It appears that the response is a String. I suspect there is a way we can make the response stream. > Add Calcite Avatica handler to Solr > --- > > Key: SOLR-9963 > URL: https://issues.apache.org/jira/browse/SOLR-9963 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Reporter: Kevin Risden >Assignee: Kevin Risden > Attachments: SOLR-9963.patch, SOLR-9963.patch, SOLR-9963.patch > > > Calcite Avatica has an http endpoint which allows Avatica drivers to connect > to the server. This can be wired in as a handler to Solr. This would allow > Solr to be used by any Avatica JDBC/ODBC driver. This depends on the Calcite > work from SOLR-8593. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
