[
https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427848#comment-15427848
]
Takeshi Yamamuro commented on SPARK-15816:
------------------------------------------
About 1.
Yea, we can support them in there; nested structs and maps are handled as
json-fomatted strings.
Array formats also are handled as strings like '{x,x,x}'.
See tests for struct/map:
https://github.com/apache/spark/compare/master...maropu:SPARK-15816#diff-8965860d928dfaf214eaf6a3f724b961R339
array:
https://github.com/apache/spark/compare/master...maropu:SPARK-15816#diff-8965860d928dfaf214eaf6a3f724b961R234
About 2.
ISTM it is not impossible though, what's the purpose to support a https mode?
> SQL server based on Postgres protocol
> -------------------------------------
>
> Key: SPARK-15816
> URL: https://issues.apache.org/jira/browse/SPARK-15816
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Reynold Xin
> Attachments: New_SQL_Server_for_Spark.pdf
>
>
> At Spark Summit today this idea came up from a discussion: it would be great
> to investigate the possibility of implementing a new SQL server using
> Postgres' protocol, in lieu of Hive ThriftServer 2. I'm creating this ticket
> to track this idea, in case others have feedback.
> This server can have a simpler architecture, and allows users to leverage a
> wide range of tools that are already available for Postgres (and many
> commercial database systems based on Postgres).
> Some of the problems we'd need to figure out are:
> 1. What is the Postgres protocol? Is there an official documentation for it?
> 2. How difficult would it be to implement that protocol in Spark (JVM in
> particular).
> 3. How does data type mapping work?
> 4. How does system commands work? Would Spark need to support all of
> Postgres' commands?
> 5. Any restrictions in supporting nested data?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
