[
https://issues.apache.org/jira/browse/SPARK-20248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962359#comment-15962359
]
Apache Spark commented on SPARK-20248:
--------------------------------------
User 'shaolinliu' has created a pull request for this issue:
https://github.com/apache/spark/pull/17581
> Spark SQL add limit parameter to enhance the reliability.
> ---------------------------------------------------------
>
> Key: SPARK-20248
> URL: https://issues.apache.org/jira/browse/SPARK-20248
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.1.0
> Environment: 2.1.0
> Reporter: shaolinliu
> Priority: Minor
>
> When we using thrift server, it is difficult to constrain the user's sql
> statement;
> When the user query a large table without limit, this will lead to thrift
> server process memory occupancy lead to service instability;
> In general, the user is not used correctly, because if you really need to
> return the whole table:
> 1, if you use this data to compute , you can complete the computation in
> the cluster and then return
> 2, if you want obtain the data, you can store it in hdfs
> For the above scene, it is recommended to add a
> "spark.sql.thriftserver.retainedResults" parameter,
> 1, when it is 0, we don not restrict user's operation
> 2, when it is greater than 0, if user query with limit, we use user's
> limit;if not we use this to limit query's result
> Priority user's limit is because, if the user consider the limit, in
> general, the user is aware of the exact meaning of this query
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
