Repository: spark Updated Branches: refs/heads/master c0b47bada -> c82fe4781
[SQL] Documentation: Explain cacheTable command add the `cacheTable` specification Author: CrazyJvm <[email protected]> Closes #1681 from CrazyJvm/sql-programming-guide-cache and squashes the following commits: 0a231e0 [CrazyJvm] grammar fixes a04020e [CrazyJvm] modify title to Cached tables 18b6594 [CrazyJvm] fix format 2cbbf58 [CrazyJvm] add cacheTable guide Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c82fe478 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c82fe478 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c82fe478 Branch: refs/heads/master Commit: c82fe4781cd0356bcfdd25c7eadf1da624bb2228 Parents: c0b47ba Author: CrazyJvm <[email protected]> Authored: Fri Aug 1 11:46:13 2014 -0700 Committer: Patrick Wendell <[email protected]> Committed: Fri Aug 1 11:46:14 2014 -0700 ---------------------------------------------------------------------- docs/sql-programming-guide.md | 10 ++++++++++ 1 file changed, 10 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/c82fe478/docs/sql-programming-guide.md ---------------------------------------------------------------------- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index a047d32..7261bad 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -769,3 +769,13 @@ To start the Spark SQL CLI, run the following in the Spark directory: Configuration of Hive is done by placing your `hive-site.xml` file in `conf/`. You may run `./bin/spark-sql --help` for a complete list of all available options. + +# Cached tables + +Spark SQL can cache tables using an in-memory columnar format by calling `cacheTable("tableName")`. +Then Spark SQL will scan only required columns and will automatically tune compression to minimize +memory usage and GC pressure. You can call `uncacheTable("tableName")` to remove the table from memory. + +Note that if you just call `cache` rather than `cacheTable`, tables will _not_ be cached in +in-memory columnar format. So we strongly recommend using `cacheTable` whenever you want to +cache tables.
