Livy's shared object mechanism mainly used to share objects between
different Livy jobs, this is mainly used for Job API. For example job A
create a object Foo which wants to be accessed by Job B, then user could
store this object Foo into JobContext with a provided name, after that Job
B could get this object by the name.
This is different from Spark's cache mechanism. What you mentioned above
(tmp table) is a Spark provided table cache mechanism, which is unrelated
Wandong Wu <wu...@husky.neu.edu> 于2018年7月11日周三 下午5:46写道：
> Dear Sir or Madam:
> I am a Livy beginner. I use Livy, because within an interactive
> session, different spark jobs could share cached RDDs or DataFrames.
> When I read some parquet files and create a table called “TmpTable”.
> The following queries will use this table. Does it mean this table has been
> If cached, where is the table cached? The table is cached in Livy or
> Spark cluster?
> Spark also supports cache function. When I read some parquet files
> and create a table called “TmpTable2”. I add such code: sql_ctx.cacheTable(
> In the next query using this table. It will be cached in Spark
> cluster. Then the following queries could use this cached table.
> What is the difference between cached in Livy and cached in Spark