Temporary tables are created in temp file space within the session. Once
the session is closed then the temporary table goes
scala> rs.registerTempTable("mytemp")
And this is the temporary file created with the above command
drwx------ - hduser supergroup 0 2016-03-11 17:09
/tmp/hive/hduser/a1a80c24-213a-41e0-bbcb-75faf38597c3
drwx------ - hduser supergroup 0 2016-03-11 17:09
/tmp/hive/hduser/a1a80c24-213a-41e0-bbcb-75faf38597c3/_tmp_space.db
Now if I open a second session, run the same query and create another
temporary table as before
scala> rs.registerTempTable("mytemp")
I would get
drwx------ - hduser supergroup 0 2016-03-11 17:22
/tmp/hive/hduser/cfc7f73c-cfc9-4cc0-a09d-823b07051cbd
drwx------ - hduser supergroup 0 2016-03-11 17:22
/tmp/hive/hduser/cfc7f73c-cfc9-4cc0-a09d-823b07051cbd/_tmp_space.db
These two tables reference different files so they never collide and they
are private to the session.
If you are familiar with #tables in Sybase or SQL Server, these temporary
tables in Spark play the same role. They are valid within that session and
private to that session
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 11 March 2016 at 15:42, ram kumar <[email protected]> wrote:
> No, I am not aware of it.
>
> Can you provide me with the details regarding this.
>
> Thanks
>
> On Fri, Mar 11, 2016 at 8:25 PM, Ted Yu <[email protected]> wrote:
>
>> temporary tables are associated with SessionState which is used
>> by SQLContext.
>>
>> Did you keep the session ?
>>
>> Cheers
>>
>> On Fri, Mar 11, 2016 at 5:02 AM, ram kumar <[email protected]>
>> wrote:
>>
>>> Hi,
>>>
>>> I registered a dataframe as a table using registerTempTable
>>> and I didn't close the Spark context.
>>>
>>> Will the table be available for longer time?
>>>
>>> Thanks
>>>
>>
>>
>
