I was just looking at that… Out of curiosity… if you make it a Hive Temp Table… who has access to the data?
Just your app, or anyone with access to the same database? (Would you be able to share data across different JVMs? ) (E.G - I have a reader who reads from source A that needs to publish the data to a bunch of minions (B) ) Would this be an option? Thx -Mike > On Feb 15, 2016, at 7:54 AM, Mich Talebzadeh > <mich.talebza...@cloudtechnologypartners.co.uk> wrote: > >> Hi, >> >> >> It is my understanding that the registered temporary tables created by >> registerTempTable() used in Spark shell built on ORC files? >> >> For example the following Data Frame just creates a logical abstraction >> >> scala> var s = HiveContext.sql("SELECT AMOUNT_SOLD, TIME_ID, CHANNEL_ID FROM >> oraclehadoop.sales") >> s: org.apache.spark.sql.DataFrame = [AMOUNT_SOLD: decimal(10,0), TIME_ID: >> timestamp, CHANNEL_ID: bigint] >> >> Then I registar this data frame as temporary table using registerTempTable() >> call >> >> s.registerTempTable("t_s") >> >> Also I believe that s.registerTempTable("t_s") creates an in-memory table >> that is scoped to the cluster in which it was created. The data is stored >> using Hive's ORC format and this tempTable is stored in memory on all nodes >> of the cluster? In other words every node in the cluster has a copy of >> tempTable in its memory? >> >> Thanks, >> >> >> -- >> Dr Mich Talebzadeh >> >> LinkedIn >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> >> http://talebzadehmich.wordpress.com >> >> NOTE: The information in this email is proprietary and confidential. This >> message is for the designated recipient only, if you are not the intended >> recipient, you should destroy it immediately. Any information in this >> message shall not be understood as given or endorsed by Cloud Technology >> Partners Ltd, its subsidiaries or their employees, unless expressly so >> stated. It is the responsibility of the recipient to ensure that this email >> is virus free, therefore neither Cloud Technology partners Ltd, its >> subsidiaries nor their employees accept any responsibility. >> > > > -- > Dr Mich Talebzadeh > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > http://talebzadehmich.wordpress.com > > NOTE: The information in this email is proprietary and confidential. This > message is for the designated recipient only, if you are not the intended > recipient, you should destroy it immediately. Any information in this message > shall not be understood as given or endorsed by Cloud Technology Partners > Ltd, its subsidiaries or their employees, unless expressly so stated. It is > the responsibility of the recipient to ensure that this email is virus free, > therefore neither Cloud Technology partners Ltd, its subsidiaries nor their > employees accept any responsibility. >