Re: Access multiple cluster

if the remote filesystem is visible from the other, than a different HDFS value, e.g hdfs://analytics:8000/historical/ can be used for reads & writes, even if your defaultFS (the one where you get max performance) is, say hdfs://processing:8000/ -performance will be slower, in both directions

Re: Access multiple cluster

Thank you guys. I will try JDBC route if I get access and let you know. On Mon, Dec 5, 2016 at 5:17 PM, Jörn Franke wrote: > If you do it frequently then you may simply copy the data to the > processing cluster. Alternatively, you could create an external table in > the

Re: Access multiple cluster

If you do it frequently then you may simply copy the data to the processing cluster. Alternatively, you could create an external table in the processing cluster to the analytics cluster. However, this has to be supported by appropriate security configuration and might be less an efficient then

Re: Access multiple cluster

The only way I think of would be accessing Hive tables through their respective thrift servers running on different clusters but not sure you can do it within Spark. Basically two different JDBC connections. HTH Dr Mich Talebzadeh LinkedIn *

Access multiple cluster

Hi Is it possible to access hive tables sitting on multiple clusters in a single spark application? We have a data processing cluster and analytics cluster. I want to join a table from analytics cluster with another table in processing cluster and finally write back in analytics cluster. Best