Not to hijack this post but how would you deal with data that is maintained by hive(Orc format file, hive created tables etc..)...Would we copy the hivemetastore(MySQL) and move that over to new cluster?
On Friday, June 19, 2015, Joep Rottinghuis <[email protected]> wrote: > You can't set up a proxy ? > You probably want to avoid writing to local file system because aside from > that being slow, it limits the size of your file to the free space on your > local disc. > > If you do need to go commando and go through a single client machine that > can see both clusters you probably want to pipe a get to a put. > > Any kind of serious data volume pulled through a "straw" is going to be > rather slow though. > > Cheers, > > Joep > > Sent from my iPhone > > On Jun 19, 2015, at 12:09 AM, Nitin Pawar <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > > yes > > On Fri, Jun 19, 2015 at 11:36 AM, Divya Gehlot <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> In thats It will be like three step process . >> 1. first cluster (secure zone) HDFS -> copytoLocal -> user local file >> system >> 2. user local space -> copy data -> second cluster user local file system >> 3. second cluster user local file system -> copyfromlocal -> second >> clusterHDFS >> >> Am I on the right track ? >> >> >> >> On 19 June 2015 at 12:38, Nitin Pawar <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> >>> What's the size of the data? >>> If you can not do distcp between clusters then other way is doing hdfs >>> get on the data and then hdfs put on another cluster >>> On 19-Jun-2015 9:56 am, "Divya Gehlot" <[email protected] >>> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >>> >>>> Hi, >>>> I need to copy data from first hadoop cluster to second hadoop cluster. >>>> I cant access second hadoop cluster from first hadoop cluster due to >>>> some security issue. >>>> Can any point me how can I do apart from distcp command. >>>> For instance >>>> Cluster 1 secured zone -> copy hdfs data to -> cluster 2 in non >>>> secured zone >>>> >>>> >>>> >>>> Thanks, >>>> Divya >>>> >>>> >>>> >> > > > -- > Nitin Pawar > >
