Re: one gobblin cluster to another gobblin cluster movement

2019-03-26 Thread Jayesh Senjaliya
Yes, exactly Shirshanka. typically companies with large infrastructure will have some levels of security zones in addition to different datacenter. if there is a source or target in higher security zone, you would have to open firewall for each source or target which could run in 100s or 1000s

Re: one gobblin cluster to another gobblin cluster movement

2019-03-26 Thread Shirshanka Das
Hi Jay, That's an interesting idea. Can you describe the use-case a bit more? Do you have a use-case where you are pulling from a source into a specific data-center and then need to ship data to the destination from a different data-center or security zone? So are you proposing a pipeline

one gobblin cluster to another gobblin cluster movement

2019-03-25 Thread Jay Sen
Hi Gobblin Devs, is there a way to do data movement between Gobblin clusters to support read/write of data across data centers or security zones? so that firewall/encryption and other tight measures only requires between Gobblin clusters and not from/to all source/targets ? Thanks Jay