Hi Mihai,
Looking to the error which you have got, it looks like that Falcon could not
able to validate the execute services of BigInsights.
To mirror the data from BigInsights (source) to HDP sandbox(target), you can
try to use Falcon HDFS replication recipe. To use Falcon HDFS replication
recipe, run it on target i.e HDP sandbox.1. Define cluster entity with services
from HDP sandbox (target) and submit on target.2. In
hdfs-replication.properties of recipe , specify source HDFS address with input
data path from BigInsights and target HDFS address with output data path. Also
in properties, specify that you want to run recipe on target for attribute
"falcon.recipe.cluster.name".
For more information on usage of HDFS replication recipe, please refer
following doc:http://falcon.apache.org/Recipes.html
I hope this will help you to mirror your data.
Please let us know, if you have any questions.
Thanks,---Peeyush
On Tuesday, 29 September 2015 7:37 AM, Mihai Popa <[email protected]>
wrote:
Hi,
I've been trying to setup mirroring between two clusters.
One runs BigInsights 3.0.2, the other Hortonworks 2.3.
I'm running Falcon on the Hortonworks cluster (which is actually just a
sandbox).
I have no trouble defining the target cluster (on the localhost), but cannot
define the source cluster (on the remote BigInsights install).
The problem is with the "execute" interface; I've tried a series of ports
starting with 9000 which is the default namenode, 9001, 50030.
<interface type='execute' endpoint='bivm.ibm.com:50030' version='2.2.0'/>
(I do have a DNS entry for bivm.ibm.com)
The error that I get is:
org.apache.falcon.entity.parser.ValidationException: Invalid Execute server or
port: bivm.ibm.com:50030
It seems like this version of BIgInsights does not come with Yarn, and from
what I see, the execute interface tries to connect to yarn's port.
Any insight (pun intended) is greatly appreciated.
thank you in advance
mihai popa
