Re: Multiple hosts for a single connection

2018-08-07 Thread Ben Laird
Hi Ash - That'd be perfect. Since conn_id isn't a primary key (since we can add via the UI), I agree that it'd be great to have the same functionality from the CLI with a flag. I can hopefully do this and submit a PR if folks would find it useful! Ben On Tue, Aug 7, 2018 at 10:28 AM, Ash Berlin-

Re: Multiple hosts for a single connection

2018-08-07 Thread Naik Kaxil
Agree that we should have the same functionality over the WebUI and CLI. We need multiple connections for a single conn-id for load balancing, so +1 for adding something like `--allow-multiple` or just a warning when a connection with the same id is created with cli. On 07/08/2018, 15:29, "As

Re: Multiple hosts for a single connection

2018-08-07 Thread Ash Berlin-Taylor
Hmm yes, it appears that the `airflow connections` CLI doesn't let you create multiple connections of the same conn_id. What the WebUI can do the CLI should be able to do also! It should allow that in some way (behind a `--allow-multiple` flag perhaps? I can see an argument for not allowing set

Re: Multiple hosts for a single connection

2018-08-07 Thread Deng Xiaodong
Hi Ben, If you would like to set multiple connections with the same *conn_id*, you can only do that in Web UI. That is, you need to build multiple entries with different host, but with the same *conn_id*. The method of setting connection in environment variables can only help set one connection f

Multiple hosts for a single connection

2018-08-07 Thread Ben Laird
Hello - I'd like to define multiple connections for the `webhdfs_default` conn_id, as we have a primary and backup host for the hadoop namenode. This is for use with the WebHDFSHook, which queries for connections and seemingly can accept a list of them from the database. ``` nn_connection