Paul, The trigger_dag cli command accepts a json "conf" parameter. To be honest I'm not familiar with the feature (perhaps Sid can provide more detail) but I think it might accomplish your goals.
J On Fri, Jul 8, 2016 at 1:11 PM Paul Minton <[email protected]> wrote: > +1 to this feature as well. > > I wonder if it would be possible to pass along some context when triggering > a job. This may be outside the scope of this thread, but it would allow for > more dynamic runs of a job. As a simple example, I may want to kick off a > job and pass along the key to a file on s3. Right now we would depend on a > initial s3 sensor, but that would require that the filename be static > across runs. > > On Thu, Jul 7, 2016 at 9:55 AM, Chris Riccomini <[email protected]> > wrote: > > > +1 > > > > On Thu, Jul 7, 2016 at 5:18 AM, Bolke de Bruin <[email protected]> > wrote: > > > > > Ideally the CLI and WebUI should both access an API that handles > > > authentication and authorization. This would resolve both issues. > > However, > > > the UI already allows for authentication and to a lesser extent > > > authorization. Thus allowing this from the UI (which we already do for > > > Celery) is not a big change. > > > > > > - Bolke > > > > > > > > > > Op 7 jul. 2016, om 11:01 heeft Alexander Alten-Lorenz < > > > [email protected]> het volgende geschreven: > > > > > > > > Sounds good, but on the other hand I'm with Maxime. Given that the > task > > > can be triggered per CLI, the functionality is available but needs a > > local > > > login. When the "run" button now would be available for everyone who > has > > > access to the UI, I can imagine that would cause some serious load > issues > > > in a production environment, especially with SLA based workflow setups. > > > > On the other hand, when the "run" button with a local executor would > > > queue the task in a control queue (like "external triggered") a admin > > could > > > finally mark them as "approved". > > > > > > > > --alex > > > > > > > >> On Jul 7, 2016, at 12:12 AM, Jeremiah Lowin <[email protected]> > > wrote: > > > >> > > > >> Perhaps it's a good chance to revisit the functionality. Right now > the > > > UI > > > >> "run" button actually runs the task via CeleryExecutor. Perhaps > > instead > > > (or > > > >> just when using a non-Celery executor) it should queue the task and > > let > > > the > > > >> Scheduler pick it up. I guess in that case it would just be sugar > for > > > >> marking a TI as QUEUED. Just a thought. > > > >> > > > >> On Wed, Jul 6, 2016 at 2:54 AM Maxime Beauchemin < > > > [email protected]> > > > >> wrote: > > > >> > > > >>> Hi, > > > >>> > > > >>> The problem is that a web server isn't the right place to run an > > > airflow > > > >>> task. From the context of the web request scope we have to somehow > > > pass a > > > >>> message to an external executor to run the task. For LocalExecutor > to > > > work > > > >>> the web server would have to start a LocalExecutor as a sub process > > and > > > >>> that doesn't sound like a great idea... > > > >>> > > > >>> Max > > > >>> > > > >>> On Tue, Jul 5, 2016 at 11:22 AM, Jason Chen < > > [email protected] > > > > > > > >>> wrote: > > > >>> > > > >>>> Hi Airflow team, > > > >>>> I am using the "LocalExecutor" and it works very well to run the > > > >>> workflow > > > >>>> I setup. > > > >>>> > > > >>>> I noticed that, from the UI, it can trigger a task to run. > > > >>>> However, I got the error "Only works with the CeleryExecutor, > sorry > > ". > > > >>>> I can ssh into airflow node and run the command line from there. > > > >>>> However, it would be nice to just run it from airflow UI. > > > >>>> Is it possible to do that (with "LocalExecutor") or it's a future > > > feature > > > >>>> to consider ? > > > >>>> > > > >>>> Thanks. > > > >>>> Jason > > > >>>> > > > >>> > > > > > > > > > > > > >
