'airflow.utils.log.logging_mixin' is not a package"
Have you encountered this before?
On Mon, Dec 18, 2017 at 8:53 PM, Feng Lu <fen...@google.com.invalid> wrote:
> Hi Kevin,
>
> Kindly see my reply inline:
>
> On Mon, Dec 18, 2017 at 3:28 PM, Kevin Lam <ke...@fathomheal
Looks like it might be related to
https://github.com/apache/incubator-airflow/commit/02ff8ae35dd16e6f23d29d7b24a5fb9c09d0b7a4?
Why isn't this fix on the v1-9 branches? Should I be using master instead?
On Tue, Dec 19, 2017 at 5:37 PM, Kevin Lam <ke...@fathomhealth.co> wrote:
>
Hi,
I'm trying to get airflow to use GCS for logging purposes and had a few
questions.
We're currently using Airflow 1.9rc2, running in a Kubernetes Airflow
deployment (similar to https://github.com/mumoshu/kube-airflow)
1/ Seems like the logging code has been going through some changes in the
Note that in 1.8 the behaviour is as expected. Thanks again!
On Fri, Nov 17, 2017 at 1:17 PM, ke...@fathomhealth.co <
ke...@fathomhealth.co> wrote:
> Hi,
>
> I'm encountering a seemingly related issue in 1.9.
>
> I have a custom Airflow Operator that I want to log from and none of the
> outputs
Sorry for spam! To clarify again, the expected behaviour is that the
logging.info calls in the custom operator would be forwarded to the
base_task_runner and then be available for viewing in the task logs in the
Airflow Web UI.
On Fri, Nov 17, 2017 at 1:20 PM, Kevin Lam <ke...@fathomhealth
Hi,
We are running airflow on kubernetes, hosted on Azure, using
https://github.com/mumoshu/kube-airflow. We are also running an autoscaler:
https://github.com/wbuchwalter/Kubernetes-acs-engine-autoscaler.
Whenever the autoscaler increase the agentPool sizes to allocate a new node
in kubernetes,
Sorry to clarify, this is now on master branch.
On Wed, Dec 20, 2017 at 10:25 AM, Kevin Lam <ke...@fathomhealth.co> wrote:
> Thanks Bolke and Feng!
>
> I seem to have a working connection with GCS but it seems there some error
> occuring in the gcs_task_handler in airflow:
&
o/<...>F2017-12-20T15%3A21%3A23.704614%2B00%3A00%2F1.log?alt=media
returned "Not Found">
On Wed, Dec 20, 2017 at 1:48 AM, Bolke de Bruin <bdbr...@gmail.com> wrote:
> Both will/should work, master is just cleaner and more manageable.
>
> B.
>
> Vers
n't match up the line numbers in this stack
> trace to either 1.9.0rc8 or 1.9.0rc2 -- both of which show the 'if old_log
> else log' on line 157
>
> -ash
>
>
> > On 20 Dec 2017, at 15:25, Kevin Lam <ke...@fathomhealth.co> wrote:
> >
> > Thanks Bolke an
Hi airflow folks,
I'm currently using Airflow v1.9 and set up logging to GCS via
http://airflow.readthedocs.io/en/latest/integration.html#gcp. I'm running
into an issue where the entire local log appears to be being written to GCS
are twice, any ideas what can cause this, or how to fix it?
To
Hi all,
My team and I have been experimenting with Airflow and Kubernetes, and
there has been a lot of activity recently with the Kubernetes Executor so
hopefully someone can help us out.
Specifically, we are using our own variant of the kubernetes executor to
run some pods on pre-emptible VMs
:44 AM, Kevin Lam <ke...@fathomhealth.co> wrote:
> I got it to work, it seems i had mismatched some code (
> airflow/config_templates/airflow_local_settings.py) from the master
> branch in the v1-9-stable branch. Thanks for your help everyone!
>
> On Wed, Dec 20, 2017 at 11
Hi,
We run Apache Airflow as a set of k8s deployments inside of a GKE cluster,
similar to the way specified in Mumoshu's github repo:
https://github.com/mumoshu/kube-airflow.
We are investigating securing our use of Airflow and are wondering about
some of Airflow's implementation details.
Hi,
We run airflow as a set of k8s deployments inside of a GKE cluster, similar
to the way specified in Mumoshu's github repo: https://github.com/
mumoshu/kube-airflow.
We are currently investigating securing our clusters using istio [1].
Specifically we are interested in adding additional
Friendly ping :). Do you think you could elaborate on the fault tolerance a
bit, Daniel? Thanks for your help!
On Wed, Sep 12, 2018 at 5:35 PM Kevin Lam wrote:
> Hi Daniel,
>
> Thanks for the reply!
>
> No we haven't looked too deeply into it. Can you elaborate a bit on h
Hi,
We run Apache Airflow in Kubernetes in a manner very similar to what is
outlined in puckel/docker-airflow [1] (Celery Executor, Redis for
messaging, Postgres).
Lately, we've encountered some of our Tasks getting stuck in a running
state, and printing out the errors:
[2018-11-20
Hi all,
We currently run Airflow as a Deployment in a kubernetes cluster. We also
use a variant of KubernetesOperator to run our DAGs.
We are investigating how to best make Airflow fault-tolerant, in part, due
to investigating the use of preemptible vms [1]. *Has there been much
discussion about
, Sep 12, 2018 at 4:59 PM Daniel Imberman
wrote:
> Hi Kevin,
>
> Have you looked into the KubernetesExecutor? We achieve fault tolerance
> using the kubernetes resourceVersion to ensure that all state is
> reproducible.
>
> On Wed, Sep 12, 2018 at 1:08 PM Kevin Lam wrote:
>
18 matches
Mail list logo
