[jira] [Commented] (AIRFLOW-1229) Make "Run Id" column clickable in Browse -> DAG Runs
[ https://issues.apache.org/jira/browse/AIRFLOW-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259852#comment-16259852 ] Alexander Bij commented on AIRFLOW-1229: I was working on the same issue and created a pull-request on Airflow. And i've added link to a the run-id which I found in your patch attachment. > Make "Run Id" column clickable in Browse -> DAG Runs > > > Key: AIRFLOW-1229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1229 > Project: Apache Airflow > Issue Type: Improvement > Components: webapp >Affects Versions: Airflow 1.8 > Environment: Python3.4 >Reporter: Erik Cederstrand >Assignee: Alexander Bij > Labels: patch > Attachments: dag_run_link.patch > > > I'm triggering a lot of DAGs manually using "airflow trigger_dag my_dag > --run_id=some_unique_id". I would like to be able in the UI to browse easily > to this specific DAG run using the "some_unique_id" label. In the graph page > of the DAG, I need to know the exact execution date, which is inconvenient, > and in the Browse -> DAG Runs page I can search by "some_unique_id", but the > "Run Ids" column is not clickable. > The attached patch makes the aforementioned column clickable, so I'm sent > directly to the graph view for that specific DAG run, not the DAG in general. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AIRFLOW-1229) Make "Run Id" column clickable in Browse -> DAG Runs
[ https://issues.apache.org/jira/browse/AIRFLOW-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Bij reassigned AIRFLOW-1229: -- Assignee: Alexander Bij > Make "Run Id" column clickable in Browse -> DAG Runs > > > Key: AIRFLOW-1229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1229 > Project: Apache Airflow > Issue Type: Improvement > Components: webapp >Affects Versions: Airflow 1.8 > Environment: Python3.4 >Reporter: Erik Cederstrand >Assignee: Alexander Bij > Labels: patch > Attachments: dag_run_link.patch > > > I'm triggering a lot of DAGs manually using "airflow trigger_dag my_dag > --run_id=some_unique_id". I would like to be able in the UI to browse easily > to this specific DAG run using the "some_unique_id" label. In the graph page > of the DAG, I need to know the exact execution date, which is inconvenient, > and in the Browse -> DAG Runs page I can search by "some_unique_id", but the > "Run Ids" column is not clickable. > The attached patch makes the aforementioned column clickable, so I'm sent > directly to the graph view for that specific DAG run, not the DAG in general. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1371) Customize log_filepath to allow files writen to NFS-shares
[
https://issues.apache.org/jira/browse/AIRFLOW-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Bij updated AIRFLOW-1371:
---
Description:
To be able to send logs to a NFS-mount the folder and files cannot contain
[reserved characters |
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
like:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* ? (question mark)
* (asterisk)
Airflow writes the logs with an ISO-datetime format. This file contains one of
these characters.
There are some solutions. What I would like is a configurable log_file_format
to create log-files.
{code}
# Fetched by views.py and written by models.py currently like this:
@property
def log_filepath(self):
iso = self.execution_date.isoformat()
log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
return "{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals()))
{code}
was:
To be able to send logs to a NFS-mount the folder and files cannot contain
[reserved characters |
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
like:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* ? (question mark)
* (asterisk)
Airflow writes the logs with an ISO-datetime format. This file contains one of
these characters.
There are some solutions. What I would like is a configurable log_file_format
to create log-files.
{code}
# Fetched by views.py and written by models.py currently like this:
@property
def log_filepath(self):
iso = self.execution_date.isoformat()
log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
return "{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals()))
{code}
> Customize log_filepath to allow files writen to NFS-shares
> --
>
> Key: AIRFLOW-1371
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1371
> Project: Apache Airflow
> Issue Type: Improvement
> Components: models
>Affects Versions: Airflow 1.8
>Reporter: Alexander Bij
>Priority: Minor
>
> To be able to send logs to a NFS-mount the folder and files cannot contain
> [reserved characters |
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
> like:
> * < (less than)
> * > (greater than)
> * : (colon)
> * " (double quote)
> * / (forward slash)
> * \ (backslash)
> * ? (question mark)
> * (asterisk)
> Airflow writes the logs with an ISO-datetime format. This file contains one
> of these characters.
> There are some solutions. What I would like is a configurable log_file_format
> to create log-files.
> {code}
> # Fetched by views.py and written by models.py currently like this:
> @property
> def log_filepath(self):
> iso = self.execution_date.isoformat()
> log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
> return "{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals()))
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1371) Customize log_filepath to allow files writen to NFS-shares
[
https://issues.apache.org/jira/browse/AIRFLOW-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Bij updated AIRFLOW-1371:
---
Description:
To be able to send logs to a NFS-mount the folder and files cannot contain
[reserved characters |
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
like:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* ? (question mark)
* (asterisk)
Airflow writes the logs with an ISO-datetime format. This file contains one of
these characters.
There are some solutions. What I would like is a configurable log_file_format
to create log-files.
{code}
# Fetched by views.py and written by models.py currently like this:
@property
def log_filepath(self):
iso = self.execution_date.isoformat()
log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
return "{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals()))
{code}
was:
To be able to send logs to a NFS-mount the folder and files cannot contain
[reserved characters |
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
like:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* ? (question mark)
* (asterisk)
Airflow writes the logs with an ISO-datetime format. This file contains one of
these characters.
There are some solutions. What I would like is a configurable log_file_format
to create log-files.
{code}
# Fetched by views.py and written by models.py currently like this:
@property
def log_filepath(self):
iso = self.execution_date.isoformat()
log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
return (
"{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals())){code}
> Customize log_filepath to allow files writen to NFS-shares
> --
>
> Key: AIRFLOW-1371
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1371
> Project: Apache Airflow
> Issue Type: Improvement
> Components: models
>Affects Versions: Airflow 1.8
>Reporter: Alexander Bij
>Priority: Minor
>
> To be able to send logs to a NFS-mount the folder and files cannot contain
> [reserved characters |
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
> like:
> * < (less than)
> * > (greater than)
> * : (colon)
> * " (double quote)
> * / (forward slash)
> * \ (backslash)
> * ? (question mark)
> * (asterisk)
> Airflow writes the logs with an ISO-datetime format. This file contains one
> of these characters.
> There are some solutions. What I would like is a configurable log_file_format
> to create log-files.
> {code}
> # Fetched by views.py and written by models.py currently like this:
> @property
> def log_filepath(self):
> iso = self.execution_date.isoformat()
> log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
> return "{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals()))
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1371) Customize log_filepath to allow files writen to NFS-shares
[
https://issues.apache.org/jira/browse/AIRFLOW-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Bij updated AIRFLOW-1371:
---
Description:
To be able to send logs to a NFS-mount the folder and files cannot contain
[reserved characters |
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
like:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* | (vertical bar or pipe)
* ? (question mark)
* (asterisk)
Airflow writes the logs with an ISO-datetime format. This file contains one of
these characters.
There are some solutions. What I would like is a configurable log_file_format
to create log-files.
{code}
# Fetched by views.py and written by models.py currently like this:
@property
def log_filepath(self):
iso = self.execution_date.isoformat()
log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
return (
"{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals())){code}
was:
To be able to send logs to a NFS-mount the folder and files cannot contain
(reserved
characters)[https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
like:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* | (vertical bar or pipe)
* ? (question mark)
* (asterisk)
Airflow writes the logs with an ISO-datetime format. This file contains one of
these characters.
There are some solutions. What I would like is a configurable log_file_format
to create log-files.
{code}
# Fetched by views.py and written by models.py currently like this:
@property
def log_filepath(self):
iso = self.execution_date.isoformat()
log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
return (
"{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals())){code}
> Customize log_filepath to allow files writen to NFS-shares
> --
>
> Key: AIRFLOW-1371
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1371
> Project: Apache Airflow
> Issue Type: Improvement
> Components: models
>Affects Versions: Airflow 1.8
>Reporter: Alexander Bij
>Priority: Minor
>
> To be able to send logs to a NFS-mount the folder and files cannot contain
> [reserved characters |
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx]
> like:
> * < (less than)
> * > (greater than)
> * : (colon)
> * " (double quote)
> * / (forward slash)
> * \ (backslash)
> * | (vertical bar or pipe)
> * ? (question mark)
> * (asterisk)
> Airflow writes the logs with an ISO-datetime format. This file contains one
> of these characters.
> There are some solutions. What I would like is a configurable log_file_format
> to create log-files.
> {code}
> # Fetched by views.py and written by models.py currently like this:
> @property
> def log_filepath(self):
> iso = self.execution_date.isoformat()
> log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
> return (
>
> "{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals())){code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AIRFLOW-1371) Customize log_filepath to allow files writen to NFS-shares
Alexander Bij created AIRFLOW-1371:
--
Summary: Customize log_filepath to allow files writen to NFS-shares
Key: AIRFLOW-1371
URL: https://issues.apache.org/jira/browse/AIRFLOW-1371
Project: Apache Airflow
Issue Type: Improvement
Components: models
Affects Versions: Airflow 1.8
Reporter: Alexander Bij
Priority: Minor
To be able to send logs to a NFS-mount the folder and files cannot contain
[reserved
characters](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx)
like:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* | (vertical bar or pipe)
* ? (question mark)
* (asterisk)
Airflow writes the logs with an ISO-datetime format. This file contains one of
these characters.
There are some solutions. What I would like is a configurable log_file_format
to create log-files.
{code:python}
# Fetched by views.py and written by models.py currently like this:
@property
def log_filepath(self):
iso = self.execution_date.isoformat()
log = os.path.expanduser(configuration.get('core', 'BASE_LOG_FOLDER'))
return (
"{log}/{self.dag_id}/{self.task_id}/{iso}.log".format(**locals())){code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (AIRFLOW-840) Python3 encoding issue in Kerberos
[
https://issues.apache.org/jira/browse/AIRFLOW-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Bij reassigned AIRFLOW-840:
-
Assignee: Alexander Bij (was: Bolke de Bruin)
> Python3 encoding issue in Kerberos
> --
>
> Key: AIRFLOW-840
> URL: https://issues.apache.org/jira/browse/AIRFLOW-840
> Project: Apache Airflow
> Issue Type: Bug
> Components: security
>Affects Versions: Airflow 1.8
> Environment: $ python --version
> Python 3.4.3
>Reporter: Erik Cederstrand
>Assignee: Alexander Bij
> Labels: security
>
> While attempting to configure Kerberos ticket renewal in a Python3
> environment, I encountered this encoding issue trying to run {{airflow
> kerberos}}:
> {code:none}
> Traceback (most recent call last):
> File "/usr/local/bin/airflow", line 15, in
> args.func(args)
> File "/usr/local/lib/python3.4/dist-packages/airflow/bin/cli.py", line 600,
> in kerberos
> airflow.security.kerberos.run()
> File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py",
> line 110, in run
> renew_from_kt()
> File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py",
> line 55, in renew_from_kt
> "\n".join(subp.stderr.readlines(
> TypeError: sequence item 0: expected str instance, bytes found
> {code}
> The issue here (ignoring for a moment why {{kinit}} is failing on my machine)
> is that Popen in Python3 returns {{bytes}} for stdin/stdout, but both are
> handled as if they are {{str}}.
> I'm unsure what the Py2/3 compat policy is at Airflow, but a simple {{from
> six import PY2}} and an if/else seems like the least intrusive fix. The
> non-PY2 path would then add something like
> {{subp.stdin.readlines().decode(errors='ignore')}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
