Github-dm-CDE commented on issue #33688:
URL: https://github.com/apache/airflow/issues/33688#issuecomment-1698604348
> Any chance to see the logs with debug turned on for tasks as asked above
@Github-dm-CDE ? And a bit more info on the deployment of yours ?
Hello, here is the code of the DAG including the logs before and after the
update to Airflow 2.7.0. Currently the debug logging is still deactivated, but
I will have a look at that in a moment. Is there a good way to enable debug
logging only for this one DAG?
DAG Config
```python
import itertools
import os
from datetime import datetime, timedelta
from pathlib import Path
from typing import Iterator
import etl_env as env
from airflow import DAG
from airflow.operators.latest_only import LatestOnlyOperator
from airflow_extensions.macros.teams_plugin import
on_failure_message_to_teams
from airflow_extensions.operators.cli_task_builder.cli_task_builder import (
CLIOperator,
CliTaskBuilder,
ConnectionBuilder,
)
from bda_prj_self_service_cas_upload.table_config import config as
self_service
from etl_env import PEX_BASE_DIR
default_args = {
"owner": "CK",
"depends_on_past": False,
"start_date": datetime(2021, 6, 1, 7, 0, 0),
"retries": 2,
"retry_delay": timedelta(minutes=5),
}
PEX_PATH = os.path.join(PEX_BASE_DIR, "viyacli.pex")
CASLIB_COUNTRIES = [country.upper() for country in env.COUNTRIES_XX]
def get_self_service_cas_upload_configs() -> Iterator[Path]:
table_config_path = os.path.dirname(self_service.__file__)
return Path(table_config_path).glob(
f"{self_service.SELF_SERVICE_CONFIG_PREFIX}*.yaml"
)
def get_caslib_teams() -> list:
caslib_teams = ["CK"]
for config in get_self_service_cas_upload_configs():
team_name =
self_service.get_team_name_from_config_filename(config.name)
caslib_teams.append(team_name.upper())
return list(set(caslib_teams))
def get_all_caslibs() -> Iterator:
yield "PUBLIC"
caslib_teams = get_caslib_teams()
for country, team in itertools.product(CASLIB_COUNTRIES, caslib_teams):
yield f"{country}_{team}"
def create_cas_table_monitoring_task(dag: DAG, caslib: str) -> CLIOperator:
return (
CliTaskBuilder()
.with_task_args(
task_id=f"collect_CAS_table_metrics_{caslib.lower()}",
dag=dag,
)
.with_command(f"{PEX_PATH} cas-monitoring")
.with_connection(
ConnectionBuilder("sas_viya", env_prefix="VIYA")
.with_login("USERNAME")
.with_password("PASSWORD")
.with_host("HOST_URL")
)
.with_connection(
ConnectionBuilder("mssql_db", env_prefix="MSSQL")
.with_login("USERNAME")
.with_password("PASSWORD")
.with_host("HOST_URL")
.with_port("HOST_PORT")
.with_schema("DATABASE")
)
.with_cli_arg(f"--ssl_cert_file {SSL_CERT_PATH}")
.with_cli_arg(f"--viya_caslib {CASLIB}")
.with_cli_arg(f"--viya_client_secret {CLIENT_SECRET}")
.with_cli_arg("--monitoring_table sas_cas_table_status")
.create()
)
def create_dag():
return DAG(
"prj_ck_cas_table_monitoring",
description=__doc__,
default_args=default_args,
schedule="*/20 * * * *",
max_active_tasks=3,
max_active_runs=1,
on_failure_callback=on_failure_message_to_teams,
)
dag = create_dag()
latest_only = LatestOnlyOperator(task_id="latest_only", dag=dag)
for caslib in get_all_caslibs():
cas_table_metrics_tasks = create_cas_table_monitoring_task(dag, caslib)
latest_only >> cas_table_metrics_tasks
```
and here are the logs for one collect_CAS_table_metrics_* task:
Airflow 2.6.3:
```bash
*** Found local files:
*** *
/var/log/airflow/dags/dag_id=prj_ck_cas_table_monitoring/run_id=scheduled__2023-08-28T09:20:00+00:00/task_id=collect_CAS_table_metrics_public/attempt=1.log
[2023-08-28, 11:40:02 CEST] {taskinstance.py:1103} INFO - Dependencies all
met for dep_context=non-requeueable deps ti=<TaskInstance:
prj_ck_cas_table_monitoring.collect_CAS_table_metrics_public
scheduled__2023-08-28T09:20:00+00:00 [queued]>
[2023-08-28, 11:40:02 CEST] {taskinstance.py:1103} INFO - Dependencies all
met for dep_context=requeueable deps ti=<TaskInstance:
prj_ck_cas_table_monitoring.collect_CAS_table_metrics_public
scheduled__2023-08-28T09:20:00+00:00 [queued]>
[2023-08-28, 11:40:02 CEST] {taskinstance.py:1308} INFO - Starting attempt 1
of 3
[2023-08-28, 11:40:02 CEST] {taskinstance.py:1327} INFO - Executing
<Task(CLIOperator): collect_CAS_table_metrics_public> on 2023-08-28
09:20:00+00:00
[2023-08-28, 11:40:02 CEST] {standard_task_runner.py:57} INFO - Started
process 3717615 to run task
[2023-08-28, 11:40:02 CEST] {standard_task_runner.py:84} INFO - Running:
['airflow', 'tasks', 'run', 'prj_ck_cas_table_monitoring',
'collect_CAS_table_metrics_public', 'scheduled__2023-08-28T09:20:00+00:00',
'--job-id', '8827796', '--raw', '--subdir',
'DAGS_FOLDER/prj_ck_cas_table_monitoring/prj_ck_cas_table_monitoring.py',
'--cfg-path', '/tmp/tmp73qdvuk2']
[2023-08-28, 11:40:02 CEST] {standard_task_runner.py:85} INFO - Job 8827796:
Subtask collect_CAS_table_metrics_public
[2023-08-28, 11:40:02 CEST] {task_command.py:410} INFO - Running
<TaskInstance: prj_ck_cas_table_monitoring.collect_CAS_table_metrics_public
scheduled__2023-08-28T09:20:00+00:00 [running]> on host OBSCURED_HOSTNAME
[2023-08-28, 11:40:02 CEST] {taskinstance.py:1545} INFO - Exporting env
vars: AIRFLOW_CTX_DAG_OWNER='CK'
AIRFLOW_CTX_DAG_ID='prj_ck_cas_table_monitoring'
AIRFLOW_CTX_TASK_ID='collect_CAS_table_metrics_public'
AIRFLOW_CTX_EXECUTION_DATE='2023-08-28T09:20:00+00:00'
AIRFLOW_CTX_TRY_NUMBER='1'
AIRFLOW_CTX_DAG_RUN_ID='scheduled__2023-08-28T09:20:00+00:00'
[2023-08-28, 11:40:02 CEST] {logging_mixin.py:150} INFO - Execute cli task
[2023-08-28, 11:40:02 CEST] {base.py:73} INFO - Using connection ID
'sas_viya_conn' for task execution.
[2023-08-28, 11:40:02 CEST] {base.py:73} INFO - Using connection ID
'cxa_mssql_db_domain_user_format' for task execution.
[2023-08-28, 11:40:02 CEST] {logging_mixin.py:150} INFO - Environment
variables: ['LANG', 'PATH', 'VIYA_USERNAME', 'VIYA_PASSWORD', 'VIYA_HOST_URL',
'MSSQL_USERNAME', 'MSSQL_PASSWORD', 'MSSQL_HOST_URL', 'MSSQL_HOST_PORT',
'MSSQL_DATABASE']
[2023-08-28, 11:40:02 CEST] {logging_mixin.py:150} INFO - Bash command:
/srv/team-workspaces/ck/pex/viyacli.pex cas-monitoring --ssl_cert_file
OBSCURED_SSL_CERT_PATH --viya_caslib PUBLIC --viya_client_secret
OBSCURED_VIYA_CLIENT_SECRET --monitoring_table sas_cas_table_status
[2023-08-28, 11:40:02 CEST] {subprocess.py:63} INFO - Tmp dir root location:
/tmp
[2023-08-28, 11:40:02 CEST] {subprocess.py:75} INFO - Running command:
['/usr/bin/bash', '-c', '/srv/team-workspaces/ck/pex/viyacli.pex cas-monitoring
--ssl_cert_file OBSCURED_SSL_CERT_PATH --viya_caslib PUBLIC
--viya_client_secret OBSCURED_VIYA_CLIENT_SECRET --monitoring_table
sas_cas_table_status']
[2023-08-28, 11:40:02 CEST] {subprocess.py:86} INFO - Output:
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - 2023-08-28
11:40:06.260 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:39
- Item missing key created - skipping for monitoring:
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - 2023-08-28
11:40:06.260 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:40
- {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "version": 3,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "name":
"OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "tableReference": {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "version": 2,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "tableUri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "sessionId":
"OBSCURED_SESSION_ID",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"sourceTableName": "OBSCURED_USER_MAPPINGTEST.sashdat",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"sourceCaslibName": "PUBLIC"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "state":
"unloaded",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "repeated": false,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"sourceLastModified": "2023-06-09T09:40:45.761Z",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "serverName":
"cas-shared-default",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "caslibName":
"PUBLIC",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "attributes": {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "owner": "cas",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "size": 8448,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "encryption":
"NONE",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "time":
"2023-06-09T10:40:46+01:00",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "group": "sas"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "links": [
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"up",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.table"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"self",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.cas.table"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"DELETE",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"delete",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"PUT",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"updateState",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/state",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/state",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"responseType": "application/json;text/plain"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"columns",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/columns",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/columns",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.column"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"dataTable",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.data.table"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - ]
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - 2023-08-28
11:40:06.260 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:39
- Item missing key created - skipping for monitoring:
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - 2023-08-28
11:40:06.260 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:40
- {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "version": 3,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "name":
"MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "tableReference": {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "version": 2,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "tableUri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "sessionId":
"OBSCURED_SESSION_ID",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"sourceTableName": "MAP_DEVICE_TYPE_FIXED.sashdat",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"sourceCaslibName": "PUBLIC"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "state":
"unloaded",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "repeated": false,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"sourceLastModified": "2023-06-09T14:18:52.223Z",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "serverName":
"cas-shared-default",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "caslibName":
"PUBLIC",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "attributes": {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "owner": "cas",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "size": 8544,
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "encryption":
"NONE",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "time":
"2023-06-09T15:18:52+01:00",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "group": "sas"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "links": [
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"up",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.table"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"self",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.cas.table"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"DELETE",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"delete",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"PUT",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"updateState",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/state",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/state",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"responseType": "application/json;text/plain"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"columns",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/columns",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/columns",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.column"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "rel":
"dataTable",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "href":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "uri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.data.table"
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - ]
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - 2023-08-28
11:40:06.260 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:45
- The following tables have been skipped: ['OBSCURED_USER_MAPPINGTEST',
'MAP_DEVICE_TYPE_FIXED']
[2023-08-28, 11:40:06 CEST] {subprocess.py:93} INFO - 2023-08-28
11:40:06.260 | WARNING | viya.mssql_utils:write_to_mssql:35 - No data was
passed to insert into the Database.
[2023-08-28, 11:40:06 CEST] {subprocess.py:97} INFO - Command exited with
return code 0
[2023-08-28, 11:40:06 CEST] {taskinstance.py:1345} INFO - Marking task as
SUCCESS. dag_id=prj_ck_cas_table_monitoring,
task_id=collect_CAS_table_metrics_public, execution_date=20230828T092000,
start_date=20230828T094002, end_date=20230828T094006
[2023-08-28, 11:40:06 CEST] {local_task_job_runner.py:225} INFO - Task
exited with return code 0
[2023-08-28, 11:40:06 CEST] {taskinstance.py:2653} INFO - 0 downstream tasks
scheduled from follow-on schedule check
```
Airflow 2.7.0:
```bash
OBSCURED_VIYA_CLIENT_SECRET Found local files:
OBSCURED_VIYA_CLIENT_SECRET *
/var/log/airflow/dags/dag_id=prj_ck_cas_table_monitoring/run_id=scheduled__2023-08-28T09:40:00+00:00/task_id=collect_CAS_table_metrics_public/attempt=1.log
[2023-08-28, 12:16:16 CEST] {taskinstance.py:1159} INFO - Dependencies all
met for dep_context=non-requeueable deps ti=<TaskInstance:
prj_ck_cas_table_monitoring.collect_CAS_table_metrics_public
scheduled__2023-08-28T09:40:00+00:00 [queued]>
[2023-08-28, 12:16:16 CEST] {taskinstance.py:1159} INFO - Dependencies all
met for dep_context=requeueable deps ti=<TaskInstance:
prj_ck_cas_table_monitoring.collect_CAS_table_metrics_public
scheduled__2023-08-28T09:40:00+00:00 [queued]>
[2023-08-28, 12:16:16 CEST] {taskinstance.py:1361} INFO - Starting attempt 1
of 3
[2023-08-28, 12:16:16 CEST] {taskinstance.py:1382} INFO - Executing
<Task(CLIOperator): collect_CAS_table_metrics_public> on 2023-08-28
09:40:00+00:00
[2023-08-28, 12:16:16 CEST] {standard_task_runner.py:57} INFO - Started
process 3763639 to run task
[2023-08-28, 12:16:16 CEST] {standard_task_runner.py:84} INFO - Running:
['airflow', 'tasks', 'run', 'prj_ck_cas_table_monitoring',
'collect_CAS_table_metrics_public', 'scheduled__2023-08-28T09:40:00+00:00',
'--job-id', '8827969', '--raw', '--subdir',
'DAGS_FOLDER/prj_ck_cas_table_monitoring/prj_ck_cas_table_monitoring.py',
'--cfg-path', '/tmp/tmpvgdhcef6']
[2023-08-28, 12:16:16 CEST] {standard_task_runner.py:85} INFO - Job 8827969:
Subtask collect_CAS_table_metrics_public
[2023-08-28, 12:16:24 CEST] {task_command.py:415} INFO - Running
<TaskInstance: prj_ck_cas_table_monitoring.collect_CAS_table_metrics_public
scheduled__2023-08-28T09:40:00+00:00 [running]> on host OBSCURED_HOSTNAME
[2023-08-28, 12:16:32 CEST] {taskinstance.py:1660} INFO - Exporting env
vars: AIRFLOW_CTX_DAG_OWNER='CK'
AIRFLOW_CTX_DAG_ID='prj_ck_cas_table_monitoring'
AIRFLOW_CTX_TASK_ID='collect_CAS_table_metrics_public'
AIRFLOW_CTX_EXECUTION_DATE='2023-08-28T09:40:00+00:00'
AIRFLOW_CTX_TRY_NUMBER='1'
AIRFLOW_CTX_DAG_RUN_ID='scheduled__2023-08-28T09:40:00+00:00'
[2023-08-28, 12:16:32 CEST] {logging_mixin.py:151} INFO - Execute cli task
[2023-08-28, 12:16:32 CEST] {base.py:73} INFO - Using connection ID
'sas_viya_conn' for task execution.
[2023-08-28, 12:16:32 CEST] {base.py:73} INFO - Using connection ID
'cxa_mssql_db_domain_user_format' for task execution.
[2023-08-28, 12:16:32 CEST] {logging_mixin.py:151} INFO - Environment
variables: ['LANG', 'PATH', 'VIYA_USERNAME', 'VIYA_PASSWORD', 'VIYA_HOST_URL',
'MSSQL_USERNAME', 'MSSQL_PASSWORD', 'MSSQL_HOST_URL', 'MSSQL_HOST_PORT',
'MSSQL_DATABASE']
[2023-08-28, 12:16:32 CEST] {logging_mixin.py:151} INFO - Bash command:
/srv/team-workspaces/ck/pex/viyacli.pex cas-monitoring --ssl_cert_file
OBSCURED_SSL_CERT_PATH --viya_caslib PUBLIC --viya_client_secret
OBSCURED_VIYA_CLIENT_SECRET --monitoring_table sas_cas_table_status
[2023-08-28, 12:16:32 CEST] {subprocess.py:63} INFO - Tmp dir root location:
/tmp
[2023-08-28, 12:16:32 CEST] {subprocess.py:75} INFO - Running command:
['/usr/bin/bash', '-c', '/srv/team-workspaces/ck/pex/viyacli.pex cas-monitoring
--ssl_cert_file OBSCURED_SSL_CERT_PATH --viya_caslib PUBLIC
--viya_client_secret OBSCURED_VIYA_CLIENT_SECRET --monitoring_table
sas_cas_table_status']
[2023-08-28, 12:16:32 CEST] {subprocess.py:86} INFO - Output:
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - 2023-08-28
12:16:35.935 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:39
- Item missing key created - skipping for monitoring:
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - 2023-08-28
12:16:35.936 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:40
- {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "version": 3,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "name":
"OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "tableReference": {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "version": 2,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "tableUri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "sessionId":
"OBSCURED_SESSION_ID",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"sourceTableName": "OBSCURED_USER_MAPPINGTEST.sashdat",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"sourceCaslibName": "PUBLIC"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "state":
"unloaded",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "repeated": false,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"sourceLastModified": "2023-06-09T09:40:45.761Z",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "serverName":
"cas-shared-default",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "caslibName":
"PUBLIC",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "attributes": {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "owner": "cas",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "size": 8448,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "encryption":
"NONE",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "time":
"2023-06-09T10:40:46+01:00",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "group": "sas"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "links": [
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"up",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.table"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"self",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.cas.table"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"DELETE",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"delete",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"PUT",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"updateState",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/state",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/state",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"responseType": "application/json;text/plain"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"columns",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/columns",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/OBSCURED_USER_MAPPINGTEST/columns",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.column"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"dataTable",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/OBSCURED_USER_MAPPINGTEST",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.data.table"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - ]
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - 2023-08-28
12:16:35.936 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:39
- Item missing key created - skipping for monitoring:
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - 2023-08-28
12:16:35.936 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:40
- {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "version": 3,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "name":
"MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "tableReference": {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "version": 2,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "tableUri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "sessionId":
"OBSCURED_SESSION_ID",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"sourceTableName": "MAP_DEVICE_TYPE_FIXED.sashdat",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"sourceCaslibName": "PUBLIC"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "state":
"unloaded",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "repeated": false,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"sourceLastModified": "2023-06-09T14:18:52.223Z",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "serverName":
"cas-shared-default",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "caslibName":
"PUBLIC",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "attributes": {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "owner": "cas",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "size": 8544,
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "encryption":
"NONE",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "time":
"2023-06-09T15:18:52+01:00",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "group": "sas"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "links": [
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"up",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.table"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"self",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.cas.table"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"DELETE",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"delete",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"PUT",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"updateState",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/state",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/state",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"responseType": "application/json;text/plain"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"columns",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/columns",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/casManagement/servers/cas-shared-default/caslibs/PUBLIC/tables/MAP_DEVICE_TYPE_FIXED/columns",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.collection",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO -
"itemType": "application/vnd.sas.cas.column"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - },
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - {
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "method":
"GET",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "rel":
"dataTable",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "href":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "uri":
"/dataTables/dataSources/cas~fs~cas-shared-default~fs~PUBLIC/tables/MAP_DEVICE_TYPE_FIXED",
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - "type":
"application/vnd.sas.data.table"
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - ]
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - }
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - 2023-08-28
12:16:35.936 | WARNING | viya.cas_monitoring.cas_monitoring:_transform_data:45
- The following tables have been skipped: ['OBSCURED_USER_MAPPINGTEST',
'MAP_DEVICE_TYPE_FIXED']
[2023-08-28, 12:16:35 CEST] {subprocess.py:93} INFO - 2023-08-28
12:16:35.936 | WARNING | viya.mssql_utils:write_to_mssql:35 - No data was
passed to insert into the Database.
[2023-08-28, 12:16:36 CEST] {subprocess.py:97} INFO - Command exited with
return code 0
[2023-08-28, 12:16:36 CEST] {taskinstance.py:1400} INFO - Marking task as
SUCCESS. dag_id=prj_ck_cas_table_monitoring,
task_id=collect_CAS_table_metrics_public, execution_date=20230828T094000,
start_date=20230828T101616, end_date=20230828T101636
[2023-08-28, 12:16:36 CEST] {local_task_job_runner.py:228} INFO - Task
exited with return code 0
[2023-08-28, 12:16:36 CEST] {taskinstance.py:2784} INFO - 0 downstream tasks
scheduled from follow-on schedule check
```
This is one of the faster running tasks in the DAG. But as you can clearly
see, the running times have quadrupled from ~4 seconds to ~20 seconds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
