[
https://issues.apache.org/jira/browse/AIRFLOW-7024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055915#comment-17055915
]
ASF GitHub Bot commented on AIRFLOW-7024:
-----------------------------------------
kaxil commented on pull request #7676: [AIRFLOW-7024] Add the verbose parameter
support to SparkSqlOperator
URL: https://github.com/apache/airflow/pull/7676
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Add the verbose parameter support to SparkSqlOperator
> -----------------------------------------------------
>
> Key: AIRFLOW-7024
> URL: https://issues.apache.org/jira/browse/AIRFLOW-7024
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.10.9
> Reporter: Kengo Seki
> Assignee: Kengo Seki
> Priority: Major
>
> The docstring for SparkSqlOperator says that it supports a {{verbose}}
> parameter, but actually it doesn't.
> {code:title=airflow/providers/apache/spark/operators/spark_sql.py}
> class SparkSqlOperator(BaseOperator):
> """
> (snip)
> :param verbose: Whether to pass the verbose flag to spark-sql
> :type verbose: bool
> :param yarn_queue: The YARN queue to submit to (Default: "default")
> :type yarn_queue: str
> """
> (snip)
> @apply_defaults
> def __init__(self,
> sql,
> conf=None,
> conn_id='spark_sql_default',
> total_executor_cores=None,
> executor_cores=None,
> executor_memory=None,
> keytab=None,
> principal=None,
> master='yarn',
> name='default-name',
> num_executors=None,
> yarn_queue='default',
> *args,
> **kwargs):
> super().__init__(*args, **kwargs)
> self._sql = sql
> self._conf = conf
> self._conn_id = conn_id
> self._total_executor_cores = total_executor_cores
> self._executor_cores = executor_cores
> self._executor_memory = executor_memory
> self._keytab = keytab
> self._principal = principal
> self._master = master
> self._name = name
> self._num_executors = num_executors
> self._yarn_queue = yarn_queue
> self._hook = None
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
