[
https://issues.apache.org/jira/browse/AIRFLOW-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AIRFLOW-1319.
---------------------------------------
Resolution: Fixed
Fix Version/s: 2.0.0
Issue resolved by pull request #2377
[https://github.com/apache/incubator-airflow/pull/2377]
> Fix misleading SparkSubmitOperator and SparkSubmitHook docstring
> ----------------------------------------------------------------
>
> Key: AIRFLOW-1319
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1319
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Reporter: Chris Sng
> Assignee: Chris Sng
> Priority: Trivial
> Labels: documentation
> Fix For: 2.0.0
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> In the community-contributed Spark submit hook and operator, it support
> {{spark-submit}}'s {{\--files}} command line option. The {{\--files}} option
> is used to submit file to each executor to be used. A good example of such
> files are serialized objects.
> However, in both docstrings
> (https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/spark_submit_operator.py#L37
> and
> https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/spark_submit_hook.py#L36),
> it provided {{hive-site.xml}} as an example. This may mislead less-informed
> developers into assuming that Hive configuration files can be submitted to
> the cluster in this manner. According to Apache Hive's documentation, hive
> configuration files are located in the directory located in the
> {{HIVE_CONF_DIR}} environment variable.
> I propose excluding this example from the docstrings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
