michalslowikowski00 commented on a change in pull request #10304: URL: https://github.com/apache/airflow/pull/10304#discussion_r478231596
########## File path: docs/howto/operator/google/cloud/dataprep.rst ########## @@ -17,7 +17,29 @@ Google Dataprep Operators ========================= -`Google Dataprep API documentation is available here <https://cloud.google.com/dataprep/docs/html/API-Reference_145281441>`__ +Dataprep is the intelligent cloud data service to visually explore, clean, and prepare data for analysis and machine learning. +Service can be use to explore and transform raw data from disparate and/or large datasets into clean and structured data for further analysis and processing. +Dataprep Job is an internal object encoding the information necessary to run a part of a Cloud Dataprep job group. +For more information about the service visit `Google Dataprep API documentation <https://cloud.google.com/dataprep/docs/html/API-Reference_145281441>`_ + +Before you begin +^^^^^^^^^^^^^^^^ +Before using Dataprep within Airflow you need to authenticate your account with TOKEN. +To get connection Dataprep with Airflow you need Dataprep token. Please follow Dataprep `instructions <https://clouddataprep.com/documentation/api#section/Authentication>`_ to do it. + +TOKEN should be added to the Connection in Airflow in JSON format. +You can check `how to do such connection <https://airflow.readthedocs.io/en/stable/howto/connection/index.html#editing-a-connection-with-the-ui>`_. + +The DataprepRunJobGroupOperator will run specified job. Operator required a recipe id. To identify the recipe id please use `API documentation for runJobGroup <https://clouddataprep.com/documentation/api#operation/runJobGroup>`_ +E.g. if the URL is /flows/10?recipe=7, the recipe id is 7. The recipe cannot be created via this operator. It can be created only via UI which is available `here <https://clouddataprep.com/>`_. +Some of parameters can be override by DAG's body request. How to do it is shown in example dag. + +See following example: +Set values for these fields: +.. code-block:: + + Conn Id: "your_conn_id" + Extra: {"extra__dataprep__token": "TOKEN"} Review comment: Yes, thank you. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
