rajaprakash91 commented on issue #16911: URL: https://github.com/apache/airflow/issues/16911#issuecomment-973894686
Oh Nice. Thank you so much. This is very useful. On Fri, Nov 19, 2021 at 1:05 PM Ashish Patel ***@***.***> wrote: > @pateash <https://github.com/pateash> Sorry, just seen your reply, yes i > was trying to update Airflow from 1.10x to 2.X. > > so what is the fix, i saw a merge request > > @rajaprakash91 <https://github.com/rajaprakash91> You can generate > ClusterConfig with the same arguments from ClusterGenerator.make() you have > been using and then pass that to the operator. > > path = f"gs://goog-dataproc-initialization-actions-{self.cfg.get('region')}/python/pip-install.sh" > > cluster_config = ClusterGenerator( > project_id=self.cfg.get('project_id'), > ........... > init_actions_uris=[ > path > ], > metadata={'PIP_PACKAGES': 'pyyaml requests pandas openpyxl'}, > properties=properties, > dag=dag > ).make() > > return DataprocClusterCreateOperator( > task_id='create_dataproc_cluster', > cluster_name='my-cluster', > project_id=self.cfg.get('project_id'), > region=self.cfg.get('region'), > cluster_config=cluster_config, > dag=dag > ) > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/airflow/issues/16911#issuecomment-973827278>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AIUYD5XHDRUPUBJKHBVKAOTUMX43NANCNFSM5ACXMNLA> > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> > or Android > <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. > > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
