mik-laj edited a comment on issue #10007: URL: https://github.com/apache/airflow/issues/10007#issuecomment-663996076
> i thought this is the project that the contribution of sagemaker is being done - isn't it ? SageMaker contributors are not an active part of this community I still think it will be difficult to fix this problem on the Airflow core. We only integrate a ready-made SDK and if the official SDK requires dependencies, this SDK should be improved to limit the number of needed dependencies. We can look for an alternative unofficial SDK, but the unofficial SDK may be the source of other problems e.g. missing features. Besides, Airflow does not require a Sagemaker SDK and it doesn't use the Sagemaker SDK anywhere. You can also prepare the config file manually based on the boto3 documentation. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_hyper_parameter_tuning_job or You can prepare the config in another environment and save it to a JSON file, and then load these config in the DAG file. Here is an example of how you use this operators without Sagemaker SDK https://github.com/apache/airflow/blob/master/tests/providers/amazon/aws/operators/test_sagemaker_tuning.py#L40-L115 Airflow doesn't require the Sagemaker SDK, nor any libraries that require numpy. Your DAG File has this dependency. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
