Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
vincbeck commented on PR #61498: URL: https://github.com/apache/airflow/pull/61498#issuecomment-3923482299 Taken over by #62147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
vincbeck closed pull request #61498: add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator URL: https://github.com/apache/airflow/pull/61498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
eladkal commented on PR #61498: URL: https://github.com/apache/airflow/pull/61498#issuecomment-3882860099 Since this PR adds new functionality it requires also adding tests to cover the new functionality -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
o-nikolas commented on code in PR #61498:
URL: https://github.com/apache/airflow/pull/61498#discussion_r2775910569
##
providers/amazon/src/airflow/providers/amazon/aws/hooks/sagemaker_unified_studio.py:
##
@@ -40,24 +40,43 @@ class SageMakerNotebookHook(BaseHook):
from airflow.providers.amazon.aws.hooks.sagemaker_unified_studio
import SageMakerNotebookHook
notebook_hook = SageMakerNotebookHook(
+execution_name="notebook_execution",
+domain_id="dzd-example123456",
+project_id="example123456",
input_config={"input_path": "path/to/notebook.ipynb",
"input_params": {"param1": "value1"}},
output_config={"output_uri": "folder/output/location/prefix",
"output_formats": "NOTEBOOK"},
-execution_name="notebook_execution",
+domain_region="us-east-1",
waiter_delay=10,
waiter_max_attempts=1440,
)
:param execution_name: The name of the notebook job to be executed, this
is same as task_id.
+:param domain_id: The domain ID for Amazon SageMaker Unified Studio.
Optional - if not provided,
+the SDK will attempt to resolve it from the MWAA environment.
Review Comment:
It is technically possible to export the right variables in a non-MWAA env,
no?
```suggestion
the SDK will attempt to resolve it from the environment.
```
Same suggestion for below and in the operator module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
vincbeck commented on code in PR #61498:
URL: https://github.com/apache/airflow/pull/61498#discussion_r2770016596
##
providers/amazon/tests/unit/amazon/aws/hooks/test_sagemaker_unified_studio.py:
##
@@ -200,28 +200,6 @@ def test_set_xcom_s3_path_negative_missing_context(self):
with pytest.raises(AirflowException, match="context is required"):
self.hook._set_xcom_s3_path(self.s3Path, {})
-def test_start_notebook_execution_default_compute(self):
Review Comment:
Makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
ruijiang-rjian commented on code in PR #61498:
URL: https://github.com/apache/airflow/pull/61498#discussion_r2769949560
##
providers/amazon/tests/unit/amazon/aws/hooks/test_sagemaker_unified_studio.py:
##
@@ -200,28 +200,6 @@ def test_set_xcom_s3_path_negative_missing_context(self):
with pytest.raises(AirflowException, match="context is required"):
self.hook._set_xcom_s3_path(self.s3Path, {})
-def test_start_notebook_execution_default_compute(self):
Review Comment:
because it's no longer setting the default compute, see line 124-125 we
removed setting default in
providers/amazon/src/airflow/providers/amazon/aws/hooks/sagemaker_unified_studio.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
ruijiang-rjian commented on code in PR #61498:
URL: https://github.com/apache/airflow/pull/61498#discussion_r2769949560
##
providers/amazon/tests/unit/amazon/aws/hooks/test_sagemaker_unified_studio.py:
##
@@ -200,28 +200,6 @@ def test_set_xcom_s3_path_negative_missing_context(self):
with pytest.raises(AirflowException, match="context is required"):
self.hook._set_xcom_s3_path(self.s3Path, {})
-def test_start_notebook_execution_default_compute(self):
Review Comment:
because it's no longer setting the default compute, see line 124-125 in
providers/amazon/src/airflow/providers/amazon/aws/hooks/sagemaker_unified_studio.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
vincbeck commented on code in PR #61498:
URL: https://github.com/apache/airflow/pull/61498#discussion_r2769937261
##
providers/amazon/tests/unit/amazon/aws/hooks/test_sagemaker_unified_studio.py:
##
@@ -200,28 +200,6 @@ def test_set_xcom_s3_path_negative_missing_context(self):
with pytest.raises(AirflowException, match="context is required"):
self.hook._set_xcom_s3_path(self.s3Path, {})
-def test_start_notebook_execution_default_compute(self):
Review Comment:
Why deleting this test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] add SageMaker Unified Studio domain_id, project_id, domain region as new parameters to SageMakerNotebookOperator [airflow]
boring-cyborg[bot] commented on PR #61498: URL: https://github.com/apache/airflow/pull/61498#issuecomment-3854565942 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst) Here are some useful points: - Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that. - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it. - Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations. - Be patient and persistent. It might take some time to get a review or get the final approval from Committers. - Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack. - Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices). - Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: [email protected] Slack: https://s.apache.org/airflow-slack -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
