mashirali opened a new issue, #31886:
URL: https://github.com/apache/airflow/issues/31886
### Apache Airflow version
2.6.1
### What happened
I was trying to add azure ml workflows in the dags but got an error
### What you think should happen instead
Everything should run fine as it ran great in local machine
### How to reproduce
[2023-06-14, 02:00:11 UTC] {datastore_client.py:991} INFO -
<azureml.core.authentication.ServicePrincipalAuthentication object at
0x7f48627941f0>
[2023-06-14, 02:00:11 UTC] {logging_mixin.py:149} INFO - Validating
arguments.
[2023-06-14, 02:00:11 UTC] {logging_mixin.py:149} INFO - Arguments validated.
[2023-06-14, 02:00:11 UTC] {logging_mixin.py:149} INFO - Successfully
obtained datastore reference and path.
[2023-06-14, 02:00:11 UTC] {logging_mixin.py:149} INFO - Uploading file to
dataset//a49db78c-c909-4319-a914-c00693d8fb9f/
[2023-06-14, 02:00:12 UTC] {taskinstance.py:1824} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/dotnetcore2/runtime.py",
line 271, in attempt_get_deps
blob_deps_to_file()
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/dotnetcore2/runtime.py",
line 263, in blob_deps_to_file
blob = request.urlopen(deps_url, context=ssl_context)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/airflow/operators/python.py",
line 181, in execute
return_value = self.execute_callable()
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/airflow/operators/python.py",
line 198, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/home/ubuntu/airflow/dags/forecast_suggest_azure.py", line 343, in
etl_forecast_suggest_azure
full_train_dataset = TabularDatasetFactory.register_pandas_dataframe(
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/data/_loggerfactory.py",
line 132, in wrapper
return func(*args, **kwargs)
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/data/dataset_factory.py",
line 653, in register_pandas_dataframe
dflow = dataprep().read_pandas_dataframe(df=sanitized_df, in_memory=True)
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/dataprep/api/readers.py",
line 537, in read_pandas_dataframe
return _read_in_memory_pandas_dataframe(df)
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/dataprep/api/readers.py",
line 588, in _read_in_memory_pandas_dataframe
read_df_step = Step('Microsoft.DPrep.ReadDataFrameFromSocketBlock', {
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/dataprep/api/step.py",
line 118, in __init__
self.arguments = arguments if isinstance(arguments, PropertyValues) else
PropertyValues.from_pod(arguments, _get_prop_descriptions(step_type))
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/typedefinitions.py",
line 72, in _get_prop_descriptions
descriptions = get_engine_api().get_block_descriptions()
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/api.py",
line 19, in get_engine_api
_engine_api = EngineAPI()
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/api.py",
line 102, in __init__
self._message_channel = launch_engine()
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/engine.py",
line 333, in launch_engine
dependencies_path = runtime.ensure_dependencies()
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/dotnetcore2/runtime.py",
line 285, in ensure_dependencies
if not attempt_get_deps(missing_pkgs):
File
"/home/ubuntu/python8env/lib/python3.8/site-packages/dotnetcore2/runtime.py",
line 279, in attempt_get_deps
raise NotImplementedError(err_msg + '\n' + _unsupported_help_msg)
NotImplementedError: Linux distribution ubuntu 22.04 does not have automatic
support.
Missing packages: {'liblttng-ust.so.0'}
.NET Core 3.1 can still be used via `dotnetcore2` if the required
dependencies are installed.
Visit https://aka.ms/dotnet-install-linux for Linux distro specific .NET
Core install instructions.
Follow your distro specific instructions to install `dotnet-runtime-*` and
replace `*` with `3.1.23`.
[2023-06-14, 02:00:12 UTC] {taskinstance.py:1345} INFO - Marking task as
FAILED. dag_id=forecast_suggest_azure, task_id=process_etl_suggest_azure,
execution_date=20230614T015414, start_date=20230614T020004,
end_date=20230614T020012
[2023-06-14, 02:00:12 UTC] {standard_task_runner.py:104} ERROR - Failed to
execute job 90 for task process_etl_suggest_azure (Linux distribution ubuntu
22.04 does not have automatic support.
Missing packages: {'liblttng-ust.so.0'}
.NET Core 3.1 can still be used via `dotnetcore2` if the required
dependencies are installed.
Visit https://aka.ms/dotnet-install-linux for Linux distro specific .NET
Core install instructions.
Follow your distro specific instructions to install `dotnet-runtime-*` and
replace `*` with `3.1.23`.
; 70892)
### Operating System
ubuntu 22.04
### Versions of Apache Airflow Providers
2.6.1
### Deployment
Virtualenv installation
### Deployment details
3.8 version of python
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
