GitHub user LucasRoesler added a comment to the discussion: Sharepoint ingest
using Microsoft Graph Filesystem
I think this is likely an interaction between the ObjectStoragePath (and UPath,
which it is built upon) and the URL we pass to it. For example, after looking
into the how the Path is initialized, i updated my code to look like this
```python
logger.warning("Using ObjectStoragePath to list files")
# Build SharePoint source path
source_path = ObjectStoragePath(
f"sharepoint://{msgraph_conn_id}@{msgraph_conn_id}/{drive_id}/{source_folder}/",
)
logger.warning(f"Listing files in SharePoint: {source_path}")
logger.warning(f"File pattern: {file_pattern}")
if not source_path.exists():
raise FileNotFoundError(f"SharePoint source folder does not exist:
{source_path}")
```
I now get logs after the initialization of the Path, but still the same error
```text
[2025-11-24, 15:40:10] WARNING - Using ObjectStoragePath to list files:
source="unusual_prefix_c7e44d46fe65fb6e0814d4788b8cf147a1576372_sharepoint_to_blob_dag"
[2025-11-24, 15:40:10] WARNING - Listing files in SharePoint:
sharepoint://sharepoint@sharepoint/private-drive-id/folder-prefix:
source="unusual_prefix_c7e44d46fe65fb6e0814d4788b8cf147a1576372_sharepoint_to_blob_dag"
[2025-11-24, 15:40:10] WARNING - File pattern: *:
source="unusual_prefix_c7e44d46fe65fb6e0814d4788b8cf147a1576372_sharepoint_to_blob_dag"
[2025-11-24, 15:40:10] INFO - Connection: sharepoint,
Connection(conn_id='sharepoint', conn_type='msgraph', description=None,
host='secret-d265-4611-8dbb-secret', schema=None,
login='secret-63e2-4f4d-8270-secret', password='***', port=None, extra='{\n
"scope": [\n "https://graph.microsoft.com/.default"\n ]\n}'):
chan="stdout": source="task"
[2025-11-24, 15:40:10] WARNING - MSGraphFS options: {'client_id':
'secret-63e2-4f4d-8270-secret', 'client_secret': '***', 'tenant_id':
'secret-d265-4611-8dbb-secret', 'scope':
['https://graph.microsoft.com/.default']}:
source="airflow.providers.microsoft.azure.fs.msgraph"
[2025-11-24, 15:40:10] WARNING - OAuth2 client params: {'client_id':
'secret-63e2-4f4d-8270-secret', 'client_secret': '***', 'tenant_id':
'secret-d265-4611-8dbb-secret', 'scope':
['https://graph.microsoft.com/.default']}:
source="airflow.providers.microsoft.azure.fs.msgraph"
[2025-11-24, 15:40:10] WARNING - MSGDriveFS drive_id=None client_id=None
tenant_id=None client_secret=None oauth2_client_params={'client_id':
'secret-63e2-4f4d-8270-secret', 'client_secret': '***', 'tenant_id':
'secret-d265-4611-8dbb-secret', 'scope':
['https://graph.microsoft.com/.default']}: source="msgraphfs.core"
[2025-11-24, 15:40:10] WARNING - MSGDriveFS initialized in
multi_site_mode=True: source="msgraphfs.core"
[2025-11-24, 15:40:10] WARNING - MSGDriveFS oauth2_client_params={'client_id':
'secret-63e2-4f4d-8270-secret', 'client_secret': '***', 'tenant_id':
'secret-d265-4611-8dbb-secret', 'scope':
['https://graph.microsoft.com/.default']}: source="msgraphfs.core"
[2025-11-24, 15:40:10] WARNING - MSGDriveFS post-super site_name=None
drive_name=None drive_id=None: source="msgraphfs.core"
[2025-11-24, 15:40:10] WARNING - MSGDriveFS drive_id=None
client_id=secret-63e2-4f4d-8270-secret tenant_id=None client_secret=***
oauth2_client_params=None: source="msgraphfs.core"
[2025-11-24, 15:40:10] WARNING - MSGDriveFS initialized in
multi_site_mode=False: source="msgraphfs.core"
[2025-11-24, 15:40:10] ERROR - Task failed with exception: source="task"
ValueError: Either oauth2_client_params must be provided, or all of client_id,
tenant_id, and client_secret must be provided (either as parameters or
environment variables MSGRAPHFS_CLIENT_ID/AZURE_CLIENT_ID,
MSGRAPHFS_TENANT_ID/AZURE_TENANT_ID,
MSGRAPHFS_CLIENT_SECRET/AZURE_CLIENT_SECRET)
```
GitHub link:
https://github.com/apache/airflow/discussions/58221#discussioncomment-15063476
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
