Re: Apache Drill - Azure blob connection time out

[Charles Givre]

HI Eric,
Welcome to Drill.   A few questions:

1.  Can you update to Drill 1.21.1?  Drill 1.19 is fairly old.
2.  What happens when you do a `show databases` query?  You didn't mention the 
size of the file you are querying but my gut is saying that this is a 
connection issue.

Best,
-- C

> On Sep 14, 2023, at 10:49 AM, Odland, Eric (Allianz Technology of America) 
> <eric.odl...@na.allianz-technology.com> wrote:
> 
> We are trying to use Apache Drill to run a query against a JSON file on our 
> Azure Blob storage account. I’ve followed all of the documentation steps for 
> setting up the Azure blob access found here: 
> https://drill.apache.org/docs/azure-blob-storage-plugin/. I’ve confirmed the 
> machine can access the file via a SAS URL so there is no firewall blocking 
> Azure blob. The query runs for around 20-25 minutes and then throws the below 
> error. The Azure key is setup in the core-site.xml file. We’re using 
> apache-drill-1.19.0 along with the following JAR files: 
> hadoop.azure-2.7.7.jar and azure-storage-8.0.0.jar.
> Any ideas to resolve this issue?
> 
> Query:
> SELECT * FROM azureblob.`RSP_DST_AMP_Extract_All.json` LIMIT 1
> 
> Error:
> RESOURCE ERROR: Failed to load schema for "azureblob"!
> 
> org.apache.hadoop.fs.azure.AzureException: 
> com.microsoft.azure.storage.StorageException: An unknown failure occurred : 
> Connection timed out: connect
> 
> Storage plugin:
> 
> 
> Configuration:
> {
>   "type": "file",
>   "connection": 
> "wasbs://stadst-raw-extra...@stadstdev3azltools01.blob.core.windows.net",
>   "workspaces": {
>     "tmp": {
>       "location": "/tmp",
>       "writable": true,
>       "defaultInputFormat": null,
>       "allowAccessOutsideWorkspace": false
>     },
>     "root": {
>       "location": "/",
>       "writable": false,
>       "defaultInputFormat": null,
>       "allowAccessOutsideWorkspace": false
>     }
>   },
>   "formats": {
>     "parquet": {
>       "type": "parquet"
>     },
>     "json": {
>       "type": "json",
>       "extensions": [
>         "json"
>       ]
>     },
>     "sequencefile": {
>       "type": "sequencefile",
>       "extensions": [
>         "seq"
>       ]
>     },
>     "csvh": {
>       "type": "text",
>       "extensions": [
>         "csvh"
>       ],
>       "extractHeader": true
>     },
>     "psv": {
>       "type": "text",
>       "extensions": [
>         "tbl"
>       ],
>       "fieldDelimiter": "|"
>     },
>     "avro": {
>       "type": "avro",
>       "extensions": [
>         "avro"
>       ]
>     },
>     "tsv": {
>       "type": "text",
>       "extensions": [
>         "tsv"
>       ],
>       "fieldDelimiter": "\t"
>     },
>     "csv": {
>       "type": "text",
>       "extensions": [
>         "csv"
>       ],
>       "extractHeader": true
>     }
>   },
>   "enabled": true
> }
> CONFIDENTIALITY NOTICE: The information in this message, and any files 
> transmitted with it, is confidential, may be legally privileged, and intended 
> only for the use of the individual(s) named above. Be aware that the use of 
> any confidential or personal information may be restricted by state and 
> federal privacy laws. If you are not the intended recipient, do not further 
> disseminate this message. If this message was received in error, please 
> notify the sender and delete it.

signature.asc
Description: Message signed with OpenPGP