janhoy commented on code in PR #3750: URL: https://github.com/apache/solr/pull/3750#discussion_r3458518356
########## solr/solr-ref-guide/modules/deployment-guide/pages/backup-restore.adoc: ########## @@ -826,3 +826,124 @@ https://docs.aws.amazon.com/sdkref/latest/guide/settings-global.html[These optio * Retries ** RetryMode (`LEGACY`, `STANDARD`, `ADAPTIVE`) ** Max Attempts + +=== AzureBlobBackupRepository + +Stores and retrieves backup files in a Microsoft Azure Blob Storage container. + +This is provided via the `azure-blob-repository` xref:configuration-guide:solr-modules.adoc[Solr Module] that needs to be enabled before use. + +This plugin supports multiple authentication methods: connection strings, account keys, SAS tokens, and Azure Identity (Managed Identity, Service Principal, Azure CLI). +For Azure Identity, ensure the identity has the "Storage Blob Data Contributor" role on the storage account. + +[source,xml] +---- +<backup> + <repository name="azure_blob" class="org.apache.solr.azureblob.AzureBlobBackupRepository" default="false"> + <str name="azure.blob.container.name">solr-backup</str> + <str name="azure.blob.connection.string">DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;EndpointSuffix=core.windows.net</str> + </repository> +</backup> +---- + +AzureBlobBackupRepository accepts the following options for configuration: + +`azure.blob.container.name`:: ++ +[%autowidth,frame=none] +|=== +|Required |Default: none +|=== ++ +The name of the Azure Blob Storage container. The container must exist before performing backup operations. + +`azure.blob.connection.string`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Complete Azure Storage connection string. Mutually exclusive with other authentication methods. + +`azure.blob.account.name`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Azure Storage account name. Used with account key or SAS token authentication. + +`azure.blob.account.key`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Azure Storage account access key. Mutually exclusive with SAS token and Azure Identity. + +`azure.blob.sas.token`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +SAS token for time-limited access. Must include `srt=sco` and `sp=rwdlac` permissions. +The `&` characters must be XML-escaped as `&`. + +`azure.blob.endpoint`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Azure Blob Storage endpoint URL (e.g., `https://myaccount.blob.core.windows.net`). +Required for Azure Identity authentication. + +`azure.blob.tenant.id`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Azure AD tenant ID for Service Principal authentication. + +`azure.blob.client.id`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Azure AD application (client) ID for Service Principal authentication. + +`azure.blob.client.secret`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Azure AD application secret for Service Principal authentication. + +`location`:: ++ +[%autowidth,frame=none] +|=== +|Optional |Default: none +|=== ++ +Default path prefix within the container for backup storage. + +The target container must already exist; it is not created automatically. + +==== Known Limitation: Azure Identity and the Security Manager + +Azure Identity authentication (Managed Identity, Service Principal, and `DefaultAzureCredential`) does not work when Solr is started with the Java Security Manager enabled, which is the default (`SOLR_SECURITY_MANAGER_ENABLED=true`). +To use Azure Identity, set `SOLR_SECURITY_MANAGER_ENABLED=false` before starting Solr. +The Connection String, Account Key, and SAS Token authentication methods are unaffected and work with the Security Manager enabled. Review Comment: Is it an option here to give an example of what to add to `security.policy` as an alternative to disabling? ########## solr/solrj/gradle.lockfile: ########## @@ -17,13 +17,19 @@ com.github.stephenc.jcip:jcip-annotations:1.0-1=compileClasspath,compileOnlyHelp com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,errorprone,testAnnotationProcessor com.google.auto.value:auto-value-annotations:1.11.0=annotationProcessor,errorprone,testAnnotationProcessor com.google.auto:auto-common:1.2.2=annotationProcessor,errorprone,testAnnotationProcessor +com.google.code.findbugs:jsr305:3.0.2=spotless865458226 com.google.errorprone:error_prone_annotation:2.41.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.errorprone:error_prone_annotations:2.18.0=spotless865458226 Review Comment: Why do we suddenly have all these spotless dependencies added to lockfiles? I would not expect any chagnes to lockfiles outside of the new module itself? ########## solr/core/gradle.lockfile: ########## @@ -1,191 +1,191 @@ # This is a Gradle generated file for dependency locking. # Manual edits can break the build and are not advised. # This file is expected to be part of source control. -biz.aQute.bnd:biz.aQute.bnd.annotation:7.1.0=compileClasspath,testCompileClasspath -com.carrotsearch.randomizedtesting:randomizedtesting-runner:2.8.4=jarValidation,testCompileClasspath,testRuntimeClasspath -com.carrotsearch:hppc:0.10.0=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.jackson.core:jackson-annotations:2.21=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.jackson.core:jackson-core:2.21.2=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.jackson.core:jackson-databind:2.21.2=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.21.2=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.21.2=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.jackson.module:jackson-module-jakarta-xmlbind-annotations:2.21.2=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.jackson:jackson-bom:2.21.2=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.fasterxml.woodstox:woodstox-core:7.0.0=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.3=annotationProcessor,compileClasspath,errorprone,jarValidation,runtimeClasspath,runtimeLibs,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath +biz.aQute.bnd:biz.aQute.bnd.annotation:7.1.0=compileClasspath,compileClasspathCopy,testCompileClasspath,testCompileClasspathCopy +com.carrotsearch.randomizedtesting:randomizedtesting-runner:2.8.4=jarValidation,testCompileClasspath,testCompileClasspathCopy,testRuntimeClasspath,testRuntimeClasspathCopy +com.carrotsearch:hppc:0.10.0=compileClasspath,compileClasspathCopy,jarValidation,runtimeClasspath,runtimeClasspathCopy,runtimeLibs,testCompileClasspath,testCompileClasspathCopy,testRuntimeClasspath,testRuntimeClasspathCopy Review Comment: Why are `runtimeClasspathCopy`, `compileClasspathCopy`, `testCompileClasspathCopy` added to many of the gradle lock files? ########## solr/modules/azure-blob-repository/gradle.lockfile: ########## @@ -0,0 +1,195 @@ +# This is a Gradle generated file for dependency locking. +# Manual edits can break the build and are not advised. +# This file is expected to be part of source control. +com.azure:azure-core-http-okhttp:1.13.3=jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-core:1.57.1=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-identity:1.18.2=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-json:1.5.1=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-sdk-bom:1.3.6=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-storage-blob-batch:12.29.3=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-storage-blob:12.33.3=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-storage-common:12.32.2=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-storage-internal-avro:12.18.2=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.azure:azure-xml:1.2.1=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.carrotsearch.randomizedtesting:randomizedtesting-runner:2.8.4=jarValidation,testCompileClasspath,testRuntimeClasspath +com.carrotsearch:hppc:0.10.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +com.fasterxml.jackson.core:jackson-annotations:2.21=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +com.fasterxml.jackson.core:jackson-core:2.21.2=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +com.fasterxml.jackson.core:jackson-databind:2.21.2=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.21.2=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.21.2=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.21.2=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.fasterxml.jackson.module:jackson-module-jakarta-xmlbind-annotations:2.21.2=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +com.fasterxml.jackson:jackson-bom:2.21.2=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +com.fasterxml.woodstox:woodstox-core:7.0.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +com.github.ben-manes.caffeine:caffeine:3.2.3=annotationProcessor,apiHelper,errorprone,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testAnnotationProcessor,testRuntimeClasspath +com.github.docker-java:docker-java-api:3.7.0=jarValidation,testCompileClasspath,testRuntimeClasspath +com.github.docker-java:docker-java-transport-zerodep:3.7.0=jarValidation,testCompileClasspath,testRuntimeClasspath +com.github.docker-java:docker-java-transport:3.7.0=jarValidation,testCompileClasspath,testRuntimeClasspath +com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,errorprone,testAnnotationProcessor +com.google.auto.value:auto-value-annotations:1.11.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.auto:auto-common:1.2.2=annotationProcessor,errorprone,testAnnotationProcessor +com.google.errorprone:error_prone_annotation:2.41.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.errorprone:error_prone_annotations:2.41.0=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +com.google.errorprone:error_prone_annotations:2.43.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.errorprone:error_prone_check_api:2.41.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.errorprone:error_prone_core:2.41.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.googlejavaformat:google-java-format:1.27.0=annotationProcessor,errorprone,testAnnotationProcessor +com.google.guava:failureaccess:1.0.3=annotationProcessor,apiHelper,compileClasspath,errorprone,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath +com.google.guava:guava:33.5.0-jre=annotationProcessor,apiHelper,compileClasspath,errorprone,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath +com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,apiHelper,compileClasspath,errorprone,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath +com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,apiHelper,compileClasspath,errorprone,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath +com.google.protobuf:protobuf-java:3.25.8=annotationProcessor,errorprone,testAnnotationProcessor +com.j256.simplemagic:simplemagic:1.17=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +com.jayway.jsonpath:json-path:2.9.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +com.lmax:disruptor:4.0.0=solrPlatformLibs +com.microsoft.azure:msal4j-persistence-extension:1.3.0=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.microsoft.azure:msal4j:1.23.1=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.squareup.okhttp3:okhttp:4.12.0=jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.squareup.okio:okio-jvm:3.16.0=jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +com.squareup.okio:okio:3.16.0=jarValidation,testCompileClasspath,testRuntimeClasspath +com.tdunning:t-digest:3.3=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +commons-cli:commons-cli:1.11.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +commons-codec:commons-codec:1.21.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +commons-io:commons-io:2.21.0=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.dropwizard.metrics:metrics-core:4.2.33=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,errorprone,testAnnotationProcessor +io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,errorprone,testAnnotationProcessor +io.netty:netty-buffer:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-codec-base:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-common:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-handler:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-resolver:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-tcnative-boringssl-static:2.0.75.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-tcnative-classes:2.0.75.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-transport-classes-epoll:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-transport-native-epoll:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-transport-native-unix-common:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.netty:netty-transport:4.2.15.Final=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-incubator:2.22.0-alpha=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry.instrumentation:opentelemetry-instrumentation-api:2.22.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry.instrumentation:opentelemetry-runtime-telemetry-java17:2.22.0-alpha=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry.instrumentation:opentelemetry-runtime-telemetry-java8:2.22.0-alpha=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry.semconv:opentelemetry-semconv:1.37.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry:opentelemetry-api-incubator:1.56.0-alpha=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry:opentelemetry-api:1.56.0=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-common:1.56.0=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-context:1.56.0=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-exporter-prometheus:1.56.0-alpha=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-common:1.56.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-metrics:1.56.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-trace:1.56.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk:1.56.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.projectreactor:reactor-core:3.7.14=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +io.prometheus:prometheus-metrics-exposition-formats:1.1.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.prometheus:prometheus-metrics-model:1.1.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.sgr:s2-geometry-library-java:1.0.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +io.swagger.core.v3:swagger-annotations-jakarta:2.2.22=apiHelper,compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testCompileClasspath,testRuntimeClasspath +jakarta.activation:jakarta.activation-api:2.1.3=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +jakarta.annotation:jakarta.annotation-api:3.0.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +jakarta.inject:jakarta.inject-api:2.0.1=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +jakarta.servlet:jakarta.servlet-api:6.1.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +jakarta.validation:jakarta.validation-api:3.1.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +jakarta.ws.rs:jakarta.ws.rs-api:4.0.0=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +jakarta.xml.bind:jakarta.xml.bind-api:4.0.2=apiHelper,jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath +javax.inject:javax.inject:1=annotationProcessor,errorprone,testAnnotationProcessor +junit:junit:4.13.2=jarValidation,testCompileClasspath,testRuntimeClasspath +net.java.dev.jna:jna-platform:5.17.0=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath +net.java.dev.jna:jna:5.18.1=compileClasspath,jarValidation,runtimeClasspath,runtimeLibs,testCompileClasspath,testRuntimeClasspath Review Comment: Noticing that jna-platform introduced in this branch does not align with jna? Declare it to same ver in toml file? ########## solr/solr-ref-guide/modules/deployment-guide/pages/backup-restore.adoc: ########## @@ -826,3 +826,124 @@ https://docs.aws.amazon.com/sdkref/latest/guide/settings-global.html[These optio * Retries ** RetryMode (`LEGACY`, `STANDARD`, `ADAPTIVE`) ** Max Attempts + +=== AzureBlobBackupRepository + +Stores and retrieves backup files in a Microsoft Azure Blob Storage container. + +This is provided via the `azure-blob-repository` xref:configuration-guide:solr-modules.adoc[Solr Module] that needs to be enabled before use. + +This plugin supports multiple authentication methods: connection strings, account keys, SAS tokens, and Azure Identity (Managed Identity, Service Principal, Azure CLI). +For Azure Identity, ensure the identity has the "Storage Blob Data Contributor" role on the storage account. + +[source,xml] +---- +<backup> Review Comment: Should perhaps re-iterate that this snippet goes in `solr.xml`, even if it is mentioned several hundred lines above? ########## solr/modules/azure-blob-repository/README.md: ########## @@ -0,0 +1,118 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +# Apache Solr Azure Blob Storage Backup Repository + +A backup repository implementation for storing Solr backups in Azure Blob Storage. + +## Prerequisites + +- Azure Storage Account with a blob container (must already exist) +- Network access to Azure Blob Storage (HTTPS port 443) + +Enable the module: +```bash +export SOLR_MODULES=azure-blob-repository +``` + +## Configuration + +Add to `solr.xml`: + +```xml +<backup> + <repository name="azure_blob" class="org.apache.solr.azureblob.AzureBlobBackupRepository" default="false"> + <str name="azure.blob.container.name">YOUR_CONTAINER_NAME</str> + <!-- Authentication options below --> + </repository> +</backup> +``` + +## Authentication Methods + +### Connection String (Development) + +```xml +<str name="azure.blob.connection.string">DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net</str> +``` + +### Account Name + Account Key + +```xml +<str name="azure.blob.endpoint">https://YOUR_ACCOUNT.blob.core.windows.net</str> +<str name="azure.blob.account.name">YOUR_ACCOUNT</str> +<str name="azure.blob.account.key">YOUR_ACCOUNT_KEY</str> +``` + +### SAS Token (Production) + +Generate a SAS token with permissions: Read, Write, Delete, List, Add, Create (`sp=rwdlac`) and resource types: Service, Container, Object (`srt=sco`). + +```xml +<str name="azure.blob.endpoint">https://YOUR_ACCOUNT.blob.core.windows.net</str> +<str name="azure.blob.sas.token">sv=2024-11-04&ss=b&srt=sco&sp=rwdlac&...</str> +``` + +Note: Escape `&` as `&` in XML. + +### Azure Identity (Production - Recommended) + +Uses Azure AD authentication. Requires "Storage Blob Data Contributor" role on the storage account. + +```xml +<str name="azure.blob.endpoint">https://YOUR_ACCOUNT.blob.core.windows.net</str> +<!-- No credentials needed when using Managed Identity or Azure CLI --> +``` + +For Service Principal, add: +```xml +<str name="azure.blob.tenant.id">YOUR_TENANT_ID</str> +<str name="azure.blob.client.id">YOUR_CLIENT_ID</str> +<str name="azure.blob.client.secret">YOUR_CLIENT_SECRET</str> +``` + +Or set environment variables: `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`. + +## Known Limitations + +Azure Identity authentication (Service Principal, Managed Identity, `DefaultAzureCredential`) does not work when Solr is started with the Java `SecurityManager` enabled. The Azure Identity SDK relies on `doPrivileged` patterns that fail under Solr's default security policy; see the [upstream issue](https://github.com/Azure/azure-sdk-for-java/issues/37464) for details. Note that the `SecurityManager` is deprecated for removal in modern JDKs, but Solr still enables it by default via `SOLR_SECURITY_MANAGER_ENABLED=true`. + +Workaround: set `SOLR_SECURITY_MANAGER_ENABLED=false` (in `solr.in.sh` / `solr.in.cmd`, or as an environment variable) before starting Solr. The Connection String, Account Key, and SAS Token authentication methods are unaffected and work with the `SecurityManager` enabled. + +## Usage + +```bash +# Backup +curl "http://localhost:8983/solr/admin/collections?action=BACKUP&name=my-backup&collection=my-collection&repository=azure_blob&location=/"; Review Comment: There are "Usage" and "Troubleshooting" sections in this README in the source folder. Do we expect end users to see / find this file? Should some of this instead be in the ref guide? What is the thinking here? ########## solr/modules/azure-blob-repository/build.gradle: ########## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +apply plugin: 'java-library' + +description = 'Azure Blob Storage Repository' + +ext { + // Disable security manager for azure-blob-repository module tests + // Required because Testcontainers needs access to Docker socket and system properties + useSecurityManager = false +} + +dependencies { + implementation platform(project(':platform')) + api(project(':solr:core')) + implementation project(':solr:solrj') + + implementation libs.apache.lucene.core + + implementation platform(libs.azure.sdk.bom) + + // Azure Storage SDK dependencies + implementation(libs.azure.storage.blob) { + exclude group: 'com.azure', module: 'azure-core-http-netty' + } + implementation(libs.azure.storage.blob.batch) { + exclude group: 'com.azure', module: 'azure-core-http-netty' + } + implementation(libs.azure.identity) { + exclude group: 'com.azure', module: 'azure-core-http-netty' + } Review Comment: I don't think we need desktop-style credentials persistence here, can shave two jar dependencies. ```suggestion exclude group: 'com.microsoft.azure', module: 'msal4j-persistence-extension' } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
