lakshmi-manasa-g commented on a change in pull request #1323: Add docs for configs of Azure Blob SystemProducer URL: https://github.com/apache/samza/pull/1323#discussion_r397602908
########## File path: docs/learn/documentation/versioned/jobs/samza-configurations.md ########## @@ -245,6 +246,34 @@ Configs for producing to [ElasticSearch](https://www.elastic.co/products/elastic |systems.**_system-name_**.<br>bulk.flush.max.size.mb|5|The maximum aggregate size of messages in the buffered before flushing.| |systems.**_system-name_**.<br>bulk.flush.interval.ms|never|How often buffered messages should be flushed.| +#### <a name="azure-blob-storage"></a>[3.7 Azure Blob Storage](#azure-blob-storage) +Configs for producing to [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/). This section applies if you have set systems.**__system-name__**.samza.factory = `org.apache.samza.system.azureblob.AzureBlobSystemFactory`.<br> +**_system-name_** is the Azure container name you want to produce blobs to. If such a container does not exist then it is created.<br> + +|Name|Default|Description| +|--- |--- |--- | +|sensitive.systems.**_system-name_**.azureblob.account.name| |__Required:__ The Azure account name to which the Azure container belongs to. | +|sensitive.systems.**_system-name_**.azureblob.account.key| |__Required:__ Key for the Azure account specified above.| + +#### <a name="advanced-azure-blob-storage"></a>[Advanced Azure Blob Storage Configurations](#advanced-azure-blob-storage) +|Name|Default|Description| +|--- |--- |--- | +|systems.**_system-name_**.azureblob.proxy.use |"false"|if true, proxy will be used to connect to Azure.| +|systems.**_system-name_**.azureblob.proxy.hostname| |if proxy.use is true then host name of proxy.| +|systems.**_system-name_**.azureblob.proxy.port| |if proxy.use is true then port of proxy.| +|samza.azureblob.log.slowRequestMs|30 secs|The duration after which an Azure request will be logged as a warning.| +|systems.**_system-name_**.azureblob.writer.factory.class|`org.apache.samza.system.`<br>`azureblob.avro.`<br>`AzureBlobAvroWriterFactory`|Fully qualified class name of the `org.apache.samza.system.azureblob.producer.AzureBlobWriter` impl for the system producer.<br><br>The default writer creates blobs that are of type AVRO and require the messages sent to a blob to be AVRO records. The blobs created by the default writer are of type [Block Blobs](https://docs.microsoft.com/en-us/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs#about-block-blobs).<br>All the following configs are relevant to this default writer.| Review comment: yes, you are right. if a new non-default writer were to wired it, some of these would apply to it but not all - for example, the new writer might choose to create append blobs and not block blobs in which case flush threshold size and flush timeout and maybe even thread pool count dont make sense as these are all for uploading blocks of a blob. But understand your concern and removing that sentence. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
