[
https://issues.apache.org/jira/browse/SOLR-15089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17283321#comment-17283321
]
Andy Throgmorton commented on SOLR-15089:
-----------------------------------------
Great idea, [~gerlowskija]! How far along are you in coding this effort? I work
at Salesforce and we wrote+use an S3 implementation of BackupRepository for a
production Solr stack. It's not in an upstreamable state right now (e.g., uses
some internal libraries for grabbing keys/secrets, etc.), but I would be happy
to look into cleaning it up and submitting it for consideration if you haven't
started yet. Or if you've already written the code, then feel free to add me on
your code review.
In regards to testing, we use the Adobe S3Mock
(https://github.com/adobe/S3Mock) library for writing unit tests. Since this
code is fairly simple, as you mentioned, the S3 APIs it uses are all mainstream
and mockable with that framework. For larger, end-to-end integration tests,
we've also started using Minio (https://min.io/) to emulate an S3 server, but I
would think that's outside the scope of this ticket.
> Allow backup/restoration to Amazon's S3 blobstore
> --------------------------------------------------
>
> Key: SOLR-15089
> URL: https://issues.apache.org/jira/browse/SOLR-15089
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Jason Gerlowski
> Priority: Major
>
> Solr's BackupRepository interface provides an abstraction around the physical
> location/format that backups are stored in. This allows plugin writers to
> create "repositories" for a variety of storage mediums. It'd be nice if Solr
> offered more mediums out of the box though, such as some of the "blobstore"
> offerings provided by various cloud providers.
> This ticket proposes that a "BackupRepository" implementation for Amazon's
> popular 'S3' blobstore, so that Solr users can use it for backups without
> needing to write their own code.
> Amazon offers a s3 Java client with acceptable licensing, and the required
> code is relatively simple. The biggest challenge in supporting this will
> likely be procedural - integration testing requires S3 access and S3 access
> costs money. We can check with INFRA to see if there is any way to get cloud
> credits for an integration test to run in nightly Jenkins runs on the ASF
> Jenkins server. Alternatively we can try to stub out the blobstore in some
> reliable way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
