Thank you for the pointers and the description in OAK-6922. It all makes
sense and seems like a reasonable approach. I assume the description is
How does it perform compared to TarMK
a) when the entire repo doesn't fit into RAM allocated to the container ?
b) when the working set doesn't fit into RAM allocated to the container ?
Since you mentioned cost, have you done a cost based analysis of RAM vs
attached disk, assuming that TarMK has already been highly optimised to
cope with deployments where the working set may only just fit into RAM ?
IIRC the Azure attached disks mount Azure Blobs behind a kernel block
device driver and use local SSD to optimise caching (in read and write
through mode). Since there are a kernel block device they also benefit from
the linux kernel VFS Disk Cache and support memory mapping via the page
cache. So An Azure attached disk often behaves like a local SSD (IIUC). I
realise that some containerisation frameworks in Azure dont yet support
easy native Azure disk mounting (eg Mesos), but others do (eg AKS)
On 1 March 2018 at 18:40, Matt Ryan <o...@mvryan.org> wrote:
> Hi Tomek,
> Some time ago (November 2016 Oakathon IIRC) some people explored a similar
> concept using AWS (S3) instead of Azure. If you haven’t discussed with
> them already it may be worth doing so. IIRC Stefan Egli and I believe
> Michael Duerig were involved and probably some others as well.
> On March 1, 2018 at 5:42:07 AM, Tomek Rekawek (reka...@adobe.com.invalid)
> Hi Tommaso,
> so, the goal is to run the Oak in a cloud, in this case Azure. In order to
> do this in a scalable way (eg. multiple instances on a single VM,
> containerized), we need to take care of provisioning the sufficient amount
> of space for the segmentstore. Mounting the physical SSD/HDD disks (in
> Azure they’re called “Managed Disks” aka EBS in Amazon) has two drawbacks:
> * it’s expensive,
> * it’s complex (each disk is a separate /dev/sdX that has to be formatted,
> mounted, etc.)
> The point of the Azure Segment Store is to deal with these two issues, by
> replacing the need for a local file system space with a remote service,
> that will be (a) cheaper and (b) easier to provision (as it’ll be
> configured on the application layer rather than VM layer).
> Another option would be using the Azure File Storage (which mounts the SMB
> file system, not the “physical” disk). However, in this case we’d have a
> remote storage that emulates a local one and SegmentMK doesn’t really
> expect this. Rather than that it’s better to create a full-fledged remote
> storage implementation, so we can work out the issues caused by the higher
> latency, etc.
> Tomek Rękawek | Adobe Research | www.adobe.com
> > On 1 Mar 2018, at 11:16, Tommaso Teofili <tommaso.teof...@gmail.com>
> > Hi Tomek,
> > While I think it's an interesting feature, I'd be also interested to hear
> > about the user story behind your prototype.
> > Regards,
> > Tommaso
> > Il giorno gio 1 mar 2018 alle ore 10:31 Tomek Rękawek <tom...@apache.org
> > ha scritto:
> >> Hello,
> >> I prepared a prototype for the Azure-based Segment Store, which allows
> >> persist all the SegmentMK-related resources (segments, journal,
> >> etc.) on a remote service, namely the Azure Blob Storage . The whole
> >> description of the approach, data structure, etc. as well as the patch
> >> be found in OAK-6922. It uses the extension points introduced in the
> >> OAK-6921.
> >> While it’s still an experimental code, I’d like to commit it to trunk
> >> rather sooner than later. The patch is already pretty big and I’d like
> >> avoid developing it “privately” on my own branch. It’s a new, optional
> >> Maven module, which doesn’t change any existing behaviour of Oak or
> >> SegmentMK. The only change it makes externally is adding a few exports
> >> the oak-segment-tar, so it can use the SPI introduced in the OAK-6921.
> >> may narrow these exports to a single package if you think it’d be good
> >> the encapsulation.
> >> There’s a related issue OAK-7297, which introduces the new fixture for
> >> benchmark and ITs. After merging it, all the Oak integration tests pass
> >> the Azure Segment Store.
> >> Looking forward for the feedback.
> >> Regards,
> >> Tomek
> >>  https://azure.microsoft.com/en-us/services/storage/blobs/
> >> --
> >> Tomek Rękawek | Adobe Research | www.adobe.com
> >> reka...@adobe.com