Suryaa shared this patch with me that has a draft: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21666 It highlighted a bug in the asterix-podman project that needs to be fixed. I think the structure is at least a great start though.
On Fri, Sep 4, 2026 at 3:53 PM Mike Carey <[email protected]> wrote: > > Spending a bit of time on open discussions this afternoon. :-) This is VERY > cool. I can think of a couple of things that could be of potential interest: > > 1. How to get cloud mode working on a laptop (esp. a Mac) for dev and/or > tire-kicking use. > 2. How to do a multi-node deployment (e.g., in AWS). > 3. How to scale up/down the number of compute partitions. > 4. How to use an actual S3 bucket as the cloud store in laptop mode (if one > wants to persist stuff there while still doing local work - e.g., to have > one's data separate so that it survives migrations from one version of the > system to the next). > > Cheers, > Mike > > On 7/31/26 9:43 AM, Suryaa Charan Shivakumar wrote: > > Hi all, > > > > I have been working on an AsterixDB cloud-mode primer and an accompanying > > runner that executes and validates each example. The current version covers: > > > > - Native collections backed by object storage > > - Database and namespace creation > > - JSON, CSV, and Parquet COPY TO/COPY FROM > > - External JSON and Parquet collections > > - Native storage and partition introspection > > - Secondary indexes and native mutations > > - Persistence after replacing the AsterixDB container > > - Iceberg queries, aggregation, snapshots, and time travel > > - Delta Lake external collections > > - Catalog metadata inspection > > > > For the local Iceberg example, the compose environment currently uses MinIO > > for the Iceberg objects and Nessie as the catalog. Nessie was selected > > because it provides a self-contained catalog that can be exercised locally. > > > > Does Nessie seem like the right catalog to present in the introductory > > primer, or would another catalog better represent how people commonly > > deploy Iceberg with AsterixDB? > > > > I would also appreciate feedback on other cloud-mode use cases we should > > demonstrate. In particular, are users commonly interested in areas such as > > multi-node deployments, cache eviction and rehydration, schema evolution, > > partition pruning, cloud credentials and authentication, operational > > monitoring, or failure and recovery scenarios? > > > > The goal is to keep the primer approachable while ensuring it reflects the > > workflows people actually use. > > > > Thanks, > > Suryaa > > > > On Fri, Jul 31, 2026 at 9:23 AM Ian Maxon <[email protected]> wrote: > > > >> Suryaa graciously tried out the patch and was able to get it to work > >> with some finagling. One of the issues is that cropped up is we don't > >> include the necessary module permissions for cloud mode to the JVM > >> args on startup by default. I thought I had added these a long time > >> ago but I remembered wrong I suppose. I have a patch up to add these > >> by default to all packaging by including them in the generated > >> appassembler helper script: > >> https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21477/1 > >> I'll go ahead and try deploying this on a real k8s cluster soon as > >> well. I think originally I only tried this on k3s or minikube, not > >> something with a real storage provider and all that. > >> > >> On Tue, Jul 28, 2026 at 8:43 AM Suryaa Charan Shivakumar > >> <[email protected]> wrote: > >>> > >>> Hello team, > >>> > >>> I’ll try this and update the thread with feedback in a bit! > >>> > >>> Best, > >>> Suryaa > >>> > >>> On Tue, Jul 28, 2026 at 7:43 AM Ian Maxon <[email protected]> wrote: > >>> > >>>> I don't think I got any feedback on it, at least that I can remember. > >>>> If someone could try the compose file, which uses the container > >>>> packaging that can be built in asterix-podman, that would be great. > >>>> > >>>> On Mon, Jul 27, 2026 at 5:08 PM Mike Carey <[email protected]> wrote: > >>>>> > >>>>> Restarting this discussion to see where it ended up! :-) > >>>>> > >>>>> (Doing some clearing out of older saved e-mail msgs.) > >>>>> > >>>>> On 1/15/26 4:44 PM, Ian Maxon wrote: > >>>>>> Hi fellow devs, > >>>>>> I've been working on improving the relatively nonexistent > >>>>>> documentation around cloud mode and all the related features, so > >> that > >>>>>> a new release can be made showcasing all the excellent work that > >> went > >>>>>> into them. At this point I have a compose file > >>>>>> (https://s.apache.org/demo-compose) that can be used with podman > >> or > >>>>>> docker to pretty easily start up a instance of AsterixDB running on > >>>>>> top of > >>>>>> S3Mock, that can be stopped and started and retain data. It also > >> shows > >>>>>> the relevant minimal configuration parameters for cloud mode. > >>>>>> > >>>>>> With this, I was trying to figure out what the tutorial/walkthrough > >>>>>> using this setup should showcase. I was thinking of something in > >> the > >>>>>> style of SQL++ 101, but perhaps as follow-on to that document, so > >>>>>> AsterixDB 102 or something. This is what I could think of off the > >> top > >>>>>> of my head, but it feels incomplete: > >>>>>> > >>>>>> - Database/Namespace > >>>>>> - COPY FROM/TO > >>>>>> - One or more external collection types > >>>>>> - Iceberg? Not sure if this would require additions to the compose > >>>>>> file, however. > >>>>>> > >>>>>> Anyone have any ideas on what else to add? > >>>>>> > >>>>>> - Ian > >>>> > >> > > >
