Jerry, Great to see that you're making some progress. I'm not very familiar with Couchbase, but it seems like there is a drastic difference in the data model between versions 3 and 4. You may find it easier to create a separate adapter to handle Couchbase 3 if supporting it is important to you. This would solve points #1 and #2 since you could use views in Couchbase 3 and N1QL in Couchbase 4. I'm not quite sure what you're proposing in point #3. A more concrete example might be helpful.
Regarding point #4, I'd suggest not worrying about that for the time being. If your adapter doesn't explicitly handle aggregation, you can have Calcite handle this client-side for now. Performance will probably be relatively poor, but this is something that can be fixed on the future once you have a working adapter. Hope some of this helps! Cheers, -- Michael Mior [email protected] 2016-07-05 19:58 GMT-04:00 Yang, Jerry <[email protected]>: > Hi guys, > > As I started working on couchbase adapters, I’ve encountered some > questions that I’d hope to clarify prior to further implementation. > > Couchbase currently supports two major versions: 3 and 4. In coucbase 4, > N1QL, a SQL-like language is introduced. In N1QL, users can select some > fields inside values (essentially the JSON document) from a bucket, much > like what CQL of Cassandra has. It makes the implementation simple and > neat as far as Calcite is concerned; most of the SQL from Calcite can yield > similar mappings to N1QL; as long as primary index is created, “SELECT * > from [bucket_name]” is possible. > > Couchbase 3, on the other hand, is not as simple. View in couchbase > contains a mapper and a reducer, and a query is implemented via ‘view’, so > fields needs to be part of view; queries containing outside fields wouldn’t > work. Also, filters are done at the client level if not defined in view. > See below for API example. (Java8; I’d probably implement this in guava > since calcite does not support Java8 yet, but you get the idea.) > > bucket > .async() > .query(ViewQuery.from("beers_and_breweries", "by_name")) > .flatMap(AsyncViewResult::rows) > .flatMap(AsyncViewRow::document) > .filter(doc -> doc.content().getString("type").equals("beer")) > .subscribe(doc -> System.out.println(doc.content().getString("name"))); > (http://docs.couchbase.com/developer/java-2.1/java-intro.html) > > So here’smy take away. > > 1. It seems to me it makes more sense to do “SELECT * from > [view_name]” instead of bucket name; view contains set of fields to select > from. Yet it is NOT compatible with couchbase 4 and in general not a good > idea. > > 2. Should I choose to use ‘bucket’ as table, I then have to predefine > bucket name as well as view name in my model, which makes ‘FROM’ obsolete > since it doesn’t consider “FROM” field anymore. > > 3. Potentially we could have a table schema as part of the model, > much like mongodb, and then dynamic create view(s) based on table schema. > However, it is very invasive and I am not sure of the implication of the > performance. Also, it requires quite some work since it means a schema to > mapper/reducer transformer. > > 4. With aggregation like distinct or count, it would have to be done > via client side aggregation if it is not part of the view; performance > might be poor without reducer. > > Any thoughts would be greatly appreciated. > > Thanks, > Jerry > > > On 6/28/16, 9:02 AM, "[email protected] on behalf of Michael Mior" < > [email protected] on behalf of [email protected]> wrote: > > AFAIK there are not currently any plans to develop a Couchbase adapter. I > think this would be a welcome addition to Calcite. The Cassandra and > MongoDB adapters are probably a good place to start. It would be great if > you would be willing to contribute this to the project when you're done. > Feel free to ask here if you have any specific questions when you get > started! > > -- > Michael Mior > [email protected] > > 2016-06-28 11:45 GMT-04:00 Yang, Jerry <[email protected]>: > > > Hi guys, > > > > Just joined the mailing list. Please advise if this is the proper place > > for such discussion, thanks in advance. > > > > My colleague and I are using calcite to develop our project, and we need > > to support couchbase as part of the requirement. I am wondering if > > couchbase adapter is on the roadmap, or it is not planned yet. > > > > We are planning to support both couchbase 3 and 4, so if it is not on the > > roadmap, we might be able to contribute back to the project when we are > > done. > > > > Currently I have some preliminary designs. The approach to couchbase > > would be different based on major version. In couchbase 4, since it > > supports n1ql, an implementation similar to Cassandra’s CQL might be > > feasible, whereas in couchbase 3, native API calls is required. > > > > Thanks again for help and please let me know if there’s a couchbase > > adapter, otherwise we are planning to do it ourselves. > > > > Thanks, > > Jerry Yang > > >
