On Tue, Oct 27, 2015 at 12:16 PM, Juan Hernández <jhern...@redhat.com> wrote:
> On 10/27/2015 11:28 AM, Roman Mohr wrote: > > > > > > On Tue, Oct 27, 2015 at 10:47 AM, Juan Hernández <jhern...@redhat.com > > <mailto:jhern...@redhat.com>> wrote: > > > > On 10/27/2015 10:16 AM, Roman Mohr wrote: > > > > > > > > > On Mon, Oct 26, 2015 at 5:32 PM, Juan Hernández < > jhern...@redhat.com <mailto:jhern...@redhat.com> > > > <mailto:jhern...@redhat.com <mailto:jhern...@redhat.com>>> wrote: > > > > > > On 10/26/2015 04:56 PM, Roman Mohr wrote: > > > > Hi Juan, > > > > > > > > The way to specify the contract look pretty clean and nice. > > > > I would love to read a few words about the big picture. What > > is the > > > > final scenario? > > > > > > > > > > The motivation for this change is that currently we don't have > > a central > > > place where the RESTAPI is specified, rather we have several > > different > > > places, using several different technologies: > > > > > > * XML schema for the data model. > > > * JAX-RS for part of the operational model (without the > > parameters). > > > * rsdl_metadata.yaml for the parameters of the operational > model. > > > > > > This makes it difficult to infer information about the model. > For > > > example, the generators of the SDKs have to download the XML > > schema, and > > > the RSDL (which is generated from the JAX-RS interfaces using > > reflection > > > and combining it with the information from the > > rsdl_metadata.yaml file) > > > and then they have to do their own computations to extract > > what they > > > need. > > > > > > Same happens with the CLI: it has to extract the information > > it needs > > > from the Python code generated for the Python SDK, yet another > > level of > > > indirection. > > > > > > > > > You are right, that definitely needs to be cleaned up. I just want > to > > > discuss a few points below with you. > > > > > > > > > > > > We are also lacking a comprehensive reference documentation of > the > > > RESTAPI. What we currently have has been written by hand, and > > gets out > > > of sync very quickly, and we don't even notice. > > > > > > > > > Did you also consider swagger? It is made for exactly that purpose. > > > I created a demo in [1] which uses resteasy, weld, > hibernate-validator > > > and swagger to demonstrate how to do DRY with jaxrs. > > > Would be great to hear you thoughts on that. > > > > > > And there is the great swagger-ui [8] to display the documentation > > in a > > > more human readable way. > > > > > > > Yes, I considered Swagger, and rejected it because it is JSON > centric, > > and I think JSON isn't as good as Java to represent the contracts of > our > > RESTAPI. > > > > > > You just write plain jax-rs, swagger just creates a description out of > > it. So the source defining the contract is pure java (jax-rs with some > > swagger annotations for description, etc.). > > Or am I missing the point here? > > > > If I understand correctly the Swagger core is a JSON (or YAML) > specification of the API. From that you can generate JAX-RS annotated > code, not the other way around. So the specification document that you > write is a JSON document. > > Alternatively, you can use the Swagger annotations to decorate your > implementation, both the entity classes and the JAX-RS resource > implementations, and then extract the model from that. But this is > putting the implementation before the specification. That is where we > are today, and it causes multiple problems. I think it is better to have > the specification and the implementation separate. Swagger does that > well when using JSON directly, our metamodel also does it well, but > using a better language. > > > > > > > In addition we need to do these changes in a smooth way, without > causing > > big changes in the middle. For example, in the first step we need to > > preserve the JAX-RS interfaces as they are today, to avoid massive > > changes to all the resource implementations. This could be done with > > > > Swagger, but would require custom code generators. With less effort > we > > can do our own. > > > > > > This is of course generally a difficult task. But I do not know why it > > would be more difficult to write a custom swagger reader (if we even > > have to, it can read the interfaces as well) . > > They are pretty streight forward. Just look at [9], this contains the > > wole jax-rs specific code to generate the swagger documentation. > > > > But yes, I don't know every detail here of the engine and can't clearly > > say that integrating that would just streight forward (my feeling tells > > me that it would not be too hard). I am just under the impression that > > we would benefit from that. Just reduces custom magic to a minimum. > > > > Using something like Swagger would be certainly possible, and not that > hard, but it requires an effort. For example, say that we decide to use > the Swagger annotations. Then we will need to add these annotations to > all our JAX-RS resource implementations. That is a non trivial effort. > We would need to add the annotations to the entities as well. But wait, > we don't have such entities, only XML schema. So we would need a reader > that parses XML schema, and creating it requires effort. Where do we put > the documentation then? Part in the JAX-RS interfaces, part in the XML > schema. We are already there, and we ended up with no documentation at > all. In my view the sum of these efforts is higher than doing our own > metamodel. > > > > > > > Swagger UI is certainly great. I did test it and it is really good. > We > > may be able to copy some concepts. > > > > > > > > > > > To solve these issues I intend to have the specification of > > the RESTAPI > > > only in one place, and using only one technology. I decided to > > use Java > > > interfaces for that. Note however that they are just the > > support for the > > > information, like paper is the support for ink. I decided to > > use Java > > > because it is easy to create, modify and re-factor using tools > > familiar > > > to most of us. > > > > > > These source of these interfaces is analysed (using QDox, > > currently) and > > > a "model" of the RESTAPI is generated in memory. This model is > > > independent of the supporting Java source, and easy to > > consume. For > > > example, imagine that you want to list all the types available > > in the > > > model and for each one display its documentation: > > > > > > Model model = ...; > > > for (Type type : model.getTypes()) { > > > Name name = type.getName(); > > > String doc = type.getDoc(); > > > System.out.println(name + ": " + doc); > > > } > > > > > > Something like this, but more elaborate, will be part of a web > > > application that provides comprehensive reference > documentation, > > > assuming that we dedicate the time to write documentation > > comments in > > > the specification. > > > > > > I intend to use this model also to do simplify the generators > > of the > > > SDKs and the CLI. > > > > > > In addition these are some of the things that I would like to > > change in > > > the near future (for 4.0): > > > > > > * Move the specification of the parameters of operations out > > of the > > > rsdl_metadata.yaml file and into the model. For example: > > > > > > @Service > > > public VmService { > > > /** > > > * The operation to add a virtual machine. > > > */ > > > interface Add { > > > /** > > > * The representation of the virtual machine is received > > > * as parameter, and the representation of the created > > > * virtual machine is returned as result. > > > */ > > > @In @Out Vm vm(); > > > > > > /** > > > * In the future, we will be able to specify other > > > * parameters here. > > > */ > > > @In Boolean force(); > > > > > > /** > > > * Even with default values. > > > */ > > > @In default Boolean force() { return true; } > > > > > > /** > > > * And we will be able to specify constraints, which > > > * will replace the rsdl_metadata.yaml file. > > > */ > > > @Constraint > > > default boolean vmNameMustNotBeNull() { > > > return vm().name() != null; > > > } > > > } > > > } > > > > > > * Enforce the constraints automatically. If the constraints > > are in the > > > model, then we can just check them and reject requests before > > delivering > > > them to the application. Currently we do this manually (and > often > > > forget) with calls to "validate(...)" methods. > > > > > > > > > > > > Did you consider just annotating the DTOs with JSR-303 annotations > and > > > integrate a validator with jax-rs? > > > See [2] for an example. > > > > > > > This is a great way to implement a system, but the goal here isn't to > > implement it, rather to specify it. Using annotations in this way > won't > > help the generators of the SDKs, for example, to figure out what > > parameters are required, mandatory, etc. > > > > > > Swagger understands them. From my example project, swagger created that > > > > description: > > type: "string" > > minLength: 10 > > maxLength: 100 > > > > out of > > > > @Size(min=10, max=100) # jsr-303 > > private String description; > > > > and so does swagger-codegen which can generate clients in java, python, > ... > > > > This is extracting the specification from the implementation, which > isn't correct in my opinion, it should be the opposite. Not saying that > this makes Swagger bad, it is nice that it has this capability, but I > think we can do it better. > > > > > > > > > > * Generate the Java classes directly from the model. Instead > of Model -> > > > XML Schema -> Java, we can do Model -> Java. This will allow > us to solve > > > some of the XJC compiler limitations, like the horrible way we > handle > > > arrays today. > > > > > > > > > Swagger [3] is a rest documentation specification. There is also a > maven > > > plugin [4] and you can create clients for example with [5]. > > > > > > > > > > > > * Replace JAX-RS with a simpler infrastructure that supports > better > > > streaming and CDI injection. > > > > > > > > > > > > With resteasy-cdi you have pretty good injection support for > resteasy. > > > Run the demo in [1] to see it in action and look at the file at > [6]. > > > > > > > Resteasy-CDI isn't standard, it only works with Resteasy. If we rely > on > > it then we re tied to Resteasy for ever. > > > > > > Even jersey has support for that (I think it is called jeryse-gf-cdi), > > but why would we want switch? I don't think that jboss will drop > > resteasy and it also works fine outside of full blown containers. I > > don't think that this is an argument. > > > > Well, nobody thought that JBoss would drop Tomcat, and they did. Nobody > thought that Resteasy would change the SPI from 2.x to 3.x, and they did. > > We want (well, I want) to get out of JAX-RS because it doesn't support > well CDI and streaming. > By the way why do you think it does not support streaming well? As far as I know there are StreamingOutputs for specific endpoints and you can always plug in your own MessageBodyWriter which can serialize item by item of a list to return everything in a stream like fashion. If you use lazy evaluating collections like guava does for a long time for many things we would be pretty much do what you are after, right? JAX-RS is not the problem here, just standard jackson message body writers do not do it. > > > > > > > > > > > > * Add support for multiple versions of the API, using the > "Version" > > > header, and generating different Java classes for entities and > services. > > > For example, if we have versions 4 and 5 of the model as > separate > > > artifacts, then we can generate "V4Vm" and "V5Vm" entity > classes, and > > > "V4VmService" and "V5VmService" service classes. These can be > used > > > simultaneously in the server, so we can have in the same engine > > > implementations for multiple versions. > > > > > > > > > There are also many ways to do that. Here [7] is a pretty clean > way to > > > do it with jax-rs and you will have everything related in one > resource. > > > > > > > Yes, there are many ways. In my opinion it is better to use the HTTP > > "Version" header, and to forward requests to different resource > > implementations without requiring different URLs or different > > content types. > > > > Have no strong opinion there, just seemed to be a good choice regarding > > to versioning limitations in jax-rs and our use of jax-rs subresources. > > > > > > > > > > > > > The final picture isn't completely defined yet. > > > > > > Regards, > > > Juan Hernandez > > > > > > > On Mon, Oct 26, 2015 at 4:03 PM, Juan Hernández < > jhern...@redhat.com <mailto:jhern...@redhat.com> > > <mailto:jhern...@redhat.com <mailto:jhern...@redhat.com>> > > > > <mailto:jhern...@redhat.com <mailto:jhern...@redhat.com> > > <mailto:jhern...@redhat.com <mailto:jhern...@redhat.com>>>> wrote: > > > > > > > > Hello, > > > > > > > > I will soon merge the following patches that introduce a > new > > > way to > > > > specify the contracts of the RESTAPI: > > > > > > > > restapi: Introduce metamodel > > > > https://gerrit.ovirt.org/45852 > > > > > > > > restapi: Use metamodel > > > > https://gerrit.ovirt.org/46478 > > > > > > > > restapi: Generate JAX-RS interfaces from model > > > > https://gerrit.ovirt.org/47337 > > > > > > > > > > > > > > > Looks pretty much like we are replacing one way of > > annotating things > > > > with another way of specifying things. > > > > Could you elaborate what the benefit of that way of > > description is? > > > > > > > > How would I customize endpoints with e.g. @Gzip annotations? > > Would > > > I at > > > > the end still have my JAX-RS annotates resource classes? > > > > > > > > > > > > These patches introduce a new "metamodel" concept, and > move > > > the current > > > > specification of the RESTAPI based on XML schema and > JAX-RS > > > interfaces > > > > to a new "model" built on the new metamodel. > > > > > > > > > > > > What does this mean for you in practical terms? > > Currently when > > > you want > > > > to introduce or modify one of the data types used by the > > > RESTAPI you > > > > start by modifying the XML schema. Once the patches are > > merged > > > the XML > > > > schema will never be touched, as it will be automatically > > > generated from > > > > the "model". For example, imagine that you need to add a > new > > > "color" > > > > attribute to the "VM" entity. To do so with the new > > model you > > > will have > > > > to modify the following file, which is the specification > of > > > the "Vm" > > > > entity, written as a Java interface: > > > > > > > > > > > > > > > > > > https://gerrit.ovirt.org/#/c/46478/16/backend/manager/modules/restapi/model/src/main/java/types/Vm.java > > > > > > > > In that interface you will have to add a line like this: > > > > > > > > String color(); > > > > > > > > Note that this Java interface is just the specification > > of the > > > entity, > > > > it won't be used at all during runtime. Instead of that > the > > > XML schema > > > > will be generated from it, and then Java will be > generated > > > from the XML > > > > schema, as we do today (this will change in the future, > but > > > not yet). > > > > > > > > Same for the services. If you want to add a new "paint" > > action > > > to the > > > > "Vm" resource then you won't modify the JAX-RS > interfaces, > > > instead of > > > > that you will modify the following file, which is the > > > specification of > > > > the "Vm" service, written as a Java interface: > > > > > > > > > > > > > > > > > > https://gerrit.ovirt.org/#/c/47337/6/backend/manager/modules/restapi/model/src/main/java/services/VmService.java > > > > > > > > In that interface you will need to add a sub-interface > > > representing the > > > > action: > > > > > > > > interface Paint { > > > > } > > > > > > > > The JAX-RS interface will be generated from that. > > Currently these > > > > sub-interfaces are empty. In the future they will > > contain the > > > > specifications of the parameters (currently in the > > > rsdl_metadata.yml > > > > file). > > > > > > > > > > > > > > > > These changes will currently affect only the > > specification of the > > > > RESTAPI, not the implementation, so in in the > > > "Backend*Resource" classes > > > > things won't change yet. > > > > > > > > > > > > Currently I do not really understand where we are going > > here. Are we > > > > trying to get rid of rdsl? > > > > > > > > So basically two questions: > > > > > > > > 1) What is the final goal? > > > > 2) What speaks agains using Hibernate validator on Daos in > > combination > > > > with JAX-RS annotated resources (and just removing all > > interfaces, as > > > > far as I can see we only have one implementation per > > endpoint) and > > > > creating all schemas and clients through SWAGGER tooling? > > > > > > > > > > > > If you have doubts, please let me know. > > > > > > > > Regards, > > > > Juan Hernandez > > > > > > > > -- > > > > Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. > > Gorbea 3, > > > planta > > > > 3ºD, 28016 Madrid, Spain > > > > Inscrita en el Reg. Mercantil de Madrid – C.I.F. > B82657941 - > > > Red Hat > > > > S.L. > > > > _______________________________________________ > > > > Devel mailing list > > > > Devel@ovirt.org <mailto:Devel@ovirt.org> > > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>> > > > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org> > > <mailto:Devel@ovirt.org <mailto:Devel@ovirt.org>>> > > > > http://lists.ovirt.org/mailman/listinfo/devel > > > > > > > > > > > > Thanks, > > > > > > > > Roman > > > > > > > > > -- > > > Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, > > planta > > > 3ºD, 28016 Madrid, Spain > > > Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - > > Red Hat > > > S.L. > > > > > > > > > > > > I don't know if it is the right thing to do to invent something new > > > here. I personally would prefer to thread a path which is very > > common on > > > the java community. > > > I would love follow the DRY principle regarding to the stack and > the > > > code and would just use the great community projects there. > > > > > > It would also completely eliminate any custom magic. The JAX-RS > > and CDI > > > magic is pretty standard and easy to understand. > > > From my perspective, real JAX-RS resoures have the advantage of > > > > > > * being very easy to understand (there is magic, but the > > connection to > > > the real endpoint is pretty clear) > > > * being easy to customize suff, like adding @GZip to an annotation > > > * describing pretty clearly the connection between the generated > rest > > > interface and the internal services > > > > > > Finally writing hand crafted tests is also much easier. > > > > > > What are your thoughts about that? > > > > > > Best Regards, > > > Roman > > > > > > > > > [1] https://github.com/rmohr/jetty-maven-cdi-demo > > > [2] > > > > > > https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/MyDto.java > > > [3] http://swagger.io/ > > > [4] https://github.com/kongchen/swagger-maven-plugin > > > [5] https://github.com/swagger-api/swagger-codegen > > > [6] > > > > > > https://github.com/rmohr/jetty-maven-cdi-demo/blob/master/src/main/java/rmohr/examples/cdi/RestSubResource.java > > > [7] > > > > > > http://maxenglander.com/2013/04/23/basic-restful-api-versioning-in-jersey.html > > > [8] https://github.com/swagger-api/swagger-ui > > > > > > > [9] > > > https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-jaxrs/src/main/java/io/swagger/jaxrs/Reader.java > > > > -- > Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta > 3ºD, 28016 Madrid, Spain > Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. >
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel