Hi, I've now completed the Core/UI serialization changes and wrote a short documentation that explains how to create Typescript models from Java models:
https://cwiki.apache.org/confluence/display/STREAMPIPES/Core-UI+Serialization Feel free to ask if you have any questions or something is unclear! Dominik On 2020/06/18 06:34:11, Dominik Riemer <[email protected]> wrote: > Hi Philipp, > > yes, either one or two, currently it's two (one for streampipes-model and one > for streampipes-model-client), see [1]. > > Dominik > > [1] > https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-145/ui/src/app/core-model/gen > > On 2020/06/17 09:05:36, Philipp Zehnder <[email protected]> wrote: > > Hi Dominik, > > > > the code generation looks good and I look forward to use it. > > > > Do I understand it correctly, there will be one file containing the whole > > generated model, right? > > > > > > Philipp > > > > > On 10. Jun 2020, at 18:00, Dominik Riemer <[email protected]> wrote: > > > > > > Hi Philipp, > > > > > > coming back to your questions after doing more experiments and code > > > refactoring: > > > > > > * I integrated a Maven Plugin [1] to streampipes-model and > > > streampipes-model-client. It creates typescript files by executing mvn > > > typescript-generator:generate and outputs a single file to > > > target/typescript-generator. > > > * I added an annotation @TsModel, the plugin picks up every class with > > > this annotation and generates typescript classes [2]. > > > * There is a folder core-model/gen in the UI where model files need to be > > > copied to [3] > > > * The serialization layer is now based on JSON/Jackson, it's still WIP > > > and there are quite a few things that need to be modified in the UI. I'm > > > gradually replacing @Gson annotations in the REST interfaces with > > > @JacksonSerialized so that all models can be easily deserialized using > > > some utility methods provided by the typescript-generator > > > * To support serialization of polymorphic classes (e.g., lists of static > > > properties), type hints need to be added to the abstract class, see an > > > example here [4]. This is similar to the GSON adapters we had before. > > > * So coming back to your questions, in case any data models served by our > > > REST interfaces are changed, the typescript-generator needs to be run > > > (copying can probably be automated). In case classes need to be changed > > > in the UI you can either cast types to <any> or create wrapper objects. > > > In general, the new approach should make UI development much less > > > error-prone. > > > > > > It would be great if you all have a look - feedback is welcome! > > > > > > Dominik > > > > > > > > > [1] > > > https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/pom.xml > > > > > > <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/pom.xml> > > > [2] > > > https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model-shared/src/main/java/org/apache/streampipes/model/shared/annotation/TsModel.java > > > > > > <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model-shared/src/main/java/org/apache/streampipes/model/shared/annotation/TsModel.java> > > > [3] > > > https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/ui/src/app/core-model/gen/streampipes-model.ts > > > > > > <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/ui/src/app/core-model/gen/streampipes-model.ts> > > > [4] > > > https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/src/main/java/org/apache/streampipes/model/schema/EventProperty.java > > > > > > <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/src/main/java/org/apache/streampipes/model/schema/EventProperty.java> > > > > > > On 2020/06/05 08:19:47, Philipp Zehnder <[email protected] > > > <mailto:[email protected]>> wrote: > > >> Hi Dominik, > > >> > > >> +1 > > >> I totally agree, we need a unified solution for the communication of the > > >> backend and front-end. > > >> > > >> I have question regarding the TypeScript code generation. You mentioned > > >> we could generate the classes in a Maven build, right? > > >> How would the development of the UI then look like? How can a UI > > >> developer generate those classes and can they be changed or are they > > >> immutable? > > >> > > >> Philipp > > >> > > >>> On 2. Jun 2020, at 23:33, Dominik Riemer <[email protected]> wrote: > > >>> > > >>> Hi all, > > >>> > > >>> > > >>> > > >>> during the last days, while starting to migrate the editor UI module to > > >>> Angular, I did some experiments with different serialization approaches. > > >>> > > >>> > > >>> > > >>> Currently, we use two different serialization approaches in the UI: > > >>> > > >>> 1. JSON for most things and the editor module also uses JSON to > > >>> represent static properties (which are used to render the UI components > > >>> in > > >>> the pipeline element configuration) > > >>> 2. The (newer) Connect module uses JSON-LD (which is also used in > > >>> the > > >>> core to represent pipeline elements) to represent static properties. > > >>> > > >>> > > >>> > > >>> Both approaches have advantages and disadvantages: > > >>> > > >>> For (1), we use GSON for serializing JSON, which is easy and fast for > > >>> most > > >>> cases, but there is no direct mapping to TypeScript classes for complex > > >>> objects (like static property definitions with polymorphism), so that we > > >>> rely on the <any> type here in most cases (especially in the editor > > >>> module) > > >>> -> all type information is lost and UI development is more error-prone. > > >>> > > >>> For (2), we use a self-made JSON-LD serializer named Tson-LD. This works > > >>> great, but TypeScript-classes need to be manually modified every time > > >>> classes in streampipes-core change, which increases maintenance effort > > >>> and > > >>> also is rather error-prone. On the other hand, Tson-LD deserializes > > >>> JSON-LD > > >>> into TypeScript classes, which is quite convenient from a developer > > >>> perspective. > > >>> > > >>> > > >>> > > >>> As we will soon be able to completely remove AngularJS from the UI, it > > >>> also > > >>> makes sense to rethink and streamline our serialization approach > > >>> (mainly to > > >>> have a single component that cares about generating UI components from > > >>> static properties, both in Connect and the Pipeline Editor). > > >>> > > >>> > > >>> > > >>> So after doing some research and testing I'd propose the following > > >>> approach: > > >>> > > >>> * Use only JSON as a serialization format for UI-core > > >>> communication > > >>> and remove JSON-LD from the UI > > >>> * Use Jackson over GSON as a serialization library, as this better > > >>> works with the next point: > > >>> * Use typescript-generator [1] to auto-generate TypeScript classes > > >>> from Java models. There is not much configuration needed and there is a > > >>> Maven plugin which could automatically run the code generation at every > > >>> build. > > >>> > > >>> > > >>> > > >>> This approach would have less overhead, guarantees that all models are > > >>> always up-to-date and reduces maintenance effort. Drawback is that > > >>> quite a > > >>> few UI modules would need to be changed. > > >>> > > >>> > > >>> > > >>> I have already a prototype running which shows that the approach works > > >>> quite > > >>> well. But before I continue, I'm interested whether you think this is a > > >>> good > > >>> or bad idea and if you have any other best practice approaches? > > >>> > > >>> > > >>> > > >>> Dominik > > >>> > > >>> > > >>> > > >>> [1] <https://github.com/vojtechhabarta/typescript-generator> > > >>> https://github.com/vojtechhabarta/typescript-generator > > >>> > > >>> > > >>> > > >>> > > >>> > > >> > > >> ......................................................... > > >> M. Sc. Philipp Zehnder > > >> Wissenschaftlicher Mitarbeiter | Research Scientist > > >> Information Process Engineering (IPE) > > >> > > >> FZI Forschungszentrum Informatik > > >> Haid-und-Neu-Str. 10–14 > > >> 76131 Karlsruhe, Germany > > >> Tel.: +49 721 9654-805 > > >> Fax: +49 721 9654-806 > > >> > > >> [email protected] <mailto:[email protected]> <mailto:[email protected] > > >> <mailto:[email protected]>> > > >> https://www.fzi.de/mitarbeiter/philipp-zehnder > > >> > > >> ......................................................... > > >> FZI Forschungszentrum Informatik > > >> Stiftung des bürgerlichen Rechts > > >> Stiftung Az: 14-0563.1 Regierungspräsidium Karlsruhe > > >> Vorstand: Prof. Dr. Andreas Oberweis, Jan Wiesenberger, Prof. Dr.-Ing. > > >> J. Marius Zöllner > > >> Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus > > >> ......................................................... > > > > ......................................................... > > M. Sc. Philipp Zehnder > > Wissenschaftlicher Mitarbeiter | Research Scientist > > Information Process Engineering (IPE) > > > > FZI Forschungszentrum Informatik > > Haid-und-Neu-Str. 10–14 > > 76131 Karlsruhe, Germany > > Tel.: +49 721 9654-805 > > Fax: +49 721 9654-806 > > > > [email protected] <mailto:[email protected]> > > https://www.fzi.de/mitarbeiter/philipp-zehnder > > > > ......................................................... > > FZI Forschungszentrum Informatik > > Stiftung des bürgerlichen Rechts > > Stiftung Az: 14-0563.1 Regierungspräsidium Karlsruhe > > Vorstand: Prof. Dr. Andreas Oberweis, Jan Wiesenberger, Prof. Dr.-Ing. J. > > Marius Zöllner > > Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus > > ......................................................... > > > > >
