Hi Thorsten, On Mon, Oct 5, 2020 at 1:08 PM Thorsten Schöning <[email protected]> wrote:
> > - jOOQ's DSL isn't very small in terms of byte code size.[...] > > Which tells me it consumes a "lot of memory" on runtime even for > unused things, correct? Java's class loaders are lazy, so unused things might not consume a lot. I don't know your environment. I don't know the impact of jar file sizes there. Just wanted to point you to some things to look out for. Anyway. Do check out Proguard and similar tools. They're designed for such cases. Also, the jOOQ lite distribution would really be useful to you, I think (Again, https://github.com/jOOQ/jOOQ/issues/8928 for reference). The lowest hanging fruit is to remove the @Support annotation from the byte code. It already produces 68kb. Then, all the Record1 - Record22 types aren't really needed. Perhaps, we keep Record1, which is useful because single-column queries are special, e.g. for IN predicate usage. But all the rest isn't needed. If this is something you're interested in, I'm happy to discuss a sponsored project to expedite this. > Do you have an absolute number like 1 MiB or 10 or 100? > Would that really be meaningful to you? 1 MiB of what? Maximum memory usage? Maximum TLAB usage? Maximum GC throughput per second? Maximum class memory usage (permgen in old Java versions)? We don't have such numbers because any number would depend on so many things that you do, that we can't guess / assume... With an environment like yours, I'm pretty sure you will need to run continuous memory consumption tests along with your integration tests and keep an eye out for outliers of anything, including your own memory usage. A rogue HashMap can kill your device if it implements a cache that you're unaware of during dev time. > I really don't have any feeling about how much memory we talk here. > With my Raspi-powered PoC, I hosted Tomcat+Axis2 and stuff like that > and 128 MiB of RAM weren't enough anymore, but I didn't try e.g. 256 > MiB RAM and Tomcat+Axis2 might not used as well. > Tomcat does a lot of things. Do you really need a container? > > [...]It would be > > interesting to be able to construct jOOQ queries either on this server, > and > > then ship the query strings to your ARM-based environment (I'm aware of a > > customer who does this, for entirely different reasons), or use some way > > generate jOOQ queries at build-time, shipping the SQL statements in a > file. > > If at all, the latter would better fit my use case, but it's difficult > to decide currently if some intermediate file format would be of any > benefit. Those files would need to be read as well and decided if to > keep them in memory or alike. > The alternative is to use SQL string constants which are kept in bytecode and would also consume memory. But again, I think you should measure things and check out specific bottlenecks. I'm sure there are lower hanging fruit than stored SQL strings. > How would that compete with generating queries at startup time of some > device/app and caching jOOQ-instances for queries itself in memory? > The jOOQ query instance probably consumes more memory than the SQL string itself (there's a lot of overhead with object headers alone). Also, pre-generating the SQL string optimises CPU, which you probably don't have abundantly either. > The files in theory could be mapped into memory only and file system > cache would optimize things, while jOOQ- instances would always need > to be available in memory. Difficult and only worth it if really every > CPU-cycle or bit of RAM counts I guess. > Hard to say. Measure! :) > > In fact, I have thought about offering such tooling in the past. It would > > be really interesting to do all the SQL generation work at compile time, > > and then produce type safe JDBC (or R2DBC, etc.) wrapping logic, > including > > the logic that maps query results to POJOs. Perhaps something similar to > > SQLDelight, but more powerful. > > > Would that be something like what you had in mind? > > When it comes to optimization, yes. But as said, I'm not sure where I > need to optimize right now at all, e.g. CPU cycles or RAM or I/O to read > files or ... What I somewhat know is that my use case needs to INSERT > a lot and when it comes to reading, it's e.g. often about comparing > timestamps to decide if to INSERT at all or simply reading bulk > inserted binary blobs to forward them somewhere else. > > So, I wouldn't benefit too much from reading into POJOs, but caching > my queries should really be possible, as they are not too dynamic at > runtime at all. > > Whatever tooling you have in mind, it might be a good idea to consider > things like GraalVM native images. From my limited knowledge, that > seems to execute static code or alike during the generation and that > might be used by jOOQ as entry point to generate queries and cache > them in memory. > Yes, that GraalVM feature will definitely be investigated in the near future. -- You received this message because you are subscribed to the Google Groups "jOOQ User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAB4ELO6AMTfn-_vTKu9JMuoAtpOT6KRna4_X%3DCDcGRaeUp%3D8jw%40mail.gmail.com.
