Hi Niels, Thanks a lot for starting this discussion. Although the latter option is more stable, i think it is not acceptable for all the developers to execute `mvn generate-sources` first. Otherwise, the Flink project is just broken and could not run tests, Flink jobs in IDE.
So i think the version properties file is enough for now. +1 for the first option. Best, Yang Niels Basjes <ni...@basjes.nl> 于2020年4月9日周四 下午4:47写道： > Hi, > > I'm working on https://issues.apache.org/jira/browse/FLINK-16871 to make > more build time variables (like the scala version) into the code available > at runtime. > > During the review process there was discussion around a basic question: *Is > generating java code during the build ok?* > See > > - https://github.com/apache/flink/pull/11245#discussion_r400035133 > - https://github.com/apache/flink/pull/11592 > - https://github.com/apache/flink/pull/11592#issuecomment-610282450 > > As suggested by Chesnay Schepler I'm putting this question to the mailing > list. > https://github.com/apache/flink/pull/11592#issuecomment-610963947 > > The main discussion was around the ease of use when running in an IDE like > IntelliJ. > > So essentially we have two solution directions available: > > 1. *Generate a properties file and then use the classloader to load this > file as a resource and then parse it as a property file.* > This is the currently used solution direction for this part of the code. > A rough version of this (to be improved) : > > https://github.com/apache/flink/commit/47099f663b7644056e9d87b262cd4dba034f513e > This method has several effects: > 1. The developer can run the project immediately from within the IDE > as fallback values are provided if the 'actual' values are missing. > 2. This property file (with stuff that should never be overwritten) > can be modified by placing a different one in the classpath. In > fact it IS > modified in the flink-dist as it generates a new file with the same > name > into the binary distribution (I consider this to be bad). > 3. Loading resources means loading, parsing and a lot of error > handling. Lots of things "can be null" or be a default value. So the > values are unreliable and lots of code needs to handle this. In fact > when > running from IntelliJ the properties file is generated poorly most > of the > time, only during a normal maven build will it work correctly. > 2. *Generate a Java source file and then simply compile this and make it > part of the project.* > Essentially the same model as you would have when using Apache Avro, > Protobuf, Antlr 4 and javacc (several of those are used in Flink!). > A rough version of this (to be improved) : > > https://github.com/apache/flink/commit/d215e4df60dc9d647dcee1aa9a2114cbf49d0566 > This method has several effects: > 1. The developer MUST run 'mvn generate-sources' before the actual the > project immediately from within the IDE as fallback values are > provided if > the 'actual' values are missing. > 2. The code/test will not run until this step is done. > 3. Because the file is generated by a plugin it is always correct. As > a consequence all variables are always available and the downstream > users > no longer have to handle the "can be null" or "default value" > situations. > > So is generating code similar to what I created a desired change? > My opinion is that it is the better solution, the data available is more > reliable and as a consequence the rest of the code is simpler. > It would probably mean that during development of flink you should be aware > of this and do an 'extra step' to get it running. > > What do you guys think? > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes >
