Re: Static code analysis for Flink project

2018-06-14 Thread Joern Kottmann
I personally wouldn't mind digging a bit through the code and help a with fixing things. If there are some checks people feel are particular important let me know, happy to send a PR. Jörn On Thu, Jun 14, 2018 at 12:45 PM, Chesnay Schepler wrote: > Note that we attempted the same thing with

Re: Static code analysis for Flink project

2018-06-14 Thread Chesnay Schepler
Note that we attempted the same thing with spotbugs in the past but no one actually enabled more checks and fixed issues. This isn't an argument against trying it again, but to anyone who wants to introduce static checks again: Please be aware that you may end up pretty much on your own in

Re: Static code analysis for Flink project

2018-06-14 Thread Joern Kottmann
Hello, we introduced static code analysis 2 years back at Apache OpenNLP. To make this work with our also not perfect code base we added the checker to the build to make the build fail, then we disabled all checks which caused failures (or fixed them), afterwards we worked on resolving the

Re: Static code analysis for Flink project

2018-06-14 Thread Till Rohrmann
Looking at some of the bugs then these things are all valid problems. For example, the RowTypeInfo does override hashCode but not equals. Or the RocksDBKeyedStateBackend tries to close streams which might be null in case of an exception. I agree that we won't fix all the problems at once and it

Re: Static code analysis for Flink project

2018-06-13 Thread Chesnay Schepler
We will need a significant amount of exclusions/suppressions. For example, 300 of the 511 vulnerabilities are caused by our TupleX classes, with public mutable fields. The "unclosed resource" inspection is rather simplistic and only tracks the life-cycle in the scope of the method. As a result

Re: Static code analysis for Flink project

2018-06-13 Thread Piotr Nowojski
Hi, Generally speaking I would be in favour of adding some reasonable static code analysis, however this requires a lot of effort and can not be done at once. Also if such checks are firing incorrectly too often it would be annoying to manually suppress such warnings/errors. > I don't really

Re: Static code analysis for Flink project

2018-06-13 Thread Chesnay Schepler
I don't really see a benefit in enabling it /continuously/. This wouldn't be part of the build or CI processes, as we can't fail the builds since it happens too often that issues are improperly categorized. Wading through these lists is time-consuming and I very much doubt that we will do

Re: Static code analysis for Flink project

2018-06-13 Thread Till Rohrmann
Hi Alex, thanks for bringing this topic up. So far the Flink project does not use a static code analysis tool but I think it can strongly benefit from it (simply by looking at the reported bugs). There was a previous discussion about enabling the ASF Sonarcube integration for Flink [1] but it was

Re: Static code analysis for Flink project

2018-06-12 Thread Ted Yu
I took a look at some of the blocker defects. e.g. https://sonarcloud.io/project/issues?id=org.apache.flink%3Aflink-parent=AWPxETxA3e-qcckj1Sl1=false=BLOCKER=BUG For ./flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/PredefinedOptions.java ,

Static code analysis for Flink project

2018-06-12 Thread simeon.arkhipov
Hello Flink community. I am new in Flink project and probably don't understand it a lot. Could you please clarify one question to me? I download Flink sources and build it from scratch. I found checkstyle guidelines that every Flink developer should follow which is very useful. However, I