[
https://issues.apache.org/jira/browse/BEAM-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15961578#comment-15961578
]
Ahmet Altay commented on BEAM-1677:
-----------------------------------
Thank you [[email protected]]. This is a great proposal and I fully support
it. I have some concerns but I think they could be addressed:
* We need to make sure that this does not effect developers. That is:
a) It should not be required to install PyCharm as part of local development
experience. Also note that doing this would mean that developers cannot test
for all lint related issues in their development environment and this may cause
pain as well.
b) Enforcements should come in a soft way initially and we can add hard rules
slowly. Possibly we need a python sdk coding guideline. (cc'ing some people who
might be interested [~sb2nov] [~vikasrk])
* It should not cause frequent issues with Jenkins. (For example how do we
handle failures in the tool, upgraded to pycharm, any technical issues with
running it)
* Are there any license issues? (I am not sure we qualify for a free license in
this case.)
> Extend automated PR check with PyCharm's code inspector for Python-SDK
> ----------------------------------------------------------------------
>
> Key: BEAM-1677
> URL: https://issues.apache.org/jira/browse/BEAM-1677
> Project: Beam
> Issue Type: Improvement
> Components: build-system
> Reporter: Tibor Kiss
> Assignee: Ahmet Altay
> Attachments:
> inspection-results-1d8be97439051ae770f99dcb49c7e8486962b118.tar.gz
>
>
> *Proposal:*
> Leverage PyCharm's static code analysis capabilities to detect possible
> errors if a Python-SDK related PR is submitted.
> *Problem statement:*
> The Python SDK test goal has linting phase, which utilizes {{pep8}} and
> {{pylint}} to validate PEP8 compliance and do static code analysis. These
> steps are helpful but {{pylint}}'s capabilities are somewhat limited compared
> to modern IDE's (like PyCharm).
> In particular: Unresolved attributes, unbound local variables and invalid
> calls (due to argument number mismatch) type of errors are not detected by
> pylint, but they do cause runtime error. See BEAM-1649 for problems not
> detected by pylint but reported by PyCharm.
> *Technical details:*
> PyCharm provides command line interface [1] to trigger code inspection. The
> results are created as entries in XML documents, containing their location,
> severity, category and textual description of the problem.
> Since the inspector reports back typos, warnings and errors it is required
> that the result XML files are properly parsed and filtered.
> A good signal-to-noise ratio is crucial for this improvement to be useful and
> accepted, therefore it is required to make good selection of events to
> minimize false positives.
>
> The filtered results shall be posted back to GitHub through Commit Status API
> [2] to notify the contributor and reviewer of the quality of the change.
>
> The static code analysis could be done on either Apache Build Jenkins servers
> or in Travis CI.
>
> *Alternatives considered:*
> - Eclipse's PyDev plugin [3] has similar code inspection capabilities as
> PyCharm. The lack of command line interface makes it difficult to integrate
> it with the current build system.
> - SonarQUBE's Python plugin [4] was briefly investigated. The number of
> rules seems very limited compared to PyCharm's code analysis.
>
> *Required changes:*
> - PyCharm to be installed in Jenkins Slave or Travis-CI's container
> - PyCharm project & inspection profile created for Python SDK
> - Result parser & poster application/script created
> - Trigger to run the result parser & poster set up
>
> *Example Code Analysis* run on OS X (given a PyCharm project is available in
> beam/sdks/python/.idea directory):
> {code}
> beam/sdks/python $ /Applications/PyCharm.app/Contents/MacOS/pycharm inspect
> $(pwd) $(pwd)/.idea/inspectionProfiles/Project_Default.xml
> $(pwd)/inspection-results -v2 -d $(pwd)/apache_beam
> {code}
>
> *Example result:*
> {code:title=PyUnresolvedReferencesInspection.xml}
> <problem>
> <file>file://$PROJECT_DIR$/apache_beam/io/range_trackers.py</file>
> <line>342</line>
> <module>python</module>
> <entry_point TYPE="file"
> FQNAME="file://$PROJECT_DIR$/apache_beam/io/range_trackers.py" />
> <problem_class severity="WARNING"
> attribute_key="WARNING_ATTRIBUTES">Unresolved references</problem_class>
> <description>Unresolved attribute reference '_end_position' for class
> 'OrderedPositionRangeTracker'</description>
> </problem>
> {code}
>
> *References:*
> [1] PyCharm Command Line Interface:
> https://www.jetbrains.com/help/pycharm/2016.3/command-line-code-inspector.html
> [2] GitHub Commit Status API: https://developer.github.com/v3/repos/statuses/
> [3] Eclipse PyDev Plugin: http://www.pydev.org/manual_adv_code_analysis.html
> [4] SonarQUBE Python plugin:
> https://www.sonarsource.com/why-us/products/codeanalyzers/sonarpython/rules.html
>
> CC: [~altay] / [~jbonofre] / [~jasonkuster]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
