GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/11292
[SPARK-13423] [WIP] [CORE] [SQL] [STREAMING] Static analysis fixes for 2.x
## What changes were proposed in this pull request?
Make some cross-cutting code improvements according to static analysis.
These are individually up for discussion since they exist in separate commits
that can be reverted. The changes are broadly:
- Inner class should be static
- Mismatched hashCode/equals
- Overflow in compareTo
- Unchecked warnings
- Misuse of assert, vs junit.assert
- get(a) + getOrElse(b) -> getOrElse(a,b)
- Array/String .size -> .length (occasionally, -> .isEmpty / .nonEmpty) to
avoid implicit conversions
- Dead code
- tailrec
- exists(_ == ) -> contains find + nonEmpty -> exists filter + size -> count
- reduce(_+_) -> sum map + flatten -> map
The most controversial may be .size -> .length simply because of its size.
It is intended to avoid implicits that might be expensive in some places.
## How was the this patch tested?
Existing Jenkins unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srowen/spark SPARK-13423
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11292.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11292
----
commit e5b7051976469a02660a4172cff8dc2d83baa139
Author: Sean Owen <[email protected]>
Date: 2016-02-20T22:02:26Z
reduce(_+_) -> sum
map + flatten -> map
commit bb39c40bffbbed991d37bb3d6dc568521d715f92
Author: Sean Owen <[email protected]>
Date: 2016-02-20T22:05:22Z
exists(_ == ) -> contains
find + nonEmpty -> exists
filter + size -> count
commit 60377a77e4d0003070ec25a5a35228ae44737665
Author: Sean Owen <[email protected]>
Date: 2016-02-20T22:21:09Z
tailrec
commit b7d5bd0cdc35b304daf8842f15d01a344d854849
Author: Sean Owen <[email protected]>
Date: 2016-02-21T11:35:57Z
Dead code
commit 16bff2e5aab2776496556275a1d249ac9ca571a1
Author: Sean Owen <[email protected]>
Date: 2016-02-21T11:49:25Z
Array/String .size -> .length (occassionally, -> .isEmpty / .nonEmpty) to
avoid implicit conversions
commit 7b54fe53ee7e3f64cee2d5cd138e2740403bd836
Author: Sean Owen <[email protected]>
Date: 2016-02-21T14:34:17Z
Revert some incorrect uses of tailrec
commit 93fc9268f52bb8260168ff2c82d8f016478e5e48
Author: Sean Owen <[email protected]>
Date: 2016-02-21T14:34:35Z
get(a) + getOrElse(b) -> getOrElse(a,b)
commit d90aa66cd7b7ae667a74dd424f8c1ac2abd9b25e
Author: Sean Owen <[email protected]>
Date: 2016-02-21T15:18:08Z
Misuse of assert, vs junit.assert
commit b07e01f7098e65eaf55ef74ad420f2af700df38f
Author: Sean Owen <[email protected]>
Date: 2016-02-21T15:23:49Z
Unchecked warnings
commit a003eae6bf1ebe6cb29e46c4fa8ff0e40189c7c2
Author: Sean Owen <[email protected]>
Date: 2016-02-21T15:29:23Z
More misuse of assert
commit a6b1669b903ca0395b6a55a7dff50d1e0311eace
Author: Sean Owen <[email protected]>
Date: 2016-02-21T15:32:00Z
Overflow in compareTo
commit 1bcbaa51d599a3fd192fe4a208c5db9d478be690
Author: Sean Owen <[email protected]>
Date: 2016-02-21T15:33:34Z
Mismatched hashCode/equals
commit adba76cb0e516c7d6fd4127d15458d7a5cea2acf
Author: Sean Owen <[email protected]>
Date: 2016-02-21T15:35:35Z
Inner class should be static
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
