GitHub user ilganeli reopened a pull request:
https://github.com/apache/spark/pull/5236
[SPARK-5931][CORE] Use consistent naming for time properties
I've added new utility methods to do the conversion from times specified as
e.g. 120s, 240ms, 360us to convert to a consistent internal representation.
I've updated usage of these constants throughout the code to be consistent.
I believe I've captured all usages of time-based properties throughout the
code. I've also updated variable names in a number of places to reflect their
units for clarity and updated documentation where appropriate.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ilganeli/spark SPARK-5931
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5236.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5236
----
commit 59bf9e1d590e49f451aaa7feeb5165572f205358
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:15:26Z
[SPARK-5931] Updated Utils and JavaUtils classes to add helper methods to
handle time strings. Updated time strings in a few places to properly parse time
commit 404f8c3762d7e23fdf3ef8d158d68f5b5aa38360
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:25:15Z
Updated usage of spark.core.connection.ack.wait.timeout
commit 7db6d2a73ea4de5bb52bcdf8e4af7edcc06a10dd
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:27:57Z
Updated usage of spark.akka.timeout
commit 4933fda30ce468d72e646536489468e295800775
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:30:05Z
Updated usage of spark.storage.blockManagerSlaveTimeout
commit c9f5cad6a35cc2f405d874a5ca62fa3f92ef0a8b
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:39:15Z
Updated spark.shuffle.io.retryWait
commit 21ef3dd6165022a1edc396c50817641d3321b1f7
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:40:52Z
updated spark.shuffle.sasl.timeout
commit 064ebd67faf655f03d0e02763f53f1df60b3e891
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:50:51Z
Updated usage of spark.cleaner.ttl
commit 7320c87289ded3f61607070a4098d00dc5ee17be
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T21:56:23Z
updated spark.akka.heartbeat.interval
commit 272c2159195948ccae9b399126f8e85015e1aaae
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:00:57Z
Updated spark.locality.wait
commit 3352d34d925650fdba34b05fc3521e15d18fcd11
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:02:59Z
Updated spark.scheduler.maxRegisteredResourcesWaitingTime
commit 3f1cfc84e669df7c2198ed21d776fd391977f34f
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:05:04Z
Updated spark.scheduler.revive.interval
commit 6d1518e8870dc91fec86d75be0cac11d3e1dc2b4
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:07:57Z
Upated spark.speculation.interval
commit 2fcc91c7cd062ca67c5b9e1565d13533e7013c2f
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:13:51Z
Updated spark.dynamicAllocation.executorIdleTimeout
commit 5181597688a6d6f0d9be311784582d033c908225
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:17:58Z
Updated spark.dynamicAllocation.schedulerBacklogTimeout
commit c6a00954f33994c35e9d76cb0222b35ce46bab69
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:22:49Z
Updated spark.core.connection.auth.wait.timeout
commit cde9bffef6ded083a9dfa80486687ddd07e85640
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:25:44Z
Updated spark.streaming.blockInterval
commit 42477aadc1f39911d325a0c703fae2ad55a6c42a
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T22:28:32Z
Updated configuration doc with note on specifying time properties
commit 9a29d8d3f502f43c237859e9f75f5d6c76662915
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-27T23:16:58Z
Fixed misuse of time in streaming context test
commit 34f87c26dd936305fec3d00d6a8cc1e6bc48734c
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-28T05:27:58Z
Update Utils.scala
Fixed ordering to fix overlap of s with ms and us.
commit 8f741e19f66b011decb3cca285606a733a08990b
Author: Ilya Ganelin <[email protected]>
Date: 2015-03-28T05:28:50Z
Update JavaUtils.java
Fixed overlap of (s) with (ms) and (us) in cascading if.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
