This is an automated email from the ASF dual-hosted git repository.
yaniv pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-amaterasu.git.
from 0d8664c Merge pull request #12 from annoymousGiraf/master
add 3c6cf0f added logic to read ama-workers / spark drivers(am) from
configuration. Spark is first, second amaterasu configs, third is defaults.
check against maximum cluster resources is also in place.
add b108c7a added driver configuration to the SDK
add 3028980 removed .gitignore-ception refactored execData in
sparkrunnersprovider added future comment but mainly added logic in appmaster
to handle spark-specific configurations
add 279d6c7 added spark framework providers support and moved logic there.
add f8e8999 driverConfiguration not cupled to spark
add 3d51596 get cpus return cpus now :)
add dad66bc fixed some bugs
add 2d7e603 handling memory formats
new 4409529 Merge pull request #13 from eyalbenivri/AMATERASU-18
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 2 +-
.../common/configuration/ClusterConfig.scala | 4 +-
.../amaterasu/common/dataobjects/TaskData.scala | 4 ++
.../runners/spark/SparkRunnersProvider.scala | 26 +++++------
.../org/apache/amaterasu/leader/yarn/Client.java | 17 +++----
.../frameworks/FrameworkProvidersFactory.scala | 8 ++--
.../frameworks/spark/SparkSetupProvider.scala | 53 ++++++++++++++++++++--
.../leader/mesos/schedulers/JobScheduler.scala | 9 +++-
.../leader/utilities/MemoryFormatParser.scala | 18 ++++++++
.../amaterasu/leader/yarn/ApplicationMaster.scala | 28 ++++++++----
.../{ => frameworks}/FrameworkSetupProvider.java | 7 ++-
.../configuration/DriverConfiguration.java | 21 +++++++++
12 files changed, 148 insertions(+), 49 deletions(-)
create mode 100644
leader/src/main/scala/org/apache/amaterasu/leader/utilities/MemoryFormatParser.scala
rename sdk/src/main/java/org/apache/amaterasu/sdk/{ =>
frameworks}/FrameworkSetupProvider.java (52%)
create mode 100644
sdk/src/main/java/org/apache/amaterasu/sdk/frameworks/configuration/DriverConfiguration.java
--
To stop receiving notification emails like this one, please contact
[email protected].
