kramasamy closed pull request #2920: Correct case of acronyms URL: https://github.com/apache/incubator-heron/pull/2920
This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/.gitignore b/.gitignore index 12fbf25f85..6091b53704 100644 --- a/.gitignore +++ b/.gitignore @@ -134,4 +134,7 @@ website/public/ .vscode # integration_test -results/ \ No newline at end of file +results/ + +# Vagrant +vagrant/.vagrant/ diff --git a/WORKSPACE b/WORKSPACE index 411f733eca..02cd7efd1c 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -11,7 +11,7 @@ slf4j_version = "1.7.7" distributedlog_version = "0.5.0" http_client_version = "4.5.2" -# heron api server +# heron API server jetty_version = "9.4.6.v20170531" jersey_verion = "2.25.1" hk2_api = "2.5.0-b32" @@ -566,7 +566,7 @@ maven_jar( # end Kubernetes java client -# heron api server +# heron API server # jetty maven_jar( name = "org_eclipse_jetty_server", @@ -723,7 +723,7 @@ maven_jar( ) # end bookkeeper & distributedlog dependencies -# end heron api server +# end heron API server # Nomad dependencies maven_jar( diff --git a/deploy/kubernetes/general/README.md b/deploy/kubernetes/general/README.md index f6c0969d69..3b3a4d0a58 100644 --- a/deploy/kubernetes/general/README.md +++ b/deploy/kubernetes/general/README.md @@ -40,13 +40,13 @@ $ kubectl create -f https://raw.githubusercontent.com/apache/incubator-heron/mas $ kubectl create -f https://raw.githubusercontent.com/apache/incubator-heron/master/deploy/kubernetes/general/tools.yaml ``` -4. Start heron apiserver: +4. Start heron API server: ```shell $ kubectl create -f https://raw.githubusercontent.com/apache/incubator-heron/master/deploy/kubernetes/general/apiserver.yaml ``` -### Deploy via heron apiserver -We will start a proxy to the cluster and then construct a [proxy url](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-services/#manually-constructing-apiserver-proxy-urls) to access the apiserver. +### Deploy via heron API server +We will start a proxy to the cluster and then construct a [proxy url](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-services/#manually-constructing-apiserver-proxy-urls) to access the API server. 1. Start kubectl proxy: @@ -54,7 +54,7 @@ We will start a proxy to the cluster and then construct a [proxy url](https://ku $ kubectl proxy -p 8001 ``` -2. Verify we can access the apiserver: +2. Verify we can access the API server: ```shell $ curl http://localhost:8001/api/v1/proxy/namespaces/default/services/heron-apiserver:9000/api/v1/version diff --git a/deploy/kubernetes/gke/gcs-apiserver.yaml b/deploy/kubernetes/gke/gcs-apiserver.yaml index a050a49028..ee0b380833 100644 --- a/deploy/kubernetes/gke/gcs-apiserver.yaml +++ b/deploy/kubernetes/gke/gcs-apiserver.yaml @@ -116,7 +116,7 @@ spec: --- ## -## Service to expose the heron api server +## Service to expose the heron API server ## apiVersion: v1 kind: Service diff --git a/deploy/kubernetes/gke/medium.yaml b/deploy/kubernetes/gke/medium.yaml index aa228f7a4b..b140d11c7f 100644 --- a/deploy/kubernetes/gke/medium.yaml +++ b/deploy/kubernetes/gke/medium.yaml @@ -30,7 +30,7 @@ platform: gke # Number of replicas for the job binary in bookkeeper jobReplicas: 2 -# amount of memory to provide for api server +# amount of memory to provide for API server apiServerMemory: 512M # Number of replicas for storage bookies, memory and storage requirements diff --git a/deploy/kubernetes/gke/small.yaml b/deploy/kubernetes/gke/small.yaml index f4ad4b3904..070c95f0e3 100644 --- a/deploy/kubernetes/gke/small.yaml +++ b/deploy/kubernetes/gke/small.yaml @@ -30,7 +30,7 @@ platform: gke # Number of replicas for the job binary in bookkeeper jobReplicas: 2 -# amount of memory to provide for api server +# amount of memory to provide for API server apiServerMemory: 512M # Number of replicas for storage bookies, memory and storage requirements diff --git a/deploy/kubernetes/helm/templates/NOTES.txt b/deploy/kubernetes/helm/templates/NOTES.txt index 9f89435e9d..c8bb9508eb 100644 --- a/deploy/kubernetes/helm/templates/NOTES.txt +++ b/deploy/kubernetes/helm/templates/NOTES.txt @@ -3,7 +3,7 @@ You can check the status of Heron by doing the following: 1. Start kubectl proxy kubectl proxy -p 8001 -2. Verify you can access the api server +2. Verify you can access the API server curl http://localhost:8001/api/v1/proxy/namespaces/{{ .Release.Namespace }}/services/{{ .Release.Name }}-apiserver:9000/api/v1/version 3. Verify you can access the heron ui diff --git a/deploy/kubernetes/helm/values.yaml.template b/deploy/kubernetes/helm/values.yaml.template index 898a1e96e5..114da7da45 100644 --- a/deploy/kubernetes/helm/values.yaml.template +++ b/deploy/kubernetes/helm/values.yaml.template @@ -19,7 +19,7 @@ imagePullPolicy: IfNotPresent # Number of replicas for the job binary in bookkeeper jobReplicas: 1 -# amount of memory to provide for api server +# amount of memory to provide for API server apiServerMemory: 512M # Number of replicas for storage bookies, memory and storage requirements diff --git a/deploy/kubernetes/minikube/README.md b/deploy/kubernetes/minikube/README.md index 07be086f33..8c22af6885 100644 --- a/deploy/kubernetes/minikube/README.md +++ b/deploy/kubernetes/minikube/README.md @@ -39,13 +39,13 @@ $ kubectl create -f https://raw.githubusercontent.com/apache/incubator-heron/mas $ kubectl create -f https://raw.githubusercontent.com/apache/incubator-heron/master/deploy/kubernetes/minikube/tools.yaml ``` -4. Start heron apiserver: +4. Start heron API server: ```shell $ kubectl create -f https://raw.githubusercontent.com/apache/incubator-heron/master/deploy/kubernetes/minikube/apiserver.yaml ``` -### Deploy via heron apiserver -We will start a proxy to the cluster and then construct a [proxy url](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-services/#manually-constructing-apiserver-proxy-urls) to access the apiserver. +### Deploy via heron API server +We will start a proxy to the cluster and then construct a [proxy url](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-services/#manually-constructing-apiserver-proxy-urls) to access the API server. 1. Start kubectl proxy: @@ -53,7 +53,7 @@ We will start a proxy to the cluster and then construct a [proxy url](https://ku $ kubectl proxy -p 8001 ``` -2. Verify we can access the apiserver: +2. Verify we can access the API server: ```shell $ curl http://localhost:8001/api/v1/proxy/namespaces/default/services/heron-apiserver:9000/api/v1/version diff --git a/eco-heron-examples/src/java/org/apache/heron/examples/eco/heron_wordcount.yaml b/eco-heron-examples/src/java/org/apache/heron/examples/eco/heron_wordcount.yaml index 3805590a61..cd909134ab 100644 --- a/eco-heron-examples/src/java/org/apache/heron/examples/eco/heron_wordcount.yaml +++ b/eco-heron-examples/src/java/org/apache/heron/examples/eco/heron_wordcount.yaml @@ -30,12 +30,12 @@ config: topology.component.resourcemap: - id: "spout-1" - ram: 256MB # The minimum value for a component's specified ram is 256MB + ram: 256MB # The minimum value for a component's specified RAM is 256MB cpu: 0.5 disk: 4GB - id: "bolt-1" - ram: 256MB # The minimum value for a component's specified ram is 256MB + ram: 256MB # The minimum value for a component's specified RAM is 256MB cpu: 0.5 disk: 2GB diff --git a/eco-storm-examples/src/java/org/apache/heron/examples/eco/simple_wordcount.yaml b/eco-storm-examples/src/java/org/apache/heron/examples/eco/simple_wordcount.yaml index cd78152551..93f9d0f990 100644 --- a/eco-storm-examples/src/java/org/apache/heron/examples/eco/simple_wordcount.yaml +++ b/eco-storm-examples/src/java/org/apache/heron/examples/eco/simple_wordcount.yaml @@ -29,12 +29,12 @@ config: topology.component.resourcemap: - id: "spout-1" - ram: 256MB # The minimum value for a component's specified ram is 256MB + ram: 256MB # The minimum value for a component's specified RAM is 256MB cpu: 0.5 disk: 4GB - id: "bolt-1" - ram: 256MB # The minimum value for a component's specified ram is 256MB + ram: 256MB # The minimum value for a component's specified RAM is 256MB cpu: 0.5 disk: 2GB diff --git a/eco/src/java/org/apache/heron/eco/builder/ConfigBuilder.java b/eco/src/java/org/apache/heron/eco/builder/ConfigBuilder.java index 978158a2ba..e76bc73674 100644 --- a/eco/src/java/org/apache/heron/eco/builder/ConfigBuilder.java +++ b/eco/src/java/org/apache/heron/eco/builder/ConfigBuilder.java @@ -145,7 +145,7 @@ private void setComponentLevelResource(Config config, Map.Entry<String, Object> long megaBytes = extractRawValue(ramWithUom, mbIndex); if (megaBytes < MINIMUM_MB) { throw new IllegalArgumentException( - "The minimum Ram resource allocation for a component must be at least 256MB"); + "The minimum RAM resource allocation for a component must be at least 256MB"); } byteAmount = ByteAmount.fromMegabytes(megaBytes); @@ -161,7 +161,7 @@ private void setComponentLevelResource(Config config, Map.Entry<String, Object> long bytes = extractRawValue(ramWithUom, bIndex); if (bytes < MINIMUM_BYTES) { throw new IllegalArgumentException( - "The minimum Ram resource allocation for a component must be at least 256000000B"); + "The minimum RAM resource allocation for a component must be at least 256000000B"); } byteAmount = ByteAmount.fromBytes(bytes); @@ -169,7 +169,7 @@ private void setComponentLevelResource(Config config, Map.Entry<String, Object> // There is no format throw an exception throw new IllegalArgumentException( - " Please specify 'B', 'MB', 'GB' when declaring Ram Resources"); + " Please specify 'B', 'MB', 'GB' when declaring RAM Resources"); } config.setComponentRam(id, byteAmount); } diff --git a/examples/src/java/org/apache/heron/examples/api/ComponentJVMOptionsTopology.java b/examples/src/java/org/apache/heron/examples/api/ComponentJVMOptionsTopology.java index a65e8255a0..999e5eef63 100644 --- a/examples/src/java/org/apache/heron/examples/api/ComponentJVMOptionsTopology.java +++ b/examples/src/java/org/apache/heron/examples/api/ComponentJVMOptionsTopology.java @@ -78,7 +78,7 @@ public static void main(String[] args) throws Exception { conf.setContainerRamRequested(ByteAmount.fromGigabytes(2)); conf.setContainerCpuRequested(2); - // Specify the size of ram padding to per container. + // Specify the size of RAM padding to per container. // Notice, this config will be considered as a hint, // and it's up to the packing algorithm to determine whether to apply this hint conf.setContainerRamPadding(ByteAmount.fromGigabytes(2)); diff --git a/examples/src/java/org/apache/heron/examples/api/spout/TestWordSpout.java b/examples/src/java/org/apache/heron/examples/api/spout/TestWordSpout.java index e229d5b6b3..bb336506dd 100644 --- a/examples/src/java/org/apache/heron/examples/api/spout/TestWordSpout.java +++ b/examples/src/java/org/apache/heron/examples/api/spout/TestWordSpout.java @@ -65,7 +65,7 @@ public void nextTuple() { final String word = words[rand.nextInt(words.length)]; collector.emit(new Values(word)); if (!throttleDuration.isZero()) { - SysUtils.sleep(throttleDuration); // sleep to throttle back cpu usage + SysUtils.sleep(throttleDuration); // sleep to throttle back CPU usage } } diff --git a/heron/api/src/cpp/config/config.h b/heron/api/src/cpp/config/config.h index 4c5badcedb..a9c99db8fc 100644 --- a/heron/api/src/cpp/config/config.h +++ b/heron/api/src/cpp/config/config.h @@ -131,12 +131,12 @@ class Config { static const std::string TOPOLOGY_RELIABILITY_MODE; /** - * Number of cpu cores per container to be reserved for this topology + * Number of CPU cores per container to be reserved for this topology */ static const std::string TOPOLOGY_CONTAINER_CPU_REQUESTED; /** - * Amount of ram per container to be reserved for this topology. + * Amount of RAM per container to be reserved for this topology. * In bytes. */ static const std::string TOPOLOGY_CONTAINER_RAM_REQUESTED; @@ -148,12 +148,12 @@ class Config { static const std::string TOPOLOGY_CONTAINER_DISK_REQUESTED; /** - * Hint for max number of cpu cores per container to be reserved for this topology + * Hint for max number of CPU cores per container to be reserved for this topology */ static const std::string TOPOLOGY_CONTAINER_MAX_CPU_HINT; /** - * Hint for max amount of ram per container to be reserved for this topology. + * Hint for max amount of RAM per container to be reserved for this topology. * In bytes. */ static const std::string TOPOLOGY_CONTAINER_MAX_RAM_HINT; @@ -171,19 +171,19 @@ class Config { static const std::string TOPOLOGY_CONTAINER_PADDING_PERCENTAGE; /** - * Amount of ram to pad each container for this topology. + * Amount of RAM to pad each container for this topology. * In bytes. */ static const std::string TOPOLOGY_CONTAINER_RAM_PADDING; /** - * Per component cpu requirement. The format of this flag is something like + * Per component CPU requirement. The format of this flag is something like * spout0:0.2,spout1:0.2,bolt1:0.5. */ static const std::string TOPOLOGY_COMPONENT_CPUMAP; /** - * Per component ram requirement. The format of this flag is something like + * Per component RAM requirement. The format of this flag is something like * spout0:12434,spout1:345353,bolt1:545356. */ static const std::string TOPOLOGY_COMPONENT_RAMMAP; @@ -305,14 +305,14 @@ class Config { void setComponentCpu(const std::string& componentName, double cpu) { if (cpu < 0) { - throw std::runtime_error("Invalid Cpu specified for component"); + throw std::runtime_error("Invalid CPU specified for component"); } appendComponentConfig<double>(Config::TOPOLOGY_COMPONENT_CPUMAP, componentName, cpu); } void setComponentRam(const std::string& componentName, int64_t bytes) { if (bytes < 0) { - throw std::runtime_error("Invalid Ram specified for component"); + throw std::runtime_error("Invalid RAM specified for component"); } appendComponentConfig<int64_t>(Config::TOPOLOGY_COMPONENT_RAMMAP, componentName, bytes); } diff --git a/heron/api/src/java/org/apache/heron/api/Config.java b/heron/api/src/java/org/apache/heron/api/Config.java index f1495d4b47..77ce91e683 100644 --- a/heron/api/src/java/org/apache/heron/api/Config.java +++ b/heron/api/src/java/org/apache/heron/api/Config.java @@ -48,7 +48,7 @@ */ public static final String TOPOLOGY_WORKER_CHILDOPTS = "topology.worker.childopts"; /** - * Per component jvm options. The format of this flag is something like + * Per component JVM options. The format of this flag is something like * spout0:jvmopt_for_spout0,spout1:jvmopt_for_spout1. Mostly should be used * in conjunction with setComponentJvmOptions(). This is used in addition * to TOPOLOGY_WORKER_CHILDOPTS. While TOPOLOGY_WORKER_CHILDOPTS applies for @@ -153,11 +153,11 @@ public static final String TOPOLOGY_RELIABILITY_MODE = "topology.reliability.mode"; /** - * Number of cpu cores per container to be reserved for this topology + * Number of CPU cores per container to be reserved for this topology */ public static final String TOPOLOGY_CONTAINER_CPU_REQUESTED = "topology.container.cpu"; /** - * Amount of ram per container to be reserved for this topology. + * Amount of RAM per container to be reserved for this topology. * In bytes. */ public static final String TOPOLOGY_CONTAINER_RAM_REQUESTED = "topology.container.ram"; @@ -167,11 +167,11 @@ */ public static final String TOPOLOGY_CONTAINER_DISK_REQUESTED = "topology.container.disk"; /** - * Hint for max number of cpu cores per container to be reserved for this topology + * Hint for max number of CPU cores per container to be reserved for this topology */ public static final String TOPOLOGY_CONTAINER_MAX_CPU_HINT = "topology.container.max.cpu.hint"; /** - * Hint for max amount of ram per container to be reserved for this topology. + * Hint for max amount of RAM per container to be reserved for this topology. * In bytes. */ public static final String TOPOLOGY_CONTAINER_MAX_RAM_HINT = "topology.container.max.ram.hint"; @@ -187,22 +187,22 @@ public static final String TOPOLOGY_CONTAINER_PADDING_PERCENTAGE = "topology.container.padding.percentage"; /** - * Amount of ram to pad each container. + * Amount of RAM to pad each container. * In bytes. */ public static final String TOPOLOGY_CONTAINER_RAM_PADDING = "topology.container.ram.padding"; /** - * Per component ram requirement. The format of this flag is something like + * Per component RAM requirement. The format of this flag is something like * spout0:0.2,spout1:0.2,bolt1:0.5. */ public static final String TOPOLOGY_COMPONENT_CPUMAP = "topology.component.cpumap"; /** - * Per component ram requirement. The format of this flag is something like + * Per component RAM requirement. The format of this flag is something like * spout0:12434,spout1:345353,bolt1:545356. */ public static final String TOPOLOGY_COMPONENT_RAMMAP = "topology.component.rammap"; /** - * Per component ram requirement. The format of this flag is something like + * Per component RAM requirement. The format of this flag is something like * spout0:12434,spout1:345353,bolt1:545356. */ public static final String TOPOLOGY_COMPONENT_DISKMAP = "topology.component.diskmap"; diff --git a/heron/api/src/java/org/apache/heron/api/utils/TopologyUtils.java b/heron/api/src/java/org/apache/heron/api/utils/TopologyUtils.java index 73a709eff3..99b96d1541 100644 --- a/heron/api/src/java/org/apache/heron/api/utils/TopologyUtils.java +++ b/heron/api/src/java/org/apache/heron/api/utils/TopologyUtils.java @@ -181,7 +181,7 @@ public static boolean verifyTopology(TopologyAPI.Topology topology) { return false; } - // Only verify ram map string well-formed. + // Only verify RAM map string well-formed. getComponentRamMapConfig(topology); // Verify all bolts input streams exist. First get all output streams. Set<String> outputStreams = new HashSet<>(); @@ -244,7 +244,7 @@ public static String getAdditionalClassPath(TopologyAPI.Topology topology) { * Returns a empty map if the Config is not set * * @param topology the topology def - * @return a map (componentName -> ram required) + * @return a map (componentName -> RAM required) */ public static Map<String, ByteAmount> getComponentRamMapConfig(TopologyAPI.Topology topology) throws RuntimeException { diff --git a/heron/api/src/java/org/apache/heron/streamlet/Builder.java b/heron/api/src/java/org/apache/heron/streamlet/Builder.java index e4e6c6caf8..8e7c6e726a 100644 --- a/heron/api/src/java/org/apache/heron/streamlet/Builder.java +++ b/heron/api/src/java/org/apache/heron/streamlet/Builder.java @@ -24,7 +24,7 @@ /** * Builder is used to register all sources. Builder thus keeps track - * of all the starting points of the computation dag and uses this + * of all the starting points of the computation DAG and uses this * information to build the topology */ public interface Builder { diff --git a/heron/api/src/java/org/apache/heron/streamlet/impl/BuilderImpl.java b/heron/api/src/java/org/apache/heron/streamlet/impl/BuilderImpl.java index 5d9cbe6ab4..936e46b312 100644 --- a/heron/api/src/java/org/apache/heron/streamlet/impl/BuilderImpl.java +++ b/heron/api/src/java/org/apache/heron/streamlet/impl/BuilderImpl.java @@ -33,7 +33,7 @@ /** * BuilderImpl implements the Builder interface. * The builder keeps track of the all the sources of the - * computation dag and builds the Topology by traversing all + * computation DAG and builds the Topology by traversing all * the computation nodes. */ public final class BuilderImpl implements Builder { diff --git a/heron/api/src/scala/org/apache/heron/streamlet/scala/Builder.scala b/heron/api/src/scala/org/apache/heron/streamlet/scala/Builder.scala index 1233faea05..0e5f13ead6 100644 --- a/heron/api/src/scala/org/apache/heron/streamlet/scala/Builder.scala +++ b/heron/api/src/scala/org/apache/heron/streamlet/scala/Builder.scala @@ -22,7 +22,7 @@ import org.apache.heron.streamlet.scala.impl.BuilderImpl /** * Builder is used to register all sources. Builder thus keeps track - * of all the starting points of the computation dag and uses this + * of all the starting points of the computation DAG and uses this * information to build the topology */ object Builder { @@ -32,7 +32,7 @@ object Builder { /** * Builder is used to register all sources. Builder thus keeps track - * of all the starting points of the computation dag and uses this + * of all the starting points of the computation DAG and uses this * information to build the topology */ trait Builder { diff --git a/heron/common/src/cpp/config/topology-config-helper.cpp b/heron/common/src/cpp/config/topology-config-helper.cpp index f80e758eb0..db0f6ee21b 100644 --- a/heron/common/src/cpp/config/topology-config-helper.cpp +++ b/heron/common/src/cpp/config/topology-config-helper.cpp @@ -231,7 +231,7 @@ sp_double64 TopologyConfigHelper::GetContainerCpuRequested(const proto::api::Top return atof(cfg.kvs(i).value().c_str()); } } - // Hmmm.. There was no value specified. The default is to allocate one cpu + // Hmmm.. There was no value specified. The default is to allocate one CPU // per component on a stmgr sp_int32 total_parallelism = TopologyConfigHelper::GetTotalParallelism(_topology); sp_int32 nstmgrs = TopologyConfigHelper::GetNumStMgrs(_topology); diff --git a/heron/common/src/cpp/config/topology-config-helper.h b/heron/common/src/cpp/config/topology-config-helper.h index e4ba40949e..7d0f890301 100644 --- a/heron/common/src/cpp/config/topology-config-helper.h +++ b/heron/common/src/cpp/config/topology-config-helper.h @@ -81,10 +81,10 @@ class TopologyConfigHelper { // and returns a new topology structure. static proto::api::Topology* StripComponentObjects(const proto::api::Topology& _topology); - // Gets the per container cpu requested by this topology + // Gets the per container CPU requested by this topology static sp_double64 GetContainerCpuRequested(const proto::api::Topology& _topology); - // Gets the per container ram requested by this topology + // Gets the per container RAM requested by this topology static sp_int64 GetContainerRamRequested(const proto::api::Topology& _topology); // Get all the streams emitted by a component diff --git a/heron/common/src/cpp/network/baseserver.h b/heron/common/src/cpp/network/baseserver.h index 20583fff6f..4336cbe38f 100644 --- a/heron/common/src/cpp/network/baseserver.h +++ b/heron/common/src/cpp/network/baseserver.h @@ -129,7 +129,7 @@ class BaseServer { // When EventLoop invokes upon a timer void OnTimer(VCallback<> cb, EventLoop::Status status); - // Internal functions which do most of the api related activities in the + // Internal functions which do most of the API related activities in the // main thread void InternalCloseConnection(BaseConnection* _connection); void InternalAddTimer(VCallback<> cb, sp_int64 msecs); diff --git a/heron/common/src/java/org/apache/heron/common/config/SystemConfigKey.java b/heron/common/src/java/org/apache/heron/common/config/SystemConfigKey.java index a347001766..80ada01fff 100644 --- a/heron/common/src/java/org/apache/heron/common/config/SystemConfigKey.java +++ b/heron/common/src/java/org/apache/heron/common/config/SystemConfigKey.java @@ -205,7 +205,7 @@ "heron.instance.reconnect.metricsmgr.interval.sec", ChronoUnit.SECONDS), /** - * The interval in seconds to sample a system metric, for instance, jvm used memory. + * The interval in seconds to sample a system metric, for instance, JVM used memory. */ INSTANCE_METRICS_SYSTEM_SAMPLE_INTERVAL( "heron.instance.metrics.system.sample.interval.sec", ChronoUnit.SECONDS), diff --git a/heron/common/src/java/org/apache/heron/common/network/IncomingPacket.java b/heron/common/src/java/org/apache/heron/common/network/IncomingPacket.java index bc3e32e5ec..045602eccf 100644 --- a/heron/common/src/java/org/apache/heron/common/network/IncomingPacket.java +++ b/heron/common/src/java/org/apache/heron/common/network/IncomingPacket.java @@ -46,8 +46,8 @@ * <p> * 2. Experiments are done by using direct buffer and the resources saving is negligible: * -- Direct buffer would save, in our scenarios, less than 1% of RAM; - * -- Direct buffer could save 30%~50% cpu of Gateway thread. - * However, the cpu used by Gateway thread is negligible, + * -- Direct buffer could save 30%~50% CPU of Gateway thread. + * However, the CPU used by Gateway thread is negligible, * less than 2% out of the whole usage in worst case. * -- The extra copy is within JVM boundary; it is pretty fast. */ diff --git a/heron/common/src/java/org/apache/heron/common/network/OutgoingPacket.java b/heron/common/src/java/org/apache/heron/common/network/OutgoingPacket.java index 6969831dc4..2b91077882 100644 --- a/heron/common/src/java/org/apache/heron/common/network/OutgoingPacket.java +++ b/heron/common/src/java/org/apache/heron/common/network/OutgoingPacket.java @@ -45,8 +45,8 @@ * <p> * 2. Experiments are done by using direct buffer and the resources saving is negligible: * -- Direct buffer would save, in our scenarios, less than 1% of RAM; - * -- Direct buffer could save 30%~50% cpu of Gateway thread. - * However, the cpu used by Gateway thread is negligible, + * -- Direct buffer could save 30%~50% CPU of Gateway thread. + * However, the CPU used by Gateway thread is negligible, * less than 2% out of the whole usage in worst case. * -- The extra copy is within JVM boundary; it is pretty fast. */ diff --git a/heron/common/src/java/org/apache/heron/common/utils/metrics/JVMMetrics.java b/heron/common/src/java/org/apache/heron/common/utils/metrics/JVMMetrics.java index f50a67a00f..ed2384c10b 100644 --- a/heron/common/src/java/org/apache/heron/common/utils/metrics/JVMMetrics.java +++ b/heron/common/src/java/org/apache/heron/common/utils/metrics/JVMMetrics.java @@ -62,16 +62,16 @@ // Metrics for count of GC per generational collection, and the sum total of all collections. private final MultiAssignableMetric<Long> jvmGCCountPerGCType; - // Metric for total live jvm threads + // Metric for total live JVM threads private final AssignableMetric<Integer> jvmThreadCount; - // Metric for total live jvm daemon threads + // Metric for total live JVM daemon threads private final AssignableMetric<Integer> jvmDaemonThreadCount; // Metric for number of open file descriptors private final AssignableMetric<Long> fdCount; - // Metric for max file descriptors allowed per jvm process + // Metric for max file descriptors allowed per JVM process private final AssignableMetric<Long> fdLimit; // The accumulated time spending on Garbage Collection in MilliSeconds @@ -105,7 +105,7 @@ */ private MultiAssignableMetric<Long> threadsCPUTimeNs; - // The cpu time used by threads other than SlaveThread and GatewayThread + // The CPU time used by threads other than SlaveThread and GatewayThread private AssignableMetric<Long> otherThreadsCPUTimeNs; /* @@ -121,11 +121,11 @@ */ private MultiAssignableMetric<Long> threadsUserCPUTimeNs; - // The user cpu time used by threads other than SlaveThread and GatewayThread + // The user CPU time used by threads other than SlaveThread and GatewayThread private AssignableMetric<Long> otherThreadsUserCPUTimeNs; /* - * The "recent cpu usage" for the Java Virtual Machine process. + * The "recent CPU usage" for the Java Virtual Machine process. * This value is a double in the [0.0,1.0] interval. * A value of 0.0 means that none of the CPUs were running threads from the JVM process * during the recent period of time observed, @@ -274,7 +274,7 @@ public void run() { processCPUTimeNs.setValue(getProcessCPUTimeNs()); getThreadsMetrics(); - // We multiple # of processors to measure a process cpu load based on cores rather than + // We multiple # of processors to measure a process CPU load based on cores rather than // overall machine processCPULoad.update(getProcessCPULoad() * runtime.availableProcessors()); @@ -367,7 +367,7 @@ private void updateMemoryPoolMetrics() { } private void getThreadsMetrics() { - // Set the cpu usage for every single thread + // Set the CPU usage for every single thread if (threadMXBean.isThreadCpuTimeSupported()) { threadMXBean.setThreadCpuTimeEnabled(true); diff --git a/heron/common/tests/java/org/apache/heron/common/utils/TopologyUtilsTest.java b/heron/common/tests/java/org/apache/heron/common/utils/TopologyUtilsTest.java index a19a3139f6..e198ee254d 100644 --- a/heron/common/tests/java/org/apache/heron/common/utils/TopologyUtilsTest.java +++ b/heron/common/tests/java/org/apache/heron/common/utils/TopologyUtilsTest.java @@ -72,10 +72,10 @@ public void testGetComponentCpuMapDefaultValue() { Map<String, Integer> bolts = new HashMap<>(); bolts.put("bolt", componentParallelism); - // sort the component cpu map + // sort the component CPU map Map<String, Double> cpuMap = new TreeMap<>(TopologyUtils.getComponentCpuMapConfig( TopologyTests.createTopology("test", topologyConfig, spouts, bolts))); - // Component cpu map is not set, the cpuMap size should be 0 + // Component CPU map is not set, the cpuMap size should be 0 Assert.assertEquals(0, cpuMap.size()); } @@ -92,7 +92,7 @@ public void testGetComponentCpuMapAllCpuSpecified() { topologyConfig.setComponentCpu("spout", spoutCpu); topologyConfig.setComponentCpu("bolt", boltCpu); - // sort the component cpu map + // sort the component CPU map Map<String, Double> cpuMap = new TreeMap<>(TopologyUtils.getComponentCpuMapConfig( TopologyTests.createTopology("test", topologyConfig, spouts, bolts))); Assert.assertArrayEquals(new String[]{"bolt", "spout"}, cpuMap.keySet().toArray()); @@ -110,10 +110,10 @@ public void testGetComponentCpuMapSomeCpuSpecified() { double spoutCpu = 2.0f; topologyConfig.setComponentCpu("spout", spoutCpu); - // sort the component cpu map + // sort the component CPU map Map<String, Double> cpuMap = new TreeMap<>(TopologyUtils.getComponentCpuMapConfig( TopologyTests.createTopology("test", topologyConfig, spouts, bolts))); - // Component cpu map sets only spout's cpu + // Component CPU map sets only spout's CPU Assert.assertArrayEquals(new String[]{"spout"}, cpuMap.keySet().toArray()); Assert.assertArrayEquals(new Double[]{spoutCpu}, cpuMap.values().toArray()); } @@ -127,10 +127,10 @@ public void testGetComponentRamMapDefaultValue() { Map<String, Integer> bolts = new HashMap<>(); bolts.put("bolt", componentParallelism); - // sort the component ram map + // sort the component RAM map Map<String, ByteAmount> ramMap = new TreeMap<>(TopologyUtils.getComponentRamMapConfig( TopologyTests.createTopology("test", topologyConfig, spouts, bolts))); - // Component ram map is not set, the ramMap size should be 0 + // Component RAM map is not set, the ramMap size should be 0 Assert.assertEquals(0, ramMap.size()); } @@ -147,7 +147,7 @@ public void testGetComponentRamMapAllRamSpecified() { topologyConfig.setComponentRam("spout", spoutRam); topologyConfig.setComponentRam("bolt", boltRam); - // sort the component ram map + // sort the component RAM map Map<String, ByteAmount> ramMap = new TreeMap<>(TopologyUtils.getComponentRamMapConfig( TopologyTests.createTopology("test", topologyConfig, spouts, bolts))); Assert.assertArrayEquals(new String[]{"bolt", "spout"}, ramMap.keySet().toArray()); @@ -165,10 +165,10 @@ public void testGetComponentRamMapSomeRamSpecified() { ByteAmount spoutRam = ByteAmount.fromGigabytes(2); topologyConfig.setComponentRam("spout", spoutRam); - // sort the component ram map + // sort the component RAM map Map<String, ByteAmount> ramMap = new TreeMap<>(TopologyUtils.getComponentRamMapConfig( TopologyTests.createTopology("test", topologyConfig, spouts, bolts))); - // Component ram map sets only spout's ram + // Component RAM map sets only spout's RAM Assert.assertArrayEquals(new String[]{"spout"}, ramMap.keySet().toArray()); Assert.assertArrayEquals(new ByteAmount[]{spoutRam}, ramMap.values().toArray()); } diff --git a/heron/config/src/yaml/conf/aurora/heron_internals.yaml b/heron/config/src/yaml/conf/aurora/heron_internals.yaml index 8aa7081ede..0870cfbe33 100644 --- a/heron/config/src/yaml/conf/aurora/heron_internals.yaml +++ b/heron/config/src/yaml/conf/aurora/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/aurora/stateful.yaml b/heron/config/src/yaml/conf/aurora/stateful.yaml index 9fdcb10b06..76e1b0b542 100644 --- a/heron/config/src/yaml/conf/aurora/stateful.yaml +++ b/heron/config/src/yaml/conf/aurora/stateful.yaml @@ -15,7 +15,7 @@ # specific language governing permissions and limitations # under the License. -# indicate the type of storage for checkpoint is hdfs +# indicate the type of storage for checkpoint is HDFS heron.statefulstorage.classname: "org.apache.heron.statefulstorage.hdfs.HDFSStorage" heron.statefulstorage.config: diff --git a/heron/config/src/yaml/conf/examples/heron_internals.yaml b/heron/config/src/yaml/conf/examples/heron_internals.yaml index 8aa7081ede..0870cfbe33 100644 --- a/heron/config/src/yaml/conf/examples/heron_internals.yaml +++ b/heron/config/src/yaml/conf/examples/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/kubernetes/heron_internals.yaml b/heron/config/src/yaml/conf/kubernetes/heron_internals.yaml index d59ca57c77..791f76dcd7 100644 --- a/heron/config/src/yaml/conf/kubernetes/heron_internals.yaml +++ b/heron/config/src/yaml/conf/kubernetes/heron_internals.yaml @@ -268,7 +268,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/local/heron_internals.yaml b/heron/config/src/yaml/conf/local/heron_internals.yaml index 609eb81eca..e82369c303 100644 --- a/heron/config/src/yaml/conf/local/heron_internals.yaml +++ b/heron/config/src/yaml/conf/local/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/localzk/heron_internals.yaml b/heron/config/src/yaml/conf/localzk/heron_internals.yaml index 8aa7081ede..0870cfbe33 100644 --- a/heron/config/src/yaml/conf/localzk/heron_internals.yaml +++ b/heron/config/src/yaml/conf/localzk/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/marathon/heron_internals.yaml b/heron/config/src/yaml/conf/marathon/heron_internals.yaml index 8aa7081ede..0870cfbe33 100644 --- a/heron/config/src/yaml/conf/marathon/heron_internals.yaml +++ b/heron/config/src/yaml/conf/marathon/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/mesos/heron_internals.yaml b/heron/config/src/yaml/conf/mesos/heron_internals.yaml index 8aa7081ede..0870cfbe33 100644 --- a/heron/config/src/yaml/conf/mesos/heron_internals.yaml +++ b/heron/config/src/yaml/conf/mesos/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/nomad/heron_internals.yaml b/heron/config/src/yaml/conf/nomad/heron_internals.yaml index 0713529fe9..3594050870 100644 --- a/heron/config/src/yaml/conf/nomad/heron_internals.yaml +++ b/heron/config/src/yaml/conf/nomad/heron_internals.yaml @@ -268,7 +268,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/nomad/scheduler.yaml b/heron/config/src/yaml/conf/nomad/scheduler.yaml index 832ca76164..0ffd281bcb 100644 --- a/heron/config/src/yaml/conf/nomad/scheduler.yaml +++ b/heron/config/src/yaml/conf/nomad/scheduler.yaml @@ -33,8 +33,8 @@ heron.nomad.scheduler.uri: http://127.0.0.1:4646 #Invoke the IScheduler as a library directly heron.scheduler.is.service: False -# The nomad schedules cpu resources in terms of clock frequency (e.g. MHz), while Heron topologies -# specify cpu requests in term of cores. This config maps core to clock freqency. +# The nomad schedules CPU resources in terms of clock frequency (e.g. MHz), while Heron topologies +# specify CPU requests in term of cores. This config maps core to clock freqency. heron.nomad.core.freq.mapping: 2000 # specify whether to run heron via raw java execs or in docker containers diff --git a/heron/config/src/yaml/conf/sandbox/heron_internals.yaml b/heron/config/src/yaml/conf/sandbox/heron_internals.yaml index 609eb81eca..e82369c303 100644 --- a/heron/config/src/yaml/conf/sandbox/heron_internals.yaml +++ b/heron/config/src/yaml/conf/sandbox/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/slurm/heron_internals.yaml b/heron/config/src/yaml/conf/slurm/heron_internals.yaml index 3314d3e3df..0a859b4fa5 100644 --- a/heron/config/src/yaml/conf/slurm/heron_internals.yaml +++ b/heron/config/src/yaml/conf/slurm/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/standalone/heron_internals.yaml b/heron/config/src/yaml/conf/standalone/heron_internals.yaml index 0713529fe9..3594050870 100644 --- a/heron/config/src/yaml/conf/standalone/heron_internals.yaml +++ b/heron/config/src/yaml/conf/standalone/heron_internals.yaml @@ -268,7 +268,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/standalone/scheduler.yaml b/heron/config/src/yaml/conf/standalone/scheduler.yaml index a492ef2e86..32489a5cd5 100644 --- a/heron/config/src/yaml/conf/standalone/scheduler.yaml +++ b/heron/config/src/yaml/conf/standalone/scheduler.yaml @@ -33,8 +33,8 @@ heron.nomad.scheduler.uri: http://127.0.0.1:4646 #Invoke the IScheduler as a library directly heron.scheduler.is.service: False -# The nomad schedules cpu resources in terms of clock frequency (e.g. MHz), while Heron topologies -# specify cpu requests in term of cores. This config maps core to clock freqency. +# The nomad schedules CPU resources in terms of clock frequency (e.g. MHz), while Heron topologies +# specify CPU requests in term of cores. This config maps core to clock freqency. heron.nomad.core.freq.mapping: 2000 # standalone mode uses the raw_exec driver diff --git a/heron/config/src/yaml/conf/standalone/templates/scheduler.template.yaml b/heron/config/src/yaml/conf/standalone/templates/scheduler.template.yaml index b543c8d4cf..1d308e0229 100644 --- a/heron/config/src/yaml/conf/standalone/templates/scheduler.template.yaml +++ b/heron/config/src/yaml/conf/standalone/templates/scheduler.template.yaml @@ -33,8 +33,8 @@ heron.nomad.scheduler.uri: <scheduler_uri> #Invoke the IScheduler as a library directly heron.scheduler.is.service: False -# The nomad schedules cpu resources in terms of clock frequency (e.g. MHz), while Heron topologies -# specify cpu requests in term of cores. This config maps core to clock freqency. +# The nomad schedules CPU resources in terms of clock frequency (e.g. MHz), while Heron topologies +# specify CPU requests in term of cores. This config maps core to clock freqency. heron.nomad.core.freq.mapping: 2000 # standalone mode uses the raw_exec driver diff --git a/heron/config/src/yaml/conf/test/test_heron_internals.yaml b/heron/config/src/yaml/conf/test/test_heron_internals.yaml index d5cabce9c5..aa1dad39ae 100644 --- a/heron/config/src/yaml/conf/test/test_heron_internals.yaml +++ b/heron/config/src/yaml/conf/test/test_heron_internals.yaml @@ -245,7 +245,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/config/src/yaml/conf/yarn/heron_internals.yaml b/heron/config/src/yaml/conf/yarn/heron_internals.yaml index 8aa7081ede..0870cfbe33 100644 --- a/heron/config/src/yaml/conf/yarn/heron_internals.yaml +++ b/heron/config/src/yaml/conf/yarn/heron_internals.yaml @@ -264,7 +264,7 @@ heron.instance.reconnect.streammgr.times: 60 heron.instance.reconnect.metricsmgr.interval.sec: 5 heron.instance.reconnect.metricsmgr.times: 60 -# The interval in second for an instance to sample its system metrics, for instance, cpu load. +# The interval in second for an instance to sample its system metrics, for instance, CPU load. heron.instance.metrics.system.sample.interval.sec: 10 # For efficient acknowledgement diff --git a/heron/executor/src/python/heron_executor.py b/heron/executor/src/python/heron_executor.py index 677bd67277..570dfe55a0 100755 --- a/heron/executor/src/python/heron_executor.py +++ b/heron/executor/src/python/heron_executor.py @@ -227,7 +227,7 @@ def init_parsed_args(self, args): lstrip('"').rstrip('"').replace('(61)', '=').replace('=', '=')) if component_jvm_opts_in_json != "": for (k, v) in json.loads(component_jvm_opts_in_json).items(): - # In json, the component name and jvm options are still in base64 encoding + # In json, the component name and JVM options are still in base64 encoding self.component_jvm_opts[base64.b64decode(k)] = base64.b64decode(v) self.pkg_type = parsed_args.pkg_type @@ -565,7 +565,7 @@ def _get_java_instance_cmd(self, instance_info): for (instance_id, component_name, global_task_id, component_index) in instance_info: total_jvm_size = int(self.component_ram_map[component_name] / (1024 * 1024)) heap_size_mb = total_jvm_size - code_cache_size_mb - java_metasize_mb - Log.info("component name: %s, ram request: %d, total jvm size: %dM, " + Log.info("component name: %s, RAM request: %d, total JVM size: %dM, " "cache size: %dM, metaspace size: %dM" % (component_name, self.component_ram_map[component_name], total_jvm_size, code_cache_size_mb, java_metasize_mb)) diff --git a/heron/instance/src/java/org/apache/heron/instance/bolt/BoltOutputCollectorImpl.java b/heron/instance/src/java/org/apache/heron/instance/bolt/BoltOutputCollectorImpl.java index 22a6b83116..4fd14a35df 100644 --- a/heron/instance/src/java/org/apache/heron/instance/bolt/BoltOutputCollectorImpl.java +++ b/heron/instance/src/java/org/apache/heron/instance/bolt/BoltOutputCollectorImpl.java @@ -133,7 +133,7 @@ public void fail(Tuple input) { sendTuple(bldr, streamId, tuple); - // TODO:- remove this after changing the api + // TODO:- remove this after changing the API return null; } diff --git a/heron/instance/src/java/org/apache/heron/instance/spout/SpoutOutputCollectorImpl.java b/heron/instance/src/java/org/apache/heron/instance/spout/SpoutOutputCollectorImpl.java index aa051434d5..54a8543819 100644 --- a/heron/instance/src/java/org/apache/heron/instance/spout/SpoutOutputCollectorImpl.java +++ b/heron/instance/src/java/org/apache/heron/instance/spout/SpoutOutputCollectorImpl.java @@ -172,7 +172,7 @@ RootTupleInfo retireInFlight(long rootId) { sendTuple(bldr, streamId, tuple); - // TODO:- remove this after changing the api + // TODO:- remove this after changing the API return null; } diff --git a/heron/instance/src/python/instance/st_heron_instance.py b/heron/instance/src/python/instance/st_heron_instance.py index dfa2388dd1..569e5aa2e8 100644 --- a/heron/instance/src/python/instance/st_heron_instance.py +++ b/heron/instance/src/python/instance/st_heron_instance.py @@ -341,7 +341,7 @@ def main(): parser.add_argument('--sys_config', required=True, help='System Config File') parser.add_argument('--override_config', required=True, help='Override Config File') parser.add_argument('--topology_pex', required=True, help='Topology Pex File') - parser.add_argument('--max_ram', required=True, help='Maximum Ram to limit', type=int) + parser.add_argument('--max_ram', required=True, help='Maximum RAM to limit', type=int) args = parser.parse_args() @@ -384,7 +384,7 @@ def main(): "\n **Topology Pex file located at: " + args.topology_pex) Log.debug("System config: " + str(sys_config)) Log.debug("Override config: " + str(override_config)) - Log.debug("Maximum Ram: " + str(args.max_ram)) + Log.debug("Maximum RAM: " + str(args.max_ram)) heron_instance = SingleThreadHeronInstance(args.topology_name, args.topology_id, instance, args.stmgr_port, args.metricsmgr_port, diff --git a/heron/instance/src/python/utils/metrics/py_metrics.py b/heron/instance/src/python/utils/metrics/py_metrics.py index 63df983a0b..55636bdb5d 100644 --- a/heron/instance/src/python/utils/metrics/py_metrics.py +++ b/heron/instance/src/python/utils/metrics/py_metrics.py @@ -32,11 +32,11 @@ class PyMetrics(BaseMetricsHelper): """Helper class to collect PyHeron program metrics""" def __init__(self, metrics_collector): - # total sys cpu time + # total sys CPU time self.sys_cpu_time = AssignableMetrics(0) - # total user cpu time + # total user CPU time self.user_cpu_time = AssignableMetrics(0) - # threads cpu usage. Not supported + # threads CPU usage. Not supported # Add it back when we find an alternative to psutil # self.threads = MultiAssignableMetrics() # number of open file descriptors. Not supported diff --git a/heron/instance/src/python/utils/system_constants.py b/heron/instance/src/python/utils/system_constants.py index 51f4290950..aa080b6641 100644 --- a/heron/instance/src/python/utils/system_constants.py +++ b/heron/instance/src/python/utils/system_constants.py @@ -104,7 +104,7 @@ INSTANCE_RECONNECT_STREAMMGR_INTERVAL_SEC = "heron.instance.reconnect.streammgr.interval.sec" # Interval in seconds to reconnect to the metrics manager INSTANCE_RECONNECT_METRICSMGR_INTERVAL_SEC = "heron.instance.reconnect.metricsmgr.interval.sec" -# Interval in seconds to sample a system metric, for instance, jvm used memory. +# Interval in seconds to sample a system metric, for instance, JVM used memory. INSTANCE_METRICS_SYSTEM_SAMPLE_INTERVAL_SEC = "heron.instance.metrics.system.sample.interval.sec" # The lookForTimeout Interval in spout instance will be timeoutInSeconds / NBUCKETS diff --git a/heron/io/dlog/src/java/org/apache/heron/dlog/DLOutputStream.java b/heron/io/dlog/src/java/org/apache/heron/dlog/DLOutputStream.java index 2e0e278bcd..41b9cb2264 100644 --- a/heron/io/dlog/src/java/org/apache/heron/dlog/DLOutputStream.java +++ b/heron/io/dlog/src/java/org/apache/heron/dlog/DLOutputStream.java @@ -49,7 +49,7 @@ public void write(int b) throws IOException { @Override public void write(byte[] b, int off, int len) throws IOException { - // TODO: avoid array copy by using the new bytebuf api + // TODO: avoid array copy by using the new bytebuf API byte[] newData = new byte[len]; System.arraycopy(b, off, newData, 0, len); write(newData); diff --git a/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/metricscache/MetricsCacheSink.java b/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/metricscache/MetricsCacheSink.java index cec43270f6..1805b92c57 100644 --- a/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/metricscache/MetricsCacheSink.java +++ b/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/metricscache/MetricsCacheSink.java @@ -58,7 +58,7 @@ * <p> * 2. Then it would construct a long-live Service running metricsCacheClient, which could automatically * recover from uncaught exceptions, i.e. close the old one and start a new one. - * Also, it provides api to update the MetricsCacheLocation that metricsCacheClient need to connect and + * Also, it provides API to update the MetricsCacheLocation that metricsCacheClient need to connect and * restart the metricsCacheClient. * There are two scenarios we need to restart a metricsCacheClient in our case: * <p> diff --git a/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/tmaster/TMasterSink.java b/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/tmaster/TMasterSink.java index e50274ecb9..f044292bce 100644 --- a/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/tmaster/TMasterSink.java +++ b/heron/metricsmgr/src/java/org/apache/heron/metricsmgr/sink/tmaster/TMasterSink.java @@ -55,7 +55,7 @@ * <p> * 2. Then it would construct a long-live Service running TMasterClient, which could automatically * recover from uncaught exceptions, i.e. close the old one and start a new one. - * Also, it provides api to update the TMasterLocation that TMasterClient need to connect and + * Also, it provides API to update the TMasterLocation that TMasterClient need to connect and * restart the TMasterClient. * There are two scenarios we need to restart a TMasterClient in our case: * <p> diff --git a/heron/packing/src/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPacking.java b/heron/packing/src/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPacking.java index d0525377bf..85ad638bbe 100644 --- a/heron/packing/src/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPacking.java +++ b/heron/packing/src/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPacking.java @@ -76,25 +76,25 @@ * through the org.apache.heron.api.Config.TOPOLOGY_CONTAINER_PADDING_PERCENTAGE * If the parameter is not specified then a default value of 10 is used (10% of the container size) * <p> - * 4. The ram required for one instance is calculated as: + * 4. The RAM required for one instance is calculated as: * value in org.apache.heron.api.Config.TOPOLOGY_COMPONENT_RAMMAP if exists, otherwise, - * the default ram value for one instance. + * the default RAM value for one instance. * <p> - * 5. The cpu required for one instance is calculated as the default cpu value for one instance. + * 5. The CPU required for one instance is calculated as the default CPU value for one instance. * <p> * 6. The disk required for one instance is calculated as the default disk value for one instance. * <p> - * 7. The ram required for a container is calculated as: - * (ram for instances in container) + (paddingPercentage * ram for instances in container) + * 7. The RAM required for a container is calculated as: + * (RAM for instances in container) + (paddingPercentage * RAM for instances in container) * <p> - * 8. The cpu required for a container is calculated as: - * (cpu for instances in container) + (paddingPercentage * cpu for instances in container) + * 8. The CPU required for a container is calculated as: + * (CPU for instances in container) + (paddingPercentage * CPU for instances in container) * <p> * 9. The disk required for a container is calculated as: * (disk for instances in container) + ((paddingPercentage * disk for instances in container) * <p> * 10. The pack() return null if PackingPlan fails to pass the safe check, for instance, - * the size of ram for an instance is less than the minimal required value. + * the size of RAM for an instance is less than the minimal required value. */ public class FirstFitDecreasingPacking implements IPacking, IRepacking { diff --git a/heron/packing/src/java/org/apache/heron/packing/builder/Container.java b/heron/packing/src/java/org/apache/heron/packing/builder/Container.java index ff9824a878..6d2c6a5ee2 100644 --- a/heron/packing/src/java/org/apache/heron/packing/builder/Container.java +++ b/heron/packing/src/java/org/apache/heron/packing/builder/Container.java @@ -31,8 +31,8 @@ import org.apache.heron.spi.packing.Resource; /** - * Class that describes a container used to place Heron Instances with specific memory, Cpu and disk - * requirements. Each container has limited ram, CpuCores and disk resources. + * Class that describes a container used to place Heron Instances with specific memory, CPU and disk + * requirements. Each container has limited RAM, CpuCores and disk resources. */ public class Container { @@ -45,7 +45,7 @@ * Creates a container with a specific capacity which will maintain a specific percentage * of its resources for padding. * - * @param capacity the capacity of the container in terms of cpu, ram and disk + * @param capacity the capacity of the container in terms of CPU, RAM and disk * @param paddingPercentage the padding percentage */ Container(int containerId, Resource capacity, int paddingPercentage) { @@ -163,7 +163,7 @@ private void assertHasSpace(Resource resource) throws ResourceExceededException usedResources.getDisk().plus(resource.getDisk()).increaseBy(paddingPercentage); if (newRam.greaterThan(this.capacity.getRam())) { - throw new ResourceExceededException(String.format("Adding %s bytes of ram to existing %s " + throw new ResourceExceededException(String.format("Adding %s bytes of RAM to existing %s " + "bytes with %d percent padding would exceed capacity %s", resource.getRam(), usedResources.getRam(), paddingPercentage, this.capacity.getRam())); } @@ -183,7 +183,7 @@ private void assertHasSpace(Resource resource) throws ResourceExceededException * Computes the used resources of the container by taking into account the resources * allocated for each instance. * - * @return a Resource object that describes the used cpu, ram and disk in the container. + * @return a Resource object that describes the used CPU, RAM and disk in the container. */ private Resource getTotalUsedResources() { ByteAmount usedRam = ByteAmount.ZERO; diff --git a/heron/packing/src/java/org/apache/heron/packing/builder/PackingPlanBuilder.java b/heron/packing/src/java/org/apache/heron/packing/builder/PackingPlanBuilder.java index 04bb1eda97..07c462959f 100644 --- a/heron/packing/src/java/org/apache/heron/packing/builder/PackingPlanBuilder.java +++ b/heron/packing/src/java/org/apache/heron/packing/builder/PackingPlanBuilder.java @@ -345,7 +345,7 @@ private void assertResourceSettings() { } containerRam = containerRam.plus(instanceRam); - // Currently not yet support disk or cpu config for different components, + // Currently not yet support disk or CPU config for different components, // so just use the default value. ByteAmount instanceDisk = instanceDefaults.getDisk(); containerDiskInBytes = containerDiskInBytes.plus(instanceDisk); diff --git a/heron/packing/src/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPacking.java b/heron/packing/src/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPacking.java index a40880f976..67f6644dcb 100644 --- a/heron/packing/src/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPacking.java +++ b/heron/packing/src/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPacking.java @@ -70,25 +70,25 @@ * through the org.apache.heron.api.Config.TOPOLOGY_CONTAINER_PADDING_PERCENTAGE * If the parameter is not specified then a default value of 10 is used (10% of the container size) * <p> - * 4. The ram required for one instance is calculated as: + * 4. The RAM required for one instance is calculated as: * value in org.apache.heron.api.Config.TOPOLOGY_COMPONENT_RAMMAP if exists, otherwise, - * the default ram value for one instance. + * the default RAM value for one instance. * <p> - * 5. The cpu required for one instance is calculated as the default cpu value for one instance. + * 5. The CPU required for one instance is calculated as the default CPU value for one instance. * <p> * 6. The disk required for one instance is calculated as the default disk value for one instance. * <p> - * 7. The ram required for a container is calculated as: - * (ram for instances in container) + (paddingPercentage * ram for instances in container) + * 7. The RAM required for a container is calculated as: + * (RAM for instances in container) + (paddingPercentage * RAM for instances in container) * <p> - * 8. The cpu required for a container is calculated as: - * (cpu for instances in container) + (paddingPercentage * cpu for instances in container) + * 8. The CPU required for a container is calculated as: + * (CPU for instances in container) + (paddingPercentage * CPU for instances in container) * <p> * 9. The disk required for a container is calculated as: * (disk for instances in container) + ((paddingPercentage * disk for instances in container) * <p> * 10. The pack() return null if PackingPlan fails to pass the safe check, for instance, - * the size of ram for an instance is less than the minimal required value. + * the size of RAM for an instance is less than the minimal required value. */ public class ResourceCompliantRRPacking implements IPacking, IRepacking { diff --git a/heron/packing/src/java/org/apache/heron/packing/roundrobin/RoundRobinPacking.java b/heron/packing/src/java/org/apache/heron/packing/roundrobin/RoundRobinPacking.java index 0241c26b59..f52ec69354 100644 --- a/heron/packing/src/java/org/apache/heron/packing/roundrobin/RoundRobinPacking.java +++ b/heron/packing/src/java/org/apache/heron/packing/roundrobin/RoundRobinPacking.java @@ -49,7 +49,7 @@ * to take equal number of instances if # of instances is multiple of # of containers. * <p> * Following semantics are guaranteed: - * 1. Every container requires same size of resource, i.e. same cpu, ram and disk. + * 1. Every container requires same size of resource, i.e. same CPU, RAM and disk. * Consider that instances in different containers can be different, the value of size * will be aligned to the max one. * <p> @@ -62,24 +62,24 @@ * value for org.apache.heron.api.Config.TOPOLOGY_CONTAINER_DISK_REQUESTED if exists, otherwise, * (disk for instances in container) + (disk padding for heron internal process) * <p> - * 4. The cpu required for a container is calculated as: + * 4. The CPU required for a container is calculated as: * value for org.apache.heron.api.Config.TOPOLOGY_CONTAINER_CPU_REQUESTED if exists, otherwise, - * (cpu for instances in container) + (cpu padding for heron internal process) + * (CPU for instances in container) + (CPU padding for heron internal process) * <p> - * 5. The ram required for a container is calculated as: + * 5. The RAM required for a container is calculated as: * value for org.apache.heron.api.Config.TOPOLOGY_CONTAINER_RAM_REQUESTED if exists, otherwise, - * (ram for instances in container) + (ram padding for heron internal process) + * (RAM for instances in container) + (RAM padding for heron internal process) * <p> - * 6. The ram required for one instance is calculated as: + * 6. The RAM required for one instance is calculated as: * value in org.apache.heron.api.Config.TOPOLOGY_COMPONENT_RAMMAP if exists, otherwise, * - if org.apache.heron.api.Config.TOPOLOGY_CONTAINER_RAM_REQUESTED not exists: - * the default ram value for one instance + * the default RAM value for one instance * - if org.apache.heron.api.Config.TOPOLOGY_CONTAINER_RAM_REQUESTED exists: - * ((TOPOLOGY_CONTAINER_RAM_REQUESTED) - (ram padding for heron internal process) - * - (ram used by instances within TOPOLOGY_COMPONENT_RAMMAP config))) / + * ((TOPOLOGY_CONTAINER_RAM_REQUESTED) - (RAM padding for heron internal process) + * - (RAM used by instances within TOPOLOGY_COMPONENT_RAMMAP config))) / * (the # of instances in container not specified in TOPOLOGY_COMPONENT_RAMMAP config) * 7. The pack() return null if PackingPlan fails to pass the safe check, for instance, - * the size of ram for an instance is less than the minimal required value. + * the size of RAM for an instance is less than the minimal required value. */ public class RoundRobinPacking implements IPacking, IRepacking { private static final Logger LOG = Logger.getLogger(RoundRobinPacking.class.getName()); @@ -129,7 +129,7 @@ private PackingPlan packInternal(int numContainer, Map<String, Integer> parallel Map<Integer, List<InstanceId>> roundRobinAllocation = getRoundRobinAllocation(numContainer, parallelismMap); - // Get the ram map for every instance + // Get the RAM map for every instance Map<Integer, Map<InstanceId, ByteAmount>> instancesRamMap = getInstancesRamMapInContainer(roundRobinAllocation); @@ -137,7 +137,7 @@ private PackingPlan packInternal(int numContainer, Map<String, Integer> parallel double containerCpu = getContainerCpuHint(roundRobinAllocation); ByteAmount containerRamHint = getContainerRamHint(roundRobinAllocation); - LOG.info(String.format("Pack internal: container cpu hint: %f, ram hint: %s, disk hint: %s.", + LOG.info(String.format("Pack internal: container CPU hint: %f, RAM hint: %s, disk hint: %s.", containerCpu, containerDiskInBytes.toString(), containerRamHint.toString())); @@ -153,7 +153,7 @@ private PackingPlan packInternal(int numContainer, Map<String, Integer> parallel for (InstanceId instanceId : instanceList) { ByteAmount instanceRam = instancesRamMap.get(containerId).get(instanceId); - // Currently not yet support disk or cpu config for different components, + // Currently not yet support disk or CPU config for different components, // so just use the default value. ByteAmount instanceDisk = instanceDiskDefault; double instanceCpu = instanceCpuDefault; @@ -191,7 +191,7 @@ private ByteAmount getContainerRamPadding(List<TopologyAPI.Config.KeyValue> topo } /** - * Calculate the ram required by any instance in the container + * Calculate the RAM required by any instance in the container * * @param allocation the allocation of instances in different container * @return A map: (containerId -> (instanceId -> instanceRequiredRam)) @@ -222,23 +222,23 @@ private ByteAmount getContainerRamPadding(List<TopologyAPI.Config.KeyValue> topo } } - // Now we have calculated ram for instances specified in ComponentRamMap - // Then to calculate ram for the rest instances + // Now we have calculated RAM for instances specified in ComponentRamMap + // Then to calculate RAM for the rest instances int instancesToAllocate = instancesToBeAccounted.size(); if (instancesToAllocate != 0) { ByteAmount individualInstanceRam = instanceRamDefault; - // The ram map is partially set. We need to calculate ram for the rest + // The RAM map is partially set. We need to calculate RAM for the rest - // We have different strategy depending on whether container ram is specified - // If container ram is specified + // We have different strategy depending on whether container RAM is specified + // If container RAM is specified if (!containerRamHint.equals(NOT_SPECIFIED_NUMBER_VALUE)) { - // remove ram for heron internal process + // remove RAM for heron internal process ByteAmount remainingRam = containerRamHint.minus(containerRamPadding).minus(usedRam); - // Split remaining ram evenly + // Split remaining RAM evenly individualInstanceRam = remainingRam.divide(instancesToAllocate); } @@ -300,10 +300,10 @@ private int getLargestContainerSize(Map<Integer, List<InstanceId>> allocation) { } /** - * Provide cpu per container. + * Provide CPU per container. * * @param allocation packing output. - * @return cpu per container. + * @return CPU per container. */ private double getContainerCpuHint(Map<Integer, List<InstanceId>> allocation) { List<TopologyAPI.Config.KeyValue> topologyConfig = topology.getTopologyConfig().getKvsList(); @@ -336,10 +336,10 @@ private ByteAmount getContainerDiskHint(Map<Integer, List<InstanceId>> allocatio } /** - * Provide ram per container. + * Provide RAM per container. * * @param allocation packing - * @return Container ram requirement + * @return Container RAM requirement */ private ByteAmount getContainerRamHint(Map<Integer, List<InstanceId>> allocation) { List<TopologyAPI.Config.KeyValue> topologyConfig = topology.getTopologyConfig().getKvsList(); @@ -361,7 +361,7 @@ private void validatePackingPlan(PackingPlan plan) throws PackingException { // Safe check if (instancePlan.getResource().getRam().lessThan(MIN_RAM_PER_INSTANCE)) { throw new PackingException(String.format("Invalid packing plan generated. A minimum of " - + "%s ram is required, but InstancePlan for component '%s' has %s", + + "%s RAM is required, but InstancePlan for component '%s' has %s", MIN_RAM_PER_INSTANCE, instancePlan.getComponentName(), instancePlan.getResource().getRam())); } diff --git a/heron/packing/src/java/org/apache/heron/packing/utils/PackingUtils.java b/heron/packing/src/java/org/apache/heron/packing/utils/PackingUtils.java index 9eb5caa8ff..9f9f55d622 100644 --- a/heron/packing/src/java/org/apache/heron/packing/utils/PackingUtils.java +++ b/heron/packing/src/java/org/apache/heron/packing/utils/PackingUtils.java @@ -52,14 +52,14 @@ private static void assertIsValidInstance(Resource instanceResources, if (instanceResources.getRam().lessThan(minInstanceRam)) { throw new PackingException(String.format( - "Instance requires ram %s which is less than the minimum ram per instance of %s", + "Instance requires RAM %s which is less than the minimum RAM per instance of %s", instanceResources.getRam(), minInstanceRam)); } ByteAmount instanceRam = instanceResources.getRam().increaseBy(paddingPercentage); if (instanceRam.greaterThan(maxContainerResources.getRam())) { throw new PackingException(String.format( - "This instance requires containers of at least %s ram. The current max container " + "This instance requires containers of at least %s RAM. The current max container " + "size is %s", instanceRam, maxContainerResources.getRam())); } @@ -68,7 +68,7 @@ private static void assertIsValidInstance(Resource instanceResources, instanceResources.getCpu(), paddingPercentage)); if (instanceCpu > maxContainerResources.getCpu()) { throw new PackingException(String.format( - "This instance requires containers with at least %s cpu cores. The current max container" + "This instance requires containers with at least %s CPU cores. The current max container" + "size is %s cores", instanceCpu, maxContainerResources.getCpu())); } diff --git a/heron/packing/tests/java/org/apache/heron/packing/AssertPacking.java b/heron/packing/tests/java/org/apache/heron/packing/AssertPacking.java index a131be4d8c..dc54ad6680 100644 --- a/heron/packing/tests/java/org/apache/heron/packing/AssertPacking.java +++ b/heron/packing/tests/java/org/apache/heron/packing/AssertPacking.java @@ -45,9 +45,9 @@ private AssertPacking() { } /** - * Verifies that the containerPlan has at least one bolt named boltName with ram equal to + * Verifies that the containerPlan has at least one bolt named boltName with RAM equal to * expectedBoltRam and likewise for spouts. If notExpectedContainerRam is not null, verifies that - * the container ram is not that. + * the container RAM is not that. */ public static void assertContainers(Set<PackingPlan.ContainerPlan> containerPlans, String boltName, String spoutName, @@ -58,7 +58,7 @@ public static void assertContainers(Set<PackingPlan.ContainerPlan> containerPlan List<Integer> expectedInstanceIndecies = new ArrayList<>(); List<Integer> foundInstanceIndecies = new ArrayList<>(); int expectedInstanceIndex = 1; - // Ram for bolt should be the value in component ram map + // RAM for bolt should be the value in component RAM map for (PackingPlan.ContainerPlan containerPlan : containerPlans) { if (notExpectedContainerRam != null) { assertNotEquals( @@ -68,12 +68,12 @@ public static void assertContainers(Set<PackingPlan.ContainerPlan> containerPlan expectedInstanceIndecies.add(expectedInstanceIndex++); foundInstanceIndecies.add(instancePlan.getTaskId()); if (instancePlan.getComponentName().equals(boltName)) { - assertEquals("Unexpected bolt ram", expectedBoltRam, instancePlan.getResource().getRam()); + assertEquals("Unexpected bolt RAM", expectedBoltRam, instancePlan.getResource().getRam()); boltFound = true; } if (instancePlan.getComponentName().equals(spoutName)) { assertEquals( - "Unexpected spout ram", expectedSpoutRam, instancePlan.getResource().getRam()); + "Unexpected spout RAM", expectedSpoutRam, instancePlan.getResource().getRam()); spoutFound = true; } } diff --git a/heron/packing/tests/java/org/apache/heron/packing/CommonPackingTests.java b/heron/packing/tests/java/org/apache/heron/packing/CommonPackingTests.java index 55adb6edc6..c72ce42915 100644 --- a/heron/packing/tests/java/org/apache/heron/packing/CommonPackingTests.java +++ b/heron/packing/tests/java/org/apache/heron/packing/CommonPackingTests.java @@ -114,9 +114,9 @@ protected PackingPlan doDefaultScalingTest(Map<String, Integer> componentChanges * * @param testTopology Input topology * @param componentChanges parallelism changes for scale up/down - * @param boltRam ram allocated to bolts + * @param boltRam RAM allocated to bolts * @param testBoltParallelism bolt parallelism - * @param spoutRam ram allocated to spouts + * @param spoutRam RAM allocated to spouts * @param testSpoutParallelism spout parallelism * @param numContainersBeforeRepack number of containers that the initial packing plan should use * @param totalInstancesExpected number of instances expected before scaling @@ -297,14 +297,14 @@ public void testScaleDownInvalidComponent() throws Exception { } /** - * Test invalid ram for instance + * Test invalid RAM for instance */ @Test(expected = PackingException.class) public void testInvalidRamInstance() throws Exception { ByteAmount maxContainerRam = ByteAmount.fromGigabytes(10); int defaultNumInstancesperContainer = 4; - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.ZERO; topologyConfig.setContainerMaxRamHint(maxContainerRam); @@ -345,7 +345,7 @@ protected void doTestContainerCountRequested(int requestedContainers, packingPlanExplicitResourcesConfig.getContainers().size()); Assert.assertEquals(totalInstances, packingPlanExplicitResourcesConfig.getInstanceCount()); - // Ram for bolt/spout should be the value in component ram map + // RAM for bolt/spout should be the value in component RAM map for (PackingPlan.ContainerPlan containerPlan : packingPlanExplicitResourcesConfig.getContainers()) { for (PackingPlan.InstancePlan instancePlan : containerPlan.getInstances()) { diff --git a/heron/packing/tests/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPackingTest.java b/heron/packing/tests/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPackingTest.java index 4d36e38d4f..dcb22d75d5 100644 --- a/heron/packing/tests/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPackingTest.java +++ b/heron/packing/tests/java/org/apache/heron/packing/binpacking/FirstFitDecreasingPackingTest.java @@ -151,17 +151,17 @@ public void testContainerRequestedResources() throws Exception { } /** - * Test the scenario ram map config is fully set + * Test the scenario RAM map config is fully set */ @Test public void testCompleteRamMapRequested() throws Exception { // Explicit set max resources for container - // the value should be ignored, since we set the complete component ram map + // the value should be ignored, since we set the complete component RAM map ByteAmount maxContainerRam = ByteAmount.fromGigabytes(15); ByteAmount maxContainerDisk = ByteAmount.fromGigabytes(20); double maxContainerCpu = 30; - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(1); ByteAmount spoutRam = ByteAmount.fromGigabytes(2); @@ -186,13 +186,13 @@ public void testCompleteRamMapRequested() throws Exception { } /** - * Test the scenario ram map config is fully set + * Test the scenario RAM map config is fully set */ @Test public void testCompleteRamMapRequested2() throws Exception { ByteAmount maxContainerRam = ByteAmount.fromGigabytes(10); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(1); ByteAmount spoutRam = ByteAmount.fromGigabytes(2); @@ -216,14 +216,14 @@ public void testCompleteRamMapRequested2() throws Exception { } /** - * Test the scenario ram map config is partially set + * Test the scenario RAM map config is partially set */ @Test public void testPartialRamMap() throws Exception { // Explicit set resources for container ByteAmount maxContainerRam = ByteAmount.fromGigabytes(10); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(4); topologyConfig.setContainerMaxRamHint(maxContainerRam); @@ -245,7 +245,7 @@ public void testPartialRamMap() throws Exception { } /** - * Test the scenario ram map config is partially set and padding is configured + * Test the scenario RAM map config is partially set and padding is configured */ @Test public void testPartialRamMapWithPadding() throws Exception { @@ -253,7 +253,7 @@ public void testPartialRamMapWithPadding() throws Exception { // Explicit set resources for container ByteAmount maxContainerRam = ByteAmount.fromGigabytes(10); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(4); topologyConfig.setContainerMaxRamHint(maxContainerRam); @@ -316,13 +316,13 @@ public void testDefaultContainerSizeRepack() throws Exception { } /** - * Test the scenario ram map config is partially set and scaling is requested + * Test the scenario RAM map config is partially set and scaling is requested */ @Test public void testRepackPadding() throws Exception { int paddingPercentage = 50; topologyConfig.setContainerPaddingPercentage(paddingPercentage); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(4); ByteAmount maxContainerRam = ByteAmount.fromGigabytes(10); topologyConfig.setComponentRam(BOLT_NAME, boltRam); @@ -362,7 +362,7 @@ public void testRepackPadding() throws Exception { } /** - * Test the scenario ram map config is partially set and scaling is requested + * Test the scenario RAM map config is partially set and scaling is requested */ @Test public void testPartialRamMapScaling() throws Exception { diff --git a/heron/packing/tests/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPackingTest.java b/heron/packing/tests/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPackingTest.java index 74c6d01429..b927997e6b 100644 --- a/heron/packing/tests/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPackingTest.java +++ b/heron/packing/tests/java/org/apache/heron/packing/roundrobin/ResourceCompliantRRPackingTest.java @@ -153,17 +153,17 @@ public void testContainersRequestedExceedsInstanceCount() throws Exception { } /** - * Test the scenario ram map config is partially set + * Test the scenario RAM map config is partially set */ @Test public void testCompleteRamMapRequested() throws Exception { int numContainers = 2; // Explicit set resources for container - // the value should be ignored, since we set the complete component ram map + // the value should be ignored, since we set the complete component RAM map ByteAmount containerRam = ByteAmount.fromGigabytes(Long.MAX_VALUE); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(1); topologyConfig.setContainerRamRequested(containerRam); @@ -180,7 +180,7 @@ public void testCompleteRamMapRequested() throws Exception { } /** - * Test the scenario ram map config is fully set + * Test the scenario RAM map config is fully set */ @Test public void testPartialRamMap() throws Exception { @@ -189,7 +189,7 @@ public void testPartialRamMap() throws Exception { // Explicit set resources for container ByteAmount containerRam = ByteAmount.fromGigabytes(10); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(1); ByteAmount spoutRam = ByteAmount.fromGigabytes(2); @@ -242,7 +242,7 @@ public void testInsufficientContainersWithMultipleAdjustments() throws Exception // Explicit set resources for container ByteAmount containerRam = ByteAmount.fromGigabytes(3); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(1); ByteAmount spoutRam = ByteAmount.fromGigabytes(2); @@ -321,13 +321,13 @@ public void testDefaultContainerSizeRepack() throws Exception { } /** - * Test the scenario ram map config is partially set and scaling is requested + * Test the scenario RAM map config is partially set and scaling is requested */ @Test public void testRepackPadding() throws Exception { int paddingPercentage = 50; topologyConfig.setContainerPaddingPercentage(paddingPercentage); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(4); ByteAmount maxContainerRam = ByteAmount.fromGigabytes(10); topologyConfig.setComponentRam(BOLT_NAME, boltRam); @@ -371,14 +371,14 @@ public void testRepackPadding() throws Exception { } /** - * Test the scenario ram map config is partially set and scaling is requested + * Test the scenario RAM map config is partially set and scaling is requested */ @Test public void testPartialRamMapScaling() throws Exception { // Explicit set resources for container ByteAmount maxContainerRam = ByteAmount.fromGigabytes(10); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(4); topologyConfig.setContainerRamRequested(maxContainerRam); topologyConfig.setComponentRam(BOLT_NAME, boltRam); @@ -461,7 +461,7 @@ public void scaleDownAndUpWithExtraPadding() throws Exception { topologyConfig.setContainerPaddingPercentage(paddingPercentage); // Explicit set resources for container ByteAmount maxContainerRam = ByteAmount.fromGigabytes(12); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount spoutRam = ByteAmount.fromGigabytes(2); topologyConfig.setContainerRamRequested(maxContainerRam); topologyConfig.setComponentRam(SPOUT_NAME, spoutRam); @@ -505,7 +505,7 @@ public void scaleDownAndUpNoPadding() throws Exception { topologyConfig.setContainerPaddingPercentage(paddingPercentage); // Explicit set resources for container ByteAmount maxContainerRam = ByteAmount.fromGigabytes(12); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount spoutRam = ByteAmount.fromGigabytes(4); topologyConfig.setContainerRamRequested(maxContainerRam); topologyConfig.setComponentRam(SPOUT_NAME, spoutRam); diff --git a/heron/packing/tests/java/org/apache/heron/packing/roundrobin/RoundRobinPackingTest.java b/heron/packing/tests/java/org/apache/heron/packing/roundrobin/RoundRobinPackingTest.java index b05dcdd0fc..742c6a4987 100644 --- a/heron/packing/tests/java/org/apache/heron/packing/roundrobin/RoundRobinPackingTest.java +++ b/heron/packing/tests/java/org/apache/heron/packing/roundrobin/RoundRobinPackingTest.java @@ -78,7 +78,7 @@ public void testCheckFailure() throws Exception { org.apache.heron.api.Config topologyConfig = new org.apache.heron.api.Config(); topologyConfig.put(org.apache.heron.api.Config.TOPOLOGY_STMGRS, numContainers); - // Explicit set insufficient ram for container + // Explicit set insufficient RAM for container ByteAmount containerRam = ByteAmount.fromGigabytes(0); topologyConfig.setContainerRamRequested(containerRam); @@ -220,7 +220,7 @@ public void testContainerRequestedResourcesWhenRamPaddingSet() throws Exception } /** - * Test the scenario ram map config is partially set + * Test the scenario RAM map config is partially set */ @Test public void testCompleteRamMapRequested() throws Exception { @@ -234,10 +234,10 @@ public void testCompleteRamMapRequested() throws Exception { topologyConfig.put(org.apache.heron.api.Config.TOPOLOGY_STMGRS, numContainers); // Explicit set resources for container - // the value should be ignored, since we set the complete component ram map + // the value should be ignored, since we set the complete component RAM map ByteAmount containerRam = ByteAmount.fromBytes(Long.MAX_VALUE); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(1); ByteAmount spoutRam = ByteAmount.fromGigabytes(2); @@ -256,7 +256,7 @@ public void testCompleteRamMapRequested() throws Exception { } /** - * Test the scenario ram map config is partially set + * Test the scenario RAM map config is partially set */ @Test public void testPartialRamMap() throws Exception { @@ -272,7 +272,7 @@ public void testPartialRamMap() throws Exception { // Explicit set resources for container ByteAmount containerRam = ByteAmount.fromGigabytes(10); - // Explicit set component ram map + // Explicit set component RAM map ByteAmount boltRam = ByteAmount.fromGigabytes(1); topologyConfig.setContainerRamRequested(containerRam); @@ -284,7 +284,7 @@ public void testPartialRamMap() throws Exception { getRoundRobinPackingPlan(topologyExplicitRamMap); Assert.assertEquals(totalInstances, packingPlanExplicitRamMap.getInstanceCount()); - // Ram for bolt should be the value in component ram map + // RAM for bolt should be the value in component RAM map for (PackingPlan.ContainerPlan containerPlan : packingPlanExplicitRamMap.getContainers()) { Assert.assertEquals(containerRam, containerPlan.getRequiredResource().getRam()); @@ -298,7 +298,7 @@ public void testPartialRamMap() throws Exception { } // Ram for spout should be: - // (containerRam - all ram for bolt - ram for padding) / (# of spouts) + // (containerRam - all RAM for bolt - RAM for padding) / (# of spouts) int spoutCount = instancesCount - boltCount; for (PackingPlan.InstancePlan instancePlan : containerPlan.getInstances()) { if (instancePlan.getComponentName().equals(SPOUT_NAME)) { diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/aurora/AuroraScheduler.java b/heron/schedulers/src/java/org/apache/heron/scheduler/aurora/AuroraScheduler.java index fe8fcdc3ae..b9d13f828a 100644 --- a/heron/schedulers/src/java/org/apache/heron/scheduler/aurora/AuroraScheduler.java +++ b/heron/schedulers/src/java/org/apache/heron/scheduler/aurora/AuroraScheduler.java @@ -115,7 +115,7 @@ public boolean onSchedule(PackingPlan packing) { LOG.info("Launching topology in aurora"); - // Align the cpu, ram, disk to the maximal one, and set them to ScheduledResource + // Align the cpu, RAM, disk to the maximal one, and set them to ScheduledResource PackingPlan updatedPackingPlan = packing.cloneWithHomogeneousScheduledResource(); SchedulerUtils.persistUpdatedPackingPlan(Runtime.topologyName(runtime), updatedPackingPlan, Runtime.schedulerStateManagerAdaptor(runtime)); diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/AppsV1beta1Controller.java b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/AppsV1beta1Controller.java index acc7874b06..4a16adca7f 100644 --- a/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/AppsV1beta1Controller.java +++ b/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/AppsV1beta1Controller.java @@ -106,7 +106,7 @@ boolean submit(PackingPlan packingPlan) { if (!response.isSuccessful()) { LOG.log(Level.SEVERE, "Error creating topology message: " + response.message()); KubernetesUtils.logResponseBodyIfPresent(LOG, response); - // construct a message based on the k8s api server response + // construct a message based on the k8s API server response throw new TopologySubmissionException( KubernetesUtils.errorMessageFromResponse(response)); } diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/MesosScheduler.java b/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/MesosScheduler.java index 011d5ec593..25b9209b6a 100644 --- a/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/MesosScheduler.java +++ b/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/MesosScheduler.java @@ -244,7 +244,7 @@ protected BaseContainer getBaseContainer(Integer containerIndex, PackingPlan pac } /** - * Fill the the resources requirement, i.e. cpu, memory and disk for the given container. + * Fill the the resources requirement, i.e. CPU, memory and disk for the given container. * This method changes the BaseContainer passed in. * <p> * Notice: Currently we just make every container homogeneous, diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/framework/BaseContainer.java b/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/framework/BaseContainer.java index d259b97b15..8db3a1a75d 100644 --- a/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/framework/BaseContainer.java +++ b/heron/schedulers/src/java/org/apache/heron/scheduler/mesos/framework/BaseContainer.java @@ -53,7 +53,7 @@ public String description; @JsonProperty - // # of cpu cores requested + // # of CPU cores requested public double cpu; @JsonProperty diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/nomad/NomadScheduler.java b/heron/schedulers/src/java/org/apache/heron/scheduler/nomad/NomadScheduler.java index f67521909c..bc2da0161e 100644 --- a/heron/schedulers/src/java/org/apache/heron/scheduler/nomad/NomadScheduler.java +++ b/heron/schedulers/src/java/org/apache/heron/scheduler/nomad/NomadScheduler.java @@ -277,7 +277,7 @@ Task getTask(String taskName, int containerIndex, Resource containerResource) { long memoryReqMb = containerResource.getRam().asMegabytes(); resourceReqs.setMemoryMb(longToInt(memoryReqMb)); - // set cpu requirements + // set CPU requirements double coresReq = containerResource.getCpu(); double coresReqFreq = NomadContext.getCoreFreqMapping(this.localConfig) * coresReq; resourceReqs.setCpu(Integer.valueOf((int) Math.round(coresReqFreq))); diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronConfigurationOptions.java b/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronConfigurationOptions.java index 902bc806ed..fdf9ad91cd 100644 --- a/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronConfigurationOptions.java +++ b/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronConfigurationOptions.java @@ -47,7 +47,7 @@ public class HeronCorePackageName implements Name<String> { } - @NamedParameter(doc = "Component Ram Distribution") + @NamedParameter(doc = "Component RAM Distribution") public class ComponentRamMap implements Name<String> { } diff --git a/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronMasterDriver.java b/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronMasterDriver.java index 19f4176365..7322ec324e 100644 --- a/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronMasterDriver.java +++ b/heron/schedulers/src/java/org/apache/heron/scheduler/yarn/HeronMasterDriver.java @@ -95,7 +95,7 @@ * are then invoked and heron components will be activated. * <p></p> * The containers allocated by YARN may be bigger than the requested size. This is a consequence of - * a YARN scheduler optimization, which allocates resources in fixed memory and cpu increments. For + * a YARN scheduler optimization, which allocates resources in fixed memory and CPU increments. For * e.g. if the memory increment is 100 MB, YARN may allocated a container of 200 MB when 150 MB is * requested. As a result, the actual resource allocation may be little more than what was * requested. Moreover YARN does not provide any mechanism to match the container request to @@ -185,7 +185,7 @@ void scheduleHeronWorkers(PackingPlan topologyPacking) throws ContainerAllocatio } /* - * Must be invoked after workers are scheduled. TMaster needs component ram map. + * Must be invoked after workers are scheduled. TMaster needs component RAM map. */ void launchTMaster() { tMaster = buildTMaster(Executors.newSingleThreadExecutor()); @@ -209,7 +209,7 @@ void scheduleHeronWorkers(Set<ContainerPlan> containers) throws ContainerAllocat } /** - * YARN allocates resources in fixed increments of memory and cpu. As a result, the actual + * YARN allocates resources in fixed increments of memory and CPU. As a result, the actual * resource allocation may be little more than what was requested. This method finds the biggest * heron containerPlan that will fit the allocated YARN container. In some cases the YARN CPU * scheduling may be disabled, resulting in default core allocation to each container. This @@ -327,7 +327,7 @@ void requestContainerForWorker(int id, final HeronWorker worker) { int cpu = worker.cores; ByteAmount mem = worker.mem; EvaluatorRequest evaluatorRequest = createEvaluatorRequest(cpu, mem); - LOG.info(String.format("Requesting container for worker: %d, mem: %s, cpu: %d", id, mem, cpu)); + LOG.info(String.format("Requesting container for worker: %d, RAM: %s, CPU: %d", id, mem, cpu)); requestor.submit(evaluatorRequest); } diff --git a/heron/schedulers/tests/java/org/apache/heron/scheduler/mesos/framework/MesosFrameworkTest.java b/heron/schedulers/tests/java/org/apache/heron/scheduler/mesos/framework/MesosFrameworkTest.java index b405472b42..7e02f9f381 100644 --- a/heron/schedulers/tests/java/org/apache/heron/scheduler/mesos/framework/MesosFrameworkTest.java +++ b/heron/schedulers/tests/java/org/apache/heron/scheduler/mesos/framework/MesosFrameworkTest.java @@ -278,7 +278,7 @@ public void testInsufficientResourcesGenerateLaunchableTasks() throws Exception Map<Protos.Offer, TaskResources> offerResources = getOfferResources(offer); BaseContainer baseContainer = new BaseContainer(); - // request more cpu than provided + // request more CPU than provided baseContainer.cpu = 10000; baseContainer.memInMB = 0; baseContainer.diskInMB = 0; diff --git a/heron/shell/assets/jquery.js b/heron/shell/assets/jquery.js index 3774ff9861..3df8887a1a 100644 --- a/heron/shell/assets/jquery.js +++ b/heron/shell/assets/jquery.js @@ -2957,7 +2957,7 @@ jQuery.event = { // If event changes its type, use the special event handlers for the changed type special = jQuery.event.special[ type ] || {}; - // If selector defined, determine special event api type, otherwise given type + // If selector defined, determine special event API type, otherwise given type type = ( selector ? special.delegateType : special.bindType ) || type; // Update special based on newly reset type diff --git a/heron/shell/src/python/handlers/jmaphandler.py b/heron/shell/src/python/handlers/jmaphandler.py index 8b1bdd2884..d147e5c23f 100644 --- a/heron/shell/src/python/handlers/jmaphandler.py +++ b/heron/shell/src/python/handlers/jmaphandler.py @@ -27,7 +27,7 @@ class JmapHandler(tornado.web.RequestHandler): """ - Responsible for getting the jmap for a jvm process given its pid. + Responsible for getting the jmap for a JVM process given its pid. """ # pylint: disable=attribute-defined-outside-init diff --git a/heron/shell/src/python/handlers/jstackhandler.py b/heron/shell/src/python/handlers/jstackhandler.py index e4563c7544..3e5181c707 100644 --- a/heron/shell/src/python/handlers/jstackhandler.py +++ b/heron/shell/src/python/handlers/jstackhandler.py @@ -27,7 +27,7 @@ class JstackHandler(tornado.web.RequestHandler): """ - Responsible for getting the jstack for a jvm process given its pid. + Responsible for getting the jstack for a JVM process given its pid. """ # pylint: disable=attribute-defined-outside-init diff --git a/heron/shell/src/python/handlers/memoryhistogramhandler.py b/heron/shell/src/python/handlers/memoryhistogramhandler.py index f0e6b14bee..7b355cfed3 100644 --- a/heron/shell/src/python/handlers/memoryhistogramhandler.py +++ b/heron/shell/src/python/handlers/memoryhistogramhandler.py @@ -27,7 +27,7 @@ class MemoryHistogramHandler(tornado.web.RequestHandler): """ - Responsible for getting the memory histogram of a jvm process given its pid. + Responsible for getting the memory histogram of a JVM process given its pid. """ # pylint: disable=attribute-defined-outside-init diff --git a/heron/spi/src/java/org/apache/heron/spi/common/Key.java b/heron/spi/src/java/org/apache/heron/spi/common/Key.java index ab4bbd7910..f55bb8e075 100644 --- a/heron/spi/src/java/org/apache/heron/spi/common/Key.java +++ b/heron/spi/src/java/org/apache/heron/spi/common/Key.java @@ -173,8 +173,8 @@ //keys for heron configuration files on the cluster OVERRIDE_YAML("heron.config.file.override.yaml", "${HERON_CONF}/override.yaml"), - // Path to the config overrides passed into the apiserver. Only applicable to submitting - // topologies via apiserver + // Path to the config overrides passed into the API server. Only applicable to submitting + // topologies via API server APISERVER_OVERRIDE_YAML("heron.apiserver.file.override.yaml", Type.STRING), //keys for config provided user binaries diff --git a/heron/spi/src/java/org/apache/heron/spi/packing/PackingPlan.java b/heron/spi/src/java/org/apache/heron/spi/packing/PackingPlan.java index af64bcd622..316f124a4c 100644 --- a/heron/spi/src/java/org/apache/heron/spi/packing/PackingPlan.java +++ b/heron/spi/src/java/org/apache/heron/spi/packing/PackingPlan.java @@ -122,13 +122,13 @@ public Integer getInstanceCount() { } /** - * Get the formatted String describing component ram distribution from PackingPlan, + * Get the formatted String describing component RAM distribution from PackingPlan, * used by executor * - * @return String describing component ram distribution + * @return String describing component RAM distribution */ public String getComponentRamDistribution() { - // Generate a map with the minimal ram size for each component + // Generate a map with the minimal RAM size for each component Map<String, ByteAmount> ramMap = new HashMap<>(); for (ContainerPlan containerPlan : this.getContainers()) { for (InstancePlan instancePlan : containerPlan.getInstances()) { diff --git a/heron/spi/src/java/org/apache/heron/spi/packing/Resource.java b/heron/spi/src/java/org/apache/heron/spi/packing/Resource.java index 42e592177f..385680bea3 100644 --- a/heron/spi/src/java/org/apache/heron/spi/packing/Resource.java +++ b/heron/spi/src/java/org/apache/heron/spi/packing/Resource.java @@ -87,7 +87,7 @@ public Resource plus(Resource other) { } /** - * Divides a resource by another resource by dividing the cpu, memory and disk values of the resources. + * Divides a resource by another resource by dividing the CPU, memory and disk values of the resources. * It returns the maximum of the three results. */ public double divideBy(Resource other) throws RuntimeException { diff --git a/heron/spi/tests/java/org/apache/heron/spi/packing/PackingPlanTest.java b/heron/spi/tests/java/org/apache/heron/spi/packing/PackingPlanTest.java index c9068e6afb..4ca78c3feb 100644 --- a/heron/spi/tests/java/org/apache/heron/spi/packing/PackingPlanTest.java +++ b/heron/spi/tests/java/org/apache/heron/spi/packing/PackingPlanTest.java @@ -116,7 +116,7 @@ public void testPackingPlanSerde() { PackingPlan newPackingPlan = deserializer.fromProto(serializer.toProto(packingPlan)); Assert.assertEquals("Packing plan not the same after converting to protobuf object and back", newPackingPlan, packingPlan); - Assert.assertEquals("Packing plan ram distribution not the same after converting to " + Assert.assertEquals("Packing plan RAM distribution not the same after converting to " + "protobuf object and back", newPackingPlan.getComponentRamDistribution(), packingPlan.getComponentRamDistribution()); } diff --git a/heron/tools/admin/src/python/standalone.py b/heron/tools/admin/src/python/standalone.py index 6ae8c6cb90..85ee341ef6 100644 --- a/heron/tools/admin/src/python/standalone.py +++ b/heron/tools/admin/src/python/standalone.py @@ -501,7 +501,7 @@ def start_api_server(masters, cl_args): output = pid.communicate() Log.debug("return code: %s output: %s" % (return_code, output)) if return_code != 0: - Log.error("Failed to start apiserver on %s with error:\n%s" % (single_master, output[1])) + Log.error("Failed to start API server on %s with error:\n%s" % (single_master, output[1])) sys.exit(-1) wait_for_job_to_start(single_master, "apiserver") @@ -796,13 +796,13 @@ def get_nomad_slave_config_file(cl_args): def get_apiserver_job_file(cl_args): ''' - get path to api server job file + get path to API server job file ''' return "%s/standalone/resources/apiserver.hcl" % config.get_heron_conf_dir() def get_heron_tools_job_file(cl_args): ''' - get path to api server job file + get path to API server job file ''' return "%s/standalone/resources/heron_tools.hcl" % config.get_heron_conf_dir() diff --git a/heron/tools/apiserver/src/java/org/apache/heron/apiserver/Runtime.java b/heron/tools/apiserver/src/java/org/apache/heron/apiserver/Runtime.java index 6caafa24ff..b939621f49 100644 --- a/heron/tools/apiserver/src/java/org/apache/heron/apiserver/Runtime.java +++ b/heron/tools/apiserver/src/java/org/apache/heron/apiserver/Runtime.java @@ -270,7 +270,7 @@ public static void main(String[] args) throws Exception { Logging.setVerbose(verbose); Logging.configure(verbose); - LOG.debug("apiserver overrides:\n {}", cmd.getOptionProperties(Flag.Property.name)); + LOG.debug("API server overrides:\n {}", cmd.getOptionProperties(Flag.Property.name)); final String toolsHome = getToolsHome(); @@ -324,7 +324,7 @@ public static void main(String[] args) throws Exception { try { server.start(); - LOG.info("Heron apiserver started at {}", server.getURI()); + LOG.info("Heron API server started at {}", server.getURI()); server.join(); } catch (Exception ex) { diff --git a/heron/tools/apiserver/src/java/org/apache/heron/apiserver/utils/ConfigUtils.java b/heron/tools/apiserver/src/java/org/apache/heron/apiserver/utils/ConfigUtils.java index 0bb5f9d39d..8a8269b4e9 100644 --- a/heron/tools/apiserver/src/java/org/apache/heron/apiserver/utils/ConfigUtils.java +++ b/heron/tools/apiserver/src/java/org/apache/heron/apiserver/utils/ConfigUtils.java @@ -74,7 +74,7 @@ public static Config getBaseConfiguration(String heronDirectory, releaseFile, overrideConfigurationFile); // Put location of the override file in the config so that schedulers invoked by - // the apiserver can load the override configs if needed. OVERRIDE_YAML cannot be used + // the API server can load the override configs if needed. OVERRIDE_YAML cannot be used // to set this because then the location will get passed on to the heron executors return Config.newBuilder() .putAll(config) diff --git a/heron/tools/cli/src/python/cli_helper.py b/heron/tools/cli/src/python/cli_helper.py index e8c112677c..8d546c289f 100644 --- a/heron/tools/cli/src/python/cli_helper.py +++ b/heron/tools/cli/src/python/cli_helper.py @@ -97,7 +97,7 @@ def run_server(command, cl_args, action, extra_args=dict()): r = service_method(service_apiurl, data=data) s = Status.Ok if r.status_code == requests.codes.ok else Status.HeronError if r.status_code != requests.codes.ok: - Log.error(r.json().get('message', "Unknown error from api server %d" % r.status_code)) + Log.error(r.json().get('message', "Unknown error from API server %d" % r.status_code)) except (requests.exceptions.ConnectionError, requests.exceptions.HTTPError) as err: Log.error(err) return SimpleResult(Status.HeronError, err_msg, succ_msg) diff --git a/heron/tools/cli/src/python/submit.py b/heron/tools/cli/src/python/submit.py index ffb0ae43c8..35487c5ff3 100644 --- a/heron/tools/cli/src/python/submit.py +++ b/heron/tools/cli/src/python/submit.py @@ -196,7 +196,7 @@ def launch_topology_server(cl_args, topology_file, topology_defn_file, topology_ created = r.status_code is requests.codes.created s = Status.Ok if created or ok else Status.HeronError if s is Status.HeronError: - Log.error(r.json().get('message', "Unknown error from api server %d" % r.status_code)) + Log.error(r.json().get('message', "Unknown error from API server %d" % r.status_code)) elif ok: # this case happens when we request a dry_run print(r.json().get("response")) diff --git a/heron/tools/cli/src/python/version.py b/heron/tools/cli/src/python/version.py index b7ff0692b1..1371e48ebd 100644 --- a/heron/tools/cli/src/python/version.py +++ b/heron/tools/cli/src/python/version.py @@ -99,7 +99,7 @@ def run(command, parser, cl_args, unknown_args): try: r = service_method(service_apiurl) if r.status_code != requests.codes.ok: - Log.error(r.json().get('message', "Unknown error from api server %d" % r.status_code)) + Log.error(r.json().get('message', "Unknown error from API server %d" % r.status_code)) sorted_items = sorted(r.json().items(), key=lambda tup: tup[0]) for key, value in sorted_items: print("%s : %s" % (key, value)) diff --git a/heron/tools/ui/resources/static/js/topologies.js b/heron/tools/ui/resources/static/js/topologies.js index 813f861e4e..82af914304 100644 --- a/heron/tools/ui/resources/static/js/topologies.js +++ b/heron/tools/ui/resources/static/js/topologies.js @@ -375,7 +375,7 @@ var AllMetrics = React.createClass({ "component=" + this.props.comp_name; var metricnames = []; - // Explicitly add jvm uptime metric since it + // Explicitly add JVM uptime metric since it // is unique, in the sense that no stream is // associated with it. metricnames.push("__jvm-uptime-secs"); diff --git a/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/HdfsUploader.java b/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/HdfsUploader.java index a0553ed7bd..0266b4ab8e 100644 --- a/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/HdfsUploader.java +++ b/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/HdfsUploader.java @@ -38,14 +38,14 @@ private String topologyPackageLocation; private URI packageURI; - // The controller on hdfs + // The controller on HDFS private HdfsController controller; @Override public void initialize(Config ipconfig) { this.config = ipconfig; - // Instantiate the hdfs controller + // Instantiate the HDFS controller this.controller = getHdfsController(); this.destTopologyDirectoryURI = HdfsContext.hdfsTopologiesDirectoryURI(config); diff --git a/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/sample.yaml b/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/sample.yaml index 432ace615d..7bef2cb1eb 100644 --- a/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/sample.yaml +++ b/heron/uploaders/src/java/org/apache/heron/uploader/hdfs/sample.yaml @@ -18,6 +18,6 @@ # Directory of config files for local hadoop client to read from heron.uploader.hdfs.config.directory: "/home/hadoop/hadoop/conf/" -# The URI of the directory for uploading topologies in the hdfs uploader +# The URI of the directory for uploading topologies in the HDFS uploader heron.uploader.hdfs.topologies.directory.uri: "hdfs:///heron/topology/" diff --git a/heron/uploaders/src/java/org/apache/heron/uploader/s3/S3Uploader.java b/heron/uploaders/src/java/org/apache/heron/uploader/s3/S3Uploader.java index e9db745fe2..cc02c2e4d2 100644 --- a/heron/uploaders/src/java/org/apache/heron/uploader/s3/S3Uploader.java +++ b/heron/uploaders/src/java/org/apache/heron/uploader/s3/S3Uploader.java @@ -45,7 +45,7 @@ import org.apache.heron.spi.uploader.UploaderException; /** - * Provides a basic uploader class for uploading topology packages to s3. + * Provides a basic uploader class for uploading topology packages to S3. * <p> * By default this uploader will write topology packages to s3://<bucket>/<topologyName>/topology.tar.gz * trying to obtain credentials using the default credential provider chain. The package destination serves as known diff --git a/heronpy/api/__init__.py b/heronpy/api/__init__.py index 6d4c651b39..b290bddba7 100644 --- a/heronpy/api/__init__.py +++ b/heronpy/api/__init__.py @@ -1,2 +1,2 @@ -'''module for heronpy api''' +'''module for heronpy API''' __import__('pkg_resources').declare_namespace(__name__) diff --git a/heronpy/api/api_constants.py b/heronpy/api/api_constants.py index 4bf34f8030..b9e16e1a07 100644 --- a/heronpy/api/api_constants.py +++ b/heronpy/api/api_constants.py @@ -18,7 +18,7 @@ # specific language governing permissions and limitations # under the License. -''' api_constants.py: defines api constants for topology config''' +''' API_constants.py: defines API constants for topology config''' #################################################################################################### ########################### Constants for topology configuration ################################## @@ -31,7 +31,7 @@ class TopologyReliabilityMode(object): # Topology-specific options for the worker child process. TOPOLOGY_WORKER_CHILDOPTS = "topology.worker.childopts" -# Per component jvm options. +# Per component JVM options. TOPOLOGY_COMPONENT_JVMOPTS = "topology.component.jvmopts" # How often a tick tuple from the "__system" component and "__tick" stream should be sent to tasks. TOPOLOGY_TICK_TUPLE_FREQ_SECS = "topology.tick.tuple.freq.secs" @@ -60,24 +60,24 @@ class TopologyReliabilityMode(object): # clean state, i.e. ignore any checkpoint state TOPOLOGY_STATEFUL_START_CLEAN = "topology.stateful.start.clean" -# Number of cpu cores per container to be reserved for this topology. +# Number of CPU cores per container to be reserved for this topology. TOPOLOGY_CONTAINER_CPU_REQUESTED = "topology.container.cpu" -# Amount of ram per container to be reserved for this topology, in bytes. +# Amount of RAM per container to be reserved for this topology, in bytes. TOPOLOGY_CONTAINER_RAM_REQUESTED = "topology.container.ram" # Amount of disk per container to be reserved for this topology, in bytes. TOPOLOGY_CONTAINER_DISK_REQUESTED = "topology.container.disk" -# Hint for max amount of ram per container to be reserved for this topology, in bytes. +# Hint for max amount of RAM per container to be reserved for this topology, in bytes. TOPOLOGY_CONTAINER_MAX_CPU_HINT = "topology.container.max.cpu.hint" # Hint for max amount of disk per container to be reserved for this topology, in bytes. TOPOLOGY_CONTAINER_MAX_DISK_HINT = "topology.container.max.disk.hint" # Padding percentage for this container. TOPOLOGY_CONTAINER_PADDING_PERCENTAGE = "topology.container.padding.percentage" -# Amount of ram padding per container. +# Amount of RAM padding per container. TOPOLOGY_CONTAINER_RAM_PADDING = "topology.container.ram.padding" -# Per component cpu requirement. +# Per component CPU requirement. TOPOLOGY_COMPONENT_CPUMAP = "topology.component.cpumap" -# Per component ram requirement. +# Per component RAM requirement. TOPOLOGY_COMPONENT_RAMMAP = "topology.component.rammap" # Per component disk requirement. TOPOLOGY_COMPONENT_DISKMAP = "topology.component.diskmap" diff --git a/heronpy/connectors/pulsar/pulsarspout.py b/heronpy/connectors/pulsar/pulsarspout.py index 56dde2327b..83bc055429 100644 --- a/heronpy/connectors/pulsar/pulsarspout.py +++ b/heronpy/connectors/pulsar/pulsarspout.py @@ -103,9 +103,9 @@ def initialize(self, config, context): self.logConfFileName = GenerateLogConfig(context) self.logger.info("Generated LogConf at %s" % self.logConfFileName) - # We currently use the high level consumer api + # We currently use the high level consumer API # For supporting effectively once, we will need to switch - # to using lower level Reader api, when it becomes + # to using lower level Reader API, when it becomes # available in python self.client = pulsar.Client(self.pulsar_cluster, log_conf_file_path=self.logConfFileName) self.logger.info("Setup Client with cluster %s" % self.pulsar_cluster) diff --git a/heronpy/streamlet/builder.py b/heronpy/streamlet/builder.py index 8f70d1e9e1..aa2bd22530 100644 --- a/heronpy/streamlet/builder.py +++ b/heronpy/streamlet/builder.py @@ -27,15 +27,14 @@ from heronpy.streamlet.impl.generatorspout import GeneratorStreamlet class Builder(object): - """A Builder object is used to build the functional api dag in Heron. - """ + """A Builder object is used to build the functional API DAG in Heron.""" def __init__(self): """ """ self._sources = [] def new_source(self, source): - """Adds a new source to the computation dag""" + """Adds a new source to the computation DAG""" source_streamlet = None if callable(source): diff --git a/heronpy/streamlet/resources.py b/heronpy/streamlet/resources.py index bdb6c76072..75782c1c0a 100644 --- a/heronpy/streamlet/resources.py +++ b/heronpy/streamlet/resources.py @@ -22,7 +22,7 @@ class Resources(object): """Resources needed by the topology are encapsulated in this class. - Currently we deal with cpu and ram. Others can be added later. + Currently we deal with CPU and RAM. Others can be added later. """ def __init__(self): self._cpu = 0.0 diff --git a/integration_test/src/java/org/apache/heron/integration_test/common/HdfsHelper.java b/integration_test/src/java/org/apache/heron/integration_test/common/HdfsHelper.java index e5392b7b42..c1c63d4004 100644 --- a/integration_test/src/java/org/apache/heron/integration_test/common/HdfsHelper.java +++ b/integration_test/src/java/org/apache/heron/integration_test/common/HdfsHelper.java @@ -39,9 +39,9 @@ private HdfsHelper() { } /** - * Provide an InputStreamReader to read from a hdfs file + * Provide an InputStreamReader to read from a HDFS file * - * @param path is the hdfs file location + * @param path is the HDFS file location */ public static InputStreamReader getHdfsStreamReader(String path) throws IOException { Path pt = new Path(path); diff --git a/integration_test/src/java/org/apache/heron/integration_test/common/spout/HdfsStringSpout.java b/integration_test/src/java/org/apache/heron/integration_test/common/spout/HdfsStringSpout.java index 7e4347f93b..f820aaee56 100644 --- a/integration_test/src/java/org/apache/heron/integration_test/common/spout/HdfsStringSpout.java +++ b/integration_test/src/java/org/apache/heron/integration_test/common/spout/HdfsStringSpout.java @@ -32,8 +32,8 @@ import org.apache.heron.integration_test.common.HdfsHelper; /** - * Given a list of hdfs path, the spout will emit every line of of the file in String format. - * When we fetch all items from hdfs BufferedReader, we would set the BufferedReader as null. We + * Given a list of HDFS path, the spout will emit every line of of the file in String format. + * When we fetch all items from HDFS BufferedReader, we would set the BufferedReader as null. We * would like to check whether (BufferedReader == null) to see whether the data is taken completely. * <p> * Note1: The hadoop cluster config should be provided through the classpath @@ -69,7 +69,7 @@ public void open(Map<String, Object> stormConf, // Pre-condition: the number of tasks is equal to the number of files to read if (paths.length != numTasks) { throw new RuntimeException( - String.format("Number of hdfs files %d not equal to number of tasks %d", + String.format("Number of HDFS files %d not equal to number of tasks %d", paths.length, numTasks)); } @@ -80,7 +80,7 @@ public void open(Map<String, Object> stormConf, int index = context.getThisTaskIndex(); String path = paths[index]; - // read file from hdfs + // read file from HDFS br = new BufferedReader( HdfsHelper.getHdfsStreamReader(path), 1024 * 1024 diff --git a/integration_test/src/java/org/apache/heron/integration_test/core/ITerminalBolt.java b/integration_test/src/java/org/apache/heron/integration_test/core/ITerminalBolt.java index 06e0e8f835..028bbc7e8a 100644 --- a/integration_test/src/java/org/apache/heron/integration_test/core/ITerminalBolt.java +++ b/integration_test/src/java/org/apache/heron/integration_test/core/ITerminalBolt.java @@ -25,7 +25,7 @@ public interface ITerminalBolt extends Serializable { /** * Invoke to write all aggregated data to the destination - * Destination can be http URL, local file, hdfs, etc. + * Destination can be http URL, local file, HDFS, etc. */ void writeFinishedData(); } diff --git a/storm-compatibility-examples/src/java/org/apache/storm/examples/spout/TestWordSpout.java b/storm-compatibility-examples/src/java/org/apache/storm/examples/spout/TestWordSpout.java index 91f8b1d6f3..f953fc1e5d 100644 --- a/storm-compatibility-examples/src/java/org/apache/storm/examples/spout/TestWordSpout.java +++ b/storm-compatibility-examples/src/java/org/apache/storm/examples/spout/TestWordSpout.java @@ -63,7 +63,7 @@ public void nextTuple() { final String word = words[rand.nextInt(words.length)]; collector.emit(new Values(word)); if (!throttleDuration.isZero()) { - Utils.sleep(throttleDuration.toMillis()); // sleep to throttle back cpu usage + Utils.sleep(throttleDuration.toMillis()); // sleep to throttle back CPU usage } } diff --git a/third_party/python/cpplint/cpplint.py b/third_party/python/cpplint/cpplint.py index 79516ddbd5..74c75bcc42 100755 --- a/third_party/python/cpplint/cpplint.py +++ b/third_party/python/cpplint/cpplint.py @@ -5690,7 +5690,7 @@ def CheckForIncludeWhatYouUse(filename, clean_lines, include_state, error, required[header] = (linenum, template) # The following function is just a speed up, no semantics are changed. - if not '<' in line: # Reduces the cpu time usage by skipping lines. + if not '<' in line: # Reduces the CPU time usage by skipping lines. continue for pattern, template, header in _re_pattern_templates: diff --git a/website/content/docs/contributors/custom-scheduler.md b/website/content/docs/contributors/custom-scheduler.md index 026c6dd171..a672915f9f 100644 --- a/website/content/docs/contributors/custom-scheduler.md +++ b/website/content/docs/contributors/custom-scheduler.md @@ -51,7 +51,7 @@ Interface | Role | Examples [`IPacking`](/api/org/apache/heron/spi/packing/IPacking.html) | Defines the algorithm used to generate physical plan for a topology. | [RoundRobin](/api/org/apache/heron/packing/roundrobin/RoundRobinPacking.html) [`ILauncher`](/api/org/apache/heron/spi/scheduler/ILauncher.html) | Defines how the scheduler is launched | [Aurora](/api/org/apache/heron/scheduler/aurora/AuroraLauncher.html), [local](/api/org/apache/heron/scheduler/local/LocalLauncher.html) [`IScheduler`](/api/org/apache/heron/spi/scheduler/IScheduler.html) | Defines the scheduler object used to construct topologies | [local](/api/org/apache/heron/scheduler/local/LocalScheduler.html) -[`IUploader`](/api/org/apache/heron/spi/uploader/IUploader.html) | Uploads the topology to a shared location accessible to the runtime environment of the topology | [local](/api/org/apache/heron/uploader/localfs/LocalFileSystemUploader.html) [hdfs](/api/org/apache/heron/uploader/hdfs/HdfsUploader.html) [s3](/api/org/apache/heron/uploader/s3/S3Uploader.html) +[`IUploader`](/api/org/apache/heron/spi/uploader/IUploader.html) | Uploads the topology to a shared location accessible to the runtime environment of the topology | [local](/api/org/apache/heron/uploader/localfs/LocalFileSystemUploader.html) [HDFS](/api/org/apache/heron/uploader/hdfs/HdfsUploader.html) [S3](/api/org/apache/heron/uploader/s3/S3Uploader.html) Heron provides a number of built-in implementations out of box. diff --git a/website/content/docs/developers/java/eco-api.mmark b/website/content/docs/developers/java/eco-api.mmark index 077ddb60c8..e109d0765c 100644 --- a/website/content/docs/developers/java/eco-api.mmark +++ b/website/content/docs/developers/java/eco-api.mmark @@ -228,11 +228,11 @@ component level `cpu` and `disk` configs in the future. topology.component.resourcemap: - id: "spout-1" - ram: 256MB # The minimum value for a component's specified ram is 256MB + ram: 256MB # The minimum value for a component's specified RAM is 256MB - id: "bolt-1" - ram: 256MB # The minimum value for a component's specified ram is 256MB + ram: 256MB # The minimum value for a component's specified RAM is 256MB ``` @@ -273,14 +273,14 @@ underlying TopologyReliabilityMode value. Values are `ATMOST_ONCE`, `ATLEAST_ON * `"topology.reliability.mode"` : A Heron topology can be run in any one of the TopologyReliabilityMode mode. The format of this flag is the string encoded values of the underlying TopologyReliabilityMode value. -* `"topology.container.cpu"` : Number of cpu cores per container to be reserved for this topology. -* `"topology.container.ram"` : Amount of ram per container to be reserved for this topology. In bytes. +* `"topology.container.cpu"` : Number of CPU cores per container to be reserved for this topology. +* `"topology.container.ram"` : Amount of RAM per container to be reserved for this topology. In bytes. * `"topology.container.disk"` : Amount of disk per container to be reserved for this topology. In bytes. -* `"topology.container.max.cpu.hint"` : Hint for max number of cpu cores per container to be reserved for this topology. -* `"topology.container.max.ram.hint"` : Hint for max amount of ram per container to be reserved for this topology. In bytes. +* `"topology.container.max.cpu.hint"` : Hint for max number of CPU cores per container to be reserved for this topology. +* `"topology.container.max.ram.hint"` : Hint for max amount of RAM per container to be reserved for this topology. In bytes. * `"topology.container.max.disk.hint"` : Hint for max amount of disk per container to be reserved for this topology. In bytes. * `"topology.container.padding.percentage"` : Hint for max amount of disk per container to be reserved for this topology. In bytes. -* `"topology.container.ram.padding"` : Amount of ram to pad each container. In bytes. +* `"topology.container.ram.padding"` : Amount of RAM to pad each container. In bytes. * `"topology.stateful.checkpoint.interval.seconds"` : What's the checkpoint interval for stateful topologies in seconds. * `"topology.stateful.start.clean"` : Boolean flag that says that the stateful topology should start from clean state, i.e. ignore any checkpoint state. * `"topology.name"` : Name of the topology. This config is automatically set by Heron when the topology is submitted. diff --git a/website/content/docs/developers/packing/ffdpacking.md b/website/content/docs/developers/packing/ffdpacking.md index 612bb180d5..5f11df307d 100644 --- a/website/content/docs/developers/packing/ffdpacking.md +++ b/website/content/docs/developers/packing/ffdpacking.md @@ -48,7 +48,7 @@ If the requirement is not specified then a default value of 1GB is used. The Fir algorithm guarantees that the amount of memory allocated to a component is either the one specified by the user or the default one. -The parameters 2-4 determine the maximum container size with respect to RAM, cpu cores and disk. +The parameters 2-4 determine the maximum container size with respect to RAM, CPU cores and disk. If one of these parameters is not specified by the user then the hint for the corresponding maximum container resource is set to the default resource requirement of 4 Heron instances. @@ -66,7 +66,7 @@ will be allocated to it. Based on these parameters, the algorithm decides how to place the instances in the containers and how many containers to use. More specifically, the algorithm first sorts the instances in decreasing order of their RAM requirements. It then picks the instance on the head of the sorted -list and places it in the first container that has enough resources (RAM, cpu cores, disk) to +list and places it in the first container that has enough resources (RAM, CPU cores, disk) to accommodate it. If none of the existing containers have the requires resources, then a new container is allocated. Note that if an the RAM requirements of an instance exceed the value of parameter 2, then the algorithm returns an empty packing plan. After all the instances have diff --git a/website/content/docs/developers/troubleshooting.md b/website/content/docs/developers/troubleshooting.md index 698e4dd3ba..893a3d42fa 100644 --- a/website/content/docs/developers/troubleshooting.md +++ b/website/content/docs/developers/troubleshooting.md @@ -128,7 +128,7 @@ getting launched. `heron-executor.stderr` files. All commands to instantiate the instances and stream managers are redirected to these files. -2. Check jvm configs for anything amiss. +2. Check JVM configs for anything amiss. 3. If `Xmx` is too low, increase `containerRAM` or `componentRAM`. Note that because heron sets aside some RAM for its internal components, like stream diff --git a/website/content/docs/operators/deployment/configuration.md b/website/content/docs/operators/deployment/configuration.md index 534cf6df33..a6c04ed5e3 100644 --- a/website/content/docs/operators/deployment/configuration.md +++ b/website/content/docs/operators/deployment/configuration.md @@ -75,7 +75,7 @@ heron.statemgr.zookeeper.is.initialize.tree: True # Directory of config files for hadoop client to read from heron.uploader.hdfs.config.directory: "/home/hadoop/hadoop/conf/" -# The URI of the directory for uploading topologies in the hdfs +# The URI of the directory for uploading topologies in the HDFS heron.uploader.hdfs.topologies.directory.uri: "hdfs:///heron/topology/" ``` diff --git a/website/content/docs/operators/deployment/schedulers/standalone.md b/website/content/docs/operators/deployment/schedulers/standalone.md index 65593933ae..5d171be492 100644 --- a/website/content/docs/operators/deployment/schedulers/standalone.md +++ b/website/content/docs/operators/deployment/schedulers/standalone.md @@ -57,11 +57,11 @@ You should see output like this: [2018-01-22 10:37:06 -0800] [INFO]: Done starting slaves [2018-01-22 10:37:06 -0800] [INFO]: Waiting for cluster to come up... 0 [2018-01-22 10:37:08 -0800] [INFO]: Starting Heron API Server on 127.0.0.1 -[2018-01-22 10:37:08 -0800] [INFO]: Waiting for apiserver to come up... 0 -[2018-01-22 10:37:09 -0800] [INFO]: Waiting for apiserver to come up... 1 +[2018-01-22 10:37:08 -0800] [INFO]: Waiting for API server to come up... 0 +[2018-01-22 10:37:09 -0800] [INFO]: Waiting for API server to come up... 1 [2018-01-22 10:37:16 -0800] [INFO]: Done starting Heron API Server [2018-01-22 10:37:16 -0800] [INFO]: Starting Heron Tools on 127.0.0.1 -[2018-01-22 10:37:16 -0800] [INFO]: Waiting for apiserver to come up... 0 +[2018-01-22 10:37:16 -0800] [INFO]: Waiting for API server to come up... 0 [2018-01-22 10:37:17 -0800] [INFO]: Done starting Heron Tools [2018-01-22 10:37:17 -0800] [INFO]: Heron standalone cluster complete! ``` diff --git a/website/content/docs/operators/heron-shell.md b/website/content/docs/operators/heron-shell.md index 2176d72abf..e0c71ee0bf 100644 --- a/website/content/docs/operators/heron-shell.md +++ b/website/content/docs/operators/heron-shell.md @@ -28,14 +28,14 @@ files online. Viewing and downloading log files is one special case of browse. ### Pid of a process -Each instance in the topology runs as a separate jvm process. This allows us to +Each instance in the topology runs as a separate JVM process. This allows us to monitor each spout or bolt instance in isolation. To run more sophisticated operations on the process as mentioned in this list below, we need to know the process id of the process running as that instance. ### Jstack of a process -This utility runs the `jstack` command on the jvm process that is running an +This utility runs the `jstack` command on the JVM process that is running an instance. The result is passed back through the REST API which can be viewed directly in the browser. @@ -50,5 +50,5 @@ the "browse" utility. ### Memory histogram of a process This utility runs `jmap` with `-histo` option, to output the memory -histogram for the jvm process of an instance. The resulting histogram is passed +histogram for the JVM process of an instance. The resulting histogram is passed back as part of the response, and can be viewed directly in the browser. diff --git a/website/content/docs/operators/heron-tracker-api.md b/website/content/docs/operators/heron-tracker-api.md index ac2578a7ec..c533075c70 100644 --- a/website/content/docs/operators/heron-tracker-api.md +++ b/website/content/docs/operators/heron-tracker-api.md @@ -355,7 +355,7 @@ Returns all exceptions for the component of the topology. ### <a name="topologies_pid">/topologies/pid</a> -Returns the PID of the instance jvm process. +Returns the PID of the instance JVM process. #### Parameters @@ -368,7 +368,7 @@ Returns the PID of the instance jvm process. ### <a name="topologies_jstack">/topologies/jstack</a> -Returns the thread dump of the instance jvm process. +Returns the thread dump of the instance JVM process. #### Parameters @@ -395,7 +395,7 @@ Returns the path to the file that can be downloaded externally. ### <a name="topologies_histo">/topologies/histo</a> -Returns histogram for the instance jvm process. +Returns histogram for the instance JVM process. #### Parameters diff --git a/website/data/configs/kubernetes.yaml b/website/data/configs/kubernetes.yaml index 87c841b9c6..7a2fa8ddf6 100644 --- a/website/data/configs/kubernetes.yaml +++ b/website/data/configs/kubernetes.yaml @@ -228,7 +228,7 @@ configfiles: description: Interval in seconds to reconnect to the metrics manager, including the request timeout in connecting default: 60 - name: heron.instance.metrics.system.sample.interval.sec - description: The interval in second for an instance to sample its system metrics, for instance, cpu load. + description: The interval in second for an instance to sample its system metrics, for instance, CPU load. default: 10 - name: heron.instance.slave.fetch.pplan.interval.sec description: The time interval (in seconds) at which Heron instances fetch the physical plan from slaves ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
