Repository: ignite Updated Branches: refs/heads/ignite-10639 e78b161a4 -> 246de0286
IGNITE-10639 Proof of concept for internal description for packages: Navigation from internal to throttling Project: http://git-wip-us.apache.org/repos/asf/ignite/repo Commit: http://git-wip-us.apache.org/repos/asf/ignite/commit/246de028 Tree: http://git-wip-us.apache.org/repos/asf/ignite/tree/246de028 Diff: http://git-wip-us.apache.org/repos/asf/ignite/diff/246de028 Branch: refs/heads/ignite-10639 Commit: 246de02860bf04feb8e6b13c994480493ae526b6 Parents: e78b161 Author: Dmitriy Pavlov <[email protected]> Authored: Wed Dec 12 17:36:19 2018 +0300 Committer: Dmitriy Pavlov <[email protected]> Committed: Wed Dec 12 17:36:19 2018 +0300 ---------------------------------------------------------------------- .../apache/ignite/internal/processors/README.md | 29 +++++++++++++++++--- .../ignite/internal/processors/cache/README.md | 7 +++++ .../internal/processors/cache/package-info.java | 23 ++++++++++++++++ .../processors/cache/persistence/README.md | 10 +++++++ .../cache/persistence/package-info.java | 23 ++++++++++++++++ .../cache/persistence/pagemem/README.md | 10 +++++-- .../cache/persistence/pagemem/package-info.java | 2 +- 7 files changed, 96 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ignite/blob/246de028/modules/core/src/main/java/org/apache/ignite/internal/processors/README.md ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/README.md b/modules/core/src/main/java/org/apache/ignite/internal/processors/README.md index 2566e26..695ab79 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/processors/README.md +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/README.md @@ -1,6 +1,27 @@ -Package for all Ignite processors. ----------------------------------- +Ignite Processors +----------------- -Processor is Ignite component with lifecycle. This lifecycle is associated with Ignite Node lifecycle. +A processor is Ignite component with the lifecycle. This lifecycle is associated with Ignite Node lifecycle. -Despite Managers Ignite processors are not associated with an SPI. \ No newline at end of file +Despite Managers, Ignite processors are not associated with an SPI. + +Cache Processors and Implementation +----------------------------------- +Main grid function from the point of end-user view is a mapping of keys (K) to values (V) +```K->V``` +This mapping is implemented by [cache](cache) + +There is also an affinity key. Usually Key and Affinity Key are equivalent. + +But collocation of data may require transformation from Key to +```K->Affinity Key``` + +Affinity key is mapped to [cache](cache) partition +```K->Affinity Key->Partition``` + +Affinity Key to partition mapping should always be static (any conditions, any JVM). Number of partitions is constant and does not change during grid lifetime, 1024 default + +Affinity Function is also responsible for mapping from partition to target (ideal) node: +```Partition->Node``` + +There are primary nodes and backup (default # of backups = 0 for performance reasons). For replicated cache backups count = nodes count. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/246de028/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/README.md ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/README.md b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/README.md new file mode 100644 index 0000000..d27c0c8 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/README.md @@ -0,0 +1,7 @@ +Ignite Persistence +------------------ +Apache Ignite has its own [Native Persistence](persistence) - Implementation + +Ignite Cache Entries +-------------------- +Each entry represended by a subclass [GridCacheMapEntry](GridCacheMapEntry.java) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/246de028/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/package-info.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/package-info.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/package-info.java new file mode 100644 index 0000000..ab907a0 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/package-info.java @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * This package contain cache-related processors & persistence implementation. <br> + * See also + * <a href="https://github.com/apache/ignite/tree/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache";>GitHub Package Readme</a> + */ +package org.apache.ignite.internal.processors.cache; http://git-wip-us.apache.org/repos/asf/ignite/blob/246de028/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/README.md ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/README.md b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/README.md new file mode 100644 index 0000000..73ea832 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/README.md @@ -0,0 +1,10 @@ +Apache Ignite Persistence +------------------------- +See introduction in [Ignite Persistent Store - under the hood](https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood) + +Contents +-------- +This package contains +- [Page Memory](pagemem) implementation for storage enabled. +- [Write-Ahead Log](wal) implementation for storage enabled. +- [FileNames generation](filename) logic \ No newline at end of file http://git-wip-us.apache.org/repos/asf/ignite/blob/246de028/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/package-info.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/package-info.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/package-info.java new file mode 100644 index 0000000..42b07b1 --- /dev/null +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/package-info.java @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * This package contain cache with persitence implementation. <br> + * See also + * <a href="https://github.com/apache/ignite/tree/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence";>GitHub Package Readme</a> + */ +package org.apache.ignite.internal.processors.cache.persistence; http://git-wip-us.apache.org/repos/asf/ignite/blob/246de028/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/README.md ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/README.md b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/README.md index 613fe96..57e623d 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/README.md +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/README.md @@ -1,15 +1,19 @@ +Apache Ignite Native Peristence Page Memory +------------------------------------------- +This package contains page memory implementation for case persitence is enabled. Speed Based Throttling ---------------------- -For introduction please see +For an introduction, please see [wiki PagesWriteThrottling](https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-PagesWriteThrottling) -If throttling is enabled in User configuration then Speed based throttlign is applied. +If throttling is enabled in User configuration, then Speed based throttling is applied. Speed based throttling is implemented by [PagesWriteSpeedBasedThrottle.java](PagesWriteSpeedBasedThrottle.java) -This approach estimates current number of page written and pages to be written. +Throttling is not active outside of [checkpoint](../checkpoint) process. +But when checkpoint is in progress speed based-throttling approach estimates the current number of pages written and pages to be written. <img src="https://docs.google.com/drawings/d/e/2PACX-1vT1u2fuSdIItg67J02ukUGx3cY1tc9B-eebRSa0Hu4zwzkzpJdNSmSCpRD1EmGhYTCxa-kYqSDKOt-v/pub?w=425&h=589";> http://git-wip-us.apache.org/repos/asf/ignite/blob/246de028/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/package-info.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/package-info.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/package-info.java index 7697969..5bc3a20 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/package-info.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/pagemem/package-info.java @@ -19,7 +19,7 @@ * This package contains page memory implementation for case persitence is enabled. * Contained clasees with <ul> * <li>Page Memory implementation {@link org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl}.</li> - * <li>Page write throttling.</li> + * <li>Page Write Throttling.</li> * <li>Mapping of full Page IDs to place in memory segment.</li> * <li>Pages replacement with disk.</li> * </ul>
