[ignite-3] branch ignite-14647 updated: IGNITE-14647 Describe no-logging native persistence
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14647 in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/ignite-14647 by this push: new 37a1fa7 IGNITE-14647 Describe no-logging native persistence 37a1fa7 is described below commit 37a1fa71e1b8136137415e3e182d235760f57a70 Author: Alexey Goncharuk AuthorDate: Mon May 3 23:46:46 2021 +0300 IGNITE-14647 Describe no-logging native persistence --- modules/vault/README.md | 96 + 1 file changed, 96 insertions(+) diff --git a/modules/vault/README.md b/modules/vault/README.md new file mode 100644 index 000..93d961f --- /dev/null +++ b/modules/vault/README.md @@ -0,0 +1,96 @@ +# Apache Ignite Native Persistence (nextgen) +This document describes the architecture of Apache Ignite 3.x native persistence and highlights key differencies +compared to the Apache Ignite 2.x native perisitence architecture. + +## Replication log as recovery journal +Apache Ignite 3.x uses log-based replication protocol (currently, Raft) to ensure data consistency on different nodes +in the cluster. This very same log is used as a logical WAL for local state machine recovery. This change allows for +several optimizations in the page memory persistence mechanics. + +This document describes the architecture of the replication log as well as the optimized version of the native +persistence. + +## Replication log +The Raft log represents a key-value storage for which key is always a contiguously growing unbounded integer number. +In practice, it is sufficient to limit the key by a 64-bit unsigned integer. The key range consists of two segments: +committed entries (the larger part of the log) cannot be changed and are immutable, and not-yet-committed entries that +can be truncated and be overwritten. The non-committed entries are rarely overwritten in practice. Additionally, the +Raft log can be compacted: entries with key smaller than lower bound are thrown away as they were made durable in the +state machine and are no longer needed. + +The replication log storage is based on WiscKey approach of log-structured storages. Multiple Raft group logs can be +stored in a single storage, however, it is not neccessary to have a single storage for all Raft groups. The storage and +WAL of Raft log are combined and use immutable append-only segments containing log indices and state machine commands. +Segment header may contain a dictionary to reduce the size of Raft group IDs written in the segment. Each log entry +contains the log index, Raft group ID and command payload (the payload itself also contains the log entry term which +is essential to Raft protocol). Schematically, the segment structure can be represented as follows: + + +++++-+ +|| Entry 1| Entry 2 | ... | +++++-+ +| Segment header | Log index | Group ID | Payload | Log index | Group ID | Payload | ... | +++++-+ + + +New entries are always written to the end of the segment, even if it is an overwrite entry with the log index which +was already written to the same Raft group. The Raft log maintains an index of log entries for each Raft log group +which is represented as a simple array of entry offsets from the beginning of the file. The array is complemented with +the base index of the first entry in the segment. The in-memory index is flushed on disk (checkpointed) only when all +entries in the segment are committed. Schematically, the index structure may be represented as follows: + + +++--+--+-+ +||Group 1 Index | Group 2 Index | ... | +++--+--+-+ +| Index header | Len | Index Base | Offset 1 | Offset 2 | ... | Len | Index Base | Offset 1 | Offset 2 | ... | ... | +++--+--+-+ + + +Upon recovery, the Raft log component reads all log entries from non-checkpointed segments and repopulates the in-memory +index which will be later checkpointed. + +## No-logging native persistence +Given that the Raft log is available as a separate component, we can now see how the page memory-based native +perisitence uses the external log to avoid any WAL logging for recovery. + +Similar to Ignite 2.x native persistence, all data
[ignite-3] branch ignite-14647 created (now c9fa5ec)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14647 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. at c9fa5ec IGNITE-14647 Describe Raft-based rebalance process This branch includes the following new commits: new c9fa5ec IGNITE-14647 Describe Raft-based rebalance process The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite-3] 01/01: IGNITE-14647 Describe Raft-based rebalance process
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14647 in repository https://gitbox.apache.org/repos/asf/ignite-3.git commit c9fa5ec4dfd62ac9f0c7caa6350c4bcd5051f2e8 Author: Alexey Goncharuk AuthorDate: Sat Apr 24 13:11:38 2021 +0300 IGNITE-14647 Describe Raft-based rebalance process --- modules/affinity/README.md | 86 ++ 1 file changed, 86 insertions(+) diff --git a/modules/affinity/README.md b/modules/affinity/README.md new file mode 100644 index 000..2c2db7c --- /dev/null +++ b/modules/affinity/README.md @@ -0,0 +1,86 @@ +# Partitioning approach + +## Hash-based partitioning + +## Range-based partitioning + +# Data migration (rebalance) +There is a significant difference between the rebalance approach in Ignite 2.x and rebalance approach in Ignite 3.x. + +Ignite 2.x implemented rebalance process with updates being applied to the storage concurrently with data migration +process. This results in a complex interaction between the rebalance process and data update protocol (the necessity +to compare key-value versions during data migration, different entry processor application paths for cases when +rebalance is active and not active, uncertain partition state during recovery, etc). + +Ignite 3.x relies on common replication infrastructure for data replication between nodes, thus the rebalance should +be handled by means of the replication protocols. + +## Raft +Raft consensus protocol does not have a concept of rebalance. Instead, it relies on two underlying mechanisms in order +to have an ability to catch offline nodes up-to-speed and bootstrap new Raft group members: Raft log and Snapshots. +These mechanisms handle both delta (when a local node has relevant enough local state so it can be brought up to speed +by sending only recent Raft log commands) and full (when a Raft group does not have sufficient Raft log to catch up the +node, so the full state machine snapshot should be sent to the local node) rebalance scenarios. The choice between +snapshot and log-based catch-up is based on Raft log availability, however, this logic can be adjusted to a more +sophisticated heuristic. The underlying state machine should only provide the snapshot functionality. This functionality +differs for in-memory and persistent tables. + +### In-memory tables +In-memory tables do not save partitions in isolated memory regions. Instead, the partition data is written to a shared +memory pool in order to provide efficient memory utilization (otherwise, an assigned memory chunk would remain assigned +to a partition and would not be eligible for other partitions for reuse). This makes it impossible to create partition +memory snapshots on phycial level, so we need to maintain a snapshot on tuple basis. + +At any moment in time at most one in-memory partition snapshot can be maintained. + + Alternative 1 +To create an in-memory snapshot, we use an MVCC-like approach with copy-on-write technique. The partition tree is +extended to support keeping two versions of a tuple for the same key: one is the most relevant version, and another one +is snapshot version. The snapshot tuple contains snapshot ID additionally to the regular tuple data. Snapshot tuples are +only available to the snapshot iterator and must be filtered out from regular data access paths. + +When a snapshot for an in-memory partition is requested, the partition state machine checks that there is no another +active snapshot and assigns a new snapshot ID which will be used for copy-on-write tuples. When the snapshot iterator +is traversing a tree, it attempts to read both up-to-date and snapshot version of the key. If the snapshot version of +the key with the current snapshot ID exists, it must be used in the iterator. If the snapshot version of the key with +the current snapshot ID does not exist, the up-to-date version of the tuple must be used in the iterator. + +Each partition state machine update checks if there is a snapshot that is being maintained. If there is no active +snapshot, the update operation should clean an old snapshot tuple version, if any, and do the regular tuple update. If +there is an active snapshot, the update operation must first clean an old snapshot tuple version, if any. Then, if a +snapshot tuple version with the current snapshot ID does not exist, the update operation copies the current tuple value +to the snapshot version, and then completes the update (it does not copy the current value if a relevant snapshot +version already exists). + +When snapshot is no longer needed, an asynchronous process can clean up the snapshot versions from the partition. + +This approach does not induce any memory overhead when no snapshot is maintained, but may require up to 2x of partition +size under heavy load (because the whole partition may be copied to the snapshot versions
[ignite-3] branch gg-33019 deleted (was 01379be)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch gg-33019 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. was 01379be GG-33019 Introduce ProjectableFilterableTableScan for native storage integration This change permanently discards the following revisions: discard 01379be GG-33019 Introduce ProjectableFilterableTableScan for native storage integration
[ignite-3] 01/01: GG-33019 Introduce ProjectableFilterableTableScan for native storage integration
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch gg-33019
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 01379be2c57335fd08605dbfe8314e3317b03030
Author: Alexey Goncharuk
AuthorDate: Wed Apr 7 15:08:38 2021 +0300
GG-33019 Introduce ProjectableFilterableTableScan for native storage
integration
---
modules/sql/pom.xml| 81 ++
.../ignite/internal/sql/rel/IgniteTableScan.java | 36 ++
.../sql/rel/ProjectableFilterableTableScan.java| 66 ++
parent/pom.xml | 19 +
pom.xml| 1 +
5 files changed, 203 insertions(+)
diff --git a/modules/sql/pom.xml b/modules/sql/pom.xml
new file mode 100644
index 000..d9a245f
--- /dev/null
+++ b/modules/sql/pom.xml
@@ -0,0 +1,81 @@
+
+
+
+
+http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.ignite
+ignite-parent
+1
+../../parent/pom.xml
+
+
+ignite-sql
+3.0.0-SNAPSHOT
+
+
+
+org.apache.ignite
+ignite-api
+${project.version}
+
+
+
+org.apache.ignite
+ignite-schema
+${project.version}
+
+
+
+org.jetbrains
+annotations
+
+
+
+org.apache.calcite
+calcite-core
+
+
+
+org.apache.calcite
+calcite-babel
+
+
+
+org.apache.calcite
+calcite-linq4j
+
+
+
+
+org.junit.jupiter
+junit-jupiter-engine
+test
+
+
+
+org.junit.jupiter
+junit-jupiter-params
+test
+
+
+
diff --git
a/modules/sql/src/main/java/org/apache/ignite/internal/sql/rel/IgniteTableScan.java
b/modules/sql/src/main/java/org/apache/ignite/internal/sql/rel/IgniteTableScan.java
new file mode 100644
index 000..6609cb0
--- /dev/null
+++
b/modules/sql/src/main/java/org/apache/ignite/internal/sql/rel/IgniteTableScan.java
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.sql.rel;
+
+import java.util.List;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.util.ImmutableBitSet;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ *
+ */
+public class IgniteTableScan extends ProjectableFilterableTableScan {
+public IgniteTableScan(
+@Nullable List proj,
+@Nullable RexNode cond,
+@Nullable ImmutableBitSet reqColumns
+) {
+super(proj, cond, reqColumns);
+}
+}
diff --git
a/modules/sql/src/main/java/org/apache/ignite/internal/sql/rel/ProjectableFilterableTableScan.java
b/modules/sql/src/main/java/org/apache/ignite/internal/sql/rel/ProjectableFilterableTableScan.java
new file mode 100644
index 000..bd876f0
--- /dev/null
+++
b/modules/sql/src/main/java/org/apache/ignite/internal/sql/rel/ProjectableFilterableTableScan.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
[ignite-3] branch gg-33019 created (now 01379be)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch gg-33019 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. at 01379be GG-33019 Introduce ProjectableFilterableTableScan for native storage integration This branch includes the following new commits: new 01379be GG-33019 Introduce ProjectableFilterableTableScan for native storage integration The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite-3] branch ignite-14393 deleted (was f6ac843)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14393 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. was f6ac843 Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into ignite-14393 This change permanently discards the following revisions: discard f6ac843 Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into ignite-14393 discard 2687266 IGNITE-14393 Components interaction discard 558a98b IGNITE-14393 Get rid of multi-key update for table creation discard 0026274 IGNITE-14393 Updated image discard 5ba6651 IGNITE-14393 Components interactions workflow draft
[ignite-3] branch main updated: IGNITE-14393 Modules interaction description
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/main by this push: new 4018647 IGNITE-14393 Modules interaction description 4018647 is described below commit 4018647b9c3522b64e914da71c873c547cbfb0be Author: Alexey Goncharuk AuthorDate: Wed Apr 7 13:21:21 2021 +0300 IGNITE-14393 Modules interaction description --- modules/runner/README.md | 166 +++ 1 file changed, 166 insertions(+) diff --git a/modules/runner/README.md b/modules/runner/README.md new file mode 100644 index 000..47a1927 --- /dev/null +++ b/modules/runner/README.md @@ -0,0 +1,166 @@ +# Ignite cluster & node lifecycle +This document describes user-level and component-level cluster lifecycles and their mutual interaction. + +## Node lifecycle +A node maintains its' local state in the local persistent key-value storage named vault. The data stored in the vault is +semantically divided in the following categories: + * User-level local configuration properties (such as memory limits, network timeouts, etc). User-level configuration + properties can be written both at runtime (not all properties will be applied at runtime, however, - some of them will + require a full node restart) and when a node is shut down (in order to be able to change properties that prevent node + startup for some reason) + * System-level private properties (such as computed local statistics, node-local commin paths, etc). System-level + private properties are computed locally based on the information available at node locally (not based on metastorage + watched values) + * System-level distributed metastorage projected properties (such as paths to partition files, etc). System-level + projected properties are associated with one or more metastorage properties and are computed based on the local node + state and the metastorage properties values. System-level projected properties values are semantically bound to a + particular revision of the dependee properties and must be recalculated when dependees are changed (see + [reliable watch processing](#reliable-watch-processing)). + +The vault is created during the first node startup and optionally populated with the paremeters from the configuration +file passed in to the ``ignite node start`` [command](TODO link to CLI readme). Only user-level properties can be +written via the provided file. + +System-level properties are written to the storage during the first vault initialization by the node start process. +Projected properties are not initialized during the initial node startup because at this point the local node is not +aware of the distributed metastorage. The node remains in a 'zombie' state until after it learns that there is an +initialized metastorage (either via the ``ignite cluster init`` [command](TODO link to CLI readme) during the initial +cluster initialization) or from the group membershup service via gossip (implying that group membership protocol is +working at this point). + +### Node components startup +For testability purposes, we require that component dependencies are defined upfront and provided at the construction +time. This additionaly requires that component dependencies form no cycles. Therefore, components form an acyclic +directed graph that is constructed in topological sort order wrt root. + +Components created and initialized also in an order consistent with a topological sort of the components graph. This +enforces serveral rules related to the components interaction: + * Since metastorage watches can only be added during the component startup, the watch notification order is consistent + with the component initialization order. I.e. if a component `B` depdends on a component `A`, then `A` receives watch + notification prior to `B`. + * Dependent component can directly call an API method on a dependee component (because it can obtain the dependee + reference during construction). Direct inverse calls are prohibited (this is enforced by only acquiring component + references during the components construction). Nevertheless, inverse call can be implemented by means of listeners or + callbacks: the dependent component installs a listener to a dependeee, which can be later invoked. + + + + +The diagram above shows the component dependency diagram and provides an o
[ignite-3] branch ignite-14393 updated (2687266 -> f6ac843)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14393 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. from 2687266 IGNITE-14393 Components interaction add a8919fe IGNITE-14403 Added ignite-core module and IgniteUuid class add cb518de IGNITE-14149 RAFT client module. (#59) add 96b8e3a IGNITE-14035: Table access API. (#33) add 53e870d IGNITE-14198 Meta storage client interface add 8cf0bb2 IGNITE-14442 Fixed NPE in IgniteRunner caused by broken REST module. (#77) add 4d82a7e IGNITE-14438 Added README.md for cli and cli-common modules. Fixes #75 add e9ef6d4 IGNITE-14295 Network messages refactored to have a common interface and proper serializers/deserializers. (#70) add 297468b IGNITE-14180 Implemented the ability to subscribe to configuration updates. (#76) add ed2b820 IGNITE-14460 Single-node RAFT server (#80). add 15166f8 IGNITE-14457 Update Ignite 3 binary build structure (#79) new f6ac843 Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into ignite-14393 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: assembly/NOTICE| 0 assembly/README| 0 assembly/assembly.xml | 50 +++ check-rules/checkstyle-rules.xml | 11 +- modules/README.md | 7 +- modules/api/pom.xml| 38 +- .../BinaryObject.java} | 11 +- .../java/org/apache/ignite/lang/IgniteFuture.java} | 14 +- .../org/apache/ignite/table/InvocationContext.java | 68 +++ .../org/apache/ignite/table/InvokeProcessor.java | 47 +++ .../ignite/table/InvokeProcessorException.java}| 9 +- .../KeyValueBinaryView.java} | 18 +- .../java/org/apache/ignite/table/KeyValueView.java | 349 .../java/org/apache/ignite/table/RecordView.java | 42 ++ .../main/java/org/apache/ignite/table/Table.java | 92 + .../java/org/apache/ignite/table/TableView.java| 309 ++ .../main/java/org/apache/ignite/table/Tuple.java | 100 + .../TableIndex.java => table/TupleBuilder.java}| 20 +- .../org/apache/ignite/table/mapper/KeyMapper.java} | 7 +- .../org/apache/ignite/table/mapper/Mappers.java} | 37 +- .../apache/ignite/table/mapper/RecordMapper.java} | 19 +- .../apache/ignite/table/mapper/ValueMapper.java} | 24 +- .../java/org/apache/ignite/table/package-info.java | 10 +- .../presto/bytecode/DynamicClassLoader.java| 9 - .../expression/ConstantBytecodeExpression.java | 2 +- modules/cli-common/README.md | 5 + modules/cli/README.md | 32 ++ modules/configuration-annotation-processor/pom.xml | 12 - .../processor/internal/Processor.java | 31 +- .../configuration/processor/internal/Utils.java| 109 + .../storage => }/ConfigurationChangerTest.java | 28 +- .../internal/util/ConfigurationUtilTest.java | 82 +++- .../internal/validation/ValidationUtilTest.java| 2 +- .../notifications/ConfigurationListenerTest.java | 245 +++ .../sample/LocalConfigurationSchema.java | 2 +- .../sample/NetworkConfigurationSchema.java | 2 +- .../ignite/configuration/sample/UsageTest.java | 16 +- .../storage/TestConfigurationStorage.java | 25 +- modules/configuration/pom.xml | 13 - .../ignite/configuration/ConfigurationChanger.java | 209 ++ .../configuration/ConfigurationProperty.java | 14 +- .../configuration/ConfigurationRegistry.java | 59 ++- .../ignite/configuration/ConfigurationTree.java| 4 +- .../ignite/configuration/ConfigurationValue.java | 6 +- .../configuration/NamedConfigurationTree.java | 9 +- .../ignite/configuration/PropertyListener.java | 61 --- .../configuration/internal/ConfigurationNode.java | 19 +- .../internal/DynamicConfiguration.java | 9 +- .../configuration/internal/DynamicProperty.java| 16 +- .../internal/NamedListConfiguration.java | 25 +- .../ConfigurationNotificationEventImpl.java} | 41 +- .../util/ConfigurationNotificationsUtil.java | 184 + .../internal/util/ConfigurationUtil.java | 42 ++ .../notifications/ConfigurationListener.java} | 20 +- .../ConfigurationNamedListListener.java} | 45 +- .../ConfigurationNotificationEvent.java} | 52 +-- .../storage/ConfigurationStorage.java | 16 +- .../storage/ConfigurationStora
[ignite-3] 01/01: Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into ignite-14393
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-14393
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit f6ac843ccf1de28c923aa6333d505ac577c8d900
Merge: 2687266 15166f8
Author: Alexey Goncharuk
AuthorDate: Mon Apr 5 13:07:46 2021 +0300
Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into
ignite-14393
assembly/NOTICE| 0
assembly/README| 0
assembly/assembly.xml | 50 +++
check-rules/checkstyle-rules.xml | 11 +-
modules/README.md | 7 +-
modules/api/pom.xml| 38 +-
.../org/apache/ignite/binary/BinaryObject.java}| 12 +-
.../java/org/apache/ignite/lang/IgniteFuture.java} | 14 +-
.../org/apache/ignite/table/InvocationContext.java | 68 +++
.../org/apache/ignite/table/InvokeProcessor.java | 47 +++
.../ignite/table/InvokeProcessorException.java}| 9 +-
.../apache/ignite/table/KeyValueBinaryView.java} | 15 +-
.../java/org/apache/ignite/table/KeyValueView.java | 349
.../java/org/apache/ignite/table/RecordView.java | 42 ++
.../main/java/org/apache/ignite/table/Table.java | 92 +
.../java/org/apache/ignite/table/TableView.java| 309 ++
.../main/java/org/apache/ignite/table/Tuple.java | 100 +
.../org/apache/ignite/table/TupleBuilder.java} | 21 +-
.../org/apache/ignite/table/mapper/KeyMapper.java} | 8 +-
.../org/apache/ignite/table/mapper/Mappers.java} | 37 +-
.../apache/ignite/table/mapper/RecordMapper.java} | 20 +-
.../apache/ignite/table/mapper/ValueMapper.java} | 22 +-
.../org/apache/ignite/table}/package-info.java | 5 +-
.../presto/bytecode/DynamicClassLoader.java| 9 -
.../expression/ConstantBytecodeExpression.java | 2 +-
modules/cli-common/README.md | 5 +
modules/cli/README.md | 32 ++
modules/configuration-annotation-processor/pom.xml | 12 -
.../processor/internal/Processor.java | 31 +-
.../configuration/processor/internal/Utils.java| 109 +
.../storage => }/ConfigurationChangerTest.java | 28 +-
.../internal/util/ConfigurationUtilTest.java | 82 +++-
.../internal/validation/ValidationUtilTest.java| 2 +-
.../notifications/ConfigurationListenerTest.java | 245 +++
.../sample/LocalConfigurationSchema.java | 2 +-
.../sample/NetworkConfigurationSchema.java | 2 +-
.../ignite/configuration/sample/UsageTest.java | 16 +-
.../storage/TestConfigurationStorage.java | 25 +-
modules/configuration/pom.xml | 13 -
.../ignite/configuration/ConfigurationChanger.java | 209 ++
.../configuration/ConfigurationProperty.java | 14 +-
.../configuration/ConfigurationRegistry.java | 59 ++-
.../ignite/configuration/ConfigurationTree.java| 4 +-
.../ignite/configuration/ConfigurationValue.java | 6 +-
.../configuration/NamedConfigurationTree.java | 9 +-
.../ignite/configuration/PropertyListener.java | 61 ---
.../configuration/internal/ConfigurationNode.java | 19 +-
.../internal/DynamicConfiguration.java | 9 +-
.../configuration/internal/DynamicProperty.java| 16 +-
.../internal/NamedListConfiguration.java | 25 +-
.../ConfigurationNotificationEventImpl.java} | 41 +-
.../util/ConfigurationNotificationsUtil.java | 184 +
.../internal/util/ConfigurationUtil.java | 42 ++
.../ConfigurationListener.java}| 28 +-
.../ConfigurationNamedListListener.java} | 37 +-
.../ConfigurationNotificationEvent.java} | 45 +-
.../storage/ConfigurationStorage.java | 16 +-
.../storage/ConfigurationStorageListener.java | 1 +
.../apache/ignite/configuration/storage/Data.java | 22 +-
.../ignite/configuration/tree/NamedListNode.java | 5 +
.../ignite/configuration/tree/NamedListView.java | 5 +
modules/{configuration => core}/pom.xml| 36 +-
.../java/org/apache/ignite/lang/IgniteUuid.java| 135 ++
.../apache/ignite/lang/IgniteUuidGenerator.java| 84
.../java/org/apache/ignite/lang/LogWrapper.java| 80
modules/metastorage-client/pom.xml | 42 ++
.../metastorage/client/MetaStorageService.java | 345
.../ignite/metastorage/client}/package-info.java | 5 +-
modules/metastorage-common/pom.xml | 47 +++
.../metastorage/common/CompactedException.java | 59 +++
.../ignite/metastorage/common/Condition.java | 356
.../ignite/metastorage/common/Conditions.java} | 46 +--
.../apache/ignite/metastorage/common/Cursor.java} | 14 +-
.../apache/ignite/metastorage/
[ignite-3] branch ignite-14393 updated: IGNITE-14393 Components interaction
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14393 in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/ignite-14393 by this push: new 2687266 IGNITE-14393 Components interaction 2687266 is described below commit 26872663e3bcf574bd05507b08f68f6f5db0b14b Author: Alexey Goncharuk AuthorDate: Mon Apr 5 13:04:56 2021 +0300 IGNITE-14393 Components interaction --- modules/runner/README.md | 55 ++-- 1 file changed, 39 insertions(+), 16 deletions(-) diff --git a/modules/runner/README.md b/modules/runner/README.md index 6776168..47a1927 100644 --- a/modules/runner/README.md +++ b/modules/runner/README.md @@ -46,10 +46,9 @@ enforces serveral rules related to the components interaction: - + -The diagram above shows an example component dependency diagram and provides an order in which compomnents may be -initialized. +The diagram above shows the component dependency diagram and provides an order in which compomnents may be initialized. ## Cluster lifecycle For a cluster to become operational, the metastorage instance must be initialized first. The initialization command @@ -58,7 +57,27 @@ initialization command, it either creates a bootstrapped Raft instance with the group node), or writes the metastorage group member IDs to the vault as a private system-level property. After the metastorage is initialized, components start to receive and process watch events, updating the local state -according to the changes received from the watch. +according to the changes received from the watch. + +An entry point to user-initiated cluster state changes is [cluster configuration](../configuration/README.md). +Configuration module provides convenient ways for managing configuration both as Java API, as well as from ``ignite`` +command line utility. + +## Reliable configuration changes +Any configuration change is translated to a metastorage multi-update and has a single configuration change ID. This ID +is used to enforce CAS-style configuration changes and to ensure no duplicate configuration changes are executed during +the cluster runtime. To reliably process cluster configuration changes, we introduce an additional metastorage key + +``` +internal.configuration.applied= +``` + +that indicates the configuration change ID that was already processed and corresponding changes are written to the +metastorage. Whenever a node processes a configuration change, it must also conditionally update the +``internal.configuration.applied`` value checking that the previous value is smaller than the change ID being applied. +This prevents configuration changes being processed more than once. Any metastorage update that processes configuration +change must update this key to indicate that this configuraion change has been already processed. It is safe to process +the same configuration change more than once since only one update will be applied. ## Reliable watch processing All cluster state is written and maintained in the metastorage. Nodes may update some state in the metastorage, which @@ -92,30 +111,34 @@ of the watch, and not processed events are replayed. We require that each Ignite table is assigned a globally unique ID (the ID must not repeat even after the table is dropped, so we use the metastorage key revision to assign table IDs). -When a table is created, Ignite first checks that a table with the given name does not exist, then attempts to create -the following key-value pair in the metastorage via the conditional update ensuring atomic `putIfAbsent` semantics: +To create a table, a user makes a change in the configuration tree by introducing the corresponding configuration +object. This can be done either via public [configuration API](TODO link to configuration API) or via the ``ignite`` +[configuration command](TODO link to CLI readme). Configuration validator checks that a table with the same name does +not exist (and performs other necessary checks) and writes the change to the metastorage. If the update succeeds, Ignite
[ignite-3] branch ignite-14393 updated: IGNITE-14393 Get rid of multi-key update for table creation
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14393 in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/ignite-14393 by this push: new 558a98b IGNITE-14393 Get rid of multi-key update for table creation 558a98b is described below commit 558a98b08fc9fe26a28873cb5d00d490c4e126e5 Author: Alexey Goncharuk AuthorDate: Mon Mar 29 19:59:09 2021 +0300 IGNITE-14393 Get rid of multi-key update for table creation --- modules/runner/README.md | 32 +++- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/modules/runner/README.md b/modules/runner/README.md index 7226377..6776168 100644 --- a/modules/runner/README.md +++ b/modules/runner/README.md @@ -90,34 +90,32 @@ of the watch, and not processed events are replayed. ### Example: `CREATE TABLE` flow We require that each Ignite table is assigned a globally unique ID (the ID must not repeat even after the table is -dropped, so we use a growing `long` counter to assign table IDs). +dropped, so we use the metastorage key revision to assign table IDs). -When a table is created, Ignite first checks that a table with the given name does not exist, chooses the next available -table ID ``idNext`` and attempts to create the following pair of key-value pairs in the metastorage via the conditional -multi-update: +When a table is created, Ignite first checks that a table with the given name does not exist, then attempts to create +the following key-value pair in the metastorage via the conditional update ensuring atomic `putIfAbsent` semantics: ``` -internal.tables.names.= -internal.tables.=name +internal.tables.names.= ``` -If the multi-update succeeds, Ignite considers the table created. If the multi-update fails, then either the table with -the same name was concurrently created (the operation fails in this case) or the ``idNext`` was assigned to another -table with a different name (Ignite retries the operation in this case). +If the update succeeds, Ignite considers the table created and uses the returned key-value pair revision as the table +ID. If the update fails, then the table with the same name was concurrently created (the operation fails in this case). In order to process affinity calculations and assignments, the affinity manager creates a reliable watch for the following keys on metastorage group members: ``` -internal.tables. +internal.tables.names.* internal.baseline ``` Whenever a watch is fired, the affinity manager checks which key was updated. If the watch is triggered for -``internal.tables.`` key, it calculates a new affinity for the table with the given ``ID``. If the watch is -triggered for ``internal.baseline`` key, the manager recalculates affinity for all tables exsiting at the watch revision -(this can be done using the metastorage ``range(keys, upperBound)`` method providing the watch event revision as the -upper bound). The calculated affinity is written to the ``internal.tables..affinity`` key. +``internal.tables.names.`` key, it calculates a new affinity for the table with the given name (using key revision +as the table ID). If the watch is triggered for ``internal.baseline`` key, the manager recalculates affinity for all +tables exsiting at the watch revision (this can be done using the metastorage ``range(keys, upperBound)`` method +providing the watch event revision as the upper bound). The calculated affinity is written to the +``internal.tables.affinity.`` key. > Note that ideally the watch should only be processed on metastorage group > leader, thus eliminating unnecessary network > trips. Theoretically, we could have embedded this logic to the state > machine, but this would enormously complicate @@ -127,7 +125,7 @@ To handle partition assignments, partition manager creates a reliable watch for nodes: ``` -internal.tables..affinity +internal.tables.affinity. ``` Whenever a watch is fired, the node checks whether there exist new partitions assigned to the local node, and if there @@ -135,8 +133,8 @@ are, the node bootstraps corresponding Raft partition servers (i.e. allocates pa The allocation information is written to projected vault keys: ``` -local.tables...logpath=/path/to/raft/log -local.tables...storagepath=/path/to/storage/file +local.tables.partition...logpath=/path/to/raft/log +local.tables.partition...storagepath=/path/to/storage/file ``` Once the projected keys are synced to the vault, the partition manager can create partition Raft servers (initialize
[ignite-3] branch ignite-14393 updated: IGNITE-14393 Updated image
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14393 in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/ignite-14393 by this push: new 0026274 IGNITE-14393 Updated image 0026274 is described below commit 0026274db523b38e999219be2981c11a0f31d681 Author: Alexey Goncharuk AuthorDate: Thu Mar 25 10:50:56 2021 +0300 IGNITE-14393 Updated image --- modules/runner/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/runner/README.md b/modules/runner/README.md index a182dd9..7226377 100644 --- a/modules/runner/README.md +++ b/modules/runner/README.md @@ -46,7 +46,7 @@ enforces serveral rules related to the components interaction: - + The diagram above shows an example component dependency diagram and provides an order in which compomnents may be initialized.
[ignite-3] branch ignite-14393 created (now 5ba6651)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14393 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. at 5ba6651 IGNITE-14393 Components interactions workflow draft This branch includes the following new commits: new 5ba6651 IGNITE-14393 Components interactions workflow draft The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite-3] 01/01: IGNITE-14393 Components interactions workflow draft
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14393 in repository https://gitbox.apache.org/repos/asf/ignite-3.git commit 5ba6651a0da7b4637fc901efe384796bf4b5a75e Author: Alexey Goncharuk AuthorDate: Wed Mar 24 20:36:38 2021 +0300 IGNITE-14393 Components interactions workflow draft --- modules/runner/README.md | 145 +++ 1 file changed, 145 insertions(+) diff --git a/modules/runner/README.md b/modules/runner/README.md new file mode 100644 index 000..a182dd9 --- /dev/null +++ b/modules/runner/README.md @@ -0,0 +1,145 @@ +# Ignite cluster & node lifecycle +This document describes user-level and component-level cluster lifecycles and their mutual interaction. + +## Node lifecycle +A node maintains its' local state in the local persistent key-value storage named vault. The data stored in the vault is +semantically divided in the following categories: + * User-level local configuration properties (such as memory limits, network timeouts, etc). User-level configuration + properties can be written both at runtime (not all properties will be applied at runtime, however, - some of them will + require a full node restart) and when a node is shut down (in order to be able to change properties that prevent node + startup for some reason) + * System-level private properties (such as computed local statistics, node-local commin paths, etc). System-level + private properties are computed locally based on the information available at node locally (not based on metastorage + watched values) + * System-level distributed metastorage projected properties (such as paths to partition files, etc). System-level + projected properties are associated with one or more metastorage properties and are computed based on the local node + state and the metastorage properties values. System-level projected properties values are semantically bound to a + particular revision of the dependee properties and must be recalculated when dependees are changed (see + [reliable watch processing](#reliable-watch-processing)). + +The vault is created during the first node startup and optionally populated with the paremeters from the configuration +file passed in to the ``ignite node start`` [command](TODO link to CLI readme). Only user-level properties can be +written via the provided file. + +System-level properties are written to the storage during the first vault initialization by the node start process. +Projected properties are not initialized during the initial node startup because at this point the local node is not +aware of the distributed metastorage. The node remains in a 'zombie' state until after it learns that there is an +initialized metastorage (either via the ``ignite cluster init`` [command](TODO link to CLI readme) during the initial +cluster initialization) or from the group membershup service via gossip (implying that group membership protocol is +working at this point). + +### Node components startup +For testability purposes, we require that component dependencies are defined upfront and provided at the construction +time. This additionaly requires that component dependencies form no cycles. Therefore, components form an acyclic +directed graph that is constructed in topological sort order wrt root. + +Components created and initialized also in an order consistent with a topological sort of the components graph. This +enforces serveral rules related to the components interaction: + * Since metastorage watches can only be added during the component startup, the watch notification order is consistent + with the component initialization order. I.e. if a component `B` depdends on a component `A`, then `A` receives watch + notification prior to `B`. + * Dependent component can directly call an API method on a dependee component (because it can obtain the dependee + reference during construction). Direct inverse calls are prohibited (this is enforced by only acquiring component + references during the components construction). Nevertheless, inverse call can be implemented by means of listeners or + callbacks: the dependent component installs a listener to a dependeee, which can be later invoked. + + + + +The diagram above shows an example component dependency diagram and provides an order in which compomnents may be +initialized. + +## Cluster lifecycle +For a cluster to become operational, the metastorage instance must be initialized first. The initialization command +chooses a set of nodes (normally, 3 - 5 nodes) to host the distrib
[ignite-3] branch ignite-13618 updated: IGNITE-13618 Correct README.md
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-13618 in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/ignite-13618 by this push: new ab6bae6 IGNITE-13618 Correct README.md ab6bae6 is described below commit ab6bae6ba6fb4e3b3e5baa9d292f91ccfa66f727 Author: Alexey Goncharuk AuthorDate: Wed Mar 17 12:24:09 2021 +0300 IGNITE-13618 Correct README.md --- modules/schema/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/schema/README.md b/modules/schema/README.md index e640252..e5f396f 100644 --- a/modules/schema/README.md +++ b/modules/schema/README.md @@ -1,6 +1,6 @@ # Schema module -This module provides API and implementation for schema management components: +This module provides implementation for schema management components: * Public API for schema definition and evolution * Schema manager component that implements necessary machinary to translate schema management commands to corresponding
[ignite-3] branch main updated: IGNITE-14315 Use maven-flatten plugin (#68)
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 497ac43 IGNITE-14315 Use maven-flatten plugin (#68)
497ac43 is described below
commit 497ac43c9e4985085daa96cdc2d73b57e61dd626
Author: Peter Ivanov
AuthorDate: Mon Mar 15 13:12:31 2021 +0300
IGNITE-14315 Use maven-flatten plugin (#68)
---
.gitignore | 1 +
parent/pom.xml | 146 +++--
pom.xml| 84 -
3 files changed, 133 insertions(+), 98 deletions(-)
diff --git a/.gitignore b/.gitignore
index c3484a1..457eacc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,3 +2,4 @@
*.iml
target
.DS_Store
+.flattened-pom.xml
diff --git a/parent/pom.xml b/parent/pom.xml
index 3af2fbf..dc55047 100644
--- a/parent/pom.xml
+++ b/parent/pom.xml
@@ -84,6 +84,7 @@
3.8.1
2.8.2
3.0.0-M5
+1.2.2
3.14.0
3.2.1
3.0.0-M5
@@ -357,23 +358,18 @@
maven-pmd-plugin
${maven.pmd.plugin.version}
+
+
+org.codehaus.mojo
+flatten-maven-plugin
+${maven.flatten.plugin.version}
+
-
-org.apache.maven.plugins
-maven-pmd-plugin
-
-true
-
-
${project.basedir}/check-rules/pmd-rules.xml
-
-
-
-
org.codehaus.mojo
@@ -409,7 +405,7 @@
org.apache.maven.plugins
@@ -422,7 +418,7 @@
org.apache.maven.plugins
@@ -437,6 +433,128 @@
+
+
+
+org.codehaus.mojo
+flatten-maven-plugin
+
+
+
+
+flatten
+process-resources
+
+flatten
+
+
+
+
+
+flatten.clean.before
+clean
+
+clean
+
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-checkstyle-plugin
+
+
+
${project.build.sourceDirectory}
+
${project.build.testSourceDirectory}
+
+true
+true
+true
+true
+
${project.build.directory}/checkstyle.xml
+
${project.basedir}/check-rules/checkstyle-rules.xml
+
${project.basedir}/check-rules/checkstyle-suppressions.xml
+
true
+**/generated/**/*
+
+
+
+com.puppycrawl.tools
+checkstyle
+${checkstyle.puppycrawl.version}
+
+
+
+
+
+
+org.apache.rat
+apache-rat-plugin
+
+true
+
+
+
IAL20
+Ignite Apache License
2.0
+
+Licensed to the Apache Software Foundation
(ASF) under one or more
+contributor license agreements. See the
NOTICE file distributed with
+this work for additional information regarding
copyright ownership.
+The ASF licenses this file to You under the
Apache License, Version 2.0
+(the "License"); you may not use this file
except in compliance with
+the License. You may obtain a copy of the
License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to
in writing, software
+distributed under the License is distributed
on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied.
+
[ignite-3] 01/01: Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into ignite-14315
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-14315
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 51c946db74d9eada278ecd992b77b50410b877f8
Merge: c14637b 4afdbb3
Author: Alexey Goncharuk
AuthorDate: Mon Mar 15 13:06:53 2021 +0300
Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into
ignite-14315
DEVNOTES.md| 164 -
parent/pom.xml | 1 +
2 files changed, 93 insertions(+), 72 deletions(-)
diff --cc parent/pom.xml
index 925ed4b,3af2fbf..dc55047
--- a/parent/pom.xml
+++ b/parent/pom.xml
@@@ -433,127 -437,6 +433,128 @@@
+
+
+
+org.codehaus.mojo
+flatten-maven-plugin
+
+
+
+
+flatten
+process-resources
+
+flatten
+
+
+
+
+
+flatten.clean.before
+clean
+
+clean
+
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-checkstyle-plugin
+
+
+
${project.build.sourceDirectory}
+
${project.build.testSourceDirectory}
+
+true
+true
+true
+true
+
${project.build.directory}/checkstyle.xml
+
${project.basedir}/check-rules/checkstyle-rules.xml
+
${project.basedir}/check-rules/checkstyle-suppressions.xml
+
true
+**/generated/**/*
+
+
+
+com.puppycrawl.tools
+checkstyle
+${checkstyle.puppycrawl.version}
+
+
+
+
+
+
+org.apache.rat
+apache-rat-plugin
+
+
true
+
+
+
IAL20
+Ignite Apache License
2.0
+
+Licensed to the Apache Software Foundation
(ASF) under one or more
+contributor license agreements. See the
NOTICE file distributed with
+this work for additional information
regarding copyright ownership.
+The ASF licenses this file to You under the
Apache License, Version 2.0
+(the "License"); you may not use this file
except in compliance with
+the License. You may obtain a copy of the
License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed
to in writing, software
+distributed under the License is distributed
on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied.
+See the License for the specific language
governing permissions and
+limitations under the License.
+
+
+
+
+
+Ignite Apache License 2.0
+
+
+false
+
+**/target/**
+**/*.md
+docs/assets/images/**
+docs/assets/js/anchor.min.js
+**/*.json
+
modules/cli/src/**/resources//builtin_modules.conf
+
modules/configuration-annotation-processor/src/main/resources/META-INF/services/javax.annotation.processing.Processor
++**/*.iml
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-pmd-plugin
+
+true
+
+
${project.basedir}/check-rules/pmd-rules.xml
+
+
+
[ignite-3] branch ignite-14315 updated (c14637b -> 51c946d)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14315 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. from c14637b IGNITE-14315 Use maven-flatten-plugin add 4afdbb3 IGNITE-14298 DEVNOTES should contain clear instructions on how to run check-style new 51c946d Merge branch 'main' of https://gitbox.apache.org/repos/asf/ignite-3 into ignite-14315 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: DEVNOTES.md| 164 - parent/pom.xml | 1 + 2 files changed, 93 insertions(+), 72 deletions(-)
[ignite-3] 01/01: IGNITE-14315 Use maven-flatten-plugin
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-14315
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit c14637bb9bf855e05cd85d38d037ae6cc27210d9
Author: Peter Ivanov
AuthorDate: Thu Mar 11 18:22:01 2021 +0300
IGNITE-14315 Use maven-flatten-plugin
---
.gitignore | 1 +
parent/pom.xml | 145 +++--
pom.xml| 83 -
3 files changed, 132 insertions(+), 97 deletions(-)
diff --git a/.gitignore b/.gitignore
index c3484a1..457eacc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,3 +2,4 @@
*.iml
target
.DS_Store
+.flattened-pom.xml
diff --git a/parent/pom.xml b/parent/pom.xml
index 3af2fbf..925ed4b 100644
--- a/parent/pom.xml
+++ b/parent/pom.xml
@@ -84,6 +84,7 @@
3.8.1
2.8.2
3.0.0-M5
+1.2.2
3.14.0
3.2.1
3.0.0-M5
@@ -357,23 +358,18 @@
maven-pmd-plugin
${maven.pmd.plugin.version}
+
+
+org.codehaus.mojo
+flatten-maven-plugin
+${maven.flatten.plugin.version}
+
-
-org.apache.maven.plugins
-maven-pmd-plugin
-
-true
-
-
${project.basedir}/check-rules/pmd-rules.xml
-
-
-
-
org.codehaus.mojo
@@ -409,7 +405,7 @@
org.apache.maven.plugins
@@ -422,7 +418,7 @@
org.apache.maven.plugins
@@ -437,6 +433,127 @@
+
+
+
+org.codehaus.mojo
+flatten-maven-plugin
+
+
+
+
+flatten
+process-resources
+
+flatten
+
+
+
+
+
+flatten.clean.before
+clean
+
+clean
+
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-checkstyle-plugin
+
+
+
${project.build.sourceDirectory}
+
${project.build.testSourceDirectory}
+
+true
+true
+true
+true
+
${project.build.directory}/checkstyle.xml
+
${project.basedir}/check-rules/checkstyle-rules.xml
+
${project.basedir}/check-rules/checkstyle-suppressions.xml
+
true
+**/generated/**/*
+
+
+
+com.puppycrawl.tools
+checkstyle
+${checkstyle.puppycrawl.version}
+
+
+
+
+
+
+org.apache.rat
+apache-rat-plugin
+
+true
+
+
+
IAL20
+Ignite Apache License
2.0
+
+Licensed to the Apache Software Foundation
(ASF) under one or more
+contributor license agreements. See the
NOTICE file distributed with
+this work for additional information regarding
copyright ownership.
+The ASF licenses this file to You under the
Apache License, Version 2.0
+(the "License"); you may not use this file
except in compliance with
+the License. You may obtain a copy of the
License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to
in writing, software
+distributed under the License is distributed
on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied.
+See the License for the specific language
governing permissions and
+limitatio
[ignite-3] branch ignite-14315 created (now c14637b)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14315 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. at c14637b IGNITE-14315 Use maven-flatten-plugin This branch includes the following new commits: new c14637b IGNITE-14315 Use maven-flatten-plugin The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite-3] branch ignite-13618 updated: IGNITE-13618 Corrected a few checks for bytecode module, moved pieces of IEP-54 to module README.md
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13618
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/ignite-13618 by this push:
new 021f826 IGNITE-13618 Corrected a few checks for bytecode module,
moved pieces of IEP-54 to module README.md
021f826 is described below
commit 021f82695b4faa722133416599510bd071d75df7
Author: Alexey Goncharuk
AuthorDate: Thu Mar 11 20:48:21 2021 +0300
IGNITE-13618 Corrected a few checks for bytecode module, moved pieces of
IEP-54 to module README.md
---
modules/bytecode/README.md | 4 +-
.../facebook/presto/bytecode/MethodDefinition.java | 8 +-
.../presto/bytecode/MethodGenerationContext.java | 2 +-
modules/schema/README.md | 50 -
.../org/apache/ignite/internal/schema/README.md| 87 ++
.../ignite/internal/schema/package-info.java | 46
6 files changed, 139 insertions(+), 58 deletions(-)
diff --git a/modules/bytecode/README.md b/modules/bytecode/README.md
index 0135e01..3a178c8 100644
--- a/modules/bytecode/README.md
+++ b/modules/bytecode/README.md
@@ -1,4 +1,6 @@
# Apache Ignite Bytecode module
-Fork of PrestoDB Bytecode module (ver 0.243).
+Fork of [PrestoDB Bytecode module (ver
0.243)](https://github.com/prestodb/presto/tree/0.243/presto-bytecode).
* Removed unnecessary guava dependency.
* Tests migrated from TestNG to JUnit 5.
+
+This module provides a convenient thin wrapper around
[ASM](https://asm.ow2.io/) library to generate classes at runtime.
\ No newline at end of file
diff --git
a/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodDefinition.java
b/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodDefinition.java
index 0297d9b..1405765 100644
---
a/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodDefinition.java
+++
b/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodDefinition.java
@@ -11,6 +11,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package com.facebook.presto.bytecode;
import java.util.ArrayList;
@@ -67,12 +68,7 @@ public class MethodDefinition {
this.access = access;
this.name = name;
-if (returnType != null) {
-this.returnType = returnType;
-}
-else {
-this.returnType = type(void.class);
-}
+this.returnType = returnType != null ? returnType : type(void.class);
this.parameters = List.copyOf(parameters);
this.parameterTypes =
parameters.stream().map(Parameter::getType).collect(Collectors.toList());
this.parameterAnnotations = parameters.stream().map(p -> new
ArrayList()).collect(Collectors.toList());
diff --git
a/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodGenerationContext.java
b/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodGenerationContext.java
index ec2a65a..e62da9d 100644
---
a/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodGenerationContext.java
+++
b/modules/bytecode/src/main/java/com/facebook/presto/bytecode/MethodGenerationContext.java
@@ -91,7 +91,7 @@ public class MethodGenerationContext {
return true;
}
-private final class ScopeContext {
+private static final class ScopeContext {
private final Scope scope;
private final List variables;
diff --git a/modules/schema/README.md b/modules/schema/README.md
index e742dc5..e640252 100644
--- a/modules/schema/README.md
+++ b/modules/schema/README.md
@@ -1,7 +1,49 @@
# Schema module
-This module provides implementations for schema configuration API and schema
management components.
+This module provides API and implementation for schema management components:
-* Schema configuration public API implementation.
-* Distributed schema management for processing schema change events at runtime.
-* Schema version management for transparent upgrade stored data purposes
according to life-schema concept.
\ No newline at end of file
+* Public API for schema definition and evolution
+* Schema manager component that implements necessary machinary to translate
schema management commands to corresponding
+ metastorage modifications, as well as schema modification event processing
logic
+* Necessary logic to build and upgrade tuples - rows of specific schema that
encode user data in schema-defined format.
+
+## Schema-aware tables
+We require that at any moment in time an Ignite table has only one most recent
relevant schema. Upon schema
+modification, we assign a monotonically growing identifier to each version of
the cache schema. The ordering guarantees
+are provided by the underlying distributed metastorage. The history of schema
versions must be kept in the metastor
[ignite-3] branch ignite-14272 deleted (was c024eb6)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14272 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. was c024eb6 IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure This change permanently discards the following revisions: discard c024eb6 IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure discard a16573f IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure
[ignite-3] branch main updated: IGNITE-14272 Move DEVNOTES.md to proper location, add modules readme
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/main by this push: new f3775d0 IGNITE-14272 Move DEVNOTES.md to proper location, add modules readme f3775d0 is described below commit f3775d021c5f105785dff5977d47fac832f8d882 Author: Alexey Goncharuk AuthorDate: Wed Mar 10 18:23:13 2021 +0300 IGNITE-14272 Move DEVNOTES.md to proper location, add modules readme --- DEVNOTES.md| 55 +- modules/DEVNOTES.md| 47 -- modules/README.md | 19 .../configuration-annotation-processor/README.md | 5 ++ modules/network/README.md | 23 + 5 files changed, 100 insertions(+), 49 deletions(-) diff --git a/DEVNOTES.md b/DEVNOTES.md index 5575f29..0185e9b 100644 --- a/DEVNOTES.md +++ b/DEVNOTES.md @@ -1,13 +1,64 @@ -## Local Build +# Setting up and building +## Prerequisites + * Java 11 SDK + * Maven 3.6.0+ (for building) + +## Building Ignite +Ignite follows standard guidelines for multi-module maven projects, so it can be easily built using the following +command from the project root directory (you can disable the tests when building using `-DskipTests`): mvn clean install [-DskipTests] Upon completion, you will find the CLI tool under `modules/cli/target` directory. Use `ignite` on Linux and MacOS, or `ignite.exe` on Windows. -## License Headers Check +### Running tests +To run unit tests only, use the following command: +mvn test + +Before running integration tests, make sure to build the project using the ``package`` target. + +To run unit + integration tests, use the following command: + +mvn integration-test + +To run integration tests only, use the following command: + +mvn integration-test -Dskip.surefire.tests=true + +### Sanity check targets +Use the following commands to run generic sanity checks before submitting a PR. + +RAT license validation: + mvn validate + +Checkstyle code validation: + +mvn checkstyle:checkstyle:aggregate + +PMD static code analysis + +mvn compile pmd:check + + +## Setting up IntelliJ Idea project +You can quickly import Ignite project to your IDE using the root `pom.xml` file. In IntelliJ, choose `Open Project` +from the `Quick Start` box or choose `Open...` from the `File` menu and select the root `pom.xml` file. + +After opening the project in IntelliJ, double check that the Java SDK is properly configured for the project: + + * Open the `File` menu and select `Project Structure...` + * In the SDKs section, ensure that a 1.11 JDK is selected (create one if none exist) + * In the `Project` section, make sure the project language level is set to 11.0 as Ignite makes use of several Java 11 + language features + +Ignite uses machine code generation for some of it's modules. To generate necessary production code, build the project +using maven (see Building Ignite). + +## Code structure +High-level modules structure and detailed modules description can be found in the [modules readme](modules/README.md). ## Release candidate verification diff --git a/modules/DEVNOTES.md b/modules/DEVNOTES.md deleted file mode 100644 index 8e6d2a1..000 --- a/modules/DEVNOTES.md +++ /dev/null @@ -1,47 +0,0 @@ -# Setting up and building -## Prerequisites - * Java 11 SDK - * Maven 3.6.0+ (for building) - -## Building Ignite -Ignite follows standard guidelines for multi-module maven projects, so it can be easily built using the following -command from the project root directory: -```commandline -mvn clean install -``` -You can disable the tests when building using the following command: -```commandline -mvn clean install -DskipTests -``` - -### Running tests -To run unit tests only, use the following command: -```commandline -mvn test -``` - -Before running integration tests, make sure to build the project using the ``package`` target. - -To run unit + integration tests, use the following command: -```commandline -mvn integration-test -``` - -To run integration tests only, use the following command: -```commandline -mvn integration-test -Dskip.surefire.tests=true -``` - -## Setting up IntelliJ Idea project -You can quickly import Ignite project to your IDE using the root `pom.xml` file. In IntelliJ, choose `Open Project` -from the `Quick Start` box or choose `Open...` from the `File` menu and select the root `pom.xml` file. - -After opening the project in IntelliJ, double check that the Java SDK is properly configured for the project: - - * Open the `File` menu and select `Project Structure...` - * In the SDKs section, ensure that a 1.11 JDK is selected (create one if none exist) - * In the `Project` section, make sure the project language level is set
[ignite-3] branch ignite-14272 updated: IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14272 in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/ignite-14272 by this push: new c024eb6 IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure c024eb6 is described below commit c024eb664bb99162727d06b7640be75b55e13271 Author: Alexey Goncharuk AuthorDate: Wed Mar 10 18:17:04 2021 +0300 IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure --- modules/network/README.md | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/modules/network/README.md b/modules/network/README.md index faddaab..14ce10e 100644 --- a/modules/network/README.md +++ b/modules/network/README.md @@ -15,4 +15,9 @@ receiving messages across nodes in the cluster. Several delivery guarantee optio target cluster member left the cluster. Internally, the networking module preserves a queue which orders scheduled messages and persistently attempts to establish a connection and deliver scheduled messages in that particular order. Messages sent via patient mode cannot be arbitrarily dropped and are kept in a memory buffer until either they are - delivered or a destination node is reported as failed. + delivered or a destination node is reported as failed. + +On top of the described primitives, the networking module provides a higher-level request-response primitive which can +be thought of as an RPC call, implying a single response for the given request. This primitive requires that the message +being sent has a unique identifier that can be matched with response on receipt. +
[ignite-3] 01/01: IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-14272 in repository https://gitbox.apache.org/repos/asf/ignite-3.git commit a16573fe2d0ca341d4b9df8af5f76bb916eeddc8 Author: Alexey Goncharuk AuthorDate: Wed Mar 10 15:39:30 2021 +0300 IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure --- DEVNOTES.md| 55 +- modules/DEVNOTES.md| 47 -- modules/README.md | 19 .../configuration-annotation-processor/README.md | 5 ++ modules/network/README.md | 18 +++ 5 files changed, 95 insertions(+), 49 deletions(-) diff --git a/DEVNOTES.md b/DEVNOTES.md index 5575f29..0185e9b 100644 --- a/DEVNOTES.md +++ b/DEVNOTES.md @@ -1,13 +1,64 @@ -## Local Build +# Setting up and building +## Prerequisites + * Java 11 SDK + * Maven 3.6.0+ (for building) + +## Building Ignite +Ignite follows standard guidelines for multi-module maven projects, so it can be easily built using the following +command from the project root directory (you can disable the tests when building using `-DskipTests`): mvn clean install [-DskipTests] Upon completion, you will find the CLI tool under `modules/cli/target` directory. Use `ignite` on Linux and MacOS, or `ignite.exe` on Windows. -## License Headers Check +### Running tests +To run unit tests only, use the following command: +mvn test + +Before running integration tests, make sure to build the project using the ``package`` target. + +To run unit + integration tests, use the following command: + +mvn integration-test + +To run integration tests only, use the following command: + +mvn integration-test -Dskip.surefire.tests=true + +### Sanity check targets +Use the following commands to run generic sanity checks before submitting a PR. + +RAT license validation: + mvn validate + +Checkstyle code validation: + +mvn checkstyle:checkstyle:aggregate + +PMD static code analysis + +mvn compile pmd:check + + +## Setting up IntelliJ Idea project +You can quickly import Ignite project to your IDE using the root `pom.xml` file. In IntelliJ, choose `Open Project` +from the `Quick Start` box or choose `Open...` from the `File` menu and select the root `pom.xml` file. + +After opening the project in IntelliJ, double check that the Java SDK is properly configured for the project: + + * Open the `File` menu and select `Project Structure...` + * In the SDKs section, ensure that a 1.11 JDK is selected (create one if none exist) + * In the `Project` section, make sure the project language level is set to 11.0 as Ignite makes use of several Java 11 + language features + +Ignite uses machine code generation for some of it's modules. To generate necessary production code, build the project +using maven (see Building Ignite). + +## Code structure +High-level modules structure and detailed modules description can be found in the [modules readme](modules/README.md). ## Release candidate verification diff --git a/modules/DEVNOTES.md b/modules/DEVNOTES.md deleted file mode 100644 index 8e6d2a1..000 --- a/modules/DEVNOTES.md +++ /dev/null @@ -1,47 +0,0 @@ -# Setting up and building -## Prerequisites - * Java 11 SDK - * Maven 3.6.0+ (for building) - -## Building Ignite -Ignite follows standard guidelines for multi-module maven projects, so it can be easily built using the following -command from the project root directory: -```commandline -mvn clean install -``` -You can disable the tests when building using the following command: -```commandline -mvn clean install -DskipTests -``` - -### Running tests -To run unit tests only, use the following command: -```commandline -mvn test -``` - -Before running integration tests, make sure to build the project using the ``package`` target. - -To run unit + integration tests, use the following command: -```commandline -mvn integration-test -``` - -To run integration tests only, use the following command: -```commandline -mvn integration-test -Dskip.surefire.tests=true -``` - -## Setting up IntelliJ Idea project -You can quickly import Ignite project to your IDE using the root `pom.xml` file. In IntelliJ, choose `Open Project` -from the `Quick Start` box or choose `Open...` from the `File` menu and select the root `pom.xml` file. - -After opening the project in IntelliJ, double check that the Java SDK is properly configured for the project: - - * Open the `File` menu and select `Project Structure...` - * In the SDKs section, ensure that a 1.11 JDK is selected (create one if none exist) - * In the `Project` section, make sure the project language level is set to 11.0 as Ignite makes use of several Java 11 - language features - -Ignite uses machine code generation for some of it's modules. To generate necessary production
[ignite-3] branch ignite-14272 created (now a16573f)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-14272 in repository https://gitbox.apache.org/repos/asf/ignite-3.git. at a16573f IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure This branch includes the following new commits: new a16573f IGNITE-14272 DEVNOTES.md moved to proper location, added draft modules structure The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite-3] branch ignite-13618 updated: IGNITE-13618 Whitespace
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13618
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/ignite-13618 by this push:
new fc0783c IGNITE-13618 Whitespace
fc0783c is described below
commit fc0783c3a0e68308be10f89c42c9616e54a5fbca
Author: Alexey Goncharuk
AuthorDate: Wed Mar 10 15:32:24 2021 +0300
IGNITE-13618 Whitespace
---
.../apache/ignite/internal/schema/SchemaDescriptorTest.java | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaDescriptorTest.java
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaDescriptorTest.java
index 13e5158..d58ad99 100644
---
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaDescriptorTest.java
+++
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaDescriptorTest.java
@@ -28,14 +28,14 @@ public class SchemaDescriptorTest {
public void testColumnIndexedAccess() {
SchemaDescriptor desc = new SchemaDescriptor(1,
new Column[] {
-new Column("columnA",NativeType.BYTE, false),
-new Column("columnB",NativeType.UUID, false),
-new Column("columnC",NativeType.INTEGER, false),
+new Column("columnA", NativeType.BYTE, false),
+new Column("columnB", NativeType.UUID, false),
+new Column("columnC", NativeType.INTEGER, false),
},
new Column[] {
-new Column("columnD",NativeType.BYTE, false),
-new Column("columnE",NativeType.UUID, false),
-new Column("columnF",NativeType.INTEGER, false),
+new Column("columnD", NativeType.BYTE, false),
+new Column("columnE", NativeType.UUID, false),
+new Column("columnF", NativeType.INTEGER, false),
}
);
[ignite-3] branch ignite-13618 updated: IGNITE-13618 Introduce absolute column index (test)
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13618
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/ignite-13618 by this push:
new f1dd492 IGNITE-13618 Introduce absolute column index (test)
f1dd492 is described below
commit f1dd49281d1c5a0f80bf3b5153fdbe36d66afc5d
Author: Alexey Goncharuk
AuthorDate: Wed Mar 10 15:13:44 2021 +0300
IGNITE-13618 Introduce absolute column index (test)
---
.../internal/schema/SchemaDescriptorTest.java | 45 ++
1 file changed, 45 insertions(+)
diff --git
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaDescriptorTest.java
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaDescriptorTest.java
new file mode 100644
index 000..13e5158
--- /dev/null
+++
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaDescriptorTest.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+/**
+ *
+ */
+public class SchemaDescriptorTest {
+@Test
+public void testColumnIndexedAccess() {
+SchemaDescriptor desc = new SchemaDescriptor(1,
+new Column[] {
+new Column("columnA",NativeType.BYTE, false),
+new Column("columnB",NativeType.UUID, false),
+new Column("columnC",NativeType.INTEGER, false),
+},
+new Column[] {
+new Column("columnD",NativeType.BYTE, false),
+new Column("columnE",NativeType.UUID, false),
+new Column("columnF",NativeType.INTEGER, false),
+}
+);
+
+for (int i = 0; i < desc.length(); i++)
+Assertions.assertEquals(i, desc.column(i).schemaIndex());
+}
+}
[ignite-3] branch ignite-13618 updated: IGNITE-13618 Introduce absolute column index
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13618
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/ignite-13618 by this push:
new e931382 IGNITE-13618 Introduce absolute column index
e931382 is described below
commit e93138240525030c1da6cfdd4cd4991ad3e536fe
Author: Alexey Goncharuk
AuthorDate: Wed Mar 10 14:53:36 2021 +0300
IGNITE-13618 Introduce absolute column index
---
.../org/apache/ignite/internal/schema/Column.java | 28 -
.../org/apache/ignite/internal/schema/Columns.java | 13 +++-
.../ignite/internal/schema/SchemaDescriptor.java | 6 +-
.../ignite/internal/schema/TupleAssembler.java | 8 ++-
.../benchmarks/SerializerBenchmarkTest.java| 21 ---
.../apache/ignite/internal/schema/ColumnsTest.java | 69 +-
.../apache/ignite/internal/schema/TupleTest.java | 2 +-
.../schema/marshaller/JavaSerializerTest.java | 16 ++---
8 files changed, 121 insertions(+), 42 deletions(-)
diff --git
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/Column.java
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/Column.java
index b93c4d3..1758f77 100644
---
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/Column.java
+++
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/Column.java
@@ -24,6 +24,9 @@ package org.apache.ignite.internal.schema;
* flag is not taken into account when columns are compared.
*/
public class Column implements Comparable {
+/** Absolute index in schema descriptor. */
+private final int schemaIndex;
+
/**
* Column name.
*/
@@ -49,12 +52,35 @@ public class Column implements Comparable {
NativeType type,
boolean nullable
) {
+this(-1, name, type, nullable);
+}
+
+/**
+ * @param schemaIndex Absolute index of this column in its schema
descriptor.
+ * @param name Column name.
+ * @param type An instance of column data type.
+ * @param nullable If {@code false}, null values will not be allowed for
this column.
+ */
+Column(
+int schemaIndex,
+String name,
+NativeType type,
+boolean nullable
+) {
+this.schemaIndex = schemaIndex;
this.name = name;
this.type = type;
this.nullable = nullable;
}
/**
+ * @return Absolute index of this column in its schema descriptor.
+ */
+public int schemaIndex() {
+return schemaIndex;
+}
+
+/**
* @return Column name.
*/
public String name() {
@@ -106,6 +132,6 @@ public class Column implements Comparable {
/** {@inheritDoc} */
@Override public String toString() {
-return "Column [name=" + name + ", type=" + type + ", nullable=" +
nullable + ']';
+return "Column [idx=" + schemaIndex + ", name=" + name + ", type=" +
type + ", nullable=" + nullable + ']';
}
}
diff --git
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/Columns.java
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/Columns.java
index 5f0e795..0773b58 100644
---
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/Columns.java
+++
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/Columns.java
@@ -92,8 +92,8 @@ public class Columns {
*
* @param cols Array of columns.
*/
-public Columns(Column... cols) {
-this.cols = sortedCopy(cols);
+Columns(int baseSchemaIdx, Column... cols) {
+this.cols = sortedCopy(baseSchemaIdx, cols);
firstVarlenColIdx = findFirstVarlenColumn();
@@ -167,14 +167,21 @@ public class Columns {
}
/**
+ * @param schemaBaseIdx Base index of this columns object in its schema.
* @param cols User columns.
* @return A copy of user columns array sorted in column order.
*/
-private Column[] sortedCopy(Column[] cols) {
+private Column[] sortedCopy(int schemaBaseIdx, Column[] cols) {
Column[] cp = Arrays.copyOf(cols, cols.length);
Arrays.sort(cp);
+for (int i = 0; i < cp.length; i++) {
+Column c = cp[i];
+
+cp[i] = new Column(schemaBaseIdx + i, c.name(), c.type(),
c.nullable());
+}
+
return cp;
}
diff --git
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/SchemaDescriptor.java
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/SchemaDescriptor.java
index 1f43e67..f2855b1 100644
---
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/SchemaDescriptor.java
+++
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/SchemaDescriptor.java
@@ -35,10 +35,10 @@ public class SchemaDescriptor {
* @param keyCol
[ignite-3] branch main updated: IGNITE-14132 Add configuration to separate Integration and Unit tests (#50)
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 6cec7cb IGNITE-14132 Add configuration to separate Integration and
Unit tests (#50)
6cec7cb is described below
commit 6cec7cb45cb793720131ded060fd0f860bee1712
Author: Peter Ivanov
AuthorDate: Mon Mar 1 12:03:50 2021 +0300
IGNITE-14132 Add configuration to separate Integration and Unit tests (#50)
---
modules/DEVNOTES.md | 47 +++
parent/pom.xml | 1 +
2 files changed, 48 insertions(+)
diff --git a/modules/DEVNOTES.md b/modules/DEVNOTES.md
new file mode 100644
index 000..8e6d2a1
--- /dev/null
+++ b/modules/DEVNOTES.md
@@ -0,0 +1,47 @@
+# Setting up and building
+## Prerequisites
+ * Java 11 SDK
+ * Maven 3.6.0+ (for building)
+
+## Building Ignite
+Ignite follows standard guidelines for multi-module maven projects, so it can
be easily built using the following
+command from the project root directory:
+```commandline
+mvn clean install
+```
+You can disable the tests when building using the following command:
+```commandline
+mvn clean install -DskipTests
+```
+
+### Running tests
+To run unit tests only, use the following command:
+```commandline
+mvn test
+```
+
+Before running integration tests, make sure to build the project using the
``package`` target.
+
+To run unit + integration tests, use the following command:
+```commandline
+mvn integration-test
+```
+
+To run integration tests only, use the following command:
+```commandline
+mvn integration-test -Dskip.surefire.tests=true
+```
+
+## Setting up IntelliJ Idea project
+You can quickly import Ignite project to your IDE using the root `pom.xml`
file. In IntelliJ, choose `Open Project`
+from the `Quick Start` box or choose `Open...` from the `File` menu and select
the root `pom.xml` file.
+
+After opening the project in IntelliJ, double check that the Java SDK is
properly configured for the project:
+
+ * Open the `File` menu and select `Project Structure...`
+ * In the SDKs section, ensure that a 1.11 JDK is selected (create one if none
exist)
+ * In the `Project` section, make sure the project language level is set to
11.0 as Ignite makes use of several Java 11
+ language features
+
+Ignite uses machine code generation for some of it's modules. To generate
necessary production code, build the project
+using maven (see Building Ignite).
diff --git a/parent/pom.xml b/parent/pom.xml
index 573cff7..29cd9e3 100644
--- a/parent/pom.xml
+++ b/parent/pom.xml
@@ -394,6 +394,7 @@
**/IT*.java
+${skip.surefire.tests}
[ignite] branch master updated: IGNITE-14112 Revisit usages of GridClosureProcessor.runLocalSafe and GridClosureProcessor.callLocalSafe methods - Fixes #8743.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new 21f992a IGNITE-14112 Revisit usages of
GridClosureProcessor.runLocalSafe and GridClosureProcessor.callLocalSafe
methods - Fixes #8743.
21f992a is described below
commit 21f992a6958529cafd389505a05ad3506a565858
Author: Alexey Goncharuk
AuthorDate: Mon Feb 8 17:40:13 2021 +0300
IGNITE-14112 Revisit usages of GridClosureProcessor.runLocalSafe and
GridClosureProcessor.callLocalSafe methods - Fixes #8743.
Signed-off-by: Alexey Goncharuk
---
.../ignite/internal/IgniteSchedulerImpl.java | 4 +-
.../processors/cache/CacheGroupContext.java| 3 +-
.../processors/cache/GridCacheAdapter.java | 28 ++--
.../processors/cache/GridCacheIoManager.java | 3 +-
.../processors/cache/GridCacheMvccManager.java | 2 +-
.../cache/GridCachePartitionExchangeManager.java | 3 +-
.../cache/GridDeferredAckMessageSender.java| 3 +-
.../distributed/GridCacheTxRecoveryFuture.java | 3 +-
.../cache/distributed/dht/GridDhtCacheAdapter.java | 2 +-
.../dht/GridDhtTxAbstractEnlistFuture.java | 3 +-
.../dht/GridPartitionedSingleGetFuture.java| 2 +-
.../atomic/GridNearAtomicSingleUpdateFuture.java | 5 ++-
.../dht/atomic/GridNearAtomicUpdateFuture.java | 5 ++-
.../dht/preloader/GridDhtPartitionDemander.java| 5 ++-
.../preloader/GridDhtPartitionsExchangeFuture.java | 4 +-
.../dht/preloader/latch/ExchangeLatchManager.java | 3 +-
.../cache/distributed/near/GridNearTxLocal.java| 3 +-
.../processors/cache/local/GridLocalCache.java | 6 +--
.../cache/local/atomic/GridLocalAtomicCache.java | 9 ++--
.../processors/cache/mvcc/MvccProcessorImpl.java | 3 +-
.../query/GridCacheDistributedQueryFuture.java | 6 +--
.../query/GridCacheDistributedQueryManager.java| 4 +-
.../continuous/CacheContinuousQueryHandler.java| 3 +-
.../cache/transactions/IgniteTxManager.java| 5 ++-
.../PartitionCountersNeighborcastFuture.java | 3 +-
.../processors/cluster/ClusterProcessor.java | 5 ++-
.../cluster/GridClusterStateProcessor.java | 3 +-
.../continuous/GridContinuousProcessor.java| 3 +-
.../datastreamer/DataStreamProcessor.java | 3 +-
.../processors/datastreamer/DataStreamerImpl.java | 5 ++-
.../datastructures/DataStructuresProcessor.java| 5 ++-
.../marshaller/GridMarshallerMappingProcessor.java | 3 +-
.../PerformanceStatisticsProcessor.java| 3 +-
.../handlers/cache/GridCacheCommandHandler.java| 5 ++-
.../DataStructuresCommandHandler.java | 6 +--
.../rest/handlers/query/QueryCommandHandler.java | 8 ++--
.../processors/service/GridServiceProcessor.java | 3 +-
.../processors/service/ServiceDeploymentTask.java | 3 +-
.../internal/processors/task/GridTaskWorker.java | 5 ++-
.../internal/visor/query/VisorQueryUtils.java | 5 ++-
.../ignite/p2p/GridP2PLocalDeploymentSelfTest.java | 2 +-
.../query/h2/twostep/GridMapQueryExecutor.java | 50 +++---
42 files changed, 136 insertions(+), 101 deletions(-)
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/IgniteSchedulerImpl.java
b/modules/core/src/main/java/org/apache/ignite/internal/IgniteSchedulerImpl.java
index 804c0ff..6930d5c 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/IgniteSchedulerImpl.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/IgniteSchedulerImpl.java
@@ -30,6 +30,8 @@ import org.apache.ignite.IgniteScheduler;
import org.apache.ignite.internal.processors.security.OperationSecurityContext;
import org.apache.ignite.internal.processors.security.SecurityUtils;
import org.apache.ignite.internal.util.future.IgniteFutureImpl;
+import org.apache.ignite.internal.util.lang.GridPlainCallable;
+import org.apache.ignite.internal.util.lang.GridPlainRunnable;
import org.apache.ignite.internal.util.typedef.internal.A;
import org.apache.ignite.lang.IgniteFuture;
import org.apache.ignite.scheduler.SchedulerFuture;
@@ -181,7 +183,7 @@ public class IgniteSchedulerImpl implements
IgniteScheduler, Externalizable {
}
/** */
-private class SecurityAwareClosure implements Runnable, Callable,
GridInternalWrapper {
+private class SecurityAwareClosure implements GridPlainRunnable,
GridPlainCallable, GridInternalWrapper {
/** Security subject id. */
private final UUID secSubjId;
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java
index 1994457..8492982 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache
[ignite] branch sql-calcite updated: IGNITE-13729 Fix type casts for aggregates causing 'Not a boolean expression' error - Fixes #8738.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch sql-calcite
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/sql-calcite by this push:
new 2b26b33 IGNITE-13729 Fix type casts for aggregates causing 'Not a
boolean expression' error - Fixes #8738.
2b26b33 is described below
commit 2b26b335140533c35ddc0cf883ae265fae94b3b8
Author: korlov42
AuthorDate: Mon Feb 8 17:30:47 2021 +0300
IGNITE-13729 Fix type casts for aggregates causing 'Not a boolean
expression' error - Fixes #8738.
---
.../query/calcite/exec/exp/agg/Accumulators.java | 7 +-
.../calcite/exec/exp/agg/AccumulatorsFactory.java | 3 +-
.../query/calcite/AggregatesIntegrationTest.java | 130 +
.../query/calcite/CalciteQueryProcessorTest.java | 3 +-
.../ignite/testsuites/IgniteCalciteTestSuite.java | 4 +-
5 files changed, 141 insertions(+), 6 deletions(-)
diff --git
a/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/Accumulators.java
b/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/Accumulators.java
index f458fee..afb0ea3 100644
---
a/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/Accumulators.java
+++
b/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/Accumulators.java
@@ -281,7 +281,10 @@ public class Accumulators {
/** {@inheritDoc} */
@Override public void add(Object... args) {
-cnt++;
+assert F.isEmpty(args) || args.length == 1;
+
+if (F.isEmpty(args) || args[0] != null)
+cnt++;
}
/** {@inheritDoc} */
@@ -297,7 +300,7 @@ public class Accumulators {
/** {@inheritDoc} */
@Override public List argumentTypes(IgniteTypeFactory
typeFactory) {
-return
F.asList(typeFactory.createTypeWithNullability(typeFactory.createSqlType(BIGINT),
false));
+return
F.asList(typeFactory.createTypeWithNullability(typeFactory.createSqlType(ANY),
false));
}
/** {@inheritDoc} */
diff --git
a/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/AccumulatorsFactory.java
b/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/AccumulatorsFactory.java
index 09344e6..7cd8ece 100644
---
a/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/AccumulatorsFactory.java
+++
b/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/AccumulatorsFactory.java
@@ -29,6 +29,7 @@ import com.google.common.cache.LoadingCache;
import com.google.common.collect.ImmutableList;
import com.google.common.primitives.Primitives;
import org.apache.calcite.DataContext;
+import org.apache.calcite.adapter.enumerable.EnumUtils;
import org.apache.calcite.adapter.enumerable.JavaRowFormat;
import org.apache.calcite.adapter.enumerable.PhysTypeImpl;
import org.apache.calcite.adapter.enumerable.RexToLixTranslator;
@@ -112,7 +113,7 @@ public class AccumulatorsFactory implements
Supplierhttp://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.query.calcite;
+
+import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.cache.QueryEntity;
+import org.apache.ignite.cache.query.annotations.QuerySqlField;
+import org.apache.ignite.configuration.CacheConfiguration;
+import org.apache.ignite.internal.IgniteEx;
+import org.apache.ignite.internal.processors.query.QueryEngine;
+import org.apache.ignite.internal.processors.query.calcite.util.Commons;
+import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
+import org.apache.ignite.testframework.junits.WithSystemProperty;
+import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
+import org.junit.Test;
+
+/**
+ *
+ */
+@WithSystemProperty(key = "calcite.debug", value = "false")
+public class AggregatesIntegrationTest extends GridCommonAbstractTest {
+/** */
+private static IgniteEx client;
+
+/** {@inheritDoc} */
+@Override protected void beforeTestsStarted() throws Exception {
+startGrids(3);
+
+client = startClientGrid();
+}
+
+/** {@inheritDoc} */
+@Override protected void afterTest() {
+for (Ignite ign : G.allGrids()) {
+
[ignite] branch sql-calcite updated: IGNITE-13973 Fix query hang if an assertion is thrown during the query execution - Fixes #8709.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch sql-calcite
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/sql-calcite by this push:
new a823f50 IGNITE-13973 Fix query hang if an assertion is thrown during
the query execution - Fixes #8709.
a823f50 is described below
commit a823f507d6e6893a53ac7a01e10943949cfa3384
Author: korlov42
AuthorDate: Mon Feb 8 17:10:59 2021 +0300
IGNITE-13973 Fix query hang if an assertion is thrown during the query
execution - Fixes #8709.
Signed-off-by: Alexey Goncharuk
---
.../query/calcite/exec/ExchangeServiceImpl.java| 24 ++-
.../query/calcite/exec/ExecutionContext.java | 20 ++-
.../query/calcite/exec/ExecutionServiceImpl.java | 7 +-
.../query/calcite/exec/LogicalRelImplementor.java | 5 +-
.../query/calcite/exec/rel/AbstractNode.java | 4 +-
.../query/calcite/exec/rel/AggregateNode.java | 68 +++
.../exec/rel/CorrelatedNestedLoopJoinNode.java | 132 ++
.../query/calcite/exec/rel/Downstream.java | 6 +-
.../query/calcite/exec/rel/FilterNode.java | 60 +++
.../processors/query/calcite/exec/rel/Inbox.java | 66 +++
.../query/calcite/exec/rel/IndexSpoolNode.java | 51 ++
.../query/calcite/exec/rel/LimitNode.java | 37 ++--
.../query/calcite/exec/rel/MergeJoinNode.java | 106 ---
.../query/calcite/exec/rel/ModifyNode.java | 67 +++
.../query/calcite/exec/rel/NestedLoopJoinNode.java | 108
.../processors/query/calcite/exec/rel/Node.java| 3 +-
.../processors/query/calcite/exec/rel/Outbox.java | 50 ++
.../query/calcite/exec/rel/ProjectNode.java| 33 +---
.../query/calcite/exec/rel/RootNode.java | 16 +-
.../query/calcite/exec/rel/ScanNode.java | 32 ++--
.../query/calcite/exec/rel/SortNode.java | 65 +++
.../query/calcite/exec/rel/TableSpoolNode.java | 59 +++
.../query/calcite/exec/rel/UnionAllNode.java | 43 ++---
.../query/calcite/message/ErrorMessage.java| 3 +-
.../CalciteErrorHandlilngIntegrationTest.java | 196 +
.../calcite/exec/rel/ContinuousExecutionTest.java | 2 +-
.../query/calcite/exec/rel/ExecutionTest.java | 85 +
.../query/calcite/planner/PlannerTest.java | 10 +-
.../ignite/testsuites/IgniteCalciteTestSuite.java | 2 +
.../processors/query/GridQueryProcessor.java | 2 +-
modules/core/src/test/config/log4j-test.xml| 4 +
31 files changed, 696 insertions(+), 670 deletions(-)
diff --git
a/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ExchangeServiceImpl.java
b/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ExchangeServiceImpl.java
index 6ed92b6..2da1b9d 100644
---
a/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ExchangeServiceImpl.java
+++
b/modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ExchangeServiceImpl.java
@@ -159,7 +159,7 @@ public class ExchangeServiceImpl extends AbstractService
implements ExchangeServ
if (!F.isEmpty(inboxes)) {
for (Inbox inbox : inboxes) {
inbox.context().cancel();
-inbox.context().execute(inbox::close);
+inbox.context().execute(inbox::close, inbox::onError);
}
}
else if (log.isDebugEnabled()) {
@@ -177,7 +177,7 @@ public class ExchangeServiceImpl extends AbstractService
implements ExchangeServ
if (!F.isEmpty(outboxes)) {
for (Outbox outbox : outboxes) {
outbox.context().cancel();
-outbox.context().execute(outbox::close);
+outbox.context().execute(outbox::close, outbox::onError);
}
}
else if (log.isDebugEnabled()) {
@@ -193,8 +193,14 @@ public class ExchangeServiceImpl extends AbstractService
implements ExchangeServ
protected void onMessage(UUID nodeId, QueryBatchAcknowledgeMessage msg) {
Outbox outbox = mailboxRegistry().outbox(msg.queryId(),
msg.exchangeId());
-if (outbox != null)
-outbox.onAcknowledge(nodeId, msg.batchId());
+if (outbox != null) {
+try {
+outbox.onAcknowledge(nodeId, msg.batchId());
+}
+catch (Throwable t) {
+outbox.onError(t);
+}
+}
else if (log.isDebugEnabled()) {
log.debug("Stale acknowledge message received: [" +
"nodeId=" + nodeId + ", " +
@@ -218,8 +224,14 @@ public class ExchangeServiceImpl extends AbstractService
implements ExchangeServ
inbox = mailboxRegistry().register(newInbox);
[ignite] branch master updated: IGNITE-14111 Add javadoc for AbstractDataPageIO - Fixes #8742.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new 02043cc IGNITE-14111 Add javadoc for AbstractDataPageIO - Fixes #8742.
02043cc is described below
commit 02043ccf0fa42ce0892649d4303d0020e7938faf
Author: Alexey Goncharuk
AuthorDate: Tue Feb 2 16:28:24 2021 +0300
IGNITE-14111 Add javadoc for AbstractDataPageIO - Fixes #8742.
Signed-off-by: Alexey Goncharuk
---
.../persistence/tree/io/AbstractDataPageIO.java| 124 +
1 file changed, 124 insertions(+)
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java
index 4708f423..55690b8 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java
@@ -37,6 +37,130 @@ import static
org.apache.ignite.internal.util.GridUnsafe.bufferAddress;
/**
* Data pages IO.
+ *
+ * Rows in a data page are organized into two arrays growing toward each
other: items table and row data.
+ *
+ * Items table contains direct or indirect items which locate a row within the
page. Items table is stored at the
+ * beginning of a page. Each item has an item ID which serves as an external
item reference
+ * (see {@link PageIdUtils#link(long, int)}) and can be either direct or
indirect. The size of any item in the items
+ * table is 2 bytes. ID of a direct item is always the same as its index in
items table so it is not stored in the
+ * item itself. ID of an indirect item may differ from its index (see example
below) so it is stored it the item
+ * along with ID (index) of direct item.
+ *
+ * Direct and indirect items are always placed in the items table in such a
way that direct items are stored first,
+ * and indirect items are always stored after direct items. A data page
explicitly stores both direct and indirect
+ * items count (see {@link #getDirectCount(long)} and {@link
#getIndirectCount(long)}), so that the item type can be
+ * easily determined: items with indexes {@code [0, directCnt)} are always
direct and items with indexes
+ * {@code [directCnt, directCnt + indirectCnt)} are always indirect. Having
both direct and indirect items in a
+ * page allows page defragmentation without breaking external links. Total
number of rows stored in a page is equal
+ * to the number of direct items.
+ *
+ * The row data is stored at the end of the page; newer rows are stored closer
to the end of the items table.
+ * Direct Items
+ * Direct items refer a stored row directly by offset in the page:
+ *
+ *
+-+
+ * | Direct Items . (rows data)
|
+ * | (4000), (3800), (3600) . row_2_ row_1_ row_0_
|
+ * | | | |_^ ^^
|
+ * | | |_||
|
+ * | |__|
|
+ * | directCnt: 3
|
+ * | indirectCnt: 0
|
+ *
+-+
+ *
+ * Direct item ID always matches it's index in the items table. The value of a
direct item in the table is an
+ * offset of the row data within the page.
+ * Indirect Items
+ * An indirect item explicitly stores the indirect item ID (1 byte) and the
index of the direct item it refers to
+ * (1 byte). The referred direct item (referrent) stores the actual offset of
the row in the data page:
+ *
+ *
+-+
+ * | Direct Items .. Indirect items (rows data)
|
+ * |
|
+ * | ⌄|
|
+ * | (3600), (3800), (3 -> 0) row_2_ row_1_
|
+ * | | |_^___^
|
+ * | |_|
|
+ * | directCnt: 2
|
+ * | indirectCount
[ignite-3] branch main updated: IGNITE-14011 Add RAT, headers check, PMD, Idea Inspections, and code style suites for Ignite 3 (#28)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/ignite-3.git The following commit(s) were added to refs/heads/main by this push: new df8229d IGNITE-14011 Add RAT, headers check, PMD, Idea Inspections, and code style suites for Ignite 3 (#28) df8229d is described below commit df8229dac0a949ddce02a700a5544199d7b948c8 Author: Peter Ivanov AuthorDate: Wed Jan 20 15:29:45 2021 +0300 IGNITE-14011 Add RAT, headers check, PMD, Idea Inspections, and code style suites for Ignite 3 (#28) --- check-rules/checkstyle-rules.xml | 109 ++ check-rules/checkstyle-suppressions.xml| 24 ++ check-rules/pmd-rules.xml | 31 ++ modules/cli-common/pom.xml | 7 +- modules/cli/pom.xml| 17 +- modules/configuration-annotation-processor/pom.xml | 24 +- modules/configuration/pom.xml | 7 +- modules/ignite-runner/pom.xml | 19 +- pom.xml => parent/pom.xml | 120 +++ pom.xml| 385 + 10 files changed, 312 insertions(+), 431 deletions(-) diff --git a/check-rules/checkstyle-rules.xml b/check-rules/checkstyle-rules.xml new file mode 100644 index 000..36b4b86 --- /dev/null +++ b/check-rules/checkstyle-rules.xml @@ -0,0 +1,109 @@ + + + + +http://www.puppycrawl.com/dtds/configuration_1_3.dtd;> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/check-rules/checkstyle-suppressions.xml b/check-rules/checkstyle-suppressions.xml new file mode 100644 index 000..3273169 --- /dev/null +++ b/check-rules/checkstyle-suppressions.xml @@ -0,0 +1,24 @@ + + + + +https://checkstyle.org/dtds/suppressions_1_2.dtd;> + + diff --git a/check-rules/pmd-rules.xml b/check-rules/pmd-rules.xml new file mode 100644 index 000..e10ea10 --- /dev/null +++ b/check-rules/pmd-rules.xml @@ -0,0 +1,31 @@ + + + + +http://pmd.sourceforge.net/ruleset/2.0.0; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; + xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 http://pmd.sourceforge.net/ruleset_2_0_0.xsd;> + + +PMD Ruleset for Apache Ignite + + + + + diff --git a/modules/cli-common/pom.xml b/modules/cli-common/pom.xml index e932309..d6fd8c1 100644 --- a/modules/cli-common/pom.xml +++ b/modules/cli-common/pom.xml @@ -24,10 +24,11 @@ org.apache.ignite -apache-ignite -3.0.0-SNAPSHOT -../../pom.xml +ignite-parent +1 +../../parent/pom.xml ignite-cli-common +3.0.0-SNAPSHOT diff --git a/modules/cli/pom.xml b/modules/cli/pom.xml index 42a8f9d..8c6a3d2 100644 --- a/modules/cli/pom.xml +++ b/modules/cli/pom.xml @@ -26,12 +26,13 @@ org.apache.ignite -apache-ignite -3.0.0-SNAPSHOT -../../pom.xml +ignite-parent +1 +../../parent/pom.xml ignite-cli +3.0.0-SNAPSHOT @@ -234,16 +235,6 @@ true - - -org.apache.rat -apache-rat-plugin - - -**/builtin_modules.conf - - - diff --git a/modules/configuration-annotation-processor/pom.xml b/modules/configuration-annotation-processor/pom.xml index fa93d22..a986a59 100644 --- a/modules/configuration-annotation-processor/pom.xml +++ b/modules/configuration-annotation-processor/pom.xml @@ -26,12 +26,13 @@ org.apache.ignite -apache-ignite -3.0.0-SNAPSHOT -../../pom.xml +ignite-parent +1 +../../parent/pom.xml ignite-configuration-annotation-processor +3.0.0-SNAPSHOT @@ -70,6 +71,13 @@ spoon-core test + + +org.junit.jupiter +junit-jupiter-engine +test + + @@ -96,16 +104,6 @@ - - -org.apache.rat -apache-rat-plugin - - - src/main/resou
[ignite] branch master updated: IGNITE-13786 Add defragmentation-specific B+Tree optimizations - Fixes #8560.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new dc4b71b IGNITE-13786 Add defragmentation-specific B+Tree
optimizations - Fixes #8560.
dc4b71b is described below
commit dc4b71b230e289e5b0d99b00e0632c9d1d0e6e82
Author: ibessonov
AuthorDate: Thu Dec 10 18:05:04 2020 +0300
IGNITE-13786 Add defragmentation-specific B+Tree optimizations - Fixes
#8560.
Signed-off-by: Alexey Goncharuk
---
.../CachePartitionDefragmentationManager.java | 24 ++
.../cache/persistence/tree/BPlusTree.java | 23 +
.../cache/persistence/tree/util/InsertLast.java| 24 --
.../persistence/IgnitePdsDefragmentationTest.java | 2 +-
.../defragmentation/IndexingDefragmentation.java | 6 --
5 files changed, 44 insertions(+), 35 deletions(-)
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/defragmentation/CachePartitionDefragmentationManager.java
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/defragmentation/CachePartitionDefragmentationManager.java
index 75b3458..aa82a23 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/defragmentation/CachePartitionDefragmentationManager.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/defragmentation/CachePartitionDefragmentationManager.java
@@ -93,6 +93,7 @@ import static
org.apache.ignite.internal.processors.cache.persistence.Checkpoint
import static
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.DEFRAGMENTATION_MAPPING_REGION_NAME;
import static
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.DEFRAGMENTATION_PART_REGION_NAME;
import static
org.apache.ignite.internal.processors.cache.persistence.defragmentation.DefragmentationFileUtils.batchRenameDefragmentedCacheGroupPartitions;
+import static
org.apache.ignite.internal.processors.cache.persistence.defragmentation.DefragmentationFileUtils.defragmentedIndexFile;
import static
org.apache.ignite.internal.processors.cache.persistence.defragmentation.DefragmentationFileUtils.defragmentedIndexTmpFile;
import static
org.apache.ignite.internal.processors.cache.persistence.defragmentation.DefragmentationFileUtils.defragmentedPartFile;
import static
org.apache.ignite.internal.processors.cache.persistence.defragmentation.DefragmentationFileUtils.defragmentedPartMappingFile;
@@ -386,9 +387,9 @@ public class CachePartitionDefragmentationManager {
partCtx.partPageMemory
);
-partCtx.createNewCacheDataStore(offheap);
+partCtx.createNewCacheDataStore(offheap);
-copyPartitionData(partCtx, treeIter);
+copyPartitionData(partCtx, treeIter);
DefragmentationPageReadWriteManager pageMgr =
(DefragmentationPageReadWriteManager)partCtx.partPageMemory.pageManager();
@@ -450,7 +451,21 @@ public class CachePartitionDefragmentationManager {
.futureFor(CheckpointState.FINISHED);
}
+PageStore oldIdxPageStore =
filePageStoreMgr.getStore(grpId, INDEX_PARTITION);
+
idxDfrgFut = idxDfrgFut.chain(fut -> {
+if (log.isDebugEnabled()) {
+log.debug(S.toString(
+"Index partition defragmented",
+"grpId", grpId, false,
+"oldPages", oldIdxPageStore.pages(), false,
+"newPages", idxAllocationTracker.get() + 1,
false,
+"pageSize", pageSize, false,
+"partFile",
defragmentedIndexFile(workDir).getName(), false,
+"workDir", workDir, false
+));
+}
+
oldPageMem.invalidate(grpId,
PageIdAllocator.INDEX_PARTITION);
PageMemoryEx partPageMem =
(PageMemoryEx)partDataRegion.pageMemory();
@@ -476,8 +491,6 @@ public class CachePartitionDefragmentationManager {
return null;
});
-PageStore oldIdxPageStore =
filePageStoreMgr.getStore(grpId, INDEX_PARTITION);
-
status.onIndexDefragmented(
oldGrpCtx,
oldIdxPageStore.size(),
@@ -614,6 +627,9 @@ public
[ignite-extensions] annotated tag ignite-flume-ext-1.0.0 created (now aa2c59e)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flume-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at aa2c59e (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-flume-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-flink-ext-1.0.0 created (now ebb68db)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flink-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ebb68db (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-flink-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-mqtt-ext-1.0.0 created (now ce071fb)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-mqtt-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ce071fb (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-mqtt-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-pub-sub-ext-1.0.0 created (now 65d68ad)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-pub-sub-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 65d68ad (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-pub-sub-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-camel-ext-1.0.0 created (now 57ca12b)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-camel-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 57ca12b (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-camel-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-rocketmq-ext-1.0.0 created (now c04faf7)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-rocketmq-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at c04faf7 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-rocketmq-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-zeromq-ext-1.0.0 created (now 4a63143)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-zeromq-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 4a63143 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-zeromq-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-jms11-ext-1.0.0 created (now ed555c2)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-jms11-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ed555c2 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-jms11-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-storm-ext-1.0.0 created (now acd7ffd)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-storm-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at acd7ffd (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-storm-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-twitter-ext-1.0.0 created (now 9fa9446)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-twitter-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 9fa9446 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-twitter-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-kafka-ext-1.0.0 created (now a4e6ad3)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-kafka-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at a4e6ad3 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-kafka-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite] branch master updated: IGNITE-13753 Fix non-thread-safe collection in JmxMetricExporterSpi - Fixes #8492.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new ea97f8a IGNITE-13753 Fix non-thread-safe collection in
JmxMetricExporterSpi - Fixes #8492.
ea97f8a is described below
commit ea97f8ab24a74eac75b41e41bfa33ada1ee269f4
Author: Alexey Goncharuk
AuthorDate: Thu Nov 26 19:22:09 2020 +0300
IGNITE-13753 Fix non-thread-safe collection in JmxMetricExporterSpi - Fixes
#8492.
Signed-off-by: Alexey Goncharuk
---
.../spi/metric/jmx/JmxMetricExporterSpi.java | 9 +-
.../ignite/spi/metric/jmx/DummyMBeanServer.java| 291 +
.../spi/metric/jmx/JmxMetricExporterSpiTest.java | 141 ++
.../ignite/testsuites/IgniteSpiTestSuite.java | 5 +-
4 files changed, 443 insertions(+), 3 deletions(-)
diff --git
a/modules/core/src/main/java/org/apache/ignite/spi/metric/jmx/JmxMetricExporterSpi.java
b/modules/core/src/main/java/org/apache/ignite/spi/metric/jmx/JmxMetricExporterSpi.java
index 7671a81..fe56007 100644
---
a/modules/core/src/main/java/org/apache/ignite/spi/metric/jmx/JmxMetricExporterSpi.java
+++
b/modules/core/src/main/java/org/apache/ignite/spi/metric/jmx/JmxMetricExporterSpi.java
@@ -18,6 +18,7 @@
package org.apache.ignite.spi.metric.jmx;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.List;
import java.util.function.Predicate;
import javax.management.JMException;
@@ -46,7 +47,7 @@ public class JmxMetricExporterSpi extends IgniteSpiAdapter
implements MetricExpo
private @Nullable Predicate filter;
/** Registered beans. */
-private final List mBeans = new ArrayList<>();
+private final List mBeans = Collections.synchronizedList(new
ArrayList<>());
/** {@inheritDoc} */
@Override public void spiStart(@Nullable String igniteInstanceName) throws
IgniteSpiException {
@@ -127,6 +128,10 @@ public class JmxMetricExporterSpi extends IgniteSpiAdapter
implements MetricExpo
unregBean(ignite, bean);
}
+/**
+ * @param ignite Ignite instance.
+ * @param bean Bean name to unregister.
+ */
private void unregBean(Ignite ignite, ObjectName bean) {
MBeanServer jmx = ignite.configuration().getMBeanServer();
@@ -143,7 +148,7 @@ public class JmxMetricExporterSpi extends IgniteSpiAdapter
implements MetricExpo
/** {@inheritDoc} */
@Override public void setMetricRegistry(ReadOnlyMetricManager reg) {
-this.mreg = reg;
+mreg = reg;
}
/** {@inheritDoc} */
diff --git
a/modules/core/src/test/java/org/apache/ignite/spi/metric/jmx/DummyMBeanServer.java
b/modules/core/src/test/java/org/apache/ignite/spi/metric/jmx/DummyMBeanServer.java
new file mode 100644
index 000..4d9c467
--- /dev/null
+++
b/modules/core/src/test/java/org/apache/ignite/spi/metric/jmx/DummyMBeanServer.java
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.spi.metric.jmx;
+
+import java.io.ObjectInputStream;
+import java.util.Set;
+import javax.management.Attribute;
+import javax.management.AttributeList;
+import javax.management.MBeanInfo;
+import javax.management.MBeanServer;
+import javax.management.NotificationFilter;
+import javax.management.NotificationListener;
+import javax.management.ObjectInstance;
+import javax.management.ObjectName;
+import javax.management.QueryExp;
+import javax.management.loading.ClassLoaderRepository;
+
+/**
+ *
+ */
+class DummyMBeanServer implements MBeanServer {
+/** */
+public static final String[] DOMAINS = new String[0];
+
+/**
+ * {@inheritDoc}
+ */
+@Override public ObjectInstance createMBean(String clsName, ObjectName
name) {
+return null;
+}
+
+/**
+ * {@inheritDoc}
+ */
+@Override public ObjectInstance createMBean(String clsName, ObjectName
name, ObjectName ldrName) {
+return null;
+}
+
+/**
+ * {@inheritDoc}
+ */
+@Override public ObjectInstance createMBean(String clsName, ObjectNam
[ignite-extensions] annotated tag ignite-mqtt-ext-1.0.0-rc1 created (now ce071fb)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-mqtt-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ce071fb (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-mqtt-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-camel-ext-1.0.0-rc1 created (now 57ca12b)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-camel-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 57ca12b (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-camel-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-flume-ext-1.0.0-rc1 created (now aa2c59e)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flume-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at aa2c59e (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-flume-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-zeromq-ext-1.0.0-rc1 created (now 4a63143)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-zeromq-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 4a63143 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-zeromq-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-twitter-ext-1.0.0-rc1 created (now 9fa9446)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-twitter-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 9fa9446 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-twitter-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-kafka-ext-1.0.0-rc1 created (now a4e6ad3)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-kafka-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at a4e6ad3 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-kafka-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-storm-ext-1.0.0-rc1 created (now acd7ffd)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-storm-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at acd7ffd (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-storm-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-rocketmq-ext-1.0.0-rc1 created (now c04faf7)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-rocketmq-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at c04faf7 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-rocketmq-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-jms11-ext-1.0.0-rc1 created (now ed555c2)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-jms11-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ed555c2 (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-jms11-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-flink-ext-1.0.0-rc1 created (now ebb68db)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flink-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ebb68db (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-flink-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-pub-sub-ext-1.0.0-rc1 created (now 65d68ad)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-pub-sub-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 65d68ad (tag) tagging 3f45b3e32d617b652e08af614a90aad6b9ff945d (commit) by Alexey Goncharuk on Mon Nov 23 16:03:35 2020 +0300 - Log - Releasing ignite-pub-sub-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] branch streaming-all-ext-1.0.0 updated: IGNITE-13745 Streaming extensions release notes
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch streaming-all-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git The following commit(s) were added to refs/heads/streaming-all-ext-1.0.0 by this push: new 3f45b3e IGNITE-13745 Streaming extensions release notes 3f45b3e is described below commit 3f45b3e32d617b652e08af614a90aad6b9ff945d Author: Alexey Goncharuk AuthorDate: Mon Nov 23 14:58:28 2020 +0300 IGNITE-13745 Streaming extensions release notes --- RELEASE_NOTES.txt | 16 1 file changed, 16 insertions(+) diff --git a/RELEASE_NOTES.txt b/RELEASE_NOTES.txt index e691735..8451780 100644 --- a/RELEASE_NOTES.txt +++ b/RELEASE_NOTES.txt @@ -1,6 +1,22 @@ Apache Ignite Extensions Release Notes === +Apache Ignite Streamer Extension Modules 1.0 +- +The following integrations were migrated the Apache Ignite Extensions repository: +* Camel integration extension +* Flink integration extension +* Flume integration extension +* JMS integration extension +* Kafka integration extension +* MQTT integration extension +* Pub/Sub integration extension +* RocketMQ integration extension +* Storm integration extension +* Twitter integration extension +* ZeroMQ integration extension + + Apache Ignite Spring Boot Modules 1.0 -
[ignite-extensions] branch master updated: IGNITE-13745 Streaming extensions release notes
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git The following commit(s) were added to refs/heads/master by this push: new 74bc337 IGNITE-13745 Streaming extensions release notes 74bc337 is described below commit 74bc3371b2e6aeae7a02ffad18040a804e7e4242 Author: Alexey Goncharuk AuthorDate: Mon Nov 23 14:58:28 2020 +0300 IGNITE-13745 Streaming extensions release notes --- RELEASE_NOTES.txt | 16 1 file changed, 16 insertions(+) diff --git a/RELEASE_NOTES.txt b/RELEASE_NOTES.txt index e691735..8451780 100644 --- a/RELEASE_NOTES.txt +++ b/RELEASE_NOTES.txt @@ -1,6 +1,22 @@ Apache Ignite Extensions Release Notes === +Apache Ignite Streamer Extension Modules 1.0 +- +The following integrations were migrated the Apache Ignite Extensions repository: +* Camel integration extension +* Flink integration extension +* Flume integration extension +* JMS integration extension +* Kafka integration extension +* MQTT integration extension +* Pub/Sub integration extension +* RocketMQ integration extension +* Storm integration extension +* Twitter integration extension +* ZeroMQ integration extension + + Apache Ignite Spring Boot Modules 1.0 -
[ignite-extensions] annotated tag ignite-zeromq-ext-1.0.0-rc1 deleted (was 13f8c9a)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-zeromq-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-zeromq-ext-1.0.0-rc1 was deleted! *** tag was 13f8c9a The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-twitter-ext-1.0.0-rc1 deleted (was ac9f893)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-twitter-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-twitter-ext-1.0.0-rc1 was deleted! *** tag was ac9f893 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-storm-ext-1.0.0-rc1 deleted (was f84b0a1)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-storm-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-storm-ext-1.0.0-rc1 was deleted! *** tag was f84b0a1 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-rocketmq-ext-1.0.0-rc1 deleted (was 0066c9a)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-rocketmq-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-rocketmq-ext-1.0.0-rc1 was deleted! *** tag was 0066c9a The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-pub-sub-ext-1.0.0-rc1 deleted (was 17c3e78)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-pub-sub-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-pub-sub-ext-1.0.0-rc1 was deleted! *** tag was 17c3e78 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-mqtt-ext-1.0.0-rc1 deleted (was 3c924da)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-mqtt-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-mqtt-ext-1.0.0-rc1 was deleted! *** tag was 3c924da The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-kafka-ext-1.0.0-rc1 deleted (was fcf9a79)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-kafka-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-kafka-ext-1.0.0-rc1 was deleted! *** tag was fcf9a79 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-jms11-ext-1.0.0-rc1 deleted (was 03e0ddc)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-jms11-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-jms11-ext-1.0.0-rc1 was deleted! *** tag was 03e0ddc The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-flume-ext-1.0.0-rc1 deleted (was ac95c28)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flume-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-flume-ext-1.0.0-rc1 was deleted! *** tag was ac95c28 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-camel-ext-1.0.0-rc1 deleted (was 429f400)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-camel-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-camel-ext-1.0.0-rc1 was deleted! *** tag was 429f400 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-flink-ext-1.0.0-rc1 deleted (was 3fcd4e5)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flink-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. *** WARNING: tag ignite-flink-ext-1.0.0-rc1 was deleted! *** tag was 3fcd4e5 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-extensions] annotated tag ignite-mqtt-ext-1.0.0-rc1 created (now 3c924da)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-mqtt-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 3c924da (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-mqtt-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-rocketmq-ext-1.0.0-rc1 created (now 0066c9a)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-rocketmq-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 0066c9a (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-rocketmq-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-kafka-ext-1.0.0-rc1 created (now fcf9a79)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-kafka-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at fcf9a79 (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-kafka-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-twitter-ext-1.0.0-rc1 created (now ac9f893)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-twitter-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ac9f893 (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-twitter-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-pub-sub-ext-1.0.0-rc1 created (now 17c3e78)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-pub-sub-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 17c3e78 (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-pub-sub-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-jms11-ext-1.0.0-rc1 created (now 03e0ddc)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-jms11-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 03e0ddc (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-jms11-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-storm-ext-1.0.0-rc1 created (now f84b0a1)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-storm-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at f84b0a1 (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-storm-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-zeromq-ext-1.0.0-rc1 created (now 13f8c9a)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-zeromq-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 13f8c9a (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-zeromq-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-flume-ext-1.0.0-rc1 created (now ac95c28)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flume-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at ac95c28 (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-flume-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-flink-ext-1.0.0-rc1 created (now 3fcd4e5)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-flink-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 3fcd4e5 (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-flink-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] annotated tag ignite-camel-ext-1.0.0-rc1 created (now 429f400)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-camel-ext-1.0.0-rc1 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 429f400 (tag) tagging 99e1c9ed82db3cdcf85c562f96a2811218d08a9e (commit) by Alexey Goncharuk on Mon Nov 23 13:18:29 2020 +0300 - Log - Releasing ignite-camel-ext-1.0.0-rc1 --- No new revisions were added by this update.
[ignite-extensions] branch streaming-all-ext-1.0.0 created (now 99e1c9e)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch streaming-all-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at 99e1c9e Release streaming extensions v.1.0.0 This branch includes the following new commits: new 99e1c9e Release streaming extensions v.1.0.0 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite-extensions] 01/01: Release streaming extensions v.1.0.0
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch streaming-all-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git commit 99e1c9ed82db3cdcf85c562f96a2811218d08a9e Author: Alexey Goncharuk AuthorDate: Mon Nov 23 12:39:30 2020 +0300 Release streaming extensions v.1.0.0 --- modules/camel-ext/pom.xml| 2 +- modules/flink-ext/pom.xml| 2 +- modules/flume-ext/pom.xml| 2 +- modules/jms11-ext/pom.xml| 2 +- modules/kafka-ext/pom.xml| 2 +- modules/mqtt-ext/pom.xml | 2 +- modules/pub-sub-ext/pom.xml | 2 +- modules/rocketmq-ext/pom.xml | 2 +- modules/storm-ext/pom.xml| 2 +- modules/twitter-ext/pom.xml | 2 +- modules/zeromq-ext/pom.xml | 2 +- parent/pom.xml | 2 +- 12 files changed, 12 insertions(+), 12 deletions(-) diff --git a/modules/camel-ext/pom.xml b/modules/camel-ext/pom.xml index 2ae77d5..7c1838e 100644 --- a/modules/camel-ext/pom.xml +++ b/modules/camel-ext/pom.xml @@ -31,7 +31,7 @@ ignite-camel-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/flink-ext/pom.xml b/modules/flink-ext/pom.xml index fe84ca5..24118f6 100644 --- a/modules/flink-ext/pom.xml +++ b/modules/flink-ext/pom.xml @@ -31,7 +31,7 @@ ignite-flink-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/flume-ext/pom.xml b/modules/flume-ext/pom.xml index f143321..cc820f4 100644 --- a/modules/flume-ext/pom.xml +++ b/modules/flume-ext/pom.xml @@ -31,7 +31,7 @@ ignite-flume-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/jms11-ext/pom.xml b/modules/jms11-ext/pom.xml index b9e6422..77209d4 100644 --- a/modules/jms11-ext/pom.xml +++ b/modules/jms11-ext/pom.xml @@ -31,7 +31,7 @@ ignite-jms11-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/kafka-ext/pom.xml b/modules/kafka-ext/pom.xml index 8f4d223..0c49d8f 100644 --- a/modules/kafka-ext/pom.xml +++ b/modules/kafka-ext/pom.xml @@ -32,7 +32,7 @@ ignite-kafka-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/mqtt-ext/pom.xml b/modules/mqtt-ext/pom.xml index ceb02b4..f83c6a7 100644 --- a/modules/mqtt-ext/pom.xml +++ b/modules/mqtt-ext/pom.xml @@ -31,7 +31,7 @@ ignite-mqtt-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/pub-sub-ext/pom.xml b/modules/pub-sub-ext/pom.xml index f420f53..9f22f24 100644 --- a/modules/pub-sub-ext/pom.xml +++ b/modules/pub-sub-ext/pom.xml @@ -33,7 +33,7 @@ ignite-pub-sub-ext -2.9.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/rocketmq-ext/pom.xml b/modules/rocketmq-ext/pom.xml index c17fe22..bb8ccdc 100644 --- a/modules/rocketmq-ext/pom.xml +++ b/modules/rocketmq-ext/pom.xml @@ -32,7 +32,7 @@ ignite-rocketmq-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/storm-ext/pom.xml b/modules/storm-ext/pom.xml index 097b686..6296b8d 100644 --- a/modules/storm-ext/pom.xml +++ b/modules/storm-ext/pom.xml @@ -31,7 +31,7 @@ ignite-storm-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/twitter-ext/pom.xml b/modules/twitter-ext/pom.xml index b419a12..f618aeb 100644 --- a/modules/twitter-ext/pom.xml +++ b/modules/twitter-ext/pom.xml @@ -31,7 +31,7 @@ ignite-twitter-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/modules/zeromq-ext/pom.xml b/modules/zeromq-ext/pom.xml index 0bceb1b..1658010 100644 --- a/modules/zeromq-ext/pom.xml +++ b/modules/zeromq-ext/pom.xml @@ -31,7 +31,7 @@ ignite-zeromq-ext -1.0.0-SNAPSHOT +1.0.0 http://ignite.apache.org diff --git a/parent/pom.xml b/parent/pom.xml index d4c69fa..851143e 100644 --- a/parent/pom.xml +++ b/parent/pom.xml @@ -36,7 +36,7 @@ 1.8 1.8 -2.10.0-SNAPSHOT +2.9.0 apache-ignite UTF-8
[ignite] branch master updated: IGNITE-12489 Fix persistence corruption caused by invalid tag check (flags were not rotated for some pages) - Fixes #8358.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new 965be31 IGNITE-12489 Fix persistence corruption caused by invalid tag
check (flags were not rotated for some pages) - Fixes #8358.
965be31 is described below
commit 965be312a340fb6fb9f958bc4fcabda0a5e4bc16
Author: Nikita Tolstunov
AuthorDate: Tue Nov 17 13:30:41 2020 +0300
IGNITE-12489 Fix persistence corruption caused by invalid tag check (flags
were not rotated for some pages) - Fixes #8358.
Signed-off-by: Alexey Goncharuk
---
.../benchmarks/jmh/tree/BPlusTreeBenchmark.java| 6 +
.../ignite/internal/pagemem/PageIdAllocator.java | 18 ++-
.../ignite/internal/pagemem/PageIdUtils.java | 2 +-
.../pagemem/store/IgnitePageStoreManager.java | 2 +-
.../ignite/internal/pagemem/store/PageStore.java | 6 +
.../cache/IgniteCacheOffheapManagerImpl.java | 6 +-
.../processors/cache/mvcc/txlog/TxLog.java | 3 +-
.../processors/cache/mvcc/txlog/TxLogTree.java | 2 +
.../cache/persistence/DataStructure.java | 32 -
.../cache/persistence/GridCacheOffheapManager.java | 63
.../IgniteCacheDatabaseSharedManager.java | 4 +-
.../cache/persistence/IndexStorageImpl.java| 2 +
.../persistence/file/FilePageStoreFactory.java | 5 +-
.../persistence/file/FilePageStoreManager.java | 5 +-
.../persistence/freelist/AbstractFreeList.java | 34 +++--
.../cache/persistence/freelist/CacheFreeList.java | 7 +-
.../cache/persistence/freelist/PagesList.java | 124 +---
.../cache/persistence/metastorage/MetaStorage.java | 21 +--
.../persistence/metastorage/MetastorageTree.java | 5 +-
.../UpgradePendingTreeToPerPartitionTask.java | 4 +-
.../cache/persistence/pagemem/PageMemoryImpl.java | 13 +-
.../persistence/partstate/GroupPartitionId.java| 12 +-
.../partstorage/PartitionMetaStorageImpl.java | 6 +-
.../snapshot/IgniteSnapshotManager.java| 4 +-
.../cache/persistence/tree/BPlusTree.java | 9 +-
.../cache/persistence/tree/io/TrackingPageIO.java | 7 +-
.../cache/persistence/tree/reuse/ReuseList.java| 18 +++
.../persistence/tree/reuse/ReuseListImpl.java | 13 +-
.../processors/cache/tree/CacheDataTree.java | 5 +-
.../processors/cache/tree/PendingEntriesTree.java | 5 +-
.../processors/cache/verify/IdleVerifyUtility.java | 14 +-
...IoStatisticsMetricsLocalMXBeanImplSelfTest.java | 2 +-
.../internal/pagemem/impl/PageIdUtilsSelfTest.java | 41 --
.../persistence/PendingTreeCorruptionTest.java | 159 +
.../db/CheckpointBufferDeadlockTest.java | 12 ++
.../database/BPlusTreeFakeReuseSelfTest.java | 6 +
.../database/BPlusTreeReuseSelfTest.java | 3 +-
.../processors/database/BPlusTreeSelfTest.java | 1 +
.../processors/database/CacheFreeListSelfTest.java | 3 +-
.../ignite/testsuites/IgnitePdsTestSuite4.java | 3 +
.../processors/query/h2/IgniteH2Indexing.java | 2 +
.../processors/query/h2/database/H2Tree.java | 2 +
42 files changed, 559 insertions(+), 132 deletions(-)
diff --git
a/modules/benchmarks/src/main/java/org/apache/ignite/internal/benchmarks/jmh/tree/BPlusTreeBenchmark.java
b/modules/benchmarks/src/main/java/org/apache/ignite/internal/benchmarks/jmh/tree/BPlusTreeBenchmark.java
index 7a35430..af843cb 100644
---
a/modules/benchmarks/src/main/java/org/apache/ignite/internal/benchmarks/jmh/tree/BPlusTreeBenchmark.java
+++
b/modules/benchmarks/src/main/java/org/apache/ignite/internal/benchmarks/jmh/tree/BPlusTreeBenchmark.java
@@ -104,6 +104,11 @@ public class BPlusTreeBenchmark extends
JmhAbstractBenchmark {
}
/** {@inheritDoc} */
+@Override public long initRecycledPage(long pageId, byte flag, PageIO
initIO) throws IgniteCheckedException {
+return pageId;
+}
+
+/** {@inheritDoc} */
@Override public long recycledPagesCount() throws
IgniteCheckedException {
return deque.size();
}
@@ -186,6 +191,7 @@ public class BPlusTreeBenchmark extends
JmhAbstractBenchmark {
reuseList,
new IOVersions<>(new LongInnerIO()),
new IOVersions<>(new LongLeafIO()),
+PageIdAllocator.FLAG_IDX,
null,
null
);
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/PageIdAllocator.java
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/PageIdAllocator.java
index 7395695..f97ada7 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/PageIdAllocator.java
+++
b/modules/core/src/main/java/org/apache/ignite/inte
[ignite] branch ignite-13617 updated (df57872 -> 7ccf99b)
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a change to branch ignite-13617
in repository https://gitbox.apache.org/repos/asf/ignite.git.
from df57872 IGNITE-13617 Fix review comments.
new 6f7faee Add checkstyle plugin. Add license check plugin.
new 7000daf Add JUnit 5 support.
new 7ccf99b IGNITE-13617 Tuple assembler no longer require the size of
the tuple to be known in advance
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
modules/commons/pom.xml| 144 +++-
...idTypeException.java => AssemblyException.java} | 12 +-
.../ignite/internal/schema/ExpandableByteBuf.java | 253 +
.../ignite/internal/schema/TupleAssembler.java | 69 ++
.../ignite/internal/schema/package-info.java | 4 +-
.../apache/ignite/internal/schema/ColumnTest.java | 5 +-
.../apache/ignite/internal/schema/ColumnsTest.java | 38 +++-
.../internal/schema/ExpandableByteBufTest.java | 153 +
.../ignite/internal/schema/NativeTypeTest.java | 34 +--
.../ignite/internal/schema/SchemaTestSuite.java| 9 +-
.../apache/ignite/internal/schema/TupleTest.java | 29 ++-
11 files changed, 652 insertions(+), 98 deletions(-)
copy
modules/commons/src/main/java/org/apache/ignite/internal/schema/{InvalidTypeException.java
=> AssemblyException.java} (66%)
create mode 100644
modules/commons/src/main/java/org/apache/ignite/internal/schema/ExpandableByteBuf.java
create mode 100644
modules/commons/src/test/java/org/apache/ignite/internal/schema/ExpandableByteBufTest.java
[ignite] 02/03: Add JUnit 5 support.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13617
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 7000daf6837da48fa32a5aa93cbf2b2c8f8ee22d
Author: Andrew Mashenkov
AuthorDate: Wed Nov 11 22:57:14 2020 +0300
Add JUnit 5 support.
---
modules/commons/pom.xml| 20 ++--
.../apache/ignite/internal/schema/ColumnTest.java | 5 +--
.../apache/ignite/internal/schema/ColumnsTest.java | 38 --
.../ignite/internal/schema/NativeTypeTest.java | 34 ++-
.../ignite/internal/schema/SchemaTestSuite.java| 8 ++---
.../apache/ignite/internal/schema/TupleTest.java | 24 ++
6 files changed, 88 insertions(+), 41 deletions(-)
diff --git a/modules/commons/pom.xml b/modules/commons/pom.xml
index 562fd06..ad7636f 100644
--- a/modules/commons/pom.xml
+++ b/modules/commons/pom.xml
@@ -144,9 +144,23 @@
-junit
-junit
-4.12
+org.jetbrains
+annotations
+${jetbrains.annotations.version}
+
+
+
+
+org.junit.jupiter
+junit-jupiter-engine
+${junit.jupiter.version}
+test
+
+
+
+org.junit.platform
+junit-platform-runner
+${junit.platform.version}
test
diff --git
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnTest.java
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnTest.java
index 7725500..ddc5b74 100644
---
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnTest.java
+++
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnTest.java
@@ -18,15 +18,16 @@
package org.apache.ignite.internal.schema;
import java.util.Arrays;
-import org.junit.Test;
+import org.junit.jupiter.api.Test;
-import static org.junit.Assert.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
/**
*
*/
public class ColumnTest {
/**
+ *
*/
@Test
public void testCompareColumns() {
diff --git
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnsTest.java
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnsTest.java
index a0d859e..93e5e45 100644
---
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnsTest.java
+++
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/ColumnsTest.java
@@ -17,16 +17,18 @@
package org.apache.ignite.internal.schema;
-import org.junit.Assert;
-import org.junit.Test;
+import org.junit.jupiter.api.Test;
-import static org.junit.Assert.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
/**
*
*/
public class ColumnsTest {
/**
+ *
*/
@Test
public void testFixsizeIndex() {
@@ -40,13 +42,14 @@ public class ColumnsTest {
assertEquals(-1, cols.firstVarlengthColumn());
for (int c = 0; c < cols.length(); c++)
-Assert.assertTrue(cols.isFixedSize(c));
+assertTrue(cols.isFixedSize(c));
assertEquals(1, cols.nullMapSize());
assertEquals(3, cols.numberOfFixsizeColumns());
}
/**
+ *
*/
@Test
public void testVarsizeIndex() {
@@ -60,13 +63,14 @@ public class ColumnsTest {
assertEquals(0, cols.firstVarlengthColumn());
for (int c = 0; c < cols.length(); c++)
-Assert.assertFalse(cols.isFixedSize(c));
+assertFalse(cols.isFixedSize(c));
assertEquals(1, cols.nullMapSize());
assertEquals(0, cols.numberOfFixsizeColumns());
}
/**
+ *
*/
@Test
public void testMixedIndex() {
@@ -82,9 +86,9 @@ public class ColumnsTest {
for (int c = 0; c < cols.length(); c++) {
if (c < cols.firstVarlengthColumn())
-Assert.assertTrue(cols.isFixedSize(c));
+assertTrue(cols.isFixedSize(c));
else
-Assert.assertFalse(cols.isFixedSize(c));
+assertFalse(cols.isFixedSize(c));
}
assertEquals(1, cols.nullMapSize());
@@ -92,6 +96,7 @@ public class ColumnsTest {
}
/**
+ *
*/
@Test
public void testNullMapSize() {
@@ -111,6 +116,7 @@ public class ColumnsTest {
}
/**
+ *
*/
@Test
public void testFoldSizeNoVarlenIncomplete1Byte() {
@@ -128,6 +134,7 @@ public class ColumnsTest {
}
/**
+ *
*/
@Test
public void testFoldSizeNoVarlenFull1Byte() {
@@ -146,6 +153,7 @@ public class ColumnsTest {
}
/**
+ *
*/
@Test
[ignite] 03/03: IGNITE-13617 Tuple assembler no longer require the size of the tuple to be known in advance
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13617
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 7ccf99be30ac0b352349bf83357437eb85ee11e6
Author: Alexey Goncharuk
AuthorDate: Thu Nov 12 18:49:30 2020 +0300
IGNITE-13617 Tuple assembler no longer require the size of the tuple to be
known in advance
---
.../ignite/internal/schema/AssemblyException.java} | 23 +-
.../ignite/internal/schema/ExpandableByteBuf.java | 253 +
.../ignite/internal/schema/TupleAssembler.java | 69 ++
.../ignite/internal/schema/package-info.java | 4 +-
.../internal/schema/ExpandableByteBufTest.java | 153 +
.../ignite/internal/schema/SchemaTestSuite.java| 1 +
.../apache/ignite/internal/schema/TupleTest.java | 9 +-
7 files changed, 447 insertions(+), 65 deletions(-)
diff --git
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaTestSuite.java
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/AssemblyException.java
similarity index 64%
copy from
modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaTestSuite.java
copy to
modules/commons/src/main/java/org/apache/ignite/internal/schema/AssemblyException.java
index 3eded6a..8d01c91 100644
---
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaTestSuite.java
+++
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/AssemblyException.java
@@ -17,19 +17,16 @@
package org.apache.ignite.internal.schema;
-import org.junit.platform.runner.JUnitPlatform;
-import org.junit.platform.suite.api.SelectClasses;
-import org.junit.runner.RunWith;
-
/**
- *
+ * The exception is thrown when the tuple assembler encountered an
unrecoverable error during the field encoding.
+ * After the exception is thrown, the assembler remains in an invalid state
and should be discarded.
*/
-@RunWith(JUnitPlatform.class)
-@SelectClasses({
-NativeTypeTest.class,
-ColumnTest.class,
-ColumnsTest.class,
-TupleTest.class
-})
-public class SchemaTestSuite {
+public class AssemblyException extends RuntimeException {
+/**
+ * @param errMsg Error message
+ * @param cause Cause for this error.
+ */
+public AssemblyException(String errMsg, Exception cause) {
+super(errMsg, cause);
+}
}
diff --git
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/ExpandableByteBuf.java
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/ExpandableByteBuf.java
new file mode 100644
index 000..542b539
--- /dev/null
+++
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/ExpandableByteBuf.java
@@ -0,0 +1,253 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema;
+
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.CharBuffer;
+import java.nio.charset.CharacterCodingException;
+import java.nio.charset.CharsetEncoder;
+import java.nio.charset.CoderResult;
+import java.util.Arrays;
+
+/**
+ * A simple byte array wrapper to allow dynamic byte array expansion during
the tuple construction. Grows exponentially
+ * up to 1MB, then expands by 1 MB each time an expansion is required. Values
are always written in LITTLE_ENDIAN
+ * format.
+ *
+ * Additionally, it tracks the high watermark of the values ever written to
the buffer so that only written bytes are
+ * returned from the {@link #toArray()} method. If the current (expanded)
buffer size does not match the high watermark,
+ * the {@link #toArray()} method will return a smaller copy of the array to
exactly match the watermark.
+ *
+ * All write methods have an absolute position. The buffer will automatically
expand to fit the value being written. If
+ * there is a gap between previously written values and the current value, it
will be filled with zero bytes:
+ *
+ * ExpandableByteBuf b = new ExpandableByteBuf(1);
+ * b.put(0, (byte)1); // Does not expand.
+ * b.put(5, (byte)1); // Expands, meaningful bytes are [0..5]
+ *
+ * byte[] data = b.toArray(); // data.length == 6
+ *
+ *
[ignite] 01/03: Add checkstyle plugin. Add license check plugin.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13617
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 6f7faee4ff742cc301ca821bbe597999c67b18c6
Author: Andrew Mashenkov
AuthorDate: Wed Nov 11 22:56:51 2020 +0300
Add checkstyle plugin.
Add license check plugin.
---
modules/commons/pom.xml | 124 +++-
1 file changed, 122 insertions(+), 2 deletions(-)
diff --git a/modules/commons/pom.xml b/modules/commons/pom.xml
index 7264e5b..562fd06 100644
--- a/modules/commons/pom.xml
+++ b/modules/commons/pom.xml
@@ -29,6 +29,119 @@
3.0.0-SNAPSHOT
http://ignite.apache.org
+
+UTF-8
+
+20.1.0
+5.7.0
+1.7.0
+
+0.13
+3.8.1
+3.0.0-M4
+
3.1.1
+8.37
+
+
+
+
+checkstyle
+
+
+
+org.apache.maven.plugins
+maven-checkstyle-plugin
+${maven.checkstyle.plugin.version}
+
+
+style
+
+check
+
+validate
+
+true
+
true
+true
+true
+
${project.build.directory}/checkstyle-result.xml
+
../checkstyle/checkstyle.xml
+
../checkstyle/checkstyle-suppressions.xml
+
+
true
+**/generated/**/*
+
+
+
+
+
+com.puppycrawl.tools
+checkstyle
+
${checkstyle.puppycrawl.version}
+
+
+
+
+
+
+
+check-licenses
+
+
+
+org.apache.rat
+apache-rat-plugin
+${apache.rat.plugin.version}
+
+
true
+
+
+
IAL20
+Ignite Apache License
2.0
+
+Licensed to the Apache Software
Foundation (ASF) under one or more
+contributor license agreements. See
the NOTICE file distributed with
+this work for additional information
regarding copyright ownership.
+The ASF licenses this file to You
under the Apache License, Version 2.0
+(the "License"); you may not use this
file except in compliance with
+the License. You may obtain a copy of
the License at
+
+
http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or
agreed to in writing, software
+distributed under the License is
distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF
ANY KIND, either express or implied.
+See the License for the specific
language governing permissions and
+limitations under the License.
+
+
+
+
+
+Ignite Apache License
2.0
+
+
+
+
+
+validate
+
+check
+
+
+
+work/**
+
[ignite] branch ignite-13617 updated: IGNITE-13617 Fix review comments.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13617
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/ignite-13617 by this push:
new df57872 IGNITE-13617 Fix review comments.
df57872 is described below
commit df5787288379122fe0e16efc5b05f7cba0dd7817
Author: Alexey Goncharuk
AuthorDate: Fri Nov 6 14:28:34 2020 +0300
IGNITE-13617 Fix review comments.
---
.../apache/ignite/internal/schema/NativeType.java | 5 +++-
.../ignite/internal/schema/NativeTypeTest.java | 6
.../ignite/internal/schema/SchemaTestSuite.java| 35 ++
3 files changed, 45 insertions(+), 1 deletion(-)
diff --git
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/NativeType.java
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/NativeType.java
index 9caaea3..211a1e1 100644
---
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/NativeType.java
+++
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/NativeType.java
@@ -78,7 +78,10 @@ public class NativeType implements Comparable {
}
/**
- * @return Length of the type if it is a fixlen type.
+ * @return Length of the type if it is a fixlen type. For varlen types the
return value is undefined, so the user
+ * should explicitly check {@code spec().fixedLength()} before using this
method.
+ *
+ * @see NativeTypeSpec#fixedLength()
*/
public int length() {
return len;
diff --git
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/NativeTypeTest.java
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/NativeTypeTest.java
index fc7e020..d2cf2a5 100644
---
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/NativeTypeTest.java
+++
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/NativeTypeTest.java
@@ -29,11 +29,17 @@ public class NativeTypeTest {
*/
@Test
public void testCompareFixlenVarlen() {
+Assert.assertTrue(NativeType.BYTE.compareTo(NativeType.STRING) < 0);
+Assert.assertTrue(NativeType.BYTE.compareTo(NativeType.BYTES) < 0);
+
Assert.assertTrue(NativeType.INTEGER.compareTo(NativeType.STRING) < 0);
Assert.assertTrue(NativeType.INTEGER.compareTo(NativeType.BYTES) < 0);
Assert.assertTrue(NativeType.LONG.compareTo(NativeType.STRING) < 0);
Assert.assertTrue(NativeType.LONG.compareTo(NativeType.BYTES) < 0);
+
+Assert.assertTrue(NativeType.UUID.compareTo(NativeType.STRING) < 0);
+Assert.assertTrue(NativeType.UUID.compareTo(NativeType.BYTES) < 0);
}
/**
diff --git
a/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaTestSuite.java
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaTestSuite.java
new file mode 100644
index 000..93e489b
--- /dev/null
+++
b/modules/commons/src/test/java/org/apache/ignite/internal/schema/SchemaTestSuite.java
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema;
+
+import org.junit.runner.RunWith;
+import org.junit.runners.Suite;
+
+
+/**
+ *
+ */
+@RunWith(Suite.class)
+@Suite.SuiteClasses({
+NativeTypeTest.class,
+ColumnTest.class,
+ColumnsTest.class,
+TupleTest.class
+})
+public class SchemaTestSuite {
+}
[ignite] 01/01: IGNITE-13617 Add schema definition, tuple assembler and tuple classes
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch ignite-13617
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 2b0d788f2973216d1df62359dc05128d4f4ffac2
Author: Alexey Goncharuk
AuthorDate: Tue Nov 3 15:22:41 2020 +0300
IGNITE-13617 Add schema definition, tuple assembler and tuple classes
---
modules/commons/pom.xml| 53 +++
.../org/apache/ignite/internal/schema/Bitmask.java | 87 +
.../ignite/internal/schema/ByteBufferTuple.java| 91 +
.../org/apache/ignite/internal/schema/Column.java | 111 ++
.../org/apache/ignite/internal/schema/Columns.java | 272 +
.../internal/schema/InvalidTypeException.java | 30 ++
.../apache/ignite/internal/schema/NativeType.java | 133 +++
.../ignite/internal/schema/NativeTypeSpec.java | 178 +
.../ignite/internal/schema/SchemaDescriptor.java | 99 +
.../org/apache/ignite/internal/schema/Tuple.java | 420
.../ignite/internal/schema/TupleAssembler.java | 431 +
.../ignite/internal/schema/package-info.java | 68
.../apache/ignite/internal/schema/ColumnTest.java | 47 +++
.../apache/ignite/internal/schema/ColumnsTest.java | 381 ++
.../ignite/internal/schema/NativeTypeTest.java | 61 +++
.../apache/ignite/internal/schema/TupleTest.java | 369 ++
pom.xml| 1 +
17 files changed, 2832 insertions(+)
diff --git a/modules/commons/pom.xml b/modules/commons/pom.xml
new file mode 100644
index 000..7264e5b
--- /dev/null
+++ b/modules/commons/pom.xml
@@ -0,0 +1,53 @@
+
+
+
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+org.apache.ignite
+ignite-commons
+3.0.0-SNAPSHOT
+http://ignite.apache.org
+
+
+
+junit
+junit
+4.12
+test
+
+
+
+
+
+org.apache.maven.plugins
+maven-compiler-plugin
+3.7.0
+
+1.8
+1.8
+
+
+
+
+
diff --git
a/modules/commons/src/main/java/org/apache/ignite/internal/schema/Bitmask.java
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/Bitmask.java
new file mode 100644
index 000..1314d32
--- /dev/null
+++
b/modules/commons/src/main/java/org/apache/ignite/internal/schema/Bitmask.java
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema;
+
+/**
+ * A fixed-sized type representing a bitmask of n bits. The
actual size of a bitmask will round up
+ * to the smallest number of bytes required to store n bits.
+ */
+public class Bitmask extends NativeType {
+/** */
+private final int bits;
+
+/**
+ * Factory method for creating the bitmask type.
+ *
+ * @param nBits Maximum number of bits in the bitmask.
+ * @return Bitmask type.
+ */
+public static Bitmask of(int nBits) {
+return new Bitmask(nBits);
+}
+
+/**
+ * Creates a bitmask type of size bits. In tuple will round
up to the closest full byte.
+ *
+ * @param bits The number of bits in the bitmask.
+ */
+protected Bitmask(int bits) {
+super(NativeTypeSpec.BITMASK, (bits + 7) / 8);
+
+this.bits = bits;
+}
+
+/**
+ * @return Maximum number of bits to be stored in the bitmask.
+ */
+public int bits() {
+return bits;
+}
+
+/** {@inheritDoc} */
+@Override public boolean equals(Object o) {
+if (this == o)
+return true;
+
+if (o == null || getClass() != o.getClass())
+return false;
+
+Bitmask that = (Bitmask)o;
+
+return bits == that.bits;
+}
+
+/** {@inheritDoc} */
+@Override public int hashCode() {
+return bits;
+}
+
+/*
[ignite] branch ignite-13617 created (now 2b0d788)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-13617 in repository https://gitbox.apache.org/repos/asf/ignite.git. at 2b0d788 IGNITE-13617 Add schema definition, tuple assembler and tuple classes This branch includes the following new commits: new 2b0d788 IGNITE-13617 Add schema definition, tuple assembler and tuple classes The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite-extensions] annotated tag ignite-spring-boot-autoconfigure-ext-1.0.0 created (now cdb87fe)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to annotated tag ignite-spring-boot-autoconfigure-ext-1.0.0 in repository https://gitbox.apache.org/repos/asf/ignite-extensions.git. at cdb87fe (tag) tagging de621e606dc0df1716728f9bcd58ccd2bbb1f370 (commit) replaces ignite-spring-boot-1.0.0-rc1 by Alexey Goncharuk on Tue Oct 13 13:03:39 2020 +0300 - Log - Added a tag ignite-spring-boot-autoconfigure-ext-1.0.0 to match with the released module name --- No new revisions were added by this update.
[ignite] branch master updated: IGNITE-13362 Add warmup stop command to control.sh - Fixes #8201.
This is an automated email from the ASF dual-hosted git repository.
agoncharuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new 0260ba0 IGNITE-13362 Add warmup stop command to control.sh - Fixes
#8201.
0260ba0 is described below
commit 0260ba0c8fa3be6ddce9efc473f45b13d11f7784
Author: ktkalenko
AuthorDate: Thu Sep 10 17:51:29 2020 +0300
IGNITE-13362 Add warmup stop command to control.sh - Fixes #8201.
Signed-off-by: Alexey Goncharuk
---
.../ignite/internal/commandline/Command.java | 31
.../ignite/internal/commandline/CommandList.java | 5 +-
.../ignite/internal/commandline/WarmUpCommand.java | 121 +
.../commandline/CommandHandlerParsingTest.java | 36 +++-
.../apache/ignite/util/GridCommandHandlerTest.java | 58 +++
.../internal/client/GridClientBeforeNodeStart.java | 66 +++
.../ignite/internal/client/GridClientFactory.java | 33 +++-
.../client/GridClientNodeStateBeforeStart.java}| 16 +-
.../internal/client/impl/GridClientImpl.java | 190 +++--
.../impl/GridClientNodeStateBeforeStartImpl.java | 49 ++
.../impl/connection/GridClientConnection.java | 9 +
.../connection/GridClientConnectionManager.java| 12 ++
.../GridClientConnectionManagerAdapter.java| 156 +++--
.../GridClientConnectionManagerOsImpl.java | 7 +-
.../connection/GridClientNioTcpConnection.java | 8 +
.../client/router/impl/GridRouterClientImpl.java | 2 +-
.../internal/processors/rest/GridRestCommand.java | 8 +-
.../processors/rest/GridRestProcessor.java | 11 +-
.../internal/processors/rest/GridRestProtocol.java | 5 +
.../GridClientNodeStateBeforeStartRequest.java}| 16 +-
.../client/message/GridClientWarmUpRequest.java| 90 ++
.../NodeStateBeforeStartCommandHandler.java| 75
.../rest/protocols/GridRestProtocolAdapter.java| 5 +
.../rest/protocols/tcp/GridTcpRestNioListener.java | 22 +++
.../rest/protocols/tcp/GridTcpRestProtocol.java| 6 +-
.../GridRestNodeStateBeforeStartRequest.java} | 13 +-
.../rest/request/GridRestWarmUpRequest.java| 54 ++
.../main/resources/META-INF/classnames.properties | 2 +
.../cache/warmup/BlockedWarmUpConfiguration.java | 4 +-
...ockedWarmUp.java => BlockedWarmUpStrategy.java} | 6 +-
.../SimpleObservableWarmUpConfiguration.java | 2 +-
...Up.java => SimpleObservableWarmUpStrategy.java} | 2 +-
.../processors/cache/warmup/WarmUpSelfTest.java| 10 +-
.../cache/warmup/WarmUpTestPluginProvider.java | 8 +-
...ridCommandHandlerClusterByClassTest_help.output | 3 +
...andHandlerClusterByClassWithSSLTest_help.output | 3 +
.../Cache/PartitionLossTest.cs | 12 +-
37 files changed, 1003 insertions(+), 153 deletions(-)
diff --git
a/modules/control-utility/src/main/java/org/apache/ignite/internal/commandline/Command.java
b/modules/control-utility/src/main/java/org/apache/ignite/internal/commandline/Command.java
index b147933..5b00a78 100644
---
a/modules/control-utility/src/main/java/org/apache/ignite/internal/commandline/Command.java
+++
b/modules/control-utility/src/main/java/org/apache/ignite/internal/commandline/Command.java
@@ -22,6 +22,7 @@ import java.util.Map;
import java.util.logging.Logger;
import org.apache.ignite.IgniteSystemProperties;
import org.apache.ignite.internal.client.GridClient;
+import org.apache.ignite.internal.client.GridClientBeforeNodeStart;
import org.apache.ignite.internal.client.GridClientConfiguration;
import org.apache.ignite.internal.client.GridClientException;
import org.apache.ignite.internal.client.GridClientFactory;
@@ -69,6 +70,36 @@ public interface Command {
}
/**
+ * Method to create thin client for communication with node before it
starts.
+ * If node has already started, there will be an error.
+ *
+ * @param clientCfg Thin client configuration.
+ * @return Grid thin client instance which is already connected to node
before it starts.
+ * @throws Exception If error occur.
+ */
+public static GridClientBeforeNodeStart startClientBeforeNodeStart(
+GridClientConfiguration clientCfg
+) throws Exception {
+GridClientBeforeNodeStart client =
GridClientFactory.startBeforeNodeStart(clientCfg);
+
+// If connection is unsuccessful, fail before doing any operations:
+if (!client.connected()) {
+GridClientException lastErr = client.checkLastError();
+
+try {
+client.close();
+}
+catch (Throwable e) {
+lastErr.addSuppressed(e);
+}
+
+throw lastErr;
+}
+
+return client;
+}
+
+/**
* Print command usage.
*
* @param logger Logger
[ignite] 01/01: Explicitly inline core messages in message factory
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a commit to branch ignite-2.9-revert-12568 in repository https://gitbox.apache.org/repos/asf/ignite.git commit 31d324e3ea8c1b87a29b64b1506f91efeae2c9d4 Author: Alexey Goncharuk AuthorDate: Thu Sep 10 14:39:33 2020 +0300 Explicitly inline core messages in message factory --- .../communication/IgniteMessageFactoryImpl.java| 1055 1 file changed, 1055 insertions(+) diff --git a/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteMessageFactoryImpl.java b/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteMessageFactoryImpl.java index 68ce797..c5abe21 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteMessageFactoryImpl.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteMessageFactoryImpl.java @@ -23,10 +23,180 @@ import java.util.List; import java.util.function.Supplier; import org.apache.ignite.IgniteException; +import org.apache.ignite.internal.GridJobCancelRequest; +import org.apache.ignite.internal.GridJobExecuteRequest; +import org.apache.ignite.internal.GridJobExecuteResponse; +import org.apache.ignite.internal.GridJobSiblingsRequest; +import org.apache.ignite.internal.GridJobSiblingsResponse; +import org.apache.ignite.internal.GridTaskCancelRequest; +import org.apache.ignite.internal.GridTaskSessionRequest; +import org.apache.ignite.internal.IgniteDiagnosticMessage; +import org.apache.ignite.internal.binary.BinaryEnumObjectImpl; +import org.apache.ignite.internal.binary.BinaryObjectImpl; +import org.apache.ignite.internal.managers.checkpoint.GridCheckpointRequest; +import org.apache.ignite.internal.managers.deployment.GridDeploymentInfoBean; +import org.apache.ignite.internal.managers.deployment.GridDeploymentRequest; +import org.apache.ignite.internal.managers.deployment.GridDeploymentResponse; +import org.apache.ignite.internal.managers.encryption.GenerateEncryptionKeyRequest; +import org.apache.ignite.internal.managers.encryption.GenerateEncryptionKeyResponse; +import org.apache.ignite.internal.managers.eventstorage.GridEventStorageMessage; +import org.apache.ignite.internal.processors.affinity.AffinityTopologyVersion; +import org.apache.ignite.internal.processors.authentication.UserAuthenticateRequestMessage; +import org.apache.ignite.internal.processors.authentication.UserAuthenticateResponseMessage; +import org.apache.ignite.internal.processors.authentication.UserManagementOperationFinishedMessage; +import org.apache.ignite.internal.processors.cache.CacheEntryInfoCollection; +import org.apache.ignite.internal.processors.cache.CacheEntryPredicateContainsValue; +import org.apache.ignite.internal.processors.cache.CacheEntrySerializablePredicate; +import org.apache.ignite.internal.processors.cache.CacheEvictionEntry; +import org.apache.ignite.internal.processors.cache.CacheInvokeDirectResult; +import org.apache.ignite.internal.processors.cache.CacheObjectByteArrayImpl; +import org.apache.ignite.internal.processors.cache.CacheObjectImpl; +import org.apache.ignite.internal.processors.cache.GridCacheEntryInfo; +import org.apache.ignite.internal.processors.cache.GridCacheMvccEntryInfo; +import org.apache.ignite.internal.processors.cache.GridCacheReturn; +import org.apache.ignite.internal.processors.cache.GridChangeGlobalStateMessageResponse; +import org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl; +import org.apache.ignite.internal.processors.cache.WalStateAckMessage; +import org.apache.ignite.internal.processors.cache.binary.MetadataRequestMessage; +import org.apache.ignite.internal.processors.cache.binary.MetadataResponseMessage; +import org.apache.ignite.internal.processors.cache.distributed.GridCacheTtlUpdateRequest; +import org.apache.ignite.internal.processors.cache.distributed.GridCacheTxRecoveryRequest; +import org.apache.ignite.internal.processors.cache.distributed.GridCacheTxRecoveryResponse; +import org.apache.ignite.internal.processors.cache.distributed.GridDistributedLockRequest; +import org.apache.ignite.internal.processors.cache.distributed.GridDistributedLockResponse; +import org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxFinishRequest; +import org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxFinishResponse; +import org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxPrepareRequest; +import org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxPrepareResponse; +import org.apache.ignite.internal.processors.cache.distributed.GridDistributedUnlockRequest; +import org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAffinityAssignmentRequest; +import org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAffinityAssignmentResponse; +import
[ignite] branch ignite-2.9-revert-12568 created (now 31d324e)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-2.9-revert-12568 in repository https://gitbox.apache.org/repos/asf/ignite.git. at 31d324e Explicitly inline core messages in message factory This branch includes the following new commits: new 31d324e Explicitly inline core messages in message factory The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[ignite] branch ignite-2.9-revert-12568 deleted (was ed52559)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-2.9-revert-12568 in repository https://gitbox.apache.org/repos/asf/ignite.git. was ed52559 Revert "IGNITE-12568 MessageFactory is refactored in order to detect registration of message with the same direct type" This change permanently discards the following revisions: discard ed52559 Revert "IGNITE-12568 MessageFactory is refactored in order to detect registration of message with the same direct type" discard 3d57f23 Revert "IGNITE-12682 IgniteMessageFactoryImpl.registerCustom() method is removed as potentially dangerous" discard f1fcea2 Revert "IGNITE-12756 TcpCommunication SPI metrics improvement"
[ignite] branch ignite-2.9-revert-12568 created (now ed52559)
This is an automated email from the ASF dual-hosted git repository. agoncharuk pushed a change to branch ignite-2.9-revert-12568 in repository https://gitbox.apache.org/repos/asf/ignite.git. at ed52559 Revert "IGNITE-12568 MessageFactory is refactored in order to detect registration of message with the same direct type" This branch includes the following new commits: new f1fcea2 Revert "IGNITE-12756 TcpCommunication SPI metrics improvement" new 3d57f23 Revert "IGNITE-12682 IgniteMessageFactoryImpl.registerCustom() method is removed as potentially dangerous" new ed52559 Revert "IGNITE-12568 MessageFactory is refactored in order to detect registration of message with the same direct type" The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
