[geode] branch develop updated (5d1e919 -> 685ad9e)
This is an automated email from the ASF dual-hosted git repository. mivanac pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 5d1e919 GEODE-9817: Enable customized source set paths for ClassAnalysisRule (#7121) add 685ad9e GEODE-9806: added skip recovery of recovered bucket (#7107) No new revisions were added by this update. Summary of changes: .../geode/internal/cache/ProxyBucketRegion.java| 13 + .../internal/cache/ProxyBucketRegionTest.java | 62 ++ 2 files changed, 75 insertions(+)
[geode-native] branch develop updated: GEODE-9804: new register interest tests (#894)
This is an automated email from the ASF dual-hosted git repository. mmartell pushed a commit to branch develop in repository https://gitbox.apache.org/repos/asf/geode-native.git The following commit(s) were added to refs/heads/develop by this push: new c5aee1b GEODE-9804: new register interest tests (#894) c5aee1b is described below commit c5aee1b68a0cfeda2480afb4b27bd2b12fae0d8f Author: Michael Martell AuthorDate: Fri Nov 19 17:30:22 2021 -0800 GEODE-9804: new register interest tests (#894) * Fixes incorrect fetching of initial entries. * Makes InterestResultPolicy a proper enum. * Adds integration benchmark for register interest. * Fixes for MSVC errors. * Don't ask for keys if not caching locally. * Replace sleep with CacheListener events. * Use latches for syncrhonization with listener. * Change boost loglevel to warning. * Encapsulate synchronization in CountdownCacheListener. Co-authored-by: Jacob Barrett --- .../testThinClientIntResPolKeysInv.cpp | 1 - cppcache/integration/benchmark/CMakeLists.txt | 4 +- cppcache/integration/benchmark/RegionBM.cpp| 15 +- .../integration/benchmark/RegisterInterestBM.cpp | 156 ++ cppcache/integration/test/RegisterKeysTest.cpp | 258 cppcache/src/InterestResultPolicy.cpp | 31 -- cppcache/src/InterestResultPolicy.hpp | 39 +-- cppcache/src/LocalRegion.cpp | 3 +- cppcache/src/LocalRegion.hpp | 4 +- cppcache/src/TcrChunkedContext.hpp | 1 - cppcache/src/TcrMessage.cpp| 129 cppcache/src/TcrMessage.hpp| 29 +- cppcache/src/ThinClientRegion.cpp | 342 - cppcache/test/CMakeLists.txt | 1 - cppcache/test/InterestResultPolicyTest.cpp | 34 -- cppcache/test/TcrMessageTest.cpp | 123 ++-- 16 files changed, 754 insertions(+), 416 deletions(-) diff --git a/cppcache/integration-test/testThinClientIntResPolKeysInv.cpp b/cppcache/integration-test/testThinClientIntResPolKeysInv.cpp index d918856..210b6a3 100644 --- a/cppcache/integration-test/testThinClientIntResPolKeysInv.cpp +++ b/cppcache/integration-test/testThinClientIntResPolKeysInv.cpp @@ -360,7 +360,6 @@ END_TASK_DEFINITION DUNIT_TASK_DEFINITION(CLIENT2, StepEight) { auto regPtr1 = getHelper()->getRegion(regionNames[1]); -// regPtr1->registerRegex(regexWildcard); regPtr1->registerAllKeys(); verifyInvalid(regionNames[1], keys[0]); diff --git a/cppcache/integration/benchmark/CMakeLists.txt b/cppcache/integration/benchmark/CMakeLists.txt index 88254d2..0702ae4 100644 --- a/cppcache/integration/benchmark/CMakeLists.txt +++ b/cppcache/integration/benchmark/CMakeLists.txt @@ -16,7 +16,9 @@ add_executable(cpp-integration-benchmark main.cpp RegionBM.cpp - PdxTypeBM.cpp) + PdxTypeBM.cpp + RegisterInterestBM.cpp +) target_link_libraries(cpp-integration-benchmark PUBLIC diff --git a/cppcache/integration/benchmark/RegionBM.cpp b/cppcache/integration/benchmark/RegionBM.cpp index 7fef624..da1b910 100644 --- a/cppcache/integration/benchmark/RegionBM.cpp +++ b/cppcache/integration/benchmark/RegionBM.cpp @@ -19,21 +19,16 @@ #include #include -// Disable warning for "extra qualifications" here. One of the boost log -// headers triggers this warning. Note: use of disable pragma here is -// intentional - attempts to use push/pop as you ordinarily should just -// yielded a gripe from the MS tools that "warning number '4596' is not a -// valid compiler warning". re-enabling the warning after the include -// fails in the same way, so just leave it disabled for the rest of the -// file. This is safe, since the warning can only trigger inside a class -// declaration, of which there are none in this file. -#ifdef WIN32 +#ifdef _MSC_VER +#pragma warning(push) #pragma warning(disable : 4596) #endif - #include #include #include +#ifdef _MSC_VER +#pragma warning(pop) +#endif #include #include diff --git a/cppcache/integration/benchmark/RegisterInterestBM.cpp b/cppcache/integration/benchmark/RegisterInterestBM.cpp new file mode 100644 index 000..7f6154b --- /dev/null +++ b/cppcache/integration/benchmark/RegisterInterestBM.cpp @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed unde
[geode] 15/16: Refactor and cleanup legacy ProcessStats.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 6239e4d653870e85b334137c72fb786968948da0 Author: Jacob Barrett AuthorDate: Thu Nov 18 13:11:45 2021 -0800 Refactor and cleanup legacy ProcessStats. --- .../internal/MemberHealthEvaluatorJUnitTest.java | 16 +++-- .../GemFireStatSamplerIntegrationTest.java | 4 +- .../internal/statistics/LinuxSystemStatsTest.java | 12 ++-- .../internal/statistics/OsStatisticsBenchmark.java | 4 +- .../admin/internal/MemberHealthEvaluator.java | 13 ++-- .../internal/InternalDistributedSystem.java| 2 +- .../internal/statistics/GemFireStatSampler.java| 24 ++- ...ovider.java => LegacyOsStatisticsProvider.java} | 15 +++-- .../internal/statistics/OsStatisticsProvider.java | 5 ++ ...isticsProvider.java => ProcessSizeSuppler.java} | 10 +-- ...rovider.java => LinuxOsStatisticsProvider.java} | 76 -- .../LinuxProcFsStatistics.java | 2 +- .../{platform => legacy}/LinuxProcessStats.java| 15 ++--- .../{platform => legacy}/LinuxSystemStats.java | 2 +- .../statistics/oshi/OshiStatisticsProvider.java| 7 ++ .../internal/statistics/platform/ProcessStats.java | 22 ++- .../internal/beans/MemberMBeanBridge.java | 12 ++-- geode.internal.statistics.OsStatisticsProvider | 2 +- .../legacy/LegacyOsStatisticsProviderTest.java | 2 +- 19 files changed, 136 insertions(+), 109 deletions(-) diff --git a/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java b/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java index 4809b40..9e84c07 100644 --- a/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java +++ b/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java @@ -21,13 +21,14 @@ import static org.junit.Assert.assertTrue; import java.util.ArrayList; import java.util.List; +import org.jetbrains.annotations.Nullable; import org.junit.Test; import org.apache.geode.admin.GemFireHealth; import org.apache.geode.admin.GemFireHealthConfig; import org.apache.geode.internal.statistics.GemFireStatSampler; -import org.apache.geode.internal.statistics.legacy.LegacyOsStatisticsProvider; -import org.apache.geode.internal.statistics.platform.ProcessStats; +import org.apache.geode.internal.statistics.ProcessSizeSuppler; +import org.apache.geode.internal.statistics.legacy.LinuxOsStatisticsProvider; /** * Contains simple tests for the {@link MemberHealthEvaluator}. @@ -46,17 +47,18 @@ public class MemberHealthEvaluatorJUnitTest extends HealthEvaluatorTestCase { */ @Test public void testCheckVMProcessSize() throws InterruptedException { -if (LegacyOsStatisticsProvider.build().osStatsSupported()) { +if (LinuxOsStatisticsProvider.build().osStatsSupported()) { GemFireStatSampler sampler = system.getStatSampler(); assertNotNull(sampler); sampler.waitForInitialization(1); // fix: remove infinite wait - ProcessStats stats = sampler.getProcessStats(); - assertNotNull(stats); + @Nullable + ProcessSizeSuppler processSizeSuppler = sampler.getProcessSizeSuppler(); + assertNotNull(processSizeSuppler); List status = new ArrayList(); - long threshold = stats.getProcessSize() * 2; + long threshold = processSizeSuppler.getAsLong() * 2; if (threshold <= 0) { // The process size is zero on some Linux versions @@ -72,7 +74,7 @@ public class MemberHealthEvaluatorJUnitTest extends HealthEvaluatorTestCase { assertTrue(status.isEmpty()); status = new ArrayList(); - long processSize = stats.getProcessSize(); + long processSize = processSizeSuppler.getAsLong(); threshold = processSize / 2; assertTrue("Threshold (" + threshold + ") is > 0. " + "Process size is " + processSize, threshold > 0); diff --git a/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java b/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java index 08b20e1..b67febe 100644 --- a/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java +++ b/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java @@ -61,7 +61,7 @@ import org.apache.geode.StatisticsType; import org.apache.geode.distributed.DistributedSystem; import org.apache.geode.distributed.internal.InternalDistributedSystem; import org.apache.geode.internal.statistics.GemFireStatSampler.LocalStatListenerImpl; -import org.apache.geode.inter
[geode] 16/16: Initial module.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit fb2b8cf261ed9509cbba81ee70f5195fe8f0a8d9 Author: Jacob Barrett AuthorDate: Fri Nov 19 11:42:30 2021 -0800 Initial module. --- .../internal/InternalDistributedSystem.java| 3 +-- geode.internal.statistics.OsStatisticsProvider | 1 - geode-stats-legacy/build.gradle| 31 ++ .../legacy/LinuxOsStatisticsProvider.java | 0 .../statistics/legacy/LinuxProcFsStatistics.java | 0 .../statistics/legacy/LinuxProcessStats.java | 0 .../statistics/legacy/LinuxSystemStats.java| 0 geode.internal.statistics.OsStatisticsProvider | 1 - settings.gradle| 1 + 9 files changed, 33 insertions(+), 4 deletions(-) diff --git a/geode-core/src/main/java/org/apache/geode/distributed/internal/InternalDistributedSystem.java b/geode-core/src/main/java/org/apache/geode/distributed/internal/InternalDistributedSystem.java index d051dc0..84e2225 100644 --- a/geode-core/src/main/java/org/apache/geode/distributed/internal/InternalDistributedSystem.java +++ b/geode-core/src/main/java/org/apache/geode/distributed/internal/InternalDistributedSystem.java @@ -105,7 +105,6 @@ import org.apache.geode.internal.statistics.StatisticsConfig; import org.apache.geode.internal.statistics.StatisticsManager; import org.apache.geode.internal.statistics.StatisticsManagerFactory; import org.apache.geode.internal.statistics.StatisticsRegistry; -import org.apache.geode.internal.statistics.legacy.LinuxProcFsStatistics; import org.apache.geode.internal.tcp.ConnectionTable; import org.apache.geode.logging.internal.LoggingSession; import org.apache.geode.logging.internal.NullLoggingSession; @@ -737,7 +736,7 @@ public class InternalDistributedSystem extends DistributedSystem // This calculation is not exact, but seems fairly close. So far we have // not loaded much into the heap and the current RSS usage is already // included the available memory calculation. -long avail = LinuxProcFsStatistics.getAvailableMemory(logger); +long avail = 0; //LinuxProcFsStatistics.getAvailableMemory(logger); long size = offHeapMemorySize + Runtime.getRuntime().totalMemory(); lockMemory(avail, size); } diff --git a/geode-core/src/main/resources/META-INF/services/org.apache.geode.internal.statistics.OsStatisticsProvider b/geode-core/src/main/resources/META-INF/services/org.apache.geode.internal.statistics.OsStatisticsProvider index fa4cef5..4b59cde 100644 --- a/geode-core/src/main/resources/META-INF/services/org.apache.geode.internal.statistics.OsStatisticsProvider +++ b/geode-core/src/main/resources/META-INF/services/org.apache.geode.internal.statistics.OsStatisticsProvider @@ -13,5 +13,4 @@ # the License. # -org.apache.geode.internal.statistics.legacy.LinuxOsStatisticsProvider org.apache.geode.internal.statistics.oshi.OshiStatisticsProvider diff --git a/geode-stats-legacy/build.gradle b/geode-stats-legacy/build.gradle new file mode 100644 index 000..f8979ad --- /dev/null +++ b/geode-stats-legacy/build.gradle @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + + +apply from: "${rootDir}/${scriptDir}/standard-subproject-configuration.gradle" + + +apply from: "${project.projectDir}/../gradle/publish-java.gradle" +apply from: "${project.projectDir}/../gradle/warnings.gradle" + +dependencies { + api(platform(project(':boms:geode-all-bom'))) + + implementation(project(':geode-core')) + implementation(project(':geode-logging')) + + compileOnly('org.jetbrains:annotations') + +} diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/legacy/LinuxOsStatisticsProvider.java b/geode-stats-legacy/src/main/java/org/apache/geode/internal/statistics/legacy/LinuxOsStatisticsProvider.java similarity index 100% rename from geode-core/src/main/java/org/apache/geode/internal/statistics/legacy/LinuxOsStatisticsProvider.java rename to geode-stats-legacy/src/main/java/org/apache/geode/internal/sta
[geode] 13/16: Fix legacy process stats sample.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 2a3b2af8a148fe289679f6e36e91a7701acb227e Author: Jacob Barrett AuthorDate: Wed Nov 17 11:23:02 2021 -0800 Fix legacy process stats sample. --- .../geode/internal/statistics/legacy/LegacyOsStatisticsProvider.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/legacy/LegacyOsStatisticsProvider.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/legacy/LegacyOsStatisticsProvider.java index a40fadf..349b7ee 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/legacy/LegacyOsStatisticsProvider.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/legacy/LegacyOsStatisticsProvider.java @@ -120,8 +120,8 @@ public class LegacyOsStatisticsProvider implements OsStatisticsProvider { refreshSystem(systemStatistics); } - void sampleProcess() { -refreshSystem(processStatistics); + private void sampleProcess() { +refreshProcess(processStatistics); } @Override
[geode] 14/16: Fix tests.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 8c8fa8518cab3ff872563399fe188e4ca6400171 Author: Jacob Barrett AuthorDate: Thu Nov 18 11:34:14 2021 -0800 Fix tests. --- .../geode/internal/statistics/GemFireStatSamplerIntegrationTest.java| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java b/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java index d799b32..08b20e1 100644 --- a/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java +++ b/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java @@ -461,7 +461,7 @@ public class GemFireStatSamplerIntegrationTest extends StatSamplerTestCase { Properties props = createGemFireProperties(); props.setProperty(STATISTIC_ARCHIVE_FILE, archiveFileName); props.setProperty(ARCHIVE_FILE_SIZE_LIMIT, "2"); -props.setProperty(ARCHIVE_DISK_SPACE_LIMIT, "14"); +props.setProperty(ARCHIVE_DISK_SPACE_LIMIT, "140"); props.setProperty(STATISTIC_SAMPLE_RATE, String.valueOf(sampleRate)); connect(props);
[geode] 11/16: Extract legacy OS stats.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit cf79e27c0c8f88f1bc911f3a89c2003546243e7b Author: Jacob Barrett AuthorDate: Wed Nov 17 08:07:17 2021 -0800 Extract legacy OS stats. --- .../internal/MemberHealthEvaluatorJUnitTest.java | 4 +- .../GemFireStatSamplerIntegrationTest.java | 3 +- ...csBenchmark.java => OsStatisticsBenchmark.java} | 47 +++- ...arkTest.java => OsStatisticsBenchmarkTest.java} | 2 +- .../statistics/AbstractStatisticsFactory.java | 13 +- .../statistics/DummyStatisticsFactory.java | 4 +- .../statistics/DummyStatisticsRegistry.java| 2 +- .../internal/statistics/GemFireStatSampler.java| 40 +-- .../geode/internal/statistics/HostStatSampler.java | 1 - .../statistics/LocalStatisticsFactory.java | 5 +- .../internal/statistics/LocalStatisticsImpl.java | 12 +- .../{platform => }/OsStatisticsFactory.java| 5 +- .../internal/statistics/OsStatisticsProvider.java | 127 + .../statistics/OsStatisticsProviderException.java} | 10 +- .../geode/internal/statistics/StatisticsImpl.java | 22 +- .../internal/statistics/StatisticsManager.java | 1 - .../internal/statistics/StatisticsRegistry.java| 20 +- .../internal/statistics/StripedStatisticsImpl.java | 2 +- .../LegacyOsStatisticsProvider.java} | 109 .../statistics/oshi/OshiStatisticsProvider.java| 307 - .../oshi/OshiStatisticsProviderException.java | 4 - .../oshi/OshiStatisticsProviderImpl.java | 307 - .../statistics/platform/LinuxProcFsStatistics.java | 31 +-- .../internal/beans/MemberMBeanBridge.java | 5 +- .../statistics/LocalStatisticsImplTest.java| 4 +- .../statistics/OsStatisticsProviderTest.java | 58 .../internal/statistics/StatisticsImplTest.java| 4 +- .../statistics/StatisticsRegistryTest.java | 49 ++-- .../legacy/LegacyOsStatisticsProviderTest.java}| 14 +- ...plTest.java => OshiStatisticsProviderTest.java} | 34 +-- .../internal/statistics/TestStatisticsManager.java | 3 +- 31 files changed, 511 insertions(+), 738 deletions(-) diff --git a/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java b/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java index 6a574e6..4809b40 100644 --- a/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java +++ b/geode-core/src/integrationTest/java/org/apache/geode/admin/internal/MemberHealthEvaluatorJUnitTest.java @@ -26,7 +26,7 @@ import org.junit.Test; import org.apache.geode.admin.GemFireHealth; import org.apache.geode.admin.GemFireHealthConfig; import org.apache.geode.internal.statistics.GemFireStatSampler; -import org.apache.geode.internal.statistics.OsStatisticsProvider; +import org.apache.geode.internal.statistics.legacy.LegacyOsStatisticsProvider; import org.apache.geode.internal.statistics.platform.ProcessStats; /** @@ -46,7 +46,7 @@ public class MemberHealthEvaluatorJUnitTest extends HealthEvaluatorTestCase { */ @Test public void testCheckVMProcessSize() throws InterruptedException { -if (OsStatisticsProvider.build().osStatsSupported()) { +if (LegacyOsStatisticsProvider.build().osStatsSupported()) { GemFireStatSampler sampler = system.getStatSampler(); assertNotNull(sampler); diff --git a/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java b/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java index 8128567..d799b32 100644 --- a/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java +++ b/geode-core/src/integrationTest/java/org/apache/geode/internal/statistics/GemFireStatSamplerIntegrationTest.java @@ -61,6 +61,7 @@ import org.apache.geode.StatisticsType; import org.apache.geode.distributed.DistributedSystem; import org.apache.geode.distributed.internal.InternalDistributedSystem; import org.apache.geode.internal.statistics.GemFireStatSampler.LocalStatListenerImpl; +import org.apache.geode.internal.statistics.legacy.LegacyOsStatisticsProvider; import org.apache.geode.internal.statistics.platform.ProcessStats; import org.apache.geode.internal.stats50.VMStats50; import org.apache.geode.logging.internal.log4j.api.LogService; @@ -171,7 +172,7 @@ public class GemFireStatSamplerIntegrationTest extends StatSamplerTestCase { assertThat(processStats) .withFailMessage("ProcessStats were not created on" + osName) .isNotNull(); - assertThat(OsStatisticsProvider.build().osStatsSupported())
[geode] 12/16: Make service loadable.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit dd3d111f9ed72d0dc68ee97ac65595f19eb7dde5 Author: Jacob Barrett AuthorDate: Wed Nov 17 11:19:45 2021 -0800 Make service loadable. --- .../internal/statistics/OsStatisticsBenchmark.java | 3 +- .../internal/statistics/GemFireStatSampler.java| 62 +- .../internal/statistics/StatisticsRegistry.java| 3 +- .../legacy/LegacyOsStatisticsProvider.java | 2 - geode.internal.statistics.OsStatisticsProvider | 17 ++ .../statistics/StatisticsRegistryTest.java | 3 +- 6 files changed, 72 insertions(+), 18 deletions(-) diff --git a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/OsStatisticsBenchmark.java b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/OsStatisticsBenchmark.java index 9ec47f1..cf91437 100644 --- a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/OsStatisticsBenchmark.java +++ b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/OsStatisticsBenchmark.java @@ -330,7 +330,8 @@ public class OsStatisticsBenchmark { public static class NoopStatisticsProvider implements OsStatisticsFactory { @Override -public Statistics createOsStatistics(final StatisticsType type, final String textId, final long numericId) { +public Statistics createOsStatistics(final StatisticsType type, final String textId, +final long numericId) { return new NoopStatistics(); } } diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java index 6ceb510..2a1c79d 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java @@ -20,10 +20,12 @@ import java.util.HashMap; import java.util.Iterator; import java.util.List; import java.util.Map; +import java.util.ServiceLoader; import java.util.Set; import java.util.concurrent.ConcurrentHashMap; import org.apache.logging.log4j.Logger; +import org.jetbrains.annotations.NotNull; import org.apache.geode.CancelCriterion; import org.apache.geode.Statistics; @@ -35,7 +37,6 @@ import org.apache.geode.internal.GemFireVersion; import org.apache.geode.internal.admin.ListenerIdMap; import org.apache.geode.internal.admin.remote.StatListenerMessage; import org.apache.geode.internal.logging.log4j.LogMarker; -import org.apache.geode.internal.statistics.oshi.OshiStatisticsProvider; import org.apache.geode.internal.statistics.platform.ProcessStats; import org.apache.geode.logging.internal.log4j.api.LogService; import org.apache.geode.logging.internal.spi.LogFile; @@ -66,13 +67,14 @@ public class GemFireStatSampler extends HostStatSampler { private int nextListenerId = 1; private ProcessStats processStats; - private OsStatisticsProvider osStatisticsProvider = new OshiStatisticsProvider(); + private final OsStatisticsProvider[] osStatisticsProviders; public GemFireStatSampler(InternalDistributedSystem internalDistributedSystem) { this(internalDistributedSystem, null); } - public GemFireStatSampler(InternalDistributedSystem internalDistributedSystem, LogFile logFile) { + public GemFireStatSampler(final @NotNull InternalDistributedSystem internalDistributedSystem, + final @NotNull LogFile logFile) { this(internalDistributedSystem.getCancelCriterion(), new StatSamplerStats(internalDistributedSystem, internalDistributedSystem.getStatisticsManager().getPid()), @@ -84,18 +86,41 @@ public class GemFireStatSampler extends HostStatSampler { } @VisibleForTesting - public GemFireStatSampler(CancelCriterion cancelCriterion, - StatSamplerStats statSamplerStats, - LogFile logFile, - StatisticsConfig statisticsConfig, - StatisticsManager statisticsManager, - DistributionManager distributionManager, + public GemFireStatSampler(final @NotNull CancelCriterion cancelCriterion, + final @NotNull StatSamplerStats statSamplerStats, + final @NotNull LogFile logFile, + final @NotNull StatisticsConfig statisticsConfig, + final @NotNull StatisticsManager statisticsManager, + final @NotNull DistributionManager distributionManager, long systemId) { +this(cancelCriterion, statSamplerStats, logFile, statisticsConfig, statisticsManager, +distributionManager, systemId, loadOsStatisticsProviders()); + } + + private static OsStatisticsProvider[] loadOsStatisticsProviders() { +final ServiceLoader loader = +ServiceLoader.load(OsStatisticsProvider.class); +final List osStatisticsProviders = new ArrayList<>(); +for (OsStatisticsProvider osStatisticsProvider : lo
[geode] 10/16: Delete comments
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 155ef609c831d10e908835b4f5f24204c14439b4 Author: Jacob Barrett AuthorDate: Tue Nov 16 12:40:19 2021 -0800 Delete comments --- .../java/org/apache/geode/internal/statistics/GemFireStatSampler.java| 1 - 1 file changed, 1 deletion(-) diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java index 2c3c922..b39ff2f 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java @@ -241,7 +241,6 @@ public class GemFireStatSampler extends HostStatSampler { statListener.checkForChange(statListenerMessage); } else { // its stale; indicate this with a negative listener id - // fix for bug 29405 statListenerMessage.addChange(-statListener.getListenerId(), 0); } }
[geode] 09/16: Updated benchmark and test.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 64e2a8c1f310dc16c5c31158863c1d726dec65a3 Author: Jacob Barrett AuthorDate: Mon Nov 15 22:55:16 2021 -0800 Updated benchmark and test. --- .../OshiStatisticsBenchmark.java | 53 +++ .../oshi/OshiStatisticsProviderImpl.java | 17 ++- .../oshi/OshiStatisticsProviderImplTest.java | 160 + .../statistics/oshi/OshiStatisticsTest.java| 92 4 files changed, 202 insertions(+), 120 deletions(-) diff --git a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsBenchmark.java similarity index 87% rename from geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java rename to geode-core/src/jmh/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsBenchmark.java index ddf9fcf..a2f1746 100644 --- a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java +++ b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsBenchmark.java @@ -13,51 +13,47 @@ * the License. */ -package org.apache.geode.internal.statistics.platform; +package org.apache.geode.internal.statistics.oshi; import java.util.concurrent.TimeUnit; import java.util.function.DoubleSupplier; import java.util.function.IntSupplier; import java.util.function.LongSupplier; +import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.BenchmarkMode; import org.openjdk.jmh.annotations.Mode; import org.openjdk.jmh.annotations.OutputTimeUnit; import org.openjdk.jmh.annotations.Scope; +import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; -import oshi.SystemInfo; import org.apache.geode.StatisticDescriptor; +import org.apache.geode.Statistics; import org.apache.geode.StatisticsType; import org.apache.geode.internal.statistics.SuppliableStatistics; +import org.apache.geode.internal.statistics.platform.OsStatisticsFactory; @State(Scope.Benchmark) @BenchmarkMode(Mode.SampleTime) @OutputTimeUnit(TimeUnit.MILLISECONDS) public class OshiStatisticsBenchmark { - private final int pid = new SystemInfo().getOperatingSystem().getProcessId(); - private final SuppliableStatistics noopStatistics = new NoopStatistics(); - - // @Setup - // public void setup() { - // OshiStatistics.init(); - // } - // - // @Benchmark - // public void noop() { - // - // } - // - // @Benchmark - // public void refreshProcess() { - // OshiStatistics.refreshProcess(pid, noopStatistics); - // } - // - // @Benchmark - // public void refreshSystem() { - // OshiStatistics.refreshSystem(noopStatistics); - // } + private final OshiStatisticsProviderImpl oshiStatisticsProvider = + new OshiStatisticsProviderImpl(); + + @Setup + public void setup() throws OshiStatisticsProviderException { +oshiStatisticsProvider.init(new NoopStatisticsProvider(), 0); + } + +// @Benchmark +// public void noop() {} + + @Benchmark + public void sampleProcess() { +oshiStatisticsProvider.sampleProcess(); + } private static class NoopStatistics implements SuppliableStatistics { @Override @@ -313,4 +309,13 @@ public class OshiStatisticsBenchmark { return null; } } + + private static class NoopStatisticsProvider implements OsStatisticsFactory { +@Override +public Statistics createOsStatistics(final StatisticsType type, final String textId, +final long numericId, +final int osStatFlags) { + return new NoopStatistics(); +} + } } diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java index 8054881..71201fc 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java @@ -26,7 +26,7 @@ import org.apache.geode.logging.internal.log4j.api.LogService; public class OshiStatisticsProviderImpl implements OshiStatisticsProvider { private static final Logger log = LogService.getLogger(); - final SystemInfo systemInfo = new SystemInfo(); + final SystemInfo systemInfo; private int processId; private CentralProcessor processor; @@ -43,6 +43,14 @@ public class OshiStatisticsProviderImpl implements OshiStatisticsProvider { private Statistics[] processorStats; private Statistics[] networkInterfaceStats; + public OshiStatisticsProviderImpl() { +this(new SystemInfo()); + } + + OshiStatisticsProviderImpl
[geode] 08/16: Update to OSHI 5.8.3.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 29287169f5fee315ac8f90c9bcbb8f75adc5e101 Author: Jacob Barrett AuthorDate: Mon Nov 15 21:06:22 2021 -0800 Update to OSHI 5.8.3. --- .../groovy/org/apache/geode/gradle/plugins/DependencyConstraints.groovy | 1 + geode-core/build.gradle | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/buildSrc/src/main/groovy/org/apache/geode/gradle/plugins/DependencyConstraints.groovy b/buildSrc/src/main/groovy/org/apache/geode/gradle/plugins/DependencyConstraints.groovy index a12b318..815e5d2 100644 --- a/buildSrc/src/main/groovy/org/apache/geode/gradle/plugins/DependencyConstraints.groovy +++ b/buildSrc/src/main/groovy/org/apache/geode/gradle/plugins/DependencyConstraints.groovy @@ -174,6 +174,7 @@ class DependencyConstraints implements Plugin { api(group: 'redis.clients', name: 'jedis', version: '3.6.3') api(group: 'xerces', name: 'xercesImpl', version: '2.12.0') api(group: 'xml-apis', name: 'xml-apis', version: '1.4.01') +api(group: 'com.github.oshi', name: 'oshi-core', version: '5.8.3') } } diff --git a/geode-core/build.gradle b/geode-core/build.gradle index eaa2f54..e6389f5 100755 --- a/geode-core/build.gradle +++ b/geode-core/build.gradle @@ -313,7 +313,7 @@ dependencies { implementation('com.healthmarketscience.rmiio:rmiio') // OSHI OS system stats - implementation('com.github.oshi:oshi-core:5.7.4') + implementation('com.github.oshi:oshi-core') //Geode-common has annotations and other pieces used geode-core api(project(':geode-common'))
[geode] 07/16: fixes max files
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 25b39ea10d006f73cfdd3ceb308e37ef0a279650 Author: Jacob Barrett AuthorDate: Sat Jun 12 20:24:02 2021 -0700 fixes max files --- .../oshi/OshiStatisticsProviderImpl.java | 32 ++ 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java index b79a038..8054881 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java @@ -181,19 +181,24 @@ public class OshiStatisticsProviderImpl implements OshiStatisticsProvider { tcPv4Stats.getConnectionFailures()); systemStats.setLong(OperatingSystemStats.tcpv4ConnectionsReset, tcPv4Stats.getConnectionsReset()); -systemStats.setLong(OperatingSystemStats.tcpv4SegmentsSent, tcPv4Stats.getSegmentsSent()); +systemStats.setLong(OperatingSystemStats.tcpv4SegmentsSent, +tcPv4Stats.getSegmentsSent()); systemStats.setLong(OperatingSystemStats.tcpv4SegmentsReceived, tcPv4Stats.getSegmentsReceived()); systemStats.setLong(OperatingSystemStats.tcpv4SegmentsRetransmitted, tcPv4Stats.getSegmentsRetransmitted()); -systemStats.setLong(OperatingSystemStats.tcpv4InErrors, tcPv4Stats.getInErrors()); -systemStats.setLong(OperatingSystemStats.tcpv4OutResets, tcPv4Stats.getOutResets()); +systemStats.setLong(OperatingSystemStats.tcpv4InErrors, +tcPv4Stats.getInErrors()); +systemStats.setLong(OperatingSystemStats.tcpv4OutResets, +tcPv4Stats.getOutResets()); final UdpStats udPv4Stats = internetProtocolStats.getUDPv4Stats(); -systemStats.setLong(OperatingSystemStats.udpv4DatagramsSent, udPv4Stats.getDatagramsSent()); +systemStats.setLong(OperatingSystemStats.udpv4DatagramsSent, +udPv4Stats.getDatagramsSent()); systemStats.setLong(OperatingSystemStats.udpv4DatagramsReceived, udPv4Stats.getDatagramsReceived()); -systemStats.setLong(OperatingSystemStats.udpv4DatagramsNoPort, udPv4Stats.getDatagramsNoPort()); +systemStats.setLong(OperatingSystemStats.udpv4DatagramsNoPort, +udPv4Stats.getDatagramsNoPort()); systemStats.setLong(OperatingSystemStats.udpv4DatagramsReceivedErrors, udPv4Stats.getDatagramsReceivedErrors()); @@ -208,26 +213,31 @@ public class OshiStatisticsProviderImpl implements OshiStatisticsProvider { tcPv6Stats.getConnectionFailures()); systemStats.setLong(OperatingSystemStats.tcpv6ConnectionsReset, tcPv6Stats.getConnectionsReset()); -systemStats.setLong(OperatingSystemStats.tcpv6SegmentsSent, tcPv6Stats.getSegmentsSent()); +systemStats.setLong(OperatingSystemStats.tcpv6SegmentsSent, +tcPv6Stats.getSegmentsSent()); systemStats.setLong(OperatingSystemStats.tcpv6SegmentsReceived, tcPv6Stats.getSegmentsReceived()); systemStats.setLong(OperatingSystemStats.tcpv6SegmentsRetransmitted, tcPv6Stats.getSegmentsRetransmitted()); -systemStats.setLong(OperatingSystemStats.tcpv6InErrors, tcPv6Stats.getInErrors()); -systemStats.setLong(OperatingSystemStats.tcpv6OutResets, tcPv6Stats.getOutResets()); +systemStats.setLong(OperatingSystemStats.tcpv6InErrors, +tcPv6Stats.getInErrors()); +systemStats.setLong(OperatingSystemStats.tcpv6OutResets, +tcPv6Stats.getOutResets()); final UdpStats udPv6Stats = internetProtocolStats.getUDPv6Stats(); -systemStats.setLong(OperatingSystemStats.udpv6DatagramsSent, udPv6Stats.getDatagramsSent()); +systemStats.setLong(OperatingSystemStats.udpv6DatagramsSent, +udPv6Stats.getDatagramsSent()); systemStats.setLong(OperatingSystemStats.udpv6DatagramsReceived, udPv6Stats.getDatagramsReceived()); -systemStats.setLong(OperatingSystemStats.udpv6DatagramsNoPort, udPv6Stats.getDatagramsNoPort()); +systemStats.setLong(OperatingSystemStats.udpv6DatagramsNoPort, +udPv6Stats.getDatagramsNoPort()); systemStats.setLong(OperatingSystemStats.udpv6DatagramsReceivedErrors, udPv6Stats.getDatagramsReceivedErrors()); final FileSystem fileSystem = operatingSystem.getFileSystem(); systemStats.setLong(OperatingSystemStats.openFileDescriptors, fileSystem.getOpenFileDescriptors()); -systemStats.setLong(OperatingSystemStats.openFileDescriptors, +systemStats.setLong(OperatingSystemStats.maxFileDescriptors, fileSystem.getMaxFileDescriptors()); }
[geode] 06/16: spotless
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit a8dca191b4f3eed02714a39efe976914c8ffb079 Author: Jacob Barrett AuthorDate: Sat Jun 12 20:14:39 2021 -0700 spotless --- .../platform/OshiStatisticsBenchmark.java | 46 +++ .../internal/statistics/GemFireStatSampler.java| 56 .../statistics/oshi/NetworkInterfaceStats.java | 2 +- .../statistics/oshi/OperatingSystemStats.java | 2 +- .../statistics/oshi/OshiStatisticsProvider.java| 2 +- .../oshi/OshiStatisticsProviderImpl.java | 89 +++- .../internal/statistics/oshi/ProcessStats.java | 2 +- .../internal/statistics/oshi/ProcessorStats.java | 2 +- .../statistics/oshi/OshiStatisticsTest.java| 151 ++--- 9 files changed, 182 insertions(+), 170 deletions(-) diff --git a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java index 6bfb079..ddf9fcf 100644 --- a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java +++ b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java @@ -20,12 +20,10 @@ import java.util.function.DoubleSupplier; import java.util.function.IntSupplier; import java.util.function.LongSupplier; -import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.BenchmarkMode; import org.openjdk.jmh.annotations.Mode; import org.openjdk.jmh.annotations.OutputTimeUnit; import org.openjdk.jmh.annotations.Scope; -import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; import oshi.SystemInfo; @@ -41,25 +39,25 @@ public class OshiStatisticsBenchmark { private final int pid = new SystemInfo().getOperatingSystem().getProcessId(); private final SuppliableStatistics noopStatistics = new NoopStatistics(); -// @Setup -// public void setup() { -//OshiStatistics.init(); -// } -// -// @Benchmark -// public void noop() { -// -// } -// -// @Benchmark -// public void refreshProcess() { -//OshiStatistics.refreshProcess(pid, noopStatistics); -// } -// -// @Benchmark -// public void refreshSystem() { -//OshiStatistics.refreshSystem(noopStatistics); -// } + // @Setup + // public void setup() { + // OshiStatistics.init(); + // } + // + // @Benchmark + // public void noop() { + // + // } + // + // @Benchmark + // public void refreshProcess() { + // OshiStatistics.refreshProcess(pid, noopStatistics); + // } + // + // @Benchmark + // public void refreshSystem() { + // OshiStatistics.refreshSystem(noopStatistics); + // } private static class NoopStatistics implements SuppliableStatistics { @Override @@ -279,7 +277,7 @@ public class OshiStatisticsBenchmark { @Override public IntSupplier setIntSupplier(final StatisticDescriptor descriptor, - final IntSupplier supplier) { +final IntSupplier supplier) { return null; } @@ -295,7 +293,7 @@ public class OshiStatisticsBenchmark { @Override public LongSupplier setLongSupplier(final StatisticDescriptor descriptor, -final LongSupplier supplier) { +final LongSupplier supplier) { return null; } @@ -311,7 +309,7 @@ public class OshiStatisticsBenchmark { @Override public DoubleSupplier setDoubleSupplier(final StatisticDescriptor descriptor, -final DoubleSupplier supplier) { +final DoubleSupplier supplier) { return null; } } diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java index a6d8e97..2c3c922 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java @@ -69,7 +69,7 @@ public class GemFireStatSampler extends HostStatSampler { private int nextListenerId = 1; private ProcessStats processStats; -// private OsStatisticsProvider osStatisticsProvider = OsStatisticsProvider.build(); + // private OsStatisticsProvider osStatisticsProvider = OsStatisticsProvider.build(); private OshiStatisticsProvider oshiStatisticsProvider = new OshiStatisticsProviderImpl(); public GemFireStatSampler(InternalDistributedSystem internalDistributedSystem) { @@ -282,17 +282,17 @@ public class GemFireStatSampler extends HostStatSampler { try { oshiStatisticsProvider.init(getOsStatisticsFactory(), pid); } catch (OshiStatisticsProviderExce
[geode] 05/16: adds tcp/udp stats adds file handle stats.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 599dbb54f918a54c3aa7a5f6c18810eb1e45eb97 Author: Jacob Barrett AuthorDate: Sat Jun 12 20:12:46 2021 -0700 adds tcp/udp stats adds file handle stats. --- .../statistics/oshi/OperatingSystemStats.java | 150 + .../oshi/OshiStatisticsProviderImpl.java | 46 +++ 2 files changed, 196 insertions(+) diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OperatingSystemStats.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OperatingSystemStats.java index 49f819d..1d4bba0 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OperatingSystemStats.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OperatingSystemStats.java @@ -52,6 +52,36 @@ public class OperatingSystemStats { static final int virtualInUse; static final int swapPagesIn; static final int swapPagesOut; + static final int tcpv4ConnectionsEstablished; + static final int tcpv4ConnectionsActive; + static final int tcpv4ConnectionsPassive; + static final int tcpv4ConnectionFailures; + static final int tcpv4ConnectionsReset; + static final int tcpv4SegmentsSent; + static final int tcpv4SegmentsReceived; + static final int tcpv4SegmentsRetransmitted; + static final int tcpv4InErrors; + static final int tcpv4OutResets; + static final int udpv4DatagramsSent; + static final int udpv4DatagramsReceived; + static final int udpv4DatagramsNoPort; + static final int udpv4DatagramsReceivedErrors; + static final int tcpv6ConnectionsEstablished; + static final int tcpv6ConnectionsActive; + static final int tcpv6ConnectionsPassive; + static final int tcpv6ConnectionFailures; + static final int tcpv6ConnectionsReset; + static final int tcpv6SegmentsSent; + static final int tcpv6SegmentsReceived; + static final int tcpv6SegmentsRetransmitted; + static final int tcpv6InErrors; + static final int tcpv6OutResets; + static final int udpv6DatagramsSent; + static final int udpv6DatagramsReceived; + static final int udpv6DatagramsNoPort; + static final int udpv6DatagramsReceivedErrors; + static final int openFileDescriptors; + static final int maxFileDescriptors; @Immutable private static final StatisticsType statisticsType; @@ -144,6 +174,96 @@ public class OperatingSystemStats { f.createLongCounter("swapPagesOut", "Number of pages read from paging/swap file(s) to resolve hard page faults.", "pages"), +f.createLongGauge("tcpv4ConnectionsEstablished", +"Connection Failures is the number of times TCP connections have made a direct transition to the CLOSED state from the SYN-SENT state or the SYN-RCVD state, plus the number of times TCP connections have made a direct transition to the LISTEN state from the SYN-RCVD state.", +"connections"), +f.createLongCounter("tcpv4ConnectionsActive", +"Connections Active is the number of times TCP connections have made a direct transition to the SYN-SENT state from the CLOSED state. In other words, it shows a number of connections which are initiated by the local computer. The value is a cumulative total.", +"connections"), +f.createLongCounter("tcpv4ConnectionsPassive", +"Connections Passive is the number of times TCP connections have made a direct transition to the SYN-RCVD state from the LISTEN state. In other words, it shows a number of connections to the local computer, which are initiated by remote computers. The value is a cumulative total.", +"connections"), +f.createLongCounter("tcpv4ConnectionFailures", +"Connections Established is the number of TCP connections for which the current state is either ESTABLISHED or CLOSE-WAIT.", +"connections"), +f.createLongCounter("tcpv4ConnectionsReset", +"Connections Reset is the number of times TCP connections have made a direct transition to the CLOSED state from either the ESTABLISHED state or the CLOSE-WAIT state.", +"connections"), +f.createLongCounter("tcpv4SegmentsSent", +"Segments Sent is the number of segments sent, including those on current connections, but excluding those containing only retransmitted bytes.", +"segments"), +f.createLongCounter("tcpv4SegmentsReceived", +"Segments Received is the number of segments received, including those received in error. This count includes segments received on currently established connections.", +"segments"), +f.createLongCounter("tcpv4SegmentsRetransmitted", +
[geode] 04/16: adds network intraces.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit c46ec0d862c984508c72eee6f9e3c16c51f175ea Author: Jacob Barrett AuthorDate: Sat Jun 12 09:32:04 2021 -0700 adds network intraces. --- .../statistics/oshi/NetworkInterfaceStats.java | 98 ++ .../oshi/OshiStatisticsProviderImpl.java | 48 +-- 2 files changed, 141 insertions(+), 5 deletions(-) diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/NetworkInterfaceStats.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/NetworkInterfaceStats.java new file mode 100644 index 000..95d4528 --- /dev/null +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/NetworkInterfaceStats.java @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.geode.internal.statistics.oshi; + +import org.jetbrains.annotations.NotNull; + +import org.apache.geode.StatisticDescriptor; +import org.apache.geode.StatisticsType; +import org.apache.geode.StatisticsTypeFactory; +import org.apache.geode.annotations.Immutable; +import org.apache.geode.internal.statistics.StatisticsTypeFactoryImpl; + +public class NetworkInterfaceStats { + static final int mtu; + static final int bytesReceived; + static final int bytesSent; + static final int packetsReceived; + static final int packetsSent; + static final int inErrors; + static final int outErrors; + static final int inDrops; + static final int collisions; + static final int speed; + + @Immutable + private static final StatisticsType statisticsType; + + static { +final StatisticsTypeFactory f = StatisticsTypeFactoryImpl.singleton(); + +statisticsType = f.createType("NetworkInterfaceStats", "Stats for a network interface.", +new StatisticDescriptor[]{ +f.createLongGauge("mtu", +"The interface Maximum Transmission Unit (MTU).", +"bytes"), +f.createLongCounter("bytesReceived", +"The bytes received.", +"bytes"), +f.createLongCounter("bytesSent", +"The bytes sent.", +"bytes"), +f.createLongCounter("packetsReceived", +"The packets received", +"packets"), +f.createLongCounter("packetsSent", +"The packets sent.", +"packets"), +f.createLongCounter("inErrors", +"Input errors", +"packets"), +f.createLongCounter("outErrors", +"Output errors", +"packets"), +f.createLongCounter("inDrops", +"Incoming/Received dropped packets.", +"packets"), +f.createLongCounter("collisions", +"Packet collisions.", +"packets"), +f.createLongGauge("speed", +"The speed of the network interface in bits per second.", +"bits/s"), +}); + +mtu = statisticsType.nameToId("mtu"); +bytesReceived = statisticsType.nameToId("bytesReceived"); +bytesSent = statisticsType.nameToId("bytesSent"); +packetsReceived = statisticsType.nameToId("packetsReceived"); +packetsSent = statisticsType.nameToId("packetsSent"); +inErrors = statisticsType.nameToId("inErrors"); +outErrors = statisticsType.nameToId("outErrors"); +inDrops = statisticsType.nameToId("inDrops"); +collisions = statisticsType.nameToId("collisions"); +speed = statisticsType.nameToId("speed"); + } + + private NetworkInterfaceStats() { +// no instances allowed + } + + public static @NotNull StatisticsType getType() { +return statisticsType; + } + +} diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatisticsProviderImpl.java index dcaf5b4..ce02b38 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/oshi/OshiStatis
[geode] 03/16: Refactor interface with more stats.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 0f965eb98d6ace4a862d0e94db73bba5cabc3350 Author: Jacob Barrett AuthorDate: Sat Jun 12 07:39:02 2021 -0700 Refactor interface with more stats. --- .../platform/OshiStatisticsBenchmark.java | 41 +++-- .../internal/statistics/GemFireStatSampler.java| 94 +-- .../geode/internal/statistics/HostStatSampler.java | 6 +- .../internal/statistics/OsStatisticsProvider.java | 62 ++- .../OperatingSystemStats.java} | 63 +++- .../statistics/oshi/OshiStatisticsProvider.java| 14 ++ .../oshi/OshiStatisticsProviderException.java | 4 + .../oshi/OshiStatisticsProviderImpl.java | 179 + .../ProcessStats.java} | 59 +-- .../internal/statistics/oshi/ProcessorStats.java | 98 +++ .../statistics/platform/OshiStatistics.java| 109 - .../internal/beans/MemberMBeanBridge.java | 2 +- .../statistics/oshi/OshiStatisticsTest.java| 103 .../statistics/platform/OshiStatisticsTest.java| 103 14 files changed, 582 insertions(+), 355 deletions(-) diff --git a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java index d6fbb07..6bfb079 100644 --- a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java +++ b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java @@ -22,14 +22,11 @@ import java.util.function.LongSupplier; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.BenchmarkMode; -import org.openjdk.jmh.annotations.Fork; -import org.openjdk.jmh.annotations.Measurement; import org.openjdk.jmh.annotations.Mode; import org.openjdk.jmh.annotations.OutputTimeUnit; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.State; -import org.openjdk.jmh.annotations.Warmup; import oshi.SystemInfo; import org.apache.geode.StatisticDescriptor; @@ -44,25 +41,25 @@ public class OshiStatisticsBenchmark { private final int pid = new SystemInfo().getOperatingSystem().getProcessId(); private final SuppliableStatistics noopStatistics = new NoopStatistics(); - @Setup - public void setup() { -OshiStatistics.init(); - } - - @Benchmark - public void noop() { - - } - - @Benchmark - public void refreshProcess() { -OshiStatistics.refreshProcess(pid, noopStatistics); - } - - @Benchmark - public void refreshSystem() { -OshiStatistics.refreshSystem(noopStatistics); - } +// @Setup +// public void setup() { +//OshiStatistics.init(); +// } +// +// @Benchmark +// public void noop() { +// +// } +// +// @Benchmark +// public void refreshProcess() { +//OshiStatistics.refreshProcess(pid, noopStatistics); +// } +// +// @Benchmark +// public void refreshSystem() { +//OshiStatistics.refreshSystem(noopStatistics); +// } private static class NoopStatistics implements SuppliableStatistics { @Override diff --git a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java index f7ef22a..a6d8e97 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java +++ b/geode-core/src/main/java/org/apache/geode/internal/statistics/GemFireStatSampler.java @@ -35,6 +35,9 @@ import org.apache.geode.internal.GemFireVersion; import org.apache.geode.internal.admin.ListenerIdMap; import org.apache.geode.internal.admin.remote.StatListenerMessage; import org.apache.geode.internal.logging.log4j.LogMarker; +import org.apache.geode.internal.statistics.oshi.OshiStatisticsProvider; +import org.apache.geode.internal.statistics.oshi.OshiStatisticsProviderException; +import org.apache.geode.internal.statistics.oshi.OshiStatisticsProviderImpl; import org.apache.geode.internal.statistics.platform.OsStatisticsFactory; import org.apache.geode.internal.statistics.platform.ProcessStats; import org.apache.geode.logging.internal.log4j.api.LogService; @@ -66,7 +69,8 @@ public class GemFireStatSampler extends HostStatSampler { private int nextListenerId = 1; private ProcessStats processStats; - private OsStatisticsProvider osStatisticsProvider = OsStatisticsProvider.build(); +// private OsStatisticsProvider osStatisticsProvider = OsStatisticsProvider.build(); + private OshiStatisticsProvider oshiStatisticsProvider = new OshiStatisticsProviderImpl(); public GemFireStatSampler(InternalDistributedSystem internalDistri
[geode] 02/16: Basic OSHI based stats.
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 2a8bdb42d7a7713b22fd4550ff2915eee16139ab Author: Jacob Barrett AuthorDate: Fri Jun 11 07:28:08 2021 -0700 Basic OSHI based stats. --- geode-core/build.gradle| 4 +- .../platform/OshiStatisticsBenchmark.java | 321 + .../platform/OshiStatisticsBenchmarkTest.java | 25 ++ .../internal/statistics/OsStatisticsProvider.java | 62 +++- .../statistics/platform/OshiProcessStats.java | 82 ++ .../statistics/platform/OshiStatistics.java| 109 +++ .../statistics/platform/OshiSystemStats.java | 136 + .../internal/beans/MemberMBeanBridge.java | 2 +- .../statistics/platform/OshiStatisticsTest.java| 103 +++ 9 files changed, 832 insertions(+), 12 deletions(-) diff --git a/geode-core/build.gradle b/geode-core/build.gradle index c9c9b7b..eaa2f54 100755 --- a/geode-core/build.gradle +++ b/geode-core/build.gradle @@ -312,6 +312,9 @@ dependencies { //RMIIO is used for uploading jar files and copying them between locator an servers implementation('com.healthmarketscience.rmiio:rmiio') + // OSHI OS system stats + implementation('com.github.oshi:oshi-core:5.7.4') + //Geode-common has annotations and other pieces used geode-core api(project(':geode-common')) implementation(project(':geode-logging')) @@ -324,7 +327,6 @@ dependencies { //copied into it, so it is an API dependency api(project(':geode-management')) - jcaImplementation(sourceSets.main.output) testImplementation(project(':geode-junit')) { diff --git a/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java new file mode 100644 index 000..d6fbb07 --- /dev/null +++ b/geode-core/src/jmh/java/org/apache/geode/internal/statistics/platform/OshiStatisticsBenchmark.java @@ -0,0 +1,321 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.geode.internal.statistics.platform; + +import java.util.concurrent.TimeUnit; +import java.util.function.DoubleSupplier; +import java.util.function.IntSupplier; +import java.util.function.LongSupplier; + +import org.openjdk.jmh.annotations.Benchmark; +import org.openjdk.jmh.annotations.BenchmarkMode; +import org.openjdk.jmh.annotations.Fork; +import org.openjdk.jmh.annotations.Measurement; +import org.openjdk.jmh.annotations.Mode; +import org.openjdk.jmh.annotations.OutputTimeUnit; +import org.openjdk.jmh.annotations.Scope; +import org.openjdk.jmh.annotations.Setup; +import org.openjdk.jmh.annotations.State; +import org.openjdk.jmh.annotations.Warmup; +import oshi.SystemInfo; + +import org.apache.geode.StatisticDescriptor; +import org.apache.geode.StatisticsType; +import org.apache.geode.internal.statistics.SuppliableStatistics; + +@State(Scope.Benchmark) +@BenchmarkMode(Mode.SampleTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +public class OshiStatisticsBenchmark { + + private final int pid = new SystemInfo().getOperatingSystem().getProcessId(); + private final SuppliableStatistics noopStatistics = new NoopStatistics(); + + @Setup + public void setup() { +OshiStatistics.init(); + } + + @Benchmark + public void noop() { + + } + + @Benchmark + public void refreshProcess() { +OshiStatistics.refreshProcess(pid, noopStatistics); + } + + @Benchmark + public void refreshSystem() { +OshiStatistics.refreshSystem(noopStatistics); + } + + private static class NoopStatistics implements SuppliableStatistics { +@Override +public int updateSuppliedValues() { + return 0; +} + +@Override +public void close() { + +} + +@Override +public int nameToId(final String name) { + return 0; +} + +@Override +public StatisticDescriptor nameToDescriptor(final String name) { + return null; +} + +@Override +public long getUniqueId() { + return 0; +} + +@Override +public StatisticsTyp
[geode] 01/16: MacOSX to macOS
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a commit to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git commit 0a485c43b1ea3a260fb4f0dfa75bc0229caf52b7 Author: Jacob Barrett AuthorDate: Thu Jun 10 16:29:19 2021 -0700 MacOSX to macOS --- .../src/main/java/org/apache/geode/internal/lang/SystemUtils.java | 6 +++--- .../java/org/apache/geode/internal/lang/SystemUtilsJUnitTest.java | 8 .../test/java/org/apache/geode/internal/util/HostNameTest.java| 8 .../geode/management/internal/cli/functions/NetstatFunction.java | 4 ++-- .../org/apache/geode/management/internal/cli/util/JdkTool.java| 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/geode-core/src/main/java/org/apache/geode/internal/lang/SystemUtils.java b/geode-core/src/main/java/org/apache/geode/internal/lang/SystemUtils.java index 53f8660..64e079b 100644 --- a/geode-core/src/main/java/org/apache/geode/internal/lang/SystemUtils.java +++ b/geode-core/src/main/java/org/apache/geode/internal/lang/SystemUtils.java @@ -44,7 +44,7 @@ public class SystemUtils { // Operating System Names public static final String LINUX_OS_NAME = "Linux"; - public static final String MAC_OSX_NAME = "Mac"; + public static final String MAC_OS_NAME = "Mac"; public static final String WINDOWS_OS_NAME = "Windows"; public static final String SOLARIS_OS_NAME = "SunOS"; @@ -152,8 +152,8 @@ public class SystemUtils { * OSX. * @see #isOS(String) */ - public static boolean isMacOSX() { -return isOS(MAC_OSX_NAME); + public static boolean isMacOS() { +return isOS(MAC_OS_NAME); } /** diff --git a/geode-core/src/test/java/org/apache/geode/internal/lang/SystemUtilsJUnitTest.java b/geode-core/src/test/java/org/apache/geode/internal/lang/SystemUtilsJUnitTest.java index 5676252..0f5b8eb 100644 --- a/geode-core/src/test/java/org/apache/geode/internal/lang/SystemUtilsJUnitTest.java +++ b/geode-core/src/test/java/org/apache/geode/internal/lang/SystemUtilsJUnitTest.java @@ -18,7 +18,7 @@ import static org.apache.geode.internal.lang.SystemUtils.APPLE_JVM_VENDOR_NAME; import static org.apache.geode.internal.lang.SystemUtils.IBM_J9_JVM_NAME; import static org.apache.geode.internal.lang.SystemUtils.JAVA_HOTSPOT_JVM_NAME; import static org.apache.geode.internal.lang.SystemUtils.LINUX_OS_NAME; -import static org.apache.geode.internal.lang.SystemUtils.MAC_OSX_NAME; +import static org.apache.geode.internal.lang.SystemUtils.MAC_OS_NAME; import static org.apache.geode.internal.lang.SystemUtils.ORACLE_JROCKIT_JVM_NAME; import static org.apache.geode.internal.lang.SystemUtils.ORACLE_JVM_VENDOR_NAME; import static org.apache.geode.internal.lang.SystemUtils.WINDOWS_OS_NAME; @@ -32,7 +32,7 @@ import static org.apache.geode.internal.lang.SystemUtils.isHotSpotVM; import static org.apache.geode.internal.lang.SystemUtils.isJ9VM; import static org.apache.geode.internal.lang.SystemUtils.isJRockitVM; import static org.apache.geode.internal.lang.SystemUtils.isLinux; -import static org.apache.geode.internal.lang.SystemUtils.isMacOSX; +import static org.apache.geode.internal.lang.SystemUtils.isMacOS; import static org.apache.geode.internal.lang.SystemUtils.isOracleJVM; import static org.apache.geode.internal.lang.SystemUtils.isWindows; import static org.assertj.core.api.Assertions.assertThat; @@ -99,8 +99,8 @@ public class SystemUtilsJUnitTest { @Test public void testIsMacOSX() { final boolean expected = - ManagementFactory.getOperatingSystemMXBean().getName().contains(MAC_OSX_NAME); -assertEquals(expected, isMacOSX()); + ManagementFactory.getOperatingSystemMXBean().getName().contains(MAC_OS_NAME); +assertEquals(expected, isMacOS()); } @Test diff --git a/geode-core/src/test/java/org/apache/geode/internal/util/HostNameTest.java b/geode-core/src/test/java/org/apache/geode/internal/util/HostNameTest.java index fa502db..fd0450c 100644 --- a/geode-core/src/test/java/org/apache/geode/internal/util/HostNameTest.java +++ b/geode-core/src/test/java/org/apache/geode/internal/util/HostNameTest.java @@ -15,7 +15,7 @@ package org.apache.geode.internal.util; import static org.apache.geode.internal.lang.SystemUtils.LINUX_OS_NAME; -import static org.apache.geode.internal.lang.SystemUtils.MAC_OSX_NAME; +import static org.apache.geode.internal.lang.SystemUtils.MAC_OS_NAME; import static org.apache.geode.internal.lang.SystemUtils.SOLARIS_OS_NAME; import static org.apache.geode.internal.lang.SystemUtils.WINDOWS_OS_NAME; import static org.apache.geode.internal.lang.SystemUtils.isWindows; @@ -54,7 +54,7 @@ public class HostNameTest { } @Test - @Parameters({MAC_OSX_NAME, LINUX_OS_NAME, SOLARIS_OS_NAME, WINDOWS_OS_NAME}) + @Parameters({MAC_OS_NAME, LINUX_OS_NAME, SOLARIS_OS_NAME, WINDOWS_OS_NAME}) public void shouldExecHostNameIfEnvValueNotAvailableOnOS(String osN
[geode] branch wip/oshi-multios-stats-module created (now fb2b8cf)
This is an automated email from the ASF dual-hosted git repository. jbarrett pushed a change to branch wip/oshi-multios-stats-module in repository https://gitbox.apache.org/repos/asf/geode.git. at fb2b8cf Initial module. This branch includes the following new commits: new 0a485c4 MacOSX to macOS new 2a8bdb4 Basic OSHI based stats. new 0f965eb Refactor interface with more stats. new c46ec0d adds network intraces. new 599dbb5 adds tcp/udp stats adds file handle stats. new a8dca19 spotless new 25b39ea fixes max files new 2928716 Update to OSHI 5.8.3. new 64e2a8c Updated benchmark and test. new 155ef60 Delete comments new cf79e27 Extract legacy OS stats. new dd3d111 Make service loadable. new 2a3b2af Fix legacy process stats sample. new 8c8fa85 Fix tests. new 6239e4d Refactor and cleanup legacy ProcessStats. new fb2b8cf Initial module. The 16 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.