Merge branch cassandra-3.11 into trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17e602b7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17e602b7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17e602b7 Branch: refs/heads/trunk Commit: 17e602b7d248c61634c31c93b7d244f35cfcefd3 Parents: 8e95534 a7c45be Author: Benjamin Lerer <b.le...@gmail.com> Authored: Tue Dec 12 10:52:23 2017 +0100 Committer: Benjamin Lerer <b.le...@gmail.com> Committed: Tue Dec 12 10:52:23 2017 +0100 ---------------------------------------------------------------------- CHANGES.txt | 2 +- NEWS.txt | 13 + bin/cassandra | 19 +- conf/cassandra-env.ps1 | 914 ++++++++++--------- conf/cassandra-env.sh | 12 + .../apache/cassandra/service/StartupChecks.java | 74 ++ .../org/apache/cassandra/utils/HeapUtils.java | 93 +- .../cassandra/utils/JVMStabilityInspector.java | 24 +- .../utils/JVMStabilityInspectorTest.java | 28 +- 9 files changed, 638 insertions(+), 541 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/CHANGES.txt ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/NEWS.txt ---------------------------------------------------------------------- diff --cc NEWS.txt index 7bb9152,0c32278..259507e --- a/NEWS.txt +++ b/NEWS.txt @@@ -13,103 -13,27 +13,116 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. +4.0 +=== + +New features +------------ + - An experimental option to compare all merkle trees together has been added - for example, in + a 3 node cluster with 2 replicas identical and 1 out-of-date, with this option enabled, the + out-of-date replica will only stream a single copy from up-to-date replica. Enable it by adding + "-os" to nodetool repair. See CASSANDRA-3200. + - The currentTimestamp, currentDate, currentTime and currentTimeUUID functions have been added. + See CASSANDRA-13132 + - Support for arithmetic operations between `timestamp`/`date` and `duration` has been added. + See CASSANDRA-11936 + - Support for arithmetic operations on number has been added. See CASSANDRA-11935 + - Preview expected streaming required for a repair (nodetool repair --preview), and validate the + consistency of repaired data between nodes (nodetool repair --validate). See CASSANDRA-13257 + - Support for selecting Map values and Set elements has been added for SELECT queries. See CASSANDRA-7396 + - Change-Data-Capture has been modified to make CommitLogSegments available + immediately upon creation via hard-linking the files. This means that incomplete + segments will be available in cdc_raw rather than fully flushed. See documentation + and CASSANDRA-12148 for more detail. + - The initial build of materialized views can be parallelized. The number of concurrent builder + threads is specified by the property `cassandra.yaml:concurrent_materialized_view_builders`. + This property can be modified at runtime through both JMX and the new `setconcurrentviewbuilders` + and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 for more details. + - There is now a binary full query log based on Chronicle Queue that can be controlled using + nodetool enablefullquerylog, disablefullquerylog, and resetfullquerylog. The log + contains all queries invoked, approximate time they were invoked, any parameters necessary + to bind wildcard values, and all query options. A human readable version of the log can be + dumped or tailed using the new bin/fqltool utility. The full query log is designed to be safe + to use in production and limits utilization of heap memory and disk space with limits + you can specify when enabling the log. + See nodetool and fqltool help text for more information. + - SSTableDump now supports the -l option to output each partition as it's own json object + See CASSANDRA-13848 for more detail + +Upgrading +--------- + - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables + have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 3.0/3.11. + Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL. + Syntax for creating compact tables is also deprecated. + - Support for legacy auth tables in the system_auth keyspace (users, + permissions, credentials) and the migration code has been removed. Migration + of these legacy auth tables must have been completed before the upgrade to + 4.0 and the legacy tables must have been removed. See the 'Upgrading' section + for version 2.2 for migration instructions. + - Cassandra 4.0 removed support for the deprecated Thrift interface. Amongst + other things, this implies the removal of all yaml options related to thrift + ('start_rpc', rpc_port, ...). + - Cassandra 4.0 removed support for any pre-3.0 format. This means you + cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade to + a 3.0.x/3.x version first (and run upgradesstable). In particular, this + mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you + will need to upgrade those sstable in 3.0.x/3.x first. + - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, previous + versions will causes issues during rolling upgrades (CASSANDRA-13274). + - Cassandra will no longer allow invalid keyspace replication options, such + as invalid datacenter names for NetworkTopologyStrategy. Operators MUST + add new nodes to a datacenter before they can set set ALTER or CREATE + keyspace replication policies using that datacenter. Existing keyspaces + will continue to operate, but CREATE and ALTER will validate that all + datacenters specified exist in the cluster. + - Cassandra 4.0 fixes a problem with incremental repair which caused repaired + data to be inconsistent between nodes. The fix changes the behavior of both + full and incremental repairs. For full repairs, data is no longer marked + repaired. For incremental repairs, anticompaction is run at the beginning + of the repair, instead of at the end. If incremental repair was being used + prior to upgrading, a full repair should be run after upgrading to resolve + any inconsistencies. + - Config option index_interval has been removed (it was deprecated since 2.0) + - Deprecated repair JMX APIs are removed. + - The version of snappy-java has been upgraded to 1.1.2.6 + - the miniumum value for internode message timeouts is 10ms. Previously, any + positive value was allowed. See cassandra.yaml entries like + read_request_timeout_in_ms for more details. + - Cassandra 4.0 allows a single port to be used for both secure and insecure + connections between cassandra nodes (CASSANDRA-10404). See the yaml for + specific property changes, and see the security doc for full details. + - Due to the parallelization of the initial build of materialized views, + the per token range view building status is stored in the new table + `system.view_builds_in_progress`. The old table `system.views_builds_in_progress` + is no longer used and can be removed. See CASSANDRA-12245 for more details. + - Config option commitlog_sync_batch_window_in_ms has been deprecated as it's + documentation has been incorrect and the setting itself near useless. + Batch mode remains a valid commit log mode, however. + - There is a new commit log mode, group, which is similar to batch mode + but blocks for up to a configurable number of milliseconds between disk flushes. + +Materialized Views +------------------- + - Following a discussion regarding concerns about the design and safety of Materialized Views, the C* development + community no longer recommends them for production use, and considers them experimental. Warnings messages will + now be logged when they are created. (See https://www.mail-archive.com/dev@cassandra.apache.org/msg11511.html) + - An 'enable_materialized_views' flag has been added to cassandra.yaml to allow operators to prevent creation of + views + + 3.11.2 + ====== + + Upgrading + --------- + - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will + rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options + are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM + options are used. See CASSANDRA-13006 for more details + - Cassandra is not logging anymore by default an Heap histogram on OutOfMemoryError. To enable that behavior + set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'true'. See CASSANDRA-13006 + for more details. + -Materialized Views -------------------- - - Following a discussion regarding concerns about the design and safety of Materialized Views, the C* development - community no longer recommends them for production use, and considers them experimental. Warnings messages will - now be logged when they are created. (See https://www.mail-archive.com/dev@cassandra.apache.org/msg11511.html) - - An 'enable_materialized_views' flag has been added to cassandra.yaml to allow operators to prevent creation of - views - 3.11.1 ====== http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/conf/cassandra-env.ps1 ---------------------------------------------------------------------- diff --cc conf/cassandra-env.ps1 index 806eabc,49d03ce..72d632b --- a/conf/cassandra-env.ps1 +++ b/conf/cassandra-env.ps1 @@@ -1,381 -1,381 +1,381 @@@ - # - # Licensed to the Apache Software Foundation (ASF) under one or more - # contributor license agreements. See the NOTICE file distributed with - # this work for additional information regarding copyright ownership. - # The ASF licenses this file to You under the Apache License, Version 2.0 - # (the "License"); you may not use this file except in compliance with - # the License. You may obtain a copy of the License at - # - # http://www.apache.org/licenses/LICENSE-2.0 - # - # Unless required by applicable law or agreed to in writing, software - # distributed under the License is distributed on an "AS IS" BASIS, - # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - # See the License for the specific language governing permissions and - # limitations under the License. - - # NOTE: All param tuning can be done in the SetCassandraEnvironment Function below - - #----------------------------------------------------------------------------- - Function SetCassandraHome() - { - if (! $env:CASSANDRA_HOME) - { - $cwd = [System.IO.Directory]::GetCurrentDirectory() - $cwd = Split-Path $cwd -parent - $env:CASSANDRA_HOME = $cwd -replace "\\", "/" - } - } - - #----------------------------------------------------------------------------- - Function SetCassandraMain() - { - if (! $env:CASSANDRA_MAIN) - { - $env:CASSANDRA_MAIN="org.apache.cassandra.service.CassandraDaemon" - } - } - - #----------------------------------------------------------------------------- - Function BuildClassPath - { - $cp = """$env:CASSANDRA_HOME\conf""" - foreach ($file in Get-ChildItem "$env:CASSANDRA_HOME\lib\*.jar") - { - $file = $file -replace "\\", "/" - $cp = $cp + ";" + """$file""" - } - - # Add build/classes/main so it works in development - $cp = $cp + ";" + """$env:CASSANDRA_HOME\build\classes\main""" - $env:CLASSPATH=$cp - } - - #----------------------------------------------------------------------------- - Function CalculateHeapSizes - { - # Check if swapping is enabled on the host and warn if so - reference CASSANDRA-7316 - - $osInfo = Get-WmiObject -class "Win32_computersystem" - $autoPage = $osInfo.AutomaticManagedPageFile - - if ($autoPage) - { - echo "*---------------------------------------------------------------------*" - echo "*---------------------------------------------------------------------*" - echo "" - echo " WARNING! Automatic page file configuration detected." - echo " It is recommended that you disable swap when running Cassandra" - echo " for performance and stability reasons." - echo "" - echo "*---------------------------------------------------------------------*" - echo "*---------------------------------------------------------------------*" - } - else - { - $pageFileInfo = Get-WmiObject -class "Win32_PageFileSetting" -EnableAllPrivileges - $pageFileCount = $PageFileInfo.Count - if ($pageFileInfo) - { - $files = @() - $sizes = @() - $hasSizes = $FALSE - - # PageFileCount isn't populated and obj comes back as single if there's only 1 - if ([string]::IsNullOrEmpty($PageFileCount)) - { - $PageFileCount = 1 - $files += $PageFileInfo.Name - if ($PageFileInfo.MaximumSize -ne 0) - { - $hasSizes = $TRUE - $sizes += $PageFileInfo.MaximumSize - } - } - else - { - for ($i = 0; $i -le $PageFileCount; $i++) - { - $files += $PageFileInfo[$i].Name - if ($PageFileInfo[$i].MaximumSize -ne 0) - { - $hasSizes = $TRUE - $sizes += $PageFileInfo[$i].MaximumSize - } - } - } - - echo "*---------------------------------------------------------------------*" - echo "*---------------------------------------------------------------------*" - echo "" - echo " WARNING! $PageFileCount swap file(s) detected" - for ($i = 0; $i -lt $PageFileCount; $i++) - { - $toPrint = " Name: " + $files[$i] - if ($hasSizes) - { - $toPrint = $toPrint + " Size: " + $sizes[$i] - $toPrint = $toPrint -replace [Environment]::NewLine, "" - } - echo $toPrint - } - echo " It is recommended that you disable swap when running Cassandra" - echo " for performance and stability reasons." - echo "" - echo "*---------------------------------------------------------------------*" - echo "*---------------------------------------------------------------------*" - } - } - - # Validate that we need to run this function and that our config is good - if ($env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE) - { - return - } - - if ((($env:MAX_HEAP_SIZE -and !$env:HEAP_NEWSIZE) -or (!$env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE)) -and ($using_cms -eq $true)) - { - echo "Please set or unset MAX_HEAP_SIZE and HEAP_NEWSIZE in pairs. Aborting startup." - exit 1 - } - - $memObject = Get-WMIObject -class win32_physicalmemory - if ($memObject -eq $null) - { - echo "WARNING! Could not determine system memory. Defaulting to 2G heap, 512M newgen. Manually override in conf\jvm.options for different heap values." - $env:MAX_HEAP_SIZE = "2048M" - $env:HEAP_NEWSIZE = "512M" - return - } - - $memory = ($memObject | Measure-Object Capacity -Sum).sum - $memoryMB = [Math]::Truncate($memory / (1024*1024)) - - $cpu = gwmi Win32_ComputerSystem | Select-Object NumberOfLogicalProcessors - $systemCores = $cpu.NumberOfLogicalProcessors - - # set max heap size based on the following - # max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB)) - # calculate 1/2 ram and cap to 1024MB - # calculate 1/4 ram and cap to 8192MB - # pick the max - $halfMem = [Math]::Truncate($memoryMB / 2) - $quarterMem = [Math]::Truncate($halfMem / 2) - - if ($halfMem -gt 1024) - { - $halfMem = 1024 - } - if ($quarterMem -gt 8192) - { - $quarterMem = 8192 - } - - $maxHeapMB = "" - if ($halfMem -gt $quarterMem) - { - $maxHeapMB = $halfMem - } - else - { - $maxHeapMB = $quarterMem - } - $env:MAX_HEAP_SIZE = [System.Convert]::ToString($maxHeapMB) + "M" - - # Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4 - $maxYGPerCore = 100 - $maxYGTotal = $maxYGPerCore * $systemCores - $desiredYG = [Math]::Truncate($maxHeapMB / 4) - - if ($desiredYG -gt $maxYGTotal) - { - $env:HEAP_NEWSIZE = [System.Convert]::ToString($maxYGTotal) + "M" - } - else - { - $env:HEAP_NEWSIZE = [System.Convert]::ToString($desiredYG) + "M" - } - } - - #----------------------------------------------------------------------------- - Function ParseJVMInfo - { - # grab info about the JVM - $pinfo = New-Object System.Diagnostics.ProcessStartInfo - $pinfo.FileName = "$env:JAVA_BIN" - $pinfo.RedirectStandardError = $true - $pinfo.RedirectStandardOutput = $true - $pinfo.UseShellExecute = $false - $pinfo.Arguments = "-d64 -version" - $p = New-Object System.Diagnostics.Process - $p.StartInfo = $pinfo - $p.Start() | Out-Null - $p.WaitForExit() - $stderr = $p.StandardError.ReadToEnd() - - $env:JVM_ARCH = "64-bit" - - if ($stderr.Contains("Error")) - { - # 32-bit JVM. re-run w/out -d64 - echo "Failed 64-bit check. Re-running to get version from 32-bit" - $pinfo.Arguments = "-version" - $p = New-Object System.Diagnostics.Process - $p.StartInfo = $pinfo - $p.Start() | Out-Null - $p.WaitForExit() - $stderr = $p.StandardError.ReadToEnd() - $env:JVM_ARCH = "32-bit" - } - - $sa = $stderr.Split("""") - $env:JVM_VERSION = $sa[1] - - if ($stderr.Contains("OpenJDK")) - { - $env:JVM_VENDOR = "OpenJDK" - } - elseif ($stderr.Contains("Java(TM)")) - { - $env:JVM_VENDOR = "Oracle" - } - else - { - $JVM_VENDOR = "other" - } - - $pa = $sa[1].Split("_") - $subVersion = $pa[1] - # Deal with -b (build) versions - if ($subVersion -contains '-') - { - $patchAndBuild = $subVersion.Split("-") - $subVersion = $patchAndBuild[0] - } - $env:JVM_PATCH_VERSION = $subVersion - } - - #----------------------------------------------------------------------------- - Function SetCassandraEnvironment - { - if (Test-Path Env:\JAVA_HOME) - { - $env:JAVA_BIN = "$env:JAVA_HOME\bin\java.exe" - } - elseif (Get-Command "java.exe") - { - $env:JAVA_BIN = "java.exe" - } - else - { - echo "ERROR! No JAVA_HOME set and could not find java.exe in the path." - exit - } - SetCassandraHome - $env:CASSANDRA_CONF = "$env:CASSANDRA_HOME\conf" - $env:CASSANDRA_PARAMS="-Dcassandra -Dlogback.configurationFile=logback.xml" - - $logdir = "$env:CASSANDRA_HOME\logs" - $storagedir = "$env:CASSANDRA_HOME\data" - $env:CASSANDRA_PARAMS = $env:CASSANDRA_PARAMS + " -Dcassandra.logdir=""$logdir"" -Dcassandra.storagedir=""$storagedir""" - - SetCassandraMain - BuildClassPath - - # Override these to set the amount of memory to allocate to the JVM at - # start-up. For production use you may wish to adjust this for your - # environment. MAX_HEAP_SIZE is the total amount of memory dedicated - # to the Java heap. HEAP_NEWSIZE refers to the size of the young - # generation. Both MAX_HEAP_SIZE and HEAP_NEWSIZE should be either set - # or not (if you set one, set the other). - # - # The main trade-off for the young generation is that the larger it - # is, the longer GC pause times will be. The shorter it is, the more - # expensive GC will be (usually). - # - # The example HEAP_NEWSIZE assumes a modern 8-core+ machine for decent - # times. If in doubt, and if you do not particularly want to tweak, go - # 100 MB per physical CPU core. - - #GC log path has to be defined here since it needs to find CASSANDRA_HOME - $env:JVM_OPTS="$env:JVM_OPTS -Xloggc:""$env:CASSANDRA_HOME/logs/gc.log""" - - # Read user-defined JVM options from jvm.options file - $content = Get-Content "$env:CASSANDRA_CONF\jvm.options" - for ($i = 0; $i -lt $content.Count; $i++) - { - $line = $content[$i] - if ($line.StartsWith("-")) - { - $env:JVM_OPTS = "$env:JVM_OPTS $line" - } - } - - $defined_xmn = $env:JVM_OPTS -like '*Xmn*' - $defined_xmx = $env:JVM_OPTS -like '*Xmx*' - $defined_xms = $env:JVM_OPTS -like '*Xms*' - $using_cms = $env:JVM_OPTS -like '*UseConcMarkSweepGC*' - - #$env:MAX_HEAP_SIZE="4096M" - #$env:HEAP_NEWSIZE="800M" - CalculateHeapSizes - - ParseJVMInfo - - # We only set -Xms and -Xmx if they were not defined on jvm.options file - # If defined, both Xmx and Xms should be defined together. - if (($defined_xmx -eq $false) -and ($defined_xms -eq $false)) - { - $env:JVM_OPTS="$env:JVM_OPTS -Xms$env:MAX_HEAP_SIZE" - $env:JVM_OPTS="$env:JVM_OPTS -Xmx$env:MAX_HEAP_SIZE" - } - elseif (($defined_xmx -eq $false) -or ($defined_xms -eq $false)) - { - echo "Please set or unset -Xmx and -Xms flags in pairs on jvm.options file." - exit - } - - # We only set -Xmn flag if it was not defined in jvm.options file - # and if the CMS GC is being used - # If defined, both Xmn and Xmx should be defined together. - if (($defined_xmn -eq $true) -and ($defined_xmx -eq $false)) - { - echo "Please set or unset -Xmx and -Xmn flags in pairs on jvm.options file." - exit - } - elseif (($defined_xmn -eq $false) -and ($using_cms -eq $true)) - { - $env:JVM_OPTS="$env:JVM_OPTS -Xmn$env:HEAP_NEWSIZE" - } - - if (($env:JVM_ARCH -eq "64-Bit") -and ($using_cms -eq $true)) - { - $env:JVM_OPTS="$env:JVM_OPTS -XX:+UseCondCardMark" - } - - # Add sigar env - see Cassandra-7838 - $env:JVM_OPTS = "$env:JVM_OPTS -Djava.library.path=""$env:CASSANDRA_HOME\lib\sigar-bin""" - - # Confirm we're on high performance power plan, warn if not - # Change to $true to suppress this warning - $suppressPowerWarning = $false - if (!$suppressPowerWarning) - { - $currentProfile = powercfg /GETACTIVESCHEME - if (!$currentProfile.Contains("High performance")) - { - echo "*---------------------------------------------------------------------*" - echo "*---------------------------------------------------------------------*" - echo "" - echo " WARNING! Detected a power profile other than High Performance." - echo " Performance of this node will suffer." - echo " Modify conf\cassandra.env.ps1 to suppress this warning." - echo "" - echo "*---------------------------------------------------------------------*" - echo "*---------------------------------------------------------------------*" - } - } - + # + # Licensed to the Apache Software Foundation (ASF) under one or more + # contributor license agreements. See the NOTICE file distributed with + # this work for additional information regarding copyright ownership. + # The ASF licenses this file to You under the Apache License, Version 2.0 + # (the "License"); you may not use this file except in compliance with + # the License. You may obtain a copy of the License at + # + # http://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an "AS IS" BASIS, + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + # See the License for the specific language governing permissions and + # limitations under the License. + + # NOTE: All param tuning can be done in the SetCassandraEnvironment Function below + + #----------------------------------------------------------------------------- + Function SetCassandraHome() + { + if (! $env:CASSANDRA_HOME) + { + $cwd = [System.IO.Directory]::GetCurrentDirectory() + $cwd = Split-Path $cwd -parent + $env:CASSANDRA_HOME = $cwd -replace "\\", "/" + } + } + + #----------------------------------------------------------------------------- + Function SetCassandraMain() + { + if (! $env:CASSANDRA_MAIN) + { + $env:CASSANDRA_MAIN="org.apache.cassandra.service.CassandraDaemon" + } + } + + #----------------------------------------------------------------------------- + Function BuildClassPath + { + $cp = """$env:CASSANDRA_HOME\conf""" + foreach ($file in Get-ChildItem "$env:CASSANDRA_HOME\lib\*.jar") + { + $file = $file -replace "\\", "/" + $cp = $cp + ";" + """$file""" + } + + # Add build/classes/main so it works in development - $cp = $cp + ";" + """$env:CASSANDRA_HOME\build\classes\main"";""$env:CASSANDRA_HOME\build\classes\thrift""" ++ $cp = $cp + ";" + """$env:CASSANDRA_HOME\build\classes\main""" + $env:CLASSPATH=$cp + } + + #----------------------------------------------------------------------------- + Function CalculateHeapSizes + { + # Check if swapping is enabled on the host and warn if so - reference CASSANDRA-7316 + + $osInfo = Get-WmiObject -class "Win32_computersystem" + $autoPage = $osInfo.AutomaticManagedPageFile + + if ($autoPage) + { + echo "*---------------------------------------------------------------------*" + echo "*---------------------------------------------------------------------*" + echo "" + echo " WARNING! Automatic page file configuration detected." + echo " It is recommended that you disable swap when running Cassandra" + echo " for performance and stability reasons." + echo "" + echo "*---------------------------------------------------------------------*" + echo "*---------------------------------------------------------------------*" + } + else + { + $pageFileInfo = Get-WmiObject -class "Win32_PageFileSetting" -EnableAllPrivileges + $pageFileCount = $PageFileInfo.Count + if ($pageFileInfo) + { + $files = @() + $sizes = @() + $hasSizes = $FALSE + + # PageFileCount isn't populated and obj comes back as single if there's only 1 + if ([string]::IsNullOrEmpty($PageFileCount)) + { + $PageFileCount = 1 + $files += $PageFileInfo.Name + if ($PageFileInfo.MaximumSize -ne 0) + { + $hasSizes = $TRUE + $sizes += $PageFileInfo.MaximumSize + } + } + else + { + for ($i = 0; $i -le $PageFileCount; $i++) + { + $files += $PageFileInfo[$i].Name + if ($PageFileInfo[$i].MaximumSize -ne 0) + { + $hasSizes = $TRUE + $sizes += $PageFileInfo[$i].MaximumSize + } + } + } + + echo "*---------------------------------------------------------------------*" + echo "*---------------------------------------------------------------------*" + echo "" + echo " WARNING! $PageFileCount swap file(s) detected" + for ($i = 0; $i -lt $PageFileCount; $i++) + { + $toPrint = " Name: " + $files[$i] + if ($hasSizes) + { + $toPrint = $toPrint + " Size: " + $sizes[$i] + $toPrint = $toPrint -replace [Environment]::NewLine, "" + } + echo $toPrint + } + echo " It is recommended that you disable swap when running Cassandra" + echo " for performance and stability reasons." + echo "" + echo "*---------------------------------------------------------------------*" + echo "*---------------------------------------------------------------------*" + } + } + + # Validate that we need to run this function and that our config is good + if ($env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE) + { + return + } + + if ((($env:MAX_HEAP_SIZE -and !$env:HEAP_NEWSIZE) -or (!$env:MAX_HEAP_SIZE -and $env:HEAP_NEWSIZE)) -and ($using_cms -eq $true)) + { + echo "Please set or unset MAX_HEAP_SIZE and HEAP_NEWSIZE in pairs. Aborting startup." + exit 1 + } + + $memObject = Get-WMIObject -class win32_physicalmemory + if ($memObject -eq $null) + { + echo "WARNING! Could not determine system memory. Defaulting to 2G heap, 512M newgen. Manually override in conf\jvm.options for different heap values." + $env:MAX_HEAP_SIZE = "2048M" + $env:HEAP_NEWSIZE = "512M" + return + } + + $memory = ($memObject | Measure-Object Capacity -Sum).sum + $memoryMB = [Math]::Truncate($memory / (1024*1024)) + + $cpu = gwmi Win32_ComputerSystem | Select-Object NumberOfLogicalProcessors + $systemCores = $cpu.NumberOfLogicalProcessors + + # set max heap size based on the following + # max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB)) + # calculate 1/2 ram and cap to 1024MB + # calculate 1/4 ram and cap to 8192MB + # pick the max + $halfMem = [Math]::Truncate($memoryMB / 2) + $quarterMem = [Math]::Truncate($halfMem / 2) + + if ($halfMem -gt 1024) + { + $halfMem = 1024 + } + if ($quarterMem -gt 8192) + { + $quarterMem = 8192 + } + + $maxHeapMB = "" + if ($halfMem -gt $quarterMem) + { + $maxHeapMB = $halfMem + } + else + { + $maxHeapMB = $quarterMem + } + $env:MAX_HEAP_SIZE = [System.Convert]::ToString($maxHeapMB) + "M" + + # Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4 + $maxYGPerCore = 100 + $maxYGTotal = $maxYGPerCore * $systemCores + $desiredYG = [Math]::Truncate($maxHeapMB / 4) + + if ($desiredYG -gt $maxYGTotal) + { + $env:HEAP_NEWSIZE = [System.Convert]::ToString($maxYGTotal) + "M" + } + else + { + $env:HEAP_NEWSIZE = [System.Convert]::ToString($desiredYG) + "M" + } + } + + #----------------------------------------------------------------------------- + Function ParseJVMInfo + { + # grab info about the JVM + $pinfo = New-Object System.Diagnostics.ProcessStartInfo + $pinfo.FileName = "$env:JAVA_BIN" + $pinfo.RedirectStandardError = $true + $pinfo.RedirectStandardOutput = $true + $pinfo.UseShellExecute = $false + $pinfo.Arguments = "-d64 -version" + $p = New-Object System.Diagnostics.Process + $p.StartInfo = $pinfo + $p.Start() | Out-Null + $p.WaitForExit() + $stderr = $p.StandardError.ReadToEnd() + + $env:JVM_ARCH = "64-bit" + + if ($stderr.Contains("Error")) + { + # 32-bit JVM. re-run w/out -d64 + echo "Failed 64-bit check. Re-running to get version from 32-bit" + $pinfo.Arguments = "-version" + $p = New-Object System.Diagnostics.Process + $p.StartInfo = $pinfo + $p.Start() | Out-Null + $p.WaitForExit() + $stderr = $p.StandardError.ReadToEnd() + $env:JVM_ARCH = "32-bit" + } + + $sa = $stderr.Split("""") + $env:JVM_VERSION = $sa[1] + + if ($stderr.Contains("OpenJDK")) + { + $env:JVM_VENDOR = "OpenJDK" + } + elseif ($stderr.Contains("Java(TM)")) + { + $env:JVM_VENDOR = "Oracle" + } + else + { + $JVM_VENDOR = "other" + } + + $pa = $sa[1].Split("_") + $subVersion = $pa[1] + # Deal with -b (build) versions + if ($subVersion -contains '-') + { + $patchAndBuild = $subVersion.Split("-") + $subVersion = $patchAndBuild[0] + } + $env:JVM_PATCH_VERSION = $subVersion + } + + #----------------------------------------------------------------------------- + Function SetCassandraEnvironment + { + if (Test-Path Env:\JAVA_HOME) + { + $env:JAVA_BIN = "$env:JAVA_HOME\bin\java.exe" + } + elseif (Get-Command "java.exe") + { + $env:JAVA_BIN = "java.exe" + } + else + { + echo "ERROR! No JAVA_HOME set and could not find java.exe in the path." + exit + } + SetCassandraHome + $env:CASSANDRA_CONF = "$env:CASSANDRA_HOME\conf" + $env:CASSANDRA_PARAMS="-Dcassandra -Dlogback.configurationFile=logback.xml" + + $logdir = "$env:CASSANDRA_HOME\logs" + $storagedir = "$env:CASSANDRA_HOME\data" + $env:CASSANDRA_PARAMS = $env:CASSANDRA_PARAMS + " -Dcassandra.logdir=""$logdir"" -Dcassandra.storagedir=""$storagedir""" + + SetCassandraMain + BuildClassPath + + # Override these to set the amount of memory to allocate to the JVM at + # start-up. For production use you may wish to adjust this for your + # environment. MAX_HEAP_SIZE is the total amount of memory dedicated + # to the Java heap. HEAP_NEWSIZE refers to the size of the young + # generation. Both MAX_HEAP_SIZE and HEAP_NEWSIZE should be either set + # or not (if you set one, set the other). + # + # The main trade-off for the young generation is that the larger it + # is, the longer GC pause times will be. The shorter it is, the more + # expensive GC will be (usually). + # + # The example HEAP_NEWSIZE assumes a modern 8-core+ machine for decent + # times. If in doubt, and if you do not particularly want to tweak, go + # 100 MB per physical CPU core. + + #GC log path has to be defined here since it needs to find CASSANDRA_HOME + $env:JVM_OPTS="$env:JVM_OPTS -Xloggc:""$env:CASSANDRA_HOME/logs/gc.log""" + + # Read user-defined JVM options from jvm.options file + $content = Get-Content "$env:CASSANDRA_CONF\jvm.options" + for ($i = 0; $i -lt $content.Count; $i++) + { + $line = $content[$i] + if ($line.StartsWith("-")) + { + $env:JVM_OPTS = "$env:JVM_OPTS $line" + } + } + + $defined_xmn = $env:JVM_OPTS -like '*Xmn*' + $defined_xmx = $env:JVM_OPTS -like '*Xmx*' + $defined_xms = $env:JVM_OPTS -like '*Xms*' + $using_cms = $env:JVM_OPTS -like '*UseConcMarkSweepGC*' + + #$env:MAX_HEAP_SIZE="4096M" + #$env:HEAP_NEWSIZE="800M" + CalculateHeapSizes + + ParseJVMInfo + + # We only set -Xms and -Xmx if they were not defined on jvm.options file + # If defined, both Xmx and Xms should be defined together. + if (($defined_xmx -eq $false) -and ($defined_xms -eq $false)) + { + $env:JVM_OPTS="$env:JVM_OPTS -Xms$env:MAX_HEAP_SIZE" + $env:JVM_OPTS="$env:JVM_OPTS -Xmx$env:MAX_HEAP_SIZE" + } + elseif (($defined_xmx -eq $false) -or ($defined_xms -eq $false)) + { + echo "Please set or unset -Xmx and -Xms flags in pairs on jvm.options file." + exit + } + + # We only set -Xmn flag if it was not defined in jvm.options file + # and if the CMS GC is being used + # If defined, both Xmn and Xmx should be defined together. + if (($defined_xmn -eq $true) -and ($defined_xmx -eq $false)) + { + echo "Please set or unset -Xmx and -Xmn flags in pairs on jvm.options file." + exit + } + elseif (($defined_xmn -eq $false) -and ($using_cms -eq $true)) + { + $env:JVM_OPTS="$env:JVM_OPTS -Xmn$env:HEAP_NEWSIZE" + } + + if (($env:JVM_ARCH -eq "64-Bit") -and ($using_cms -eq $true)) + { + $env:JVM_OPTS="$env:JVM_OPTS -XX:+UseCondCardMark" + } + + # Add sigar env - see Cassandra-7838 + $env:JVM_OPTS = "$env:JVM_OPTS -Djava.library.path=""$env:CASSANDRA_HOME\lib\sigar-bin""" + + # Confirm we're on high performance power plan, warn if not + # Change to $true to suppress this warning + $suppressPowerWarning = $false + if (!$suppressPowerWarning) + { + $currentProfile = powercfg /GETACTIVESCHEME + if (!$currentProfile.Contains("High performance")) + { + echo "*---------------------------------------------------------------------*" + echo "*---------------------------------------------------------------------*" + echo "" + echo " WARNING! Detected a power profile other than High Performance." + echo " Performance of this node will suffer." + echo " Modify conf\cassandra.env.ps1 to suppress this warning." + echo "" + echo "*---------------------------------------------------------------------*" + echo "*---------------------------------------------------------------------*" + } + } + # provides hints to the JIT compiler $env:JVM_OPTS = "$env:JVM_OPTS -XX:CompileCommandFile=$env:CASSANDRA_CONF\hotspot_compiler" http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/conf/cassandra-env.sh ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/17e602b7/src/java/org/apache/cassandra/service/StartupChecks.java ---------------------------------------------------------------------- --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
