[
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuqi Jin updated CASSANDRA-15794:
----------------------------------
Discovered By: User Report
Severity: Normal
Since Version: 3.11.4
Description:
We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the
following problems.
* We started a single 3.11.4 C* node.
* We ran cassandra-stress like this
{code:java}
./cassandra-stress write n = 30 -rate threads = 10 -node 172.17.0.2 {code}
* We stopped this node, and started a C* node running C* compiled from trunk
(git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
* New C* failed to start with the following error message
{code:java}
ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while
loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 -
Error while loading schema: java.lang.IllegalArgumentException: Compact Tables
are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... DROP
COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to migrate off
Compact Storage. at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
(java.lang.IllegalArgumentException) encountered during startup: Compact
Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520
CassandraDaemon.java:792 - Exception encountered during
startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in
Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE`
command supplied in 3.x/3.11 Cassandra in order to migrate off Compact Storage.
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
* We stopped the trunk version C* and started the 3.11.4 version C*.
* 3.11.4 C* failed to start with the following error messages:
{code:java}
Exception (java.lang.IllegalStateException) encountered during startup: Unknown
commitlog version 7Exception (java.lang.IllegalStateException) encountered
during startup: Unknown commitlog version 7ERROR [main] 2020-05-07 01:13:26,798
CassandraDaemon.java:749 - Exception encountered during
startupjava.lang.IllegalStateException: Unknown commitlog version 7 at
org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:167)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:148)
~[main/:na] at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324)
[main/:na] at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:620)
[main/:na] at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:732)
[main/:na]
{code}
We manually deleted all new version commit logs and it worked. However, it's
not obvious whether manually deleting these commit logs are safe.
* And when we run the "ALTER ... DROP COMPACT STORAGE" command on old version
node as the error message suggested, non-existent columns appeared with null
value.
original table:
{code:java}
key | C0
| C1
| C2
| C3 | C4
------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------
0x36384b30374f33314c30 |
0x3ab9b990a5dbbe3dbf0354d35bc32122ab09f963a9a29b04434fac1ab16af8992ed0 |
0xdd9f9d6c207f3720d01cc2f9ab4e83a5c59226c779de8cf79cfe8f1a6c5a092fbee4 |
0x9116a0bb80af7f73d803ceeb78f8aada9555e78936b26678202f6fb1f4993df92089 |
0xbde6fe3df138b0428bcec8ad4ab0dcf256b5bd57c2b6ac42523913522d78f710ee98 |
0x63f1cac68e33a7375a8370fbf300dcc70f76052c3ce84588f3b7d69ef6bc470db986
0x30374b37384e364c3531 |
0xce7990de95e1516101cbbd6ca3bdc2819e799c8f9b1bfd1b08aa1d1edf09dd409b7d |
0xc99b2076286ee4d4be742508653ed1178fb04192ae192d31745235e57dead6bf7f45 |
0xb492df82f1f2055af30694f135d3c99b0eac4e8d7d4d8e8b2d8ce49a9a3e50e3c63c |
0xc42bcb9b1a215a8d9629887bee918437fd580f0d15c48e1402fe11f6caab069e95aa |
0x329f193b16024ea72ace70571848e56b36496a05896454d13e1696c5c21053b5bcbb{code}
After dropping compact storage:
{code:java}
key | column1 | C0
| C1
| C2
| C3
| C4
| value
------------------------+---------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+-------
0x36384b30374f33314c30 | null |
0x3ab9b990a5dbbe3dbf0354d35bc32122ab09f963a9a29b04434fac1ab16af8992ed0 |
0xdd9f9d6c207f3720d01cc2f9ab4e83a5c59226c779de8cf79cfe8f1a6c5a092fbee4 |
0x9116a0bb80af7f73d803ceeb78f8aada9555e78936b26678202f6fb1f4993df92089 |
0xbde6fe3df138b0428bcec8ad4ab0dcf256b5bd57c2b6ac42523913522d78f710ee98 |
0x63f1cac68e33a7375a8370fbf300dcc70f76052c3ce84588f3b7d69ef6bc470db986 | null
0x30374b37384e364c3531 | null |
0xce7990de95e1516101cbbd6ca3bdc2819e799c8f9b1bfd1b08aa1d1edf09dd409b7d |
0xc99b2076286ee4d4be742508653ed1178fb04192ae192d31745235e57dead6bf7f45 |
0xb492df82f1f2055af30694f135d3c99b0eac4e8d7d4d8e8b2d8ce49a9a3e50e3c63c |
0xc42bcb9b1a215a8d9629887bee918437fd580f0d15c48e1402fe11f6caab069e95aa |
0x329f193b16024ea72ace70571848e56b36496a05896454d13e1696c5c21053b5bcbb | null
{code}
was:
We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the
following problems.
* We started a single 3.11.4 C* node.
* We ran cassandra-stress like this
{code:java}
./cassandra-stress write n = 30 -rate threads = 10 -node 172.17.0.2 {code}
* We stopped this node, and started a C* node running C* compiled from trunk
(git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
* New C* failed to start with the following error message
{code:java}
ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while
loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 -
Error while loading schema: java.lang.IllegalArgumentException: Compact Tables
are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... DROP
COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to migrate off
Compact Storage. at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
(java.lang.IllegalArgumentException) encountered during startup: Compact
Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520
CassandraDaemon.java:792 - Exception encountered during
startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in
Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE`
command supplied in 3.x/3.11 Cassandra in order to migrate off Compact Storage.
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
at
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
* We stopped the trunk version C* and started the 3.11.4 version C*.
* 3.11.4 C* failed to start with the following error messages:
{code:java}
Exception (java.lang.IllegalStateException) encountered during startup: Unknown
commitlog version 7Exception (java.lang.IllegalStateException) encountered
during startup: Unknown commitlog version 7ERROR [main] 2020-05-07 01:13:26,798
CassandraDaemon.java:749 - Exception encountered during
startupjava.lang.IllegalStateException: Unknown commitlog version 7 at
org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:167)
~[main/:na] at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:148)
~[main/:na] at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324)
[main/:na] at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:620)
[main/:na] at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:732)
[main/:na]
{code}
We manually deleted all new version commit logs and it worked. However, it's
not obvious whether manually deleting these commit logs are safe.
Summary: Upgraded C* (4.x) fail to start because of Compact Tables &
dropping compact tables in downgraded C* (3.11.4) introduces non-existent
columns (was: Upgraded C* (4.x) fail to start because of Compact Tables &
downgraded C* (3.11.4) fail to start because of commit log)
> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact
> tables in downgraded C* (3.11.4) introduces non-existent columns
> ---------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
> Issue Type: Bug
> Reporter: Zhuqi Jin
> Priority: Normal
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the
> following problems.
> * We started a single 3.11.4 C* node.
> * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node 172.17.0.2 {code}
> * We stopped this node, and started a C* node running C* compiled from trunk
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
> * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245
> - Error while loading schema: java.lang.IllegalArgumentException: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
> (java.lang.IllegalArgumentException) encountered during startup: Compact
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ...
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520
> CassandraDaemon.java:792 - Exception encountered during
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE`
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact
> Storage. at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
> at
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
> at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
> * We stopped the trunk version C* and started the 3.11.4 version C*.
> * 3.11.4 C* failed to start with the following error messages:
> {code:java}
> Exception (java.lang.IllegalStateException) encountered during startup:
> Unknown commitlog version 7Exception (java.lang.IllegalStateException)
> encountered during startup: Unknown commitlog version 7ERROR [main]
> 2020-05-07 01:13:26,798 CassandraDaemon.java:749 - Exception encountered
> during startupjava.lang.IllegalStateException: Unknown commitlog version 7 at
> org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:167)
> ~[main/:na] at
> org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:148)
> ~[main/:na] at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324)
> [main/:na] at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:620)
> [main/:na] at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:732)
> [main/:na]
> {code}
> We manually deleted all new version commit logs and it worked. However, it's
> not obvious whether manually deleting these commit logs are safe.
> * And when we run the "ALTER ... DROP COMPACT STORAGE" command on old
> version node as the error message suggested, non-existent columns appeared
> with null value.
> original table:
> {code:java}
> key | C0
> | C1
> | C2
> | C3
> | C4
> ------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------
> 0x36384b30374f33314c30 |
> 0x3ab9b990a5dbbe3dbf0354d35bc32122ab09f963a9a29b04434fac1ab16af8992ed0 |
> 0xdd9f9d6c207f3720d01cc2f9ab4e83a5c59226c779de8cf79cfe8f1a6c5a092fbee4 |
> 0x9116a0bb80af7f73d803ceeb78f8aada9555e78936b26678202f6fb1f4993df92089 |
> 0xbde6fe3df138b0428bcec8ad4ab0dcf256b5bd57c2b6ac42523913522d78f710ee98 |
> 0x63f1cac68e33a7375a8370fbf300dcc70f76052c3ce84588f3b7d69ef6bc470db986
> 0x30374b37384e364c3531 |
> 0xce7990de95e1516101cbbd6ca3bdc2819e799c8f9b1bfd1b08aa1d1edf09dd409b7d |
> 0xc99b2076286ee4d4be742508653ed1178fb04192ae192d31745235e57dead6bf7f45 |
> 0xb492df82f1f2055af30694f135d3c99b0eac4e8d7d4d8e8b2d8ce49a9a3e50e3c63c |
> 0xc42bcb9b1a215a8d9629887bee918437fd580f0d15c48e1402fe11f6caab069e95aa |
> 0x329f193b16024ea72ace70571848e56b36496a05896454d13e1696c5c21053b5bcbb{code}
> After dropping compact storage:
> {code:java}
> key | column1 | C0
> | C1
> | C2
> | C3
> | C4
> | value
> ------------------------+---------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+-------
> 0x36384b30374f33314c30 | null |
> 0x3ab9b990a5dbbe3dbf0354d35bc32122ab09f963a9a29b04434fac1ab16af8992ed0 |
> 0xdd9f9d6c207f3720d01cc2f9ab4e83a5c59226c779de8cf79cfe8f1a6c5a092fbee4 |
> 0x9116a0bb80af7f73d803ceeb78f8aada9555e78936b26678202f6fb1f4993df92089 |
> 0xbde6fe3df138b0428bcec8ad4ab0dcf256b5bd57c2b6ac42523913522d78f710ee98 |
> 0x63f1cac68e33a7375a8370fbf300dcc70f76052c3ce84588f3b7d69ef6bc470db986 | null
> 0x30374b37384e364c3531 | null |
> 0xce7990de95e1516101cbbd6ca3bdc2819e799c8f9b1bfd1b08aa1d1edf09dd409b7d |
> 0xc99b2076286ee4d4be742508653ed1178fb04192ae192d31745235e57dead6bf7f45 |
> 0xb492df82f1f2055af30694f135d3c99b0eac4e8d7d4d8e8b2d8ce49a9a3e50e3c63c |
> 0xc42bcb9b1a215a8d9629887bee918437fd580f0d15c48e1402fe11f6caab069e95aa |
> 0x329f193b16024ea72ace70571848e56b36496a05896454d13e1696c5c21053b5bcbb | null
> {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
