[jira] [Created] (CASSANDRA-14624) Website: Add blog post for Faster Streaming
Dinesh Joshi created CASSANDRA-14624: Summary: Website: Add blog post for Faster Streaming Key: CASSANDRA-14624 URL: https://issues.apache.org/jira/browse/CASSANDRA-14624 Project: Cassandra Issue Type: Task Components: Documentation and Website Reporter: Dinesh Joshi Assignee: Dinesh Joshi Attachments: faster-streaming-blog.patch Please add a new blog post entry on the Cassandra website. It describes the recent work on performance optimizations in Cassandra related to streaming enhancements. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14523:
--
Resolution: Fixed
Fix Version/s: (was: 4.x)
4.0
Status: Resolved (was: Patch Available)
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |10 | 0
> | 0 | 0
> scheduled_optional_tasks | 0 | 2147483647 | 5 | 8
> | 0 | 0
> scheduled_summary_builder | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_tasks | 0 | 2147483647 | 194 | 74
> | 0 | 0
>secondary_index_management | 0 | 1 | 0 | 0
> | 0 | 0
> validation_executor | 0 | 2147483647 | 0 | 0
> | 0 | 0
> view_build_executor | 0 | 1 | 0 | 0
> | 0 | 0
> view_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570571#comment-16570571
]
Aleksey Yeschenko commented on CASSANDRA-14523:
---
Committed to trunk as
[d5ae2ae481545b1fb2332b46013088f2f8cea636|https://github.com/apache/cassandra/commit/d5ae2ae481545b1fb2332b46013088f2f8cea636],
thanks.
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.x
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |10 | 0
> | 0 | 0
> scheduled_optional_tasks | 0 | 2147483647 | 5 | 8
> | 0 | 0
> scheduled_summary_builder | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_tasks | 0 | 2147483647 | 194 | 74
> | 0 | 0
>secondary_index_management | 0 | 1 | 0 | 0
> | 0 | 0
> validation_executor | 0 | 2147483647 | 0 | 0
> | 0 | 0
> view_build_executor | 0 | 1 | 0 | 0
> | 0 | 0
> view_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> {code}
--
This
cassandra git commit: Add a virtual table to expose thread pools
Repository: cassandra
Updated Branches:
refs/heads/trunk fcb87a2d1 -> d5ae2ae48
Add a virtual table to expose thread pools
patch by Chris Lohfink; reviewed by Aleksey Yeschenko for
CASSANDRA-14523
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d5ae2ae4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d5ae2ae4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d5ae2ae4
Branch: refs/heads/trunk
Commit: d5ae2ae481545b1fb2332b46013088f2f8cea636
Parents: fcb87a2
Author: Chris Lohfink
Authored: Fri Jun 22 10:48:01 2018 -0500
Committer: Aleksey Yeshchenko
Committed: Mon Aug 6 19:06:16 2018 +0100
--
CHANGES.txt | 1 +
.../DebuggableThreadPoolExecutor.java | 12 ++
.../JMXEnabledThreadPoolExecutor.java | 2 +-
.../concurrent/LocalAwareExecutorService.java | 46 -
.../cassandra/concurrent/SEPExecutor.java | 28 ++-
.../apache/cassandra/concurrent/SEPWorker.java | 2 +-
.../cassandra/concurrent/StageManager.java | 12 ++
.../db/virtual/SystemViewsKeyspace.java | 5 +-
.../cassandra/db/virtual/ThreadPoolsTable.java | 85 +
.../metrics/CassandraMetricsRegistry.java | 33 +++-
.../apache/cassandra/metrics/SEPMetrics.java| 109
.../metrics/ThreadPoolMetricNameFactory.java| 45 -
.../cassandra/metrics/ThreadPoolMetrics.java| 171 +++
.../org/apache/cassandra/tools/NodeProbe.java | 61 ++-
.../apache/cassandra/utils/StatusLogger.java| 25 ++-
.../org/apache/cassandra/cql3/ViewLongTest.java | 4 +-
.../apache/cassandra/cql3/ViewComplexTest.java | 4 +-
.../cassandra/cql3/ViewFilteringTest.java | 4 +-
.../apache/cassandra/cql3/ViewSchemaTest.java | 4 +-
.../org/apache/cassandra/cql3/ViewTest.java | 4 +-
20 files changed, 354 insertions(+), 303 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5ae2ae4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3b35608..103da7b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Add a virtual table to expose thread pools (CASSANDRA-14523)
* Add a virtual table to expose caches (CASSANDRA-14538)
* Fix toDate function for timestamp arguments (CASSANDRA-14502)
* Revert running dtests by default in circleci (CASSANDRA-14614)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5ae2ae4/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 92cbbf4..ef0f43c 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -213,6 +213,18 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
super.beforeExecute(t, r);
}
+@Override
+public int getActiveTaskCount()
+{
+return getActiveCount();
+}
+
+@Override
+public int getPendingTaskCount()
+{
+return getQueue().size();
+}
+
/**
* Send @param t and any exception wrapped by @param r to the default
uncaught exception handler,
* or log them if none such is set up
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5ae2ae4/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
index 2dafb4f..278b399 100644
--- a/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
@@ -79,7 +79,7 @@ public class JMXEnabledThreadPoolExecutor extends
DebuggableThreadPoolExecutor i
{
super(corePoolSize, maxPoolSize, keepAliveTime, unit, workQueue,
threadFactory);
super.prestartAllCoreThreads();
-metrics = new ThreadPoolMetrics(this, jmxPath, threadFactory.id);
+metrics = new ThreadPoolMetrics(this, jmxPath,
threadFactory.id).register();
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
mbeanName = "org.apache.cassandra." + jmxPath + ":type=" +
threadFactory.id;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5ae2ae4/src/java/org/apache/cassandra/concurrent/LocalAwareExecutorService.java
[jira] [Commented] (CASSANDRA-14466) Enable Direct I/O
[ https://issues.apache.org/jira/browse/CASSANDRA-14466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570483#comment-16570483 ] Mulugeta Mammo commented on CASSANDRA-14466: [~snazy] now that JDK-11 support has been resolved, can we review this patch again? > Enable Direct I/O > -- > > Key: CASSANDRA-14466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14466 > Project: Cassandra > Issue Type: New Feature > Components: Local Write-Read Paths >Reporter: Mulugeta Mammo >Priority: Major > Attachments: direct_io.patch > > > Hi, > JDK 10 introduced a new API for Direct IO that enables applications to bypass > the file system cache and potentially improve performance. Details of this > feature can be found at [https://bugs.openjdk.java.net/browse/JDK-8164900]. > This patch uses the JDK 10 API to enable Direct IO for the Cassandra read > path. By default, we have disabled this feature; but it can be enabled using > a new configuration parameter, enable_direct_io_for_read_path. We have > conducted a Cassandra read-only stress test and measured a throughput gain of > up to 60% on flash drives. > The patch requires JDK 10 Cassandra Support - > https://issues.apache.org/jira/browse/CASSANDRA-9608 > Please review the patch and let us know your feedback. > Thanks, > [^direct_io.patch] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14522) sstableloader should use discovered broadcast address to connect intra-cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14522:
---
Fix Version/s: 3.11.4
> sstableloader should use discovered broadcast address to connect intra-cluster
> --
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
> Fix For: 3.0.18, 3.11.4
>
> Attachments: CASSANDRA-14522.patch
>
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
[jira] [Updated] (CASSANDRA-14522) sstableloader should use discovered broadcast address to connect intra-cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14522:
---
Fix Version/s: 3.0.18
> sstableloader should use discovered broadcast address to connect intra-cluster
> --
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
> Fix For: 3.0.18
>
> Attachments: CASSANDRA-14522.patch
>
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
>
[jira] [Created] (CASSANDRA-14623) sstableloader should use discovered broadcast address to connect intra-cluster in 2.2
Ariel Weisberg created CASSANDRA-14623: -- Summary: sstableloader should use discovered broadcast address to connect intra-cluster in 2.2 Key: CASSANDRA-14623 URL: https://issues.apache.org/jira/browse/CASSANDRA-14623 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Ariel Weisberg CASSANDRA-14522 fixes this in 3.0 and 3.11. It wasn't a bug in trunk at the time. Currently it connects on the interface specified the clients to connect to and not the storage interface. If they are different it selects the wrong interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14522) sstableloader should use discovered broadcast address to connect intra-cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14522:
---
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as
[3d48cbc74cb50e02e85ed51ccf6a3a46690c3a99|https://github.com/apache/cassandra/commit/3d48cbc74cb50e02e85ed51ccf6a3a46690c3a99].
Thanks!
> sstableloader should use discovered broadcast address to connect intra-cluster
> --
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
> Attachments: CASSANDRA-14522.patch
>
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
>
[2/6] cassandra git commit: sstableloader should use discovered broadcast address to connect intra-cluster
sstableloader should use discovered broadcast address to connect intra-cluster Patch by Jeremy Ow; Reviewed by Ariel Weisberg Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d48cbc7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d48cbc7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d48cbc7 Branch: refs/heads/cassandra-3.11 Commit: 3d48cbc74cb50e02e85ed51ccf6a3a46690c3a99 Parents: c4f1f408 Author: Jeremy Ow Authored: Thu Aug 2 12:14:18 2018 -0400 Committer: Ariel Weisberg Committed: Mon Aug 6 12:04:54 2018 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d48cbc7/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index e4dbbda..7e4d88c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.18 + * sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522) * Fix reading columns with non-UTF names from schema (CASSANDRA-14468) Merged from 2.2: * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377) http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d48cbc7/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java -- diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java index 3e0d97b..1d65aa0 100644 --- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java +++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java @@ -85,7 +85,7 @@ public class NativeSSTableLoaderClient extends SSTableLoader.Client Range range = new Range<>(tokenFactory.fromString(tokenRange.getStart().getValue().toString()), tokenFactory.fromString(tokenRange.getEnd().getValue().toString())); for (Host endpoint : endpoints) -addRangeForEndpoint(range, endpoint.getAddress()); +addRangeForEndpoint(range, endpoint.getBroadcastAddress()); } Types types = fetchTypes(keyspace, session); - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f4c390b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f4c390b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f4c390b Branch: refs/heads/cassandra-3.11 Commit: 4f4c390bc105c5e407bfe8b04c282ac28862e451 Parents: f8acbd2 3d48cbc Author: Ariel Weisberg Authored: Mon Aug 6 12:06:46 2018 -0400 Committer: Ariel Weisberg Committed: Mon Aug 6 12:06:46 2018 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f4c390b/CHANGES.txt -- diff --cc CHANGES.txt index ab64ab9,7e4d88c..e2d1681 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,5 -1,5 +1,6 @@@ -3.0.18 +3.11.4 +Merged from 3.0: + * sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522) * Fix reading columns with non-UTF names from schema (CASSANDRA-14468) Merged from 2.2: * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f4c390b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[1/6] cassandra git commit: sstableloader should use discovered broadcast address to connect intra-cluster
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 c4f1f4080 -> 3d48cbc74 refs/heads/cassandra-3.11 f8acbd278 -> 4f4c390bc refs/heads/trunk 45c3b461e -> fcb87a2d1 sstableloader should use discovered broadcast address to connect intra-cluster Patch by Jeremy Ow; Reviewed by Ariel Weisberg Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d48cbc7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d48cbc7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d48cbc7 Branch: refs/heads/cassandra-3.0 Commit: 3d48cbc74cb50e02e85ed51ccf6a3a46690c3a99 Parents: c4f1f408 Author: Jeremy Ow Authored: Thu Aug 2 12:14:18 2018 -0400 Committer: Ariel Weisberg Committed: Mon Aug 6 12:04:54 2018 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d48cbc7/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index e4dbbda..7e4d88c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.18 + * sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522) * Fix reading columns with non-UTF names from schema (CASSANDRA-14468) Merged from 2.2: * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377) http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d48cbc7/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java -- diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java index 3e0d97b..1d65aa0 100644 --- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java +++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java @@ -85,7 +85,7 @@ public class NativeSSTableLoaderClient extends SSTableLoader.Client Range range = new Range<>(tokenFactory.fromString(tokenRange.getStart().getValue().toString()), tokenFactory.fromString(tokenRange.getEnd().getValue().toString())); for (Host endpoint : endpoints) -addRangeForEndpoint(range, endpoint.getAddress()); +addRangeForEndpoint(range, endpoint.getBroadcastAddress()); } Types types = fetchTypes(keyspace, session); - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f4c390b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f4c390b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f4c390b Branch: refs/heads/trunk Commit: 4f4c390bc105c5e407bfe8b04c282ac28862e451 Parents: f8acbd2 3d48cbc Author: Ariel Weisberg Authored: Mon Aug 6 12:06:46 2018 -0400 Committer: Ariel Weisberg Committed: Mon Aug 6 12:06:46 2018 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f4c390b/CHANGES.txt -- diff --cc CHANGES.txt index ab64ab9,7e4d88c..e2d1681 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,5 -1,5 +1,6 @@@ -3.0.18 +3.11.4 +Merged from 3.0: + * sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522) * Fix reading columns with non-UTF names from schema (CASSANDRA-14468) Merged from 2.2: * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f4c390b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fcb87a2d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fcb87a2d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fcb87a2d Branch: refs/heads/trunk Commit: fcb87a2d1c56418d3cc33bd0df36f243c6f1566b Parents: 45c3b46 4f4c390 Author: Ariel Weisberg Authored: Mon Aug 6 12:07:08 2018 -0400 Committer: Ariel Weisberg Committed: Mon Aug 6 12:07:08 2018 -0400 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[3/6] cassandra git commit: sstableloader should use discovered broadcast address to connect intra-cluster
sstableloader should use discovered broadcast address to connect intra-cluster Patch by Jeremy Ow; Reviewed by Ariel Weisberg Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d48cbc7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d48cbc7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d48cbc7 Branch: refs/heads/trunk Commit: 3d48cbc74cb50e02e85ed51ccf6a3a46690c3a99 Parents: c4f1f408 Author: Jeremy Ow Authored: Thu Aug 2 12:14:18 2018 -0400 Committer: Ariel Weisberg Committed: Mon Aug 6 12:04:54 2018 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d48cbc7/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index e4dbbda..7e4d88c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.18 + * sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522) * Fix reading columns with non-UTF names from schema (CASSANDRA-14468) Merged from 2.2: * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377) http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d48cbc7/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java -- diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java index 3e0d97b..1d65aa0 100644 --- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java +++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java @@ -85,7 +85,7 @@ public class NativeSSTableLoaderClient extends SSTableLoader.Client Range range = new Range<>(tokenFactory.fromString(tokenRange.getStart().getValue().toString()), tokenFactory.fromString(tokenRange.getEnd().getValue().toString())); for (Host endpoint : endpoints) -addRangeForEndpoint(range, endpoint.getAddress()); +addRangeForEndpoint(range, endpoint.getBroadcastAddress()); } Types types = fetchTypes(keyspace, session); - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14522) sstableloader should use discovered broadcast address to connect intracluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14522:
---
Summary: sstableloader should use discovered broadcast address to connect
intracluster (was: sstableloader should use discovered broadcast address to
connect to intracluster)
> sstableloader should use discovered broadcast address to connect intracluster
> -
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
> Attachments: CASSANDRA-14522.patch
>
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
>
[jira] [Updated] (CASSANDRA-14522) sstableloader should use discovered broadcast address to connect intra-cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14522:
---
Summary: sstableloader should use discovered broadcast address to connect
intra-cluster (was: sstableloader should use discovered broadcast address to
connect intracluster)
> sstableloader should use discovered broadcast address to connect intra-cluster
> --
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
> Attachments: CASSANDRA-14522.patch
>
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
>
[jira] [Comment Edited] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570370#comment-16570370
]
Aleksey Yeschenko edited comment on CASSANDRA-14523 at 8/6/18 3:53 PM:
---
[~cnlwsu] made most of the requested changes. I pushed a bit more
[here|https://github.com/iamaleksey/cassandra/commits/14523-4.0]. It cleans up
existing things a bit, makes them more consistent, and implements an override
for single-partition-key reads, among other things.
It's going through CI now,
[here|https://circleci.com/workflow-run/f3956836-29d7-4570-aef2-07fa2b6cae11].
If CI is happy, and you are ok with the latest changes, [~cnlwsu], I'll commit
this version.
was (Author: iamaleksey):
[~cnlwsu] made most of the requested changes. I pushed a bit more
[here|https://github.com/iamaleksey/cassandra/commits/14523-4.0]. It cleans up
existing things a bit, makes them more consistent, and implements an override
for single-partition-key reads, among other things.
It's going through CI now,
[here|https://circleci.com/workflow-run/92851a7d-3846-4233-a8b3-7a06a305ad44].
If CI is happy, and you are ok with the latest changes, [~cnlwsu], I'll commit
this version.
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.x
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |
[jira] [Commented] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570370#comment-16570370
]
Aleksey Yeschenko commented on CASSANDRA-14523:
---
[~cnlwsu] made most of the requested changes. I pushed a bit more
[here|https://github.com/iamaleksey/cassandra/commits/14523-4.0]. It cleans up
existing things a bit, makes them more consistent, and implements an override
for single-partition-key reads, among other things.
It's going through CI now,
[here|https://circleci.com/workflow-run/92851a7d-3846-4233-a8b3-7a06a305ad44].
If CI is happy, and you are ok with the latest changes, [~cnlwsu], I'll commit
this version.
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.x
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |10 | 0
> | 0 | 0
> scheduled_optional_tasks | 0 | 2147483647 | 5 | 8
> | 0 | 0
> scheduled_summary_builder | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_tasks | 0 | 2147483647 | 194 | 74
> | 0 | 0
>secondary_index_management | 0 | 1 | 0 | 0
> | 0 | 0
> validation_executor | 0 |
[jira] [Commented] (CASSANDRA-14522) sstableloader should use discovered broadcast address to connect to intracluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570346#comment-16570346
]
Ariel Weisberg commented on CASSANDRA-14522:
I'm going to commit and close this for 3.0 and 3.11 and create a separate
ticket for 2.2 since it is pretty involved to figure out how to do it right
there.
[3.0
CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/cassandra-14522-3.0]
[3.11
CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/cassandra-14522-3.11]
dtest failures appear unrelated.
> sstableloader should use discovered broadcast address to connect to
> intracluster
>
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
> Attachments: CASSANDRA-14522.patch
>
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
>
[jira] [Updated] (CASSANDRA-14522) sstableloader should use discovered broadcast address to connect to intracluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-14522:
---
Status: Ready to Commit (was: Patch Available)
> sstableloader should use discovered broadcast address to connect to
> intracluster
>
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
> Attachments: CASSANDRA-14522.patch
>
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
>
[jira] [Commented] (CASSANDRA-14622) Run compaction on cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569863#comment-16569863
]
Wasil Shaikh commented on CASSANDRA-14622:
--
You mean, i should draft the concerns & mail it.
> Run compaction on cluster
> --
>
> Key: CASSANDRA-14622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14622
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: CentOS Linux release 7.1.1503 (Core)
>Reporter: Wasil Shaikh
>Priority: Major
> Labels: Cassandra
>
> Dear Team,
> I have setup a cassandra cluster for 3 m/c.
> [root@ip-172-31-21-222 ~]# nodetool status
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Tokens Owns Host ID Rack
> UN 172.31.21.223 4.01 TB 256 ? 9d2a408a-264f-423f-8af2-e1d4cb608998 rack1
> UN 172.31.21.222 4.5 TB 256 ? c3d24437-3e03-41d7-a2af-2ed53bcf2dfc rack1
> UN 172.31.21.221 4.26 TB 256 ? aaf4fbaf-837b-44e2-98ba-5eb2b3e6b62b rack1
> Note: Non-system keyspaces don't have the same replication settings,
> effective ownership information is meaningless
> [root@ip-172-31-21-222 ~]#
>
> Now i have deleted around 40-45 GB data but space seems is not vacant.
> I'm using SizeTieredCompactionStrategy and below is my table defination.
>
> CREATE TABLE test.att_103 (
> md5 text PRIMARY KEY,
> arcsize int,
> att_str blob,
> attsize int,
> udate timestamp
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '\{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = \{'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = \{'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> I have dumped data using sstable2json command to verify whether my column
> data is deleted or not.
> e.g
> {"key": "9b7937de43aa22fc3bf18ec72d86140b",
> "cells": [["att_str",1533305633,1533305633552000,"d"],
> ["udate","2018-08-03 14:13Z",1533305633552000]]}
>
> Now can you help me here regarding how to run compaction manually instead of
> waiting for 10 days of grace period.
>
> Thanks
> Wasil.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14622) Run compaction on cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson resolved CASSANDRA-14622.
-
Resolution: Invalid
Fix Version/s: (was: 2.2.1)
questions like this should go to u...@cassandra.apache.org
> Run compaction on cluster
> --
>
> Key: CASSANDRA-14622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14622
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: CentOS Linux release 7.1.1503 (Core)
>Reporter: Wasil Shaikh
>Priority: Major
> Labels: Cassandra
>
> Dear Team,
> I have setup a cassandra cluster for 3 m/c.
> [root@ip-172-31-21-222 ~]# nodetool status
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Tokens Owns Host ID Rack
> UN 172.31.21.223 4.01 TB 256 ? 9d2a408a-264f-423f-8af2-e1d4cb608998 rack1
> UN 172.31.21.222 4.5 TB 256 ? c3d24437-3e03-41d7-a2af-2ed53bcf2dfc rack1
> UN 172.31.21.221 4.26 TB 256 ? aaf4fbaf-837b-44e2-98ba-5eb2b3e6b62b rack1
> Note: Non-system keyspaces don't have the same replication settings,
> effective ownership information is meaningless
> [root@ip-172-31-21-222 ~]#
>
> Now i have deleted around 40-45 GB data but space seems is not vacant.
> I'm using SizeTieredCompactionStrategy and below is my table defination.
>
> CREATE TABLE test.att_103 (
> md5 text PRIMARY KEY,
> arcsize int,
> att_str blob,
> attsize int,
> udate timestamp
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '\{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = \{'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = \{'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> I have dumped data using sstable2json command to verify whether my column
> data is deleted or not.
> e.g
> {"key": "9b7937de43aa22fc3bf18ec72d86140b",
> "cells": [["att_str",1533305633,1533305633552000,"d"],
> ["udate","2018-08-03 14:13Z",1533305633552000]]}
>
> Now can you help me here regarding how to run compaction manually instead of
> waiting for 10 days of grace period.
>
> Thanks
> Wasil.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14622) Run compaction on cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wasil Shaikh updated CASSANDRA-14622:
-
Labels: Cassandra (was: )
> Run compaction on cluster
> --
>
> Key: CASSANDRA-14622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14622
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: CentOS Linux release 7.1.1503 (Core)
>Reporter: Wasil Shaikh
>Priority: Major
> Labels: Cassandra
> Fix For: 2.2.1
>
>
> Dear Team,
> I have setup a cassandra cluster for 3 m/c.
> [root@ip-172-31-21-222 ~]# nodetool status
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Tokens Owns Host ID Rack
> UN 172.31.21.223 4.01 TB 256 ? 9d2a408a-264f-423f-8af2-e1d4cb608998 rack1
> UN 172.31.21.222 4.5 TB 256 ? c3d24437-3e03-41d7-a2af-2ed53bcf2dfc rack1
> UN 172.31.21.221 4.26 TB 256 ? aaf4fbaf-837b-44e2-98ba-5eb2b3e6b62b rack1
> Note: Non-system keyspaces don't have the same replication settings,
> effective ownership information is meaningless
> [root@ip-172-31-21-222 ~]#
>
> Now i have deleted around 40-45 GB data but space seems is not vacant.
> I'm using SizeTieredCompactionStrategy and below is my table defination.
>
> CREATE TABLE test.att_103 (
> md5 text PRIMARY KEY,
> arcsize int,
> att_str blob,
> attsize int,
> udate timestamp
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '\{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = \{'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = \{'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> I have dumped data using sstable2json command to verify whether my column
> data is deleted or not.
> e.g
> {"key": "9b7937de43aa22fc3bf18ec72d86140b",
> "cells": [["att_str",1533305633,1533305633552000,"d"],
> ["udate","2018-08-03 14:13Z",1533305633552000]]}
>
> Now can you help me here regarding how to run compaction manually instead of
> waiting for 10 days of grace period.
>
> Thanks
> Wasil.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14622) Run compaction on cluster
Wasil Shaikh created CASSANDRA-14622:
Summary: Run compaction on cluster
Key: CASSANDRA-14622
URL: https://issues.apache.org/jira/browse/CASSANDRA-14622
Project: Cassandra
Issue Type: Bug
Components: CQL
Environment: CentOS Linux release 7.1.1503 (Core)
Reporter: Wasil Shaikh
Fix For: 2.2.1
Dear Team,
I have setup a cassandra cluster for 3 m/c.
[root@ip-172-31-21-222 ~]# nodetool status
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 172.31.21.223 4.01 TB 256 ? 9d2a408a-264f-423f-8af2-e1d4cb608998 rack1
UN 172.31.21.222 4.5 TB 256 ? c3d24437-3e03-41d7-a2af-2ed53bcf2dfc rack1
UN 172.31.21.221 4.26 TB 256 ? aaf4fbaf-837b-44e2-98ba-5eb2b3e6b62b rack1
Note: Non-system keyspaces don't have the same replication settings, effective
ownership information is meaningless
[root@ip-172-31-21-222 ~]#
Now i have deleted around 40-45 GB data but space seems is not vacant.
I'm using SizeTieredCompactionStrategy and below is my table defination.
CREATE TABLE test.att_103 (
md5 text PRIMARY KEY,
arcsize int,
att_str blob,
attsize int,
udate timestamp
) WITH bloom_filter_fp_chance = 0.01
AND caching = '\{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = \{'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = \{'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
I have dumped data using sstable2json command to verify whether my column data
is deleted or not.
e.g
{"key": "9b7937de43aa22fc3bf18ec72d86140b",
"cells": [["att_str",1533305633,1533305633552000,"d"],
["udate","2018-08-03 14:13Z",1533305633552000]]}
Now can you help me here regarding how to run compaction manually instead of
waiting for 10 days of grace period.
Thanks
Wasil.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
