Thanks Josh/Keith! To answer Josh's questions, that column family only has few rows, the other column families that work fine, one of them also only has few rows, and another one has million of rows. Not using locality group or any other fancy configuration.
I tried 'debug on' per suggestion from Keith, and perhaps it's obvious to know the culprit from the output. Below are the snippets (the first one stopped after returning the result, the second one kept going indefinitely): /// on column family that works fine /// /// 2015-09-30 09:55:35,281 [shell.Shell] DEBUG: Found no scan iterators to set 2015-09-30 09:55:35,307 [impl.TabletLocatorImpl] TRACE: tid=12 oid=7 Locating tablet table=22 row= skipRow=false retry=false 2015-09-30 09:55:35,308 [impl.TabletLocatorImpl] TRACE: tid=12 oid=8 Locating tablet table=!0 row=22; skipRow=false retry=false 2015-09-30 09:55:35,308 [impl.RootTabletLocator] TRACE: tid=12 oid=9 Looking up root tablet location in zookeeper. 2015-09-30 09:55:35,310 [impl.RootTabletLocator] TRACE: tid=12 oid=9 Found root tablet at server01:10014|14eb7844d280183 in 0.001 secs 2015-09-30 09:55:35,315 [metadata.MetadataLocationObtainer] TRACE: tid=12 oid=10 Looking up in +r row=!0;22; extent=+r<< tserver=server01:10014 2015-09-30 09:55:35,320 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:55:35,359 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 562 2015-09-30 09:55:35,360 [metadata.MetadataLocationObtainer] TRACE: tid=12 oid=10 Got 4 results from +r<< in 0.045 secs 2015-09-30 09:55:35,362 [impl.TabletLocatorImpl] TRACE: tid=12 oid=8 Located tablet !0;~< at server01:10014 in 0.054 secs 2015-09-30 09:55:35,363 [metadata.MetadataLocationObtainer] TRACE: tid=12 oid=11 Looking up in !0 row=22; extent=!0;~< tserver=server01:10014 2015-09-30 09:55:35,363 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:55:35,369 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 720 2015-09-30 09:55:35,369 [metadata.MetadataLocationObtainer] TRACE: tid=12 oid=11 Got 8 results from !0;~< in 0.006 secs 2015-09-30 09:55:35,370 [impl.TabletLocatorImpl] TRACE: tid=12 oid=7 Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at server01:10014 in 0.063 secs 2015-09-30 09:55:35,370 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:55:35,371 [impl.ThriftScanner] TRACE: tid=12 oid=12 Starting scan tserver=server01:10014 tablet=22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< range=(-inf,+inf) ssil=[] ssio={} 2015-09-30 09:55:35,387 [impl.ThriftScanner] TRACE: tid=12 oid=12 Finished scanning tablet in 0.016 secs #results=0 2015-09-30 09:55:35,387 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 272 2015-09-30 09:55:35,387 [impl.TabletLocatorImpl] TRACE: tid=12 oid=13 Locating tablet table=22 row=sehttp://dbpedia.org/resource/2008%E2%80%9309_M skipRow=true retry=false 2015-09-30 09:55:35,387 [impl.TabletLocatorImpl] TRACE: tid=12 oid=13 Located tablet 22;sehttp://dbpedia.org/resource/James_Co;sehttp://dbpedia.org/resource/2008%E2%80%9309_M at server01:10014 in 0.000 secs 2015-09-30 09:55:35,388 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:55:35,388 [impl.ThriftScanner] TRACE: tid=12 oid=14 Starting scan tserver=server01:10014 tablet=22;sehttp://dbpedia.org/resource/James_Co;sehttp://dbpedia.org/resource/2008%E2%80%9309_M range=(-inf,+inf) ssil=[] ssio={} 2015-09-30 09:55:35,392 [impl.ThriftScanner] TRACE: tid=12 oid=14 Finished scanning tablet in 0.004 secs #results=0 2015-09-30 09:55:35,392 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 278 2015-09-30 09:55:35,392 [impl.TabletLocatorImpl] TRACE: tid=12 oid=15 Locating tablet table=22 row=sehttp://dbpedia.org/resource/James_Co skipRow=true retry=false 2015-09-30 09:55:35,393 [impl.TabletLocatorImpl] TRACE: tid=12 oid=15 Located tablet 22;sehttp://dbpedia.org/resource/Royal_L;sehttp://dbpedia.org/resource/James_Co at server01:10014 in 0.001 secs 2015-09-30 09:55:35,393 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:55:35,393 [impl.ThriftScanner] TRACE: tid=12 oid=16 Starting scan tserver=server01:10014 tablet=22;sehttp://dbpedia.org/resource/Royal_L;sehttp://dbpedia.org/resource/James_Co range=(-inf,+inf) ssil=[] ssio={} 2015-09-30 09:55:35,396 [impl.ThriftScanner] TRACE: tid=12 oid=16 Finished scanning tablet in 0.003 secs #results=0 2015-09-30 09:55:35,396 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 278 2015-09-30 09:55:35,396 [impl.TabletLocatorImpl] TRACE: tid=12 oid=17 Locating tablet table=22 row=sehttp://dbpedia.org/resource/Royal_L skipRow=true retry=false 2015-09-30 09:55:35,396 [impl.TabletLocatorImpl] TRACE: tid=12 oid=17 Located tablet 22<;sehttp://dbpedia.org/resource/Royal_L at server01:10014 in 0.000 secs 2015-09-30 09:55:35,397 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:55:35,397 [impl.ThriftScanner] TRACE: tid=12 oid=18 Starting scan tserver=server01:10014 tablet=22<;sehttp://dbpedia.org/resource/Royal_L range=(-inf,+inf) ssil=[] ssio={} 2015-09-30 09:55:35,410 [impl.ThriftScanner] TRACE: tid=12 oid=18 Completely finished scan in 0.013 secs #results=0 2015-09-30 09:55:35,410 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 272 2015-09-30 09:55:35,410 [admin.TableOperations] TRACE: tid=12 oid=19 Fetching list of tables... 2015-09-30 09:55:35,412 [admin.TableOperations] TRACE: tid=12 oid=19 Fetched 23 table names in 0.002 secs 2015-09-30 09:55:35,413 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:55:35,413 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 138 2015-09-30 09:55:35,413 [admin.TableOperations] TRACE: tid=12 oid=20 Fetching list of namespaces... 2015-09-30 09:55:35,414 [admin.TableOperations] TRACE: tid=12 oid=20 Fetched 2 namespaces in 0.001 secs /// on column family that hung /// /// 2015-09-30 09:59:33,609 [admin.TableOperations] TRACE: tid=12 oid=25 Checking if table BVAULT_RDF_STATS exists... 2015-09-30 09:59:33,612 [admin.TableOperations] TRACE: tid=12 oid=25 Checked existance of true in 0.002 secs 2015-09-30 09:59:33,613 [impl.ThriftTransportPool] TRACE: Creating new connection to connection to server01:10014 2015-09-30 09:59:33,628 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 4968 2015-09-30 09:59:33,628 [admin.TableOperations] TRACE: tid=12 oid=26 Checking if table BVAULT_RDF_STATS exists... 2015-09-30 09:59:33,630 [admin.TableOperations] TRACE: tid=12 oid=26 Checked existance of true in 0.002 secs 2015-09-30 09:59:33,630 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:33,644 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 4968 2015-09-30 09:59:33,644 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:33,658 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 4968 2015-09-30 09:59:33,658 [admin.TableOperations] TRACE: tid=12 oid=27 Checking if table BVAULT_RDF_STATS exists... 2015-09-30 09:59:33,660 [admin.TableOperations] TRACE: tid=12 oid=27 Checked existance of true in 0.002 secs 2015-09-30 09:59:33,661 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:33,675 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 4968 2015-09-30 09:59:33,675 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:33,676 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 130 2015-09-30 09:59:33,678 [shell.Shell] DEBUG: Found no scan iterators to set 2015-09-30 09:59:33,682 [impl.TabletLocatorImpl] TRACE: tid=12 oid=28 Locating tablet table=22 row= skipRow=false retry=false 2015-09-30 09:59:33,682 [impl.TabletLocatorImpl] TRACE: tid=12 oid=28 Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at server01:10014 in 0.000 secs 2015-09-30 09:59:33,682 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:33,682 [impl.ThriftScanner] TRACE: tid=12 oid=29 Starting scan tserver=server01:10014 tablet=22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< range=(-inf,+inf) ssil=[] ssio={} 2015-09-30 09:59:34,685 [impl.ThriftScanner] TRACE: tid=12 oid=29 Finished scan in 1.003 secs #results=0 scanid=4197848475135257903 2015-09-30 09:59:34,685 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 242 2015-09-30 09:59:34,685 [impl.TabletLocatorImpl] TRACE: tid=12 oid=30 Locating tablet table=22 row= skipRow=false retry=false 2015-09-30 09:59:34,686 [impl.TabletLocatorImpl] TRACE: tid=12 oid=30 Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at server01:10014 in 0.000 secs 2015-09-30 09:59:34,686 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:34,686 [impl.ThriftScanner] TRACE: tid=12 oid=31 Continuing scan tserver=server01:10014 scanid=4197848475135257903 2015-09-30 09:59:35,696 [impl.ThriftScanner] TRACE: tid=12 oid=31 Finished scan in 1.010 secs #results=0 scanid=4197848475135257903 2015-09-30 09:59:35,696 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 122 2015-09-30 09:59:35,696 [impl.TabletLocatorImpl] TRACE: tid=12 oid=32 Locating tablet table=22 row= skipRow=false retry=false 2015-09-30 09:59:35,697 [impl.TabletLocatorImpl] TRACE: tid=12 oid=32 Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at server01:10014 in 0.001 secs 2015-09-30 09:59:35,697 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:35,697 [impl.ThriftScanner] TRACE: tid=12 oid=33 Continuing scan tserver=server01:10014 scanid=4197848475135257903 2015-09-30 09:59:36,699 [impl.ThriftScanner] TRACE: tid=12 oid=33 Finished scan in 1.001 secs #results=0 scanid=4197848475135257903 2015-09-30 09:59:36,699 [impl.ThriftTransportPool] TRACE: Returned connection server01:10014 (120000) ioCount : 122 2015-09-30 09:59:36,699 [impl.TabletLocatorImpl] TRACE: tid=12 oid=34 Locating tablet table=22 row= skipRow=false retry=false 2015-09-30 09:59:36,699 [impl.TabletLocatorImpl] TRACE: tid=12 oid=34 Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at server01:10014 in 0.000 secs 2015-09-30 09:59:36,699 [impl.ThriftTransportPool] TRACE: Using existing connection to server01:10014 2015-09-30 09:59:36,699 [impl.ThriftScanner] TRACE: tid=12 oid=35 Continuing scan tserver=server01:10014 scanid=4197848475135257903 2015-09-30 09:59:37,701 [impl.ThriftScanner] TRACE: tid=12 oid=35 Finished scan in 1.002 secs #results=0 scanid=4197848475135257903 ... ... ... (keep going) -- View this message in context: http://apache-accumulo.1065345.n5.nabble.com/scan-command-hung-tp15286p15296.html Sent from the Developers mailing list archive at Nabble.com.
