[jira] [Created] (KUDU-1560) Implement delta flush history GC

2016-08-12 Thread Mike Percy (JIRA)
Mike Percy created KUDU-1560:


 Summary: Implement delta flush history GC
 Key: KUDU-1560
 URL: https://issues.apache.org/jira/browse/KUDU-1560
 Project: Kudu
  Issue Type: Bug
  Components: tablet
Reporter: Mike Percy
Assignee: Mike Percy






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (KUDU-1559) blocks_open_writing metric shows thousands of blocks open at once

2016-08-12 Thread Todd Lipcon (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated KUDU-1559:
--
Status: In Review  (was: Open)

> blocks_open_writing metric shows thousands of blocks open at once
> -
>
> Key: KUDU-1559
> URL: https://issues.apache.org/jira/browse/KUDU-1559
> Project: Kudu
>  Issue Type: Bug
>  Components: fs, tserver
>Affects Versions: 0.10.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
> Attachments: graph.png
>
>
> The 'block_manager_blocks_open_writing' metric is showing tens of thousands 
> of blocks open for write on my tservers in a test cluster. That seems 
> incorrect to me, since we should only have blocks open in the middle of a 
> flush. Perhaps the metric is wrong, or perhaps we are somehow leaking open 
> blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (KUDU-1559) blocks_open_writing metric shows thousands of blocks open at once

2016-08-12 Thread Todd Lipcon (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon reassigned KUDU-1559:
-

Assignee: Todd Lipcon

> blocks_open_writing metric shows thousands of blocks open at once
> -
>
> Key: KUDU-1559
> URL: https://issues.apache.org/jira/browse/KUDU-1559
> Project: Kudu
>  Issue Type: Bug
>  Components: fs, tserver
>Affects Versions: 0.10.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
> Attachments: graph.png
>
>
> The 'block_manager_blocks_open_writing' metric is showing tens of thousands 
> of blocks open for write on my tservers in a test cluster. That seems 
> incorrect to me, since we should only have blocks open in the middle of a 
> flush. Perhaps the metric is wrong, or perhaps we are somehow leaking open 
> blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (KUDU-1558) Page says (incubating)

2016-08-12 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419617#comment-15419617
 ] 

Todd Lipcon commented on KUDU-1558:
---

This is because the /docs/ is from our last release, which was an incubating 
release. We'll have another release hopefully next week, at which point we'll 
update the docs from the new one.

> Page says (incubating)
> --
>
> Key: KUDU-1558
> URL: https://issues.apache.org/jira/browse/KUDU-1558
> Project: Kudu
>  Issue Type: Bug
>  Components: website
> Environment: http://kudu.apache.org/docs/
>Reporter: Sebb
>Priority: Minor
>
> The page http://kudu.apache.org/docs/ says (incubating)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (KUDU-921) TSTabletManager: Run tablet copy client on background thread

2016-08-12 Thread Todd Lipcon (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated KUDU-921:
-
Description: StartTabletCopy() should not run the whole tablet-copy 
procedure on the caller's thread.  (was: StartRemoteBootstrap() should not run 
the whole remote bootstrap procedure on the caller's thread.)
Summary: TSTabletManager: Run tablet copy client on background thread  
(was: TSTabletManager: Run remote bootstrap on background thread)

> TSTabletManager: Run tablet copy client on background thread
> 
>
> Key: KUDU-921
> URL: https://issues.apache.org/jira/browse/KUDU-921
> Project: Kudu
>  Issue Type: Bug
>  Components: consensus
>Affects Versions: Feature Complete
>Reporter: Mike Percy
>Assignee: Mike Percy
>Priority: Critical
>
> StartTabletCopy() should not run the whole tablet-copy procedure on the 
> caller's thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (KUDU-1500) TSAN race in ListTablets vs tablet metadata loading

2016-08-12 Thread Dinesh Bhat (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Bhat updated KUDU-1500:
--
Attachment: tablet_copy-itest.txt

Attached is a logfile generated from a new test added to exercise this data 
race(and fix).

> TSAN race in ListTablets vs tablet metadata loading
> ---
>
> Key: KUDU-1500
> URL: https://issues.apache.org/jira/browse/KUDU-1500
> Project: Kudu
>  Issue Type: Bug
>  Components: tablet, tserver
>Affects Versions: 0.9.0
>Reporter: Todd Lipcon
>Assignee: Dinesh Bhat
>  Labels: newbie
> Attachments: tablet_copy-itest.txt
>
>
> {code}
> WARNING: ThreadSanitizer: data race (pid=20066)
>   Write of size 8 at 0x7d4ccd90 by thread T71 (mutexes: write M4355):
> #0 std::vector std::allocator 
> >::_M_erase_at_end(kudu::PartitionSchema::HashBucketSchema*) 
> /data/1/todd/kudu/thirdparty/installed-deps-tsan/gcc/include/c++/4.9.3/bits/stl_vector.h:1439:26
>  (libkudu_common.so+0x00158983)
> #1 std::vector std::allocator >::clear() 
> /data/1/todd/kudu/thirdparty/installed-deps-tsan/gcc/include/c++/4.9.3/bits/stl_vector.h:1212:9
>  (libkudu_common.so+0x0013d721)
> #2 kudu::PartitionSchema::Clear() 
> /data/1/todd/kudu/build/tsan/../../src/kudu/common/partition.cc:876:3 
> (libkudu_common.so+0x0012f8fc)
> #3 kudu::PartitionSchema::FromPB(kudu::PartitionSchemaPB const&, 
> kudu::Schema const&, kudu::PartitionSchema*) 
> /data/1/todd/kudu/build/tsan/../../src/kudu/common/partition.cc:129:3 
> (libkudu_common.so+0x0012f4e6)
> #4 
> kudu::tablet::TabletMetadata::LoadFromSuperBlock(kudu::tablet::TabletSuperBlockPB
>  const&) 
> /data/1/todd/kudu/build/tsan/../../src/kudu/tablet/tablet_metadata.cc:298:7 
> (libtablet.so+0x003a4561)
> #5 
> kudu::tablet::TabletMetadata::ReplaceSuperBlock(kudu::tablet::TabletSuperBlockPB
>  const&) 
> /data/1/todd/kudu/build/tsan/../../src/kudu/tablet/tablet_metadata.cc:494:3 
> (libtablet.so+0x003a7805)
> #6 kudu::tserver::RemoteBootstrapClient::Start(std::string const&, 
> kudu::HostPort const&, scoped_refptr*) 
> /data/1/todd/kudu/build/tsan/../../src/kudu/tserver/remote_bootstrap_client.cc:222:5
>  (libtserver.so+0x001148ca)
> #7 
> kudu::tserver::TSTabletManager::StartRemoteBootstrap(kudu::consensus::StartRemoteBootstrapRequestPB
>  const&, boost::optional*) 
> /data/1/todd/kudu/build/tsan/../../src/kudu/tserver/ts_tablet_manager.cc:423:3
>  (libtserver.so+0x001aa273)
> #8 
> kudu::tserver::ConsensusServiceImpl::StartRemoteBootstrap(kudu::consensus::StartRemoteBootstrapRequestPB
>  const*, kudu::consensus::StartRemoteBootstrapResponsePB*, 
> kudu::rpc::RpcContext*) 
> /data/1/todd/kudu/build/tsan/../../src/kudu/tserver/tablet_service.cc:982:14 
> (libtserver.so+0x00175d03)
> #9 
> kudu::consensus::ConsensusServiceIf::ConsensusServiceIf(scoped_refptr
>  const&)::$_8::operator()(google::protobuf::Message const*, 
> google::protobuf::Message*, kudu::rpc::RpcContext*) const 
> /data/1/todd/kudu/build/tsan/src/kudu/consensus/consensus.service.cc:188:7 
> (libconsensus_proto.so+0x0009e457)
> #10 std::_Function_handler google::protobuf::Message*, kudu::rpc::RpcContext*), 
> kudu::consensus::ConsensusServiceIf::ConsensusServiceIf(scoped_refptr
>  const&)::$_8>::_M_invoke(std::_Any_data const&, google::protobuf::Message 
> const*, google::protobuf::Message*, kudu::rpc::RpcContext*) 
> /data/1/todd/kudu/thirdparty/installed-deps-tsan/gcc/include/c++/4.9.3/functional:2039:2
>  (libconsensus_proto.so+0x0009e17a)
> #11 std::function google::protobuf::Message*, 
> kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, 
> google::protobuf::Message*, kudu::rpc::RpcContext*) const 
> /data/1/todd/kudu/thirdparty/installed-deps-tsan/gcc/include/c++/4.9.3/functional:2439:14
>  (libkrpc.so+0x00177997)
> #12 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) 
> /data/1/todd/kudu/build/tsan/../../src/kudu/rpc/service_if.cc:94:3 
> (libkrpc.so+0x0017744c)
> #13 kudu::rpc::ServicePool::RunThread() 
> /data/1/todd/kudu/build/tsan/../../src/kudu/rpc/service_pool.cc:206:5 
> (libkrpc.so+0x0017a650)
> #14 boost::_mfi::mf0 kudu::rpc::ServicePool>::operator()(kudu::rpc::ServicePool*) const 
> /usr/include/boost/bind/mem_fn_template.hpp:49:29 (libkrpc.so+0x0017d55b)
> #15 void boost::_bi::list1 
> >::operator(), 
> boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0 kudu::rpc::ServicePool>&, boost::_bi::list0&, int) 
> /usr/include/boost/bind/bind.hpp:246:9 (libkrpc.so+0x0017d438)
> #16