[
https://issues.apache.org/jira/browse/IMPALA-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-6816:
----------------------------------
Target Version: Product Backlog
> Statestore spends a lot of time in GetMinSubscriberTopicVersion()
> -----------------------------------------------------------------
>
> Key: IMPALA-6816
> URL: https://issues.apache.org/jira/browse/IMPALA-6816
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
> Affects Versions: Impala 3.0, Impala 2.12.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Minor
> Labels: admission-control, statestore
>
> {noformat}
> Samples: 13K of event 'cycles', Event count (approx.): 1200870513
> 20.23% statestored impalad [.]
> impala::Statestore::GetMinSubscriberTopicVersion(std::string const&,
> std::string*)
> 7.68% statestored [kernel.kallsyms] [k] find_busiest_group
> 3.46% statestored impalad [.]
> impala::Statestore::Subscriber::LastTopicVersionProcessed(std::string const&)
> const
> 3.26% statestored libc-2.12.so [.] __memcmp_sse4_1
> 1.41% statestored [kernel.kallsyms] [k] find_next_bit
> 1.40% statestored [kernel.kallsyms] [k] cpumask_next_and
> 1.21% statestored libpthread-2.12.so [.] pthread_mutex_lock
> 1.04% statestored libc-2.12.so [.] memcpy
> 1.01% statestored [kernel.kallsyms] [k] _spin_lock
> 0.98% statestored impalad [.] 0x000000000088f903
> 0.93% statestored impalad [.] 0x000000000088f8f5
> 0.91% statestored impalad [.] 0x000000000088f8ea
> 0.85% statestored [kernel.kallsyms] [k] ixgbe_xmit_frame_ring
> 0.77% statestored impalad [.] 0x000000000088f8e3
> 0.75% statestored impalad [.] 0x000000000088f900
> 0.75% statestored impalad [.]
> impala::Statestore::IsPrioritizedTopic(std::string const&)
> 0.73% statestored impalad [.] 0x000000000088f8fa
> 0.72% statestored impalad [.] operator new[](unsigned
> long)
> 0.68% statestored [kernel.kallsyms] [k] tcp_recvmsg
> 0.67% statestored impalad [.] 0x000000000088f8fd
> 0.66% statestored impalad [.]
> impala::Statestore::Topic::BuildDelta(std::string const&, long,
> impala::TTopicDelta*)
> 0.61% statestored [kernel.kallsyms] [k] thread_return
> 0.60% statestored impalad [.] 0x000000000088f8f2
> 0.60% statestored libstdc++.so.6 [.]
> std::string::compare(std::string const&) const
> 0.59% statestored impalad [.] 0x000000000088f8e6
> 0.56% statestored impalad [.] 0x000000000088f8ee
> 0.56% statestored libcrypto.so.1.0.1e [.] aesni_encrypt
> 0.55% statestored impalad [.] 0x000000000088f8e0
> 0.55% statestored [kernel.kallsyms] [k] tcp_transmit_skb
> 0.53% statestored [kernel.kallsyms] [k] fget_light
> 0.51% statestored impalad [.] std::_Rb_tree<void*,
> std::pair<void* const, std::shared_ptr<impala::ThriftClientImpl> >,
> std::_Select1st<std::pair<void* const, std::shared_ptr<impala::ThriftCl
> 0.50% statestored impalad [.]
> apache::thrift::transport::TVirtualTransport<apache::thrift::transport::TBufferedTransport,
> apache::thrift::transport::TBufferBase>::readAll_virt(unsigned char*
> 0.50% statestored impalad [.]
> impala::Statestore::DoSubscriberUpdate(impala::Statestore::UpdateKind, int,
> impala::Statestore::ScheduledSubscriberUpdate const&)
> 0.49% statestored libssl.so.1.0.1e [.] tls1_enc
> 0.48% statestored libssl.so.1.0.1e [.] ssl3_read_bytes
> {noformat}
> We are spending most of our time computing this for non-catalog topics, where
> it's not even used.
> There are a couple of ways we could fix this that I can think of:
> * Avoid including this information for topics where we're not interested in it
> * Cache or precompute the value somehow to avoid iterating over all
> subscribers every time
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
