[jira] [Updated] (IMPALA-6816) Statestore spends a lot of time in GetMinSubscriberTopicVersion()
[
https://issues.apache.org/jira/browse/IMPALA-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-6816:
---
Fix Version/s: Impala 2.13.0
> Statestore spends a lot of time in GetMinSubscriberTopicVersion()
> -
>
> Key: IMPALA-6816
> URL: https://issues.apache.org/jira/browse/IMPALA-6816
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Minor
> Labels: admission-control, statestore
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> {noformat}
> Samples: 13K of event 'cycles', Event count (approx.): 1200870513
> 20.23% statestored impalad [.]
> impala::Statestore::GetMinSubscriberTopicVersion(std::string const&,
> std::string*)
>7.68% statestored [kernel.kallsyms][k] find_busiest_group
>3.46% statestored impalad [.]
> impala::Statestore::Subscriber::LastTopicVersionProcessed(std::string const&)
> const
>3.26% statestored libc-2.12.so [.] __memcmp_sse4_1
>1.41% statestored [kernel.kallsyms][k] find_next_bit
>1.40% statestored [kernel.kallsyms][k] cpumask_next_and
>1.21% statestored libpthread-2.12.so [.] pthread_mutex_lock
>1.04% statestored libc-2.12.so [.] memcpy
>1.01% statestored [kernel.kallsyms][k] _spin_lock
>0.98% statestored impalad [.] 0x0088f903
>0.93% statestored impalad [.] 0x0088f8f5
>0.91% statestored impalad [.] 0x0088f8ea
>0.85% statestored [kernel.kallsyms][k] ixgbe_xmit_frame_ring
>0.77% statestored impalad [.] 0x0088f8e3
>0.75% statestored impalad [.] 0x0088f900
>0.75% statestored impalad [.]
> impala::Statestore::IsPrioritizedTopic(std::string const&)
>0.73% statestored impalad [.] 0x0088f8fa
>0.72% statestored impalad [.] operator new[](unsigned
> long)
>0.68% statestored [kernel.kallsyms][k] tcp_recvmsg
>0.67% statestored impalad [.] 0x0088f8fd
>0.66% statestored impalad [.]
> impala::Statestore::Topic::BuildDelta(std::string const&, long,
> impala::TTopicDelta*)
>0.61% statestored [kernel.kallsyms][k] thread_return
>0.60% statestored impalad [.] 0x0088f8f2
>0.60% statestored libstdc++.so.6 [.]
> std::string::compare(std::string const&) const
>0.59% statestored impalad [.] 0x0088f8e6
>0.56% statestored impalad [.] 0x0088f8ee
>0.56% statestored libcrypto.so.1.0.1e [.] aesni_encrypt
>0.55% statestored impalad [.] 0x0088f8e0
>0.55% statestored [kernel.kallsyms][k] tcp_transmit_skb
>0.53% statestored [kernel.kallsyms][k] fget_light
>0.51% statestored impalad [.] std::_Rb_tree std::pair >,
> std::_Select1st0.50% statestored impalad [.]
> apache::thrift::transport::TVirtualTransport apache::thrift::transport::TBufferBase>::readAll_virt(unsigned char*
>0.50% statestored impalad [.]
> impala::Statestore::DoSubscriberUpdate(impala::Statestore::UpdateKind, int,
> impala::Statestore::ScheduledSubscriberUpdate const&)
>0.49% statestored libssl.so.1.0.1e [.] tls1_enc
>0.48% statestored libssl.so.1.0.1e [.] ssl3_read_bytes
> {noformat}
> We are spending most of our time computing this for non-catalog topics, where
> it's not even used.
> There are a couple of ways we could fix this that I can think of:
> * Avoid including this information for topics where we're not interested in it
> * Cache or precompute the value somehow to avoid iterating over all
> subscribers every time
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6816) Statestore spends a lot of time in GetMinSubscriberTopicVersion()
[
https://issues.apache.org/jira/browse/IMPALA-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-6816:
--
Target Version: Product Backlog
> Statestore spends a lot of time in GetMinSubscriberTopicVersion()
> -
>
> Key: IMPALA-6816
> URL: https://issues.apache.org/jira/browse/IMPALA-6816
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Minor
> Labels: admission-control, statestore
>
> {noformat}
> Samples: 13K of event 'cycles', Event count (approx.): 1200870513
> 20.23% statestored impalad [.]
> impala::Statestore::GetMinSubscriberTopicVersion(std::string const&,
> std::string*)
>7.68% statestored [kernel.kallsyms][k] find_busiest_group
>3.46% statestored impalad [.]
> impala::Statestore::Subscriber::LastTopicVersionProcessed(std::string const&)
> const
>3.26% statestored libc-2.12.so [.] __memcmp_sse4_1
>1.41% statestored [kernel.kallsyms][k] find_next_bit
>1.40% statestored [kernel.kallsyms][k] cpumask_next_and
>1.21% statestored libpthread-2.12.so [.] pthread_mutex_lock
>1.04% statestored libc-2.12.so [.] memcpy
>1.01% statestored [kernel.kallsyms][k] _spin_lock
>0.98% statestored impalad [.] 0x0088f903
>0.93% statestored impalad [.] 0x0088f8f5
>0.91% statestored impalad [.] 0x0088f8ea
>0.85% statestored [kernel.kallsyms][k] ixgbe_xmit_frame_ring
>0.77% statestored impalad [.] 0x0088f8e3
>0.75% statestored impalad [.] 0x0088f900
>0.75% statestored impalad [.]
> impala::Statestore::IsPrioritizedTopic(std::string const&)
>0.73% statestored impalad [.] 0x0088f8fa
>0.72% statestored impalad [.] operator new[](unsigned
> long)
>0.68% statestored [kernel.kallsyms][k] tcp_recvmsg
>0.67% statestored impalad [.] 0x0088f8fd
>0.66% statestored impalad [.]
> impala::Statestore::Topic::BuildDelta(std::string const&, long,
> impala::TTopicDelta*)
>0.61% statestored [kernel.kallsyms][k] thread_return
>0.60% statestored impalad [.] 0x0088f8f2
>0.60% statestored libstdc++.so.6 [.]
> std::string::compare(std::string const&) const
>0.59% statestored impalad [.] 0x0088f8e6
>0.56% statestored impalad [.] 0x0088f8ee
>0.56% statestored libcrypto.so.1.0.1e [.] aesni_encrypt
>0.55% statestored impalad [.] 0x0088f8e0
>0.55% statestored [kernel.kallsyms][k] tcp_transmit_skb
>0.53% statestored [kernel.kallsyms][k] fget_light
>0.51% statestored impalad [.] std::_Rb_tree std::pair >,
> std::_Select1st0.50% statestored impalad [.]
> apache::thrift::transport::TVirtualTransport apache::thrift::transport::TBufferBase>::readAll_virt(unsigned char*
>0.50% statestored impalad [.]
> impala::Statestore::DoSubscriberUpdate(impala::Statestore::UpdateKind, int,
> impala::Statestore::ScheduledSubscriberUpdate const&)
>0.49% statestored libssl.so.1.0.1e [.] tls1_enc
>0.48% statestored libssl.so.1.0.1e [.] ssl3_read_bytes
> {noformat}
> We are spending most of our time computing this for non-catalog topics, where
> it's not even used.
> There are a couple of ways we could fix this that I can think of:
> * Avoid including this information for topics where we're not interested in it
> * Cache or precompute the value somehow to avoid iterating over all
> subscribers every time
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6816) Statestore spends a lot of time in GetMinSubscriberTopicVersion()
[
https://issues.apache.org/jira/browse/IMPALA-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-6816:
--
Labels: admission-control statestore (was: )
> Statestore spends a lot of time in GetMinSubscriberTopicVersion()
> -
>
> Key: IMPALA-6816
> URL: https://issues.apache.org/jira/browse/IMPALA-6816
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Minor
> Labels: admission-control, statestore
>
> {noformat}
> Samples: 13K of event 'cycles', Event count (approx.): 1200870513
> 20.23% statestored impalad [.]
> impala::Statestore::GetMinSubscriberTopicVersion(std::string const&,
> std::string*)
>7.68% statestored [kernel.kallsyms][k] find_busiest_group
>3.46% statestored impalad [.]
> impala::Statestore::Subscriber::LastTopicVersionProcessed(std::string const&)
> const
>3.26% statestored libc-2.12.so [.] __memcmp_sse4_1
>1.41% statestored [kernel.kallsyms][k] find_next_bit
>1.40% statestored [kernel.kallsyms][k] cpumask_next_and
>1.21% statestored libpthread-2.12.so [.] pthread_mutex_lock
>1.04% statestored libc-2.12.so [.] memcpy
>1.01% statestored [kernel.kallsyms][k] _spin_lock
>0.98% statestored impalad [.] 0x0088f903
>0.93% statestored impalad [.] 0x0088f8f5
>0.91% statestored impalad [.] 0x0088f8ea
>0.85% statestored [kernel.kallsyms][k] ixgbe_xmit_frame_ring
>0.77% statestored impalad [.] 0x0088f8e3
>0.75% statestored impalad [.] 0x0088f900
>0.75% statestored impalad [.]
> impala::Statestore::IsPrioritizedTopic(std::string const&)
>0.73% statestored impalad [.] 0x0088f8fa
>0.72% statestored impalad [.] operator new[](unsigned
> long)
>0.68% statestored [kernel.kallsyms][k] tcp_recvmsg
>0.67% statestored impalad [.] 0x0088f8fd
>0.66% statestored impalad [.]
> impala::Statestore::Topic::BuildDelta(std::string const&, long,
> impala::TTopicDelta*)
>0.61% statestored [kernel.kallsyms][k] thread_return
>0.60% statestored impalad [.] 0x0088f8f2
>0.60% statestored libstdc++.so.6 [.]
> std::string::compare(std::string const&) const
>0.59% statestored impalad [.] 0x0088f8e6
>0.56% statestored impalad [.] 0x0088f8ee
>0.56% statestored libcrypto.so.1.0.1e [.] aesni_encrypt
>0.55% statestored impalad [.] 0x0088f8e0
>0.55% statestored [kernel.kallsyms][k] tcp_transmit_skb
>0.53% statestored [kernel.kallsyms][k] fget_light
>0.51% statestored impalad [.] std::_Rb_tree std::pair >,
> std::_Select1st0.50% statestored impalad [.]
> apache::thrift::transport::TVirtualTransport apache::thrift::transport::TBufferBase>::readAll_virt(unsigned char*
>0.50% statestored impalad [.]
> impala::Statestore::DoSubscriberUpdate(impala::Statestore::UpdateKind, int,
> impala::Statestore::ScheduledSubscriberUpdate const&)
>0.49% statestored libssl.so.1.0.1e [.] tls1_enc
>0.48% statestored libssl.so.1.0.1e [.] ssl3_read_bytes
> {noformat}
> We are spending most of our time computing this for non-catalog topics, where
> it's not even used.
> There are a couple of ways we could fix this that I can think of:
> * Avoid including this information for topics where we're not interested in it
> * Cache or precompute the value somehow to avoid iterating over all
> subscribers every time
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
