Riza Suminto created IMPALA-10984:
-------------------------------------
Summary: Improve performance of FROM_UNIXTIME function.
Key: IMPALA-10984
URL: https://issues.apache.org/jira/browse/IMPALA-10984
Project: IMPALA
Issue Type: Improvement
Components: Backend
Affects Versions: Impala 4.0.0
Reporter: Riza Suminto
Assignee: Riza Suminto
FROM_UNIXTIME function is implemented by calling TimestampValue::ToString() in
TimestampFunctions::FromUnix().
We found out that evaluation of TimestampValue::ToString() can get trapped in
tcmalloc::CentralFreeList lock, as shown in this pstack
{code:java}
#0 0x000000000277d81a in base::internal::SpinLockDelay(int volatile*, int, int)
()
#1 0x00000000027d17f9 in SpinLock::SlowLock() ()
#2 0x000000000287a399 in tcmalloc::CentralFreeList::RemoveRange(void**, void**,
int) ()
#3 0x00000000028882f3 in tcmalloc::ThreadCache::FetchFromCentralCache(unsigned
long, unsigned long) ()
#4 0x00000000029c5e88 in tc_newarray ()
#5 0x00007faedc677169 in std::string::_Rep::_S_create(unsigned long, unsigned
long, std::allocator<char> const&) () from
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p4948.16676264/lib/impala/lib/libstdc++.so.6
#6 0x0000000000f769de in impala::TimestampValue::ToString() const ()
#7 0x00007faeb317e08e in ?? ()
#8 0x00007fad62af6068 in ?? ()
#9 0x00007faedc8c20c0 in ?? () from
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p4948.16676264/lib/impala/lib/libstdc++.so.6
#10 0x0000000000000000 in ?? (){code}
This is presumably due to the combination use of stringstream,
boost::gregorian::to_iso_extended_string and
boost::posix_time::to_simple_string that involve multiple string allocation and
copying.
This can be problematic when FROM_UNIXTIME is being evaluated for millions of
rows.
We should come up with better implementation that involve less string
allocation and copying.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
