Tim Armstrong has posted comments on this change.
Change subject: IMPALA-3735: Add per-fragment information to debug webpage
......................................................................
Patch Set 4:
(8 comments)
I found a crash from running a local stress test and clicking around in the web
UI a bit.
./tests/stress/concurrent_select.py --tpch-db=tpch_parquet
--fail-upon-successive-errors=100
Backtrace is:
#0 0x00007f4789d21267 in __GI_raise (sig=sig@entry=6) at
../sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007f4789d22eca in __GI_abort () at abort.c:89
#2 0x00007f4789d1a03d in __assert_fail_base (fmt=0x7f4789e7bfe8 "%s%s%s:%u:
%s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x2928e82 "px != 0",
file=file@entry=0x2928e38
"/opt/Impala-Toolchain/boost-1.57.0/include/boost/smart_ptr/scoped_ptr.hpp",
line=line@entry=99,
function=function@entry=0x292bb00
<boost::scoped_ptr<impala::Thread>::operator->() const::__PRETTY_FUNCTION__>
"T* boost::scoped_ptr<T>::operator->() const [with T = impala::Thread]") at
assert.c:92
#3 0x00007f4789d1a0f2 in __GI___assert_fail (assertion=0x2928e82 "px != 0",
file=0x2928e38
"/opt/Impala-Toolchain/boost-1.57.0/include/boost/smart_ptr/scoped_ptr.hpp",
line=99,
function=0x292bb00 <boost::scoped_ptr<impala::Thread>::operator->()
const::__PRETTY_FUNCTION__> "T* boost::scoped_ptr<T>::operator->() const [with
T = impala::Thread]")
at assert.c:101
#4 0x0000000001330365 in boost::scoped_ptr<impala::Thread>::operator->
(this=0x1ac25d978) at
/opt/Impala-Toolchain/boost-1.57.0/include/boost/smart_ptr/scoped_ptr.hpp:99
#5 0x000000000153c5c4 in impala::FragmentMgr::FragmentExecState::thread_id
(this=0x1ac25ce00) at be/src/service/fragment-exec-state.h:53
Python Exception <class 'ValueError'> Cannot find type const
impala::Webserver::ArgumentMap::_Rep_type:
#6 0x000000000153a6f5 in impala::FragmentMgr::FragmentsWebpage
(this=0xb379e40, args=std::map with 0 elements, document=0x7f460c3a33f0) at
be/src/service/fragment-mgr.cc:68
#7 0x000000000153bdf8 in
_ZZN6impala11FragmentMgrC4EPNS_7ExecEnvEENKUlRKT_PT0_E_clISt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEEN9rapidjson15GenericDocumentINSI_4UTF8IcEENSI_19MemoryPoolAllocatorINSI_12CrtAllocatorEEEEEEEDaS5_S7_
() at be/src/service/fragment-mgr.cc:39
#8 0x000000000153be2d in
boost::detail::function::void_function_obj_invoker2<impala::FragmentMgr::FragmentMgr(impala::ExecEnv*)::<lambda(const
auto:1&, auto:2*)>, void, const std::map<std::basic_string<char,
std::char_traits<char>, std::allocator<char> >, std::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char>
> >, std::allocator<std::pair<const std::basic_string<char,
std::char_traits<char>, std::allocator<char> >, std::basic_string<char,
std::char_traits<char>, std::allocator<char> > > > >&,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>
>*>::invoke(boost::detail::function::function_buffer &, const
std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char>
>, std::basic_string<char, std::char_traits<char>, std::allocator<char> >,
std::less<std::basic_string<char, std::char_traits<char>, std::allocator<!
char> > >, std::allocator<std::pairPython Exception <class 'ValueError'>
Cannot find type const std::map<std::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::less<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >,
std::allocator<std::pair<std::basic_string<char, std::char_traits<char>,
std::allocator<char> > const, std::basic_string<char, std::char_traits<char>,
std::allocator<char> > > > >::_Rep_type:
<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const,
std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > &,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > *)
(function_obj_ptr=..., a0=std::map with 0 elements, a1=0x7f460c3a33f0)
at
/opt/Impala-Toolchain/boost-1.57.0/include/boost/function/function_template.hpp:153
Python Exception <class 'ValueError'> Cannot find type const
std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char>
>, std::basic_string<char, std::char_traits<char>, std::allocator<char> >,
std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char>
> >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>,
std::allocator<char> > const, std::basic_string<char, std::char_traits<char>,
std::allocator<char> > > > >::_Rep_type:
#9 0x000000000160e688 in boost::function2<void, std::map<std::string,
std::string, std::less<std::string>, std::allocator<std::pair<std::string
const, std::string> > > const&,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >*>::operator()
(this=0xb312c30, a0=std::map with 0 elements,
a1=0x7f460c3a33f0) at
/opt/Impala-Toolchain/boost-1.57.0/include/boost/function/function_template.hpp:767
Python Exception <class 'ValueError'> Cannot find type const
impala::Webserver::ArgumentMap::_Rep_type:
#10 0x000000000160c211 in impala::Webserver::RenderUrlWithTemplate
(this=0xa7be5a0, arguments=std::map with 0 elements, url_handler=...,
output=0x7f460c3a39e0,
content_type=0x7f460c3a3b5c) at be/src/util/webserver.cc:423
#11 0x000000000160bd1e in impala::Webserver::BeginRequestCallback
(this=0xa7be5a0, connection=0x1f80af000, request_info=0x1f80af000) at
be/src/util/webserver.cc:400
#12 0x000000000160b7bf in impala::Webserver::BeginRequestCallbackStatic
(connection=0x1f80af000) at be/src/util/webserver.cc:359
#13 0x000000000161deb9 in handle_request (conn=0x1f80af000) at
be/src/thirdparty/squeasel/squeasel.c:3853
#14 0x000000000161fe56 in process_new_connection (conn=0x1f80af000) at
be/src/thirdparty/squeasel/squeasel.c:4450
#15 0x000000000162052b in worker_thread (thread_func_param=0xa11cc00) at
be/src/thirdparty/squeasel/squeasel.c:4581
#16 0x00007f478cc186aa in start_thread (arg=0x7f460c3a5700) at
pthread_create.c:333
#17 0x00007f4789df2e9d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109
http://gerrit.cloudera.org:8080/#/c/3323/4/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:
PS4, Line 629: rapidjson
Shouldn't be needed with "using namespace rapidjson"
PS4, Line 678: num_fragments_completed
Isn't this num_fragment_instances_completed?
Line 682: doc->AddMember("num_completed", num_fragments_completed,
doc->GetAllocator());
Should we be unambiguous here that it's num_instances_completed?
http://gerrit.cloudera.org:8080/#/c/3323/4/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:
Line 612: void ImpalaHttpHandler::QueryFragmentsHandler(const
Webserver::ArgumentMap& args,
I ran into some issues trying to get this to work. I was running:
select distinct * from tpch_20_parquet.lineitem;
The output was:
Fragment: 0 (0 complete of 0)
Show / Hide
Fragment: 1 (0 complete of 3)
Show / Hide
Fragment: 2 (0 complete of 3)
Show / Hide
This has two problems. First, I think we're showing the coordinator fragment
even though it doesn't have any fragment instances associated. Second, nothing
happened when I clicked "show/hide". This is on chrome on Linux. This is in the
console:
GET http://tarmstrong-box.ca.cloudera.com:25000/www/jquery/jquery-1.12.4.min.js
bootstrap.min.js:6 Uncaught Error: Bootstrap's JavaScript requires
jQuery(anonymous function) @ bootstrap.min.js:6
query_fragments?query_id=b44465bdf649dd15:64a7fa0703c4a19d:194 Uncaught
ReferenceError: $ is not defined(anonymous function) @
query_fragments?query_id=b44465bdf649dd15:64a7fa0703c4a19d:194
data:text/javascript;base64,ZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ3YnKS5vbmNsaWNrPWZ1bmN0aW9uKCl7d2luZG9…:1
Uncaught TypeError: Cannot set property 'onclick' of null(anonymous function)
@
data:text/javascript;base64,ZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ3YnKS5vbmNsaWNrPWZ1bmN0aW9uKCl7d2luZG9…:1
Do I need to rebase this on top of a different patch or something?
http://gerrit.cloudera.org:8080/#/c/3323/4/www/fragments.tmpl
File www/fragments.tmpl:
PS4, Line 32: 25000
It doesn't seem ideal to assume that the debug webpage is always on this port.
Is it difficult to propagate this info through the statestore somehow?
http://gerrit.cloudera.org:8080/#/c/3323/4/www/query_fragments.tmpl
File www/query_fragments.tmpl:
PS4, Line 64: InstanceID
Is there meant to be a space before ID?
Line 71: .append("<td><a href='http://"; + this.backend + ":25000'>"
+ this.backend + "</a></td>")
Long line
Line 78: xhr.open('GET',
"/query_fragments?query_id={{query_id}}&fragment_idx=" + idx + "&json", true);
Long line
--
To view, visit http://gerrit.cloudera.org:8080/3323
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I1d995da50c3a119b7aaf04d6f87e60e9e573a5ed
Gerrit-PatchSet: 4
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Henry Robinson <[email protected]>
Gerrit-Reviewer: Bharath Vissapragada <[email protected]>
Gerrit-Reviewer: Henry Robinson <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-HasComments: Yes
