Adar Dembo created KUDU-2716:
--------------------------------
Summary: rapidjson mishandles kMinInt32 and kMinInt64
Key: KUDU-2716
URL: https://issues.apache.org/jira/browse/KUDU-2716
Project: Kudu
Issue Type: Bug
Components: util
Affects Versions: 1.9.0
Reporter: Adar Dembo
The rapidjson code that parses numbers mishandles kMinInt32 and kMinInt64.
Moreover, it only appears to do so when compiled with -O3 (the default for
RELEASE builds) by certain compilers. I could not reproduce the mishandling
when compiling with no optimizations (DEBUG) or with -O1 (FASTDEBUG).
I can't speak to kMinInt64, but when parsing kMinInt32, rapidjson identifies
the number as a possible unsigned, unsigned long, signed, or signed long,
instead of just a possible signed or signed long. Currently, this manifests as
a SIGSEGV in jsonreader-test.cc:
{noformat}
*** Aborted at 1551300683 (unix time) try "date -d @1551300683" if you are
using GNU date ***
PC: @ 0x55d7ab64ae19 kudu::JsonReader::ExtractUint32()
*** SIGSEGV (@0x0) received by PID 10903 (TID 0x7f578ebdf840) from PID 0; stack
trace: ***
@ 0x7f578da84890 (unknown)
@ 0x55d7ab64ae19 kudu::JsonReader::ExtractUint32()
@ 0x55d7ab5f95cf kudu::JsonReaderTest_SignedAndUnsignedInts_Test::TestBody()
@ 0x55d7ab83d0cd testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x55d7ab8331f2 testing::Test::Run()
@ 0x55d7ab8332d4 testing::TestInfo::Run()
@ 0x55d7ab833417 testing::TestCase::Run()
@ 0x55d7ab8338e8 testing::internal::UnitTestImpl::RunAllTests()
@ 0x55d7ab83d5ad testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x55d7ab833a41 testing::UnitTest::Run()
@ 0x55d7ab5e5776 main
@ 0x7f578d6a2b97 __libc_start_main
@ 0x55d7ab5eba7a _start
{noformat}
The offending line is:
{noformat}
ASSERT_TRUE(r.ExtractUint32(r.root(), signed_small32,
nullptr).IsInvalidArgument());
{noformat}
Internally, our JsonReader class successfully parses the number as an unsigned,
which then leads to a crash because we passed in 'nullptr' as the result
(because we expected an earlier parsing error).
We've reproduced this with the gcc found on Ubuntu 18, but not the one in
Ubuntu 16. It also repros if you use clang-6.0 from thirdparty with RELEASE
builds.
We already knew that this was an issue as early as commit 02e82ca14, which
disabled this test in UBSAN builds, because the mishandling triggers UBSAN:
{noformat}
thirdparty/installed/common/include/rapidjson/reader.h:644:18: runtime error:
negation of -2147483648 cannot be represented in type 'int'; cast to an
unsigned type to negate this value to itself
#0 0x7f75d4db9d58 in void rapidjson::GenericReader<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::ParseNumber<0u,
rapidjson::GenericStringStream<rapidjson::UTF8<char> >,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >
>(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&)
../thirdparty/installed/common/include/rapidjson/reader.h:644:18
#1 0x7f75d4db6b84 in void rapidjson::GenericReader<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::ParseObject<0u,
rapidjson::GenericStringStream<rapidjson::UTF8<char> >,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >
>(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&)
../thirdparty/installed/common/include/rapidjson/reader.h:290:4
#2 0x7f75d4db646f in bool rapidjson::GenericReader<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::Parse<0u,
rapidjson::GenericStringStream<rapidjson::UTF8<char> >,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >
>(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&,
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&)
../thirdparty/installed/common/include/rapidjson/reader.h:243:15
#3 0x7f75d4db6105 in rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::ParseStream<0u,
rapidjson::GenericStringStream<rapidjson::UTF8<char> >
>(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&)
../thirdparty/installed/common/include/rapidjson/document.h:712:23
#4 0x7f75d4db543a in rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&
rapidjson::GenericDocument<rapidjson::UTF8<char>,
rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::Parse<0u>(char
const*) ../thirdparty/installed/common/include/rapidjson/document.h:745:10
#5 0x7f75d4db1db9 in kudu::JsonReader::Init()
../src/kudu/util/jsonreader.cc:65:13
#6 0x53aa5a in kudu::JsonReaderTest_SignedAndUnsignedInts_Test::TestBody()
../src/kudu/util/jsonreader-test.cc:159:3
#7 0x7f75d2cc3f9c in void
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*)
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402
#8 0x7f75d2cc3f9c in void
testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*)
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438
#9 0x7f75d2cb9621 in testing::Test::Run()
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474
#10 0x7f75d2cb9703 in testing::TestInfo::Run()
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656
#11 0x7f75d2cb9846 in testing::TestCase::Run()
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774
#12 0x7f75d2cb9d17 in testing::internal::UnitTestImpl::RunAllTests()
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649
#13 0x7f75d2cc447c in bool
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool>(testing::internal::UnitTestImpl*, bool
(testing::internal::UnitTestImpl::*)(), char const*)
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402
#14 0x7f75d2cc447c in bool
testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool>(testing::internal::UnitTestImpl*, bool
(testing::internal::UnitTestImpl::*)(), char const*)
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438
#15 0x7f75d2cb9e70 in testing::UnitTest::Run()
/home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257
#16 0x7f75d57ed2c6 in main ../src/kudu/util/test_main.cc:106:13
#17 0x7f75d1943b96 in __libc_start_main csu/libc-start.c:310
#18 0x4325e9 in _start
(/home/adar/Source/kudu/build/asan/bin/jsonreader-test+0x4325e9)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
thirdparty/installed/common/include/rapidjson/reader.h:644:18 in
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
