[jira] [Created] (ARROW-9038) [C++] Improve BitBlockCounter
Yibo Cai created ARROW-9038: --- Summary: [C++] Improve BitBlockCounter Key: ARROW-9038 URL: https://issues.apache.org/jira/browse/ARROW-9038 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai ARROW-9029 implements BitBlockCounter. There are chances to improve pops counting performance per this review comment: https://github.com/apache/arrow/pull/7346#discussion_r435005226 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8979) [C++] Implement bitmap word reader and writer
Yibo Cai created ARROW-8979: --- Summary: [C++] Implement bitmap word reader and writer Key: ARROW-8979 URL: https://issues.apache.org/jira/browse/ARROW-8979 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai Below three Jira tasks optimize bitmap operations(logical, copy, compare, etc) unaligned case. They use word-by-word approach instead of bit-by-bit to improve performance. There are some common code of read/write bitmap in words. It's better to implement word based bitmap reader and writer to wrap similar function and reduce code redundancy. https://issues.apache.org/jira/browse/ARROW-8553 https://issues.apache.org/jira/browse/ARROW-8843 https://issues.apache.org/jira/browse/ARROW-8844 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8974) [C++] Refine TransferBitmap template parameters
Yibo Cai created ARROW-8974: --- Summary: [C++] Refine TransferBitmap template parameters Key: ARROW-8974 URL: https://issues.apache.org/jira/browse/ARROW-8974 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai [TransferBitmap|https://github.com/apache/arrow/blob/44e723d9ac7c64739d419ad66618d2d56003d1b7/cpp/src/arrow/util/bit_util.cc#L110] has two template parameters of bool type with four combinations. Change them to function parameters can reduce code size. I think "restore_trailing_bits" cannot impact performance. "invert_bits" needs benchmark. Also, bool parameter is hard to figure out at [caller side|https://github.com/apache/arrow/blob/44e723d9ac7c64739d419ad66618d2d56003d1b7/cpp/src/arrow/util/bit_util.cc#L208], better to use meaningful defines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8844) [C++] Optimize TransferBitmap unaligned case
Yibo Cai created ARROW-8844: --- Summary: [C++] Optimize TransferBitmap unaligned case Key: ARROW-8844 URL: https://issues.apache.org/jira/browse/ARROW-8844 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai TransferBitmap(CopyBitmap, InvertBitmap) unaligned case is processed bit-by-bit[1]. Similar trick in this PR[2] may also be helpful here to improve performance by processing in words. [1] https://github.com/apache/arrow/blob/e5a33f1220705aec6a224b55d2a6f47fbd957603/cpp/src/arrow/util/bit_util.cc#L121-L134 [2] https://github.com/apache/arrow/pull/7135 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8843) [C++] Optimize BItmapEquals unaligned case
Yibo Cai created ARROW-8843: --- Summary: [C++] Optimize BItmapEquals unaligned case Key: ARROW-8843 URL: https://issues.apache.org/jira/browse/ARROW-8843 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai BitmapEquals unaligned case compares two bitmap bit-by-bit[1]. Similar tricks in this PR[2] may also be helpful here to improve performance by processing in words. [1] https://github.com/apache/arrow/blob/e5a33f1220705aec6a224b55d2a6f47fbd957603/cpp/src/arrow/util/bit_util.cc#L248-L254 [2] https://github.com/apache/arrow/pull/7135 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8728) [C++] Bitmap operation may cause buffer overflow
Yibo Cai created ARROW-8728: --- Summary: [C++] Bitmap operation may cause buffer overflow Key: ARROW-8728 URL: https://issues.apache.org/jira/browse/ARROW-8728 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai Happen to find this issue when refining bitmap operation, [this code|https://github.com/apache/arrow/blob/9b75a60658327c39383bee48fa6e5827faf2ced3/cpp/src/arrow/util/bit_util.cc#L267] may overflow out buffer. Should be "(length + left_offset % 8)"; Improve unit test to test large offset values can trigger the bug. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8537) [C++] Performance regression from ARROW-8523
Yibo Cai created ARROW-8537: --- Summary: [C++] Performance regression from ARROW-8523 Key: ARROW-8537 URL: https://issues.apache.org/jira/browse/ARROW-8537 Project: Apache Arrow Issue Type: Bug Reporter: Yibo Cai I optimized BitmapReader in [this PR|https://github.com/apache/arrow/pull/6986] and see performance uplift of BitmapReader test case. I didn't check other test cases as this change looks trivial. I reviewed all test cases just now and see big performance drop of 4 cases, details at [PR link|https://github.com/apache/arrow/pull/6986#issuecomment-616915079]. I also compared performance of code using BitmapReader, no obvious changes found. Looks we should revert that PR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8523) [C++] Optimize BitmapReader
Yibo Cai created ARROW-8523: --- Summary: [C++] Optimize BitmapReader Key: ARROW-8523 URL: https://issues.apache.org/jira/browse/ARROW-8523 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8496) [C++] Refine ByteStreamSplitDecodeScalar
Yibo Cai created ARROW-8496: --- Summary: [C++] Refine ByteStreamSplitDecodeScalar Key: ARROW-8496 URL: https://issues.apache.org/jira/browse/ARROW-8496 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8440) Refine simd header files
Yibo Cai created ARROW-8440: --- Summary: Refine simd header files Key: ARROW-8440 URL: https://issues.apache.org/jira/browse/ARROW-8440 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai This is a follow up of ARROW-8227. It aims to unify simd header files and simplify code. Currently, sse header files are included in sse_util.h, neon header files in neon_util.h and avx header files included directly in c source file. sse_util.h/neon_util.h also contain crc code which is not used by cpp files #include them. It may be better to put all simd header files in a single simd.h, and move crc code to where they are used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8438) [C++] arrow-io-memory-benchmark crashes
Yibo Cai created ARROW-8438:
---
Summary: [C++] arrow-io-memory-benchmark crashes
Key: ARROW-8438
URL: https://issues.apache.org/jira/browse/ARROW-8438
Project: Apache Arrow
Issue Type: Bug
Components: C++
Reporter: Yibo Cai
"arrow-io-memory-benchmark" SIGSEGV in latest code base. It worked at least
when my last commit 8 days ago: b1d4c86eb28267525c52f436c3a096e70b8ef6e0
stack backtrace attached
(gdb) r
Starting program: /home/cyb/share/debug/arrow-io-memory-benchmark
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
(gdb) [New Thread 0x737ff700 (LWP 29065)]
2020-04-14 14:24:40
Running /home/cyb/share/debug/arrow-io-memory-benchmark
Run on (32 X 2100 MHz CPU s)
CPU Caches:
L1 Data 32K (x16)
L1 Instruction 64K (x16)
L2 Unified 512K (x16)
L3 Unified 4096K (x16)
Load Average: 2.64, 4.39, 4.28
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may
be noisy and will incur extra overhead.
***WARNING*** Library was built as DEBUG. Timings may be affected.
Thread 1 "arrow-io-memory" received signal SIGSEGV, Segmentation fault.
0x768e67c8 in arrow::Buffer::is_mutable (this=0x0) at
../src/arrow/buffer.h:258
258 ../src/arrow/buffer.h: No such file or directory.
(gdb) bt
#0 0x768e67c8 in arrow::Buffer::is_mutable (this=0x0) at
../src/arrow/buffer.h:258
#1 0x76c3c41a in
arrow::io::FixedSizeBufferWriter::FixedSizeBufferWriterImpl::FixedSizeBufferWriterImpl
(this=0x558921f0, buffer=std::shared_ptr (empty) = {...})
at ../src/arrow/io/memory.cc:164
#2 0x76c3a575 in
arrow::io::FixedSizeBufferWriter::FixedSizeBufferWriter (this=0x7fffd660,
buffer=std::shared_ptr (empty) = {...}, __in_chrg=,
__vtt_parm=) at ../src/arrow/io/memory.cc:227
#3 0x555ebd00 in arrow::ParallelMemoryCopy (state=...) at
../src/arrow/io/memory_benchmark.cc:303
#4 0x555f80d4 in benchmark::internal::FunctionBenchmark::Run
(this=0x55891290, st=...)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_register.cc:496
#5 0x5564bcc7 in benchmark::internal::BenchmarkInstance::Run
(this=0x558939c0, iters=10, thread_id=0, timer=0x7fffd7a0,
manager=0x55894b70)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_api_internal.cc:10
#6 0x5562c0c8 in benchmark::internal::(anonymous
namespace)::RunInThread (b=0x558939c0, iters=10, thread_id=0,
manager=0x55894b70)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_runner.cc:119
#7 0x5562c95a in benchmark::internal::(anonymous
namespace)::BenchmarkRunner::DoNIterations (this=0x7fffddc0)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_runner.cc:214
#8 0x5562d0ac in benchmark::internal::(anonymous
namespace)::BenchmarkRunner::DoOneRepetition (this=0x7fffddc0,
repetition_index=0)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_runner.cc:299
#9 0x5562c558 in benchmark::internal::(anonymous
namespace)::BenchmarkRunner::BenchmarkRunner (this=0x7fffddc0, b_=...,
complexity_reports_=0x7fffdef0)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_runner.cc:161
#10 0x5562d47f in benchmark::internal::RunBenchmark (b=...,
complexity_reports=0x7fffdef0)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_runner.cc:355
#11 0x555f0ae6 in benchmark::internal::(anonymous
namespace)::RunBenchmarks (benchmarks=std::vector of length 9, capacity 12 =
{...}, display_reporter=0x55891510, file_reporter=0x0)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark.cc:265
#12 0x555f13b6 in benchmark::RunSpecifiedBenchmarks
(display_reporter=0x55891510, file_reporter=0x0)
at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark.cc:399
#13 0x555f0ef8 in benchmark::RunSpecifiedBenchmarks () at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark.cc:340
#14 0x555efc64 in main (argc=1, argv=0x7fffe398) at
/home/cyb/arrow/cpp/debug/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_main.cc:17
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-8238) [C++][Compute] Failed to build compute tests on windows with msvc2015
Yibo Cai created ARROW-8238:
---
Summary: [C++][Compute] Failed to build compute tests on windows
with msvc2015
Key: ARROW-8238
URL: https://issues.apache.org/jira/browse/ARROW-8238
Project: Apache Arrow
Issue Type: Bug
Components: C++ - Compute
Reporter: Yibo Cai
Build Arrow compute tests on Windows10 with MSVC2015:
{code:bash}
cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DARROW_COMPUTE=ON
-DARROW_BUILD_TESTS=ON ..
ninja -j3
{code}
Build failed with below message:
{code:bash}
[311/405] Linking CXX executable release\arrow-misc-test.exe
FAILED: release/arrow-misc-test.exe
cmd.exe /C "cd . &&
C:\Users\yibcai01\Miniconda3\envs\arrow-dev\Library\bin\cmake.exe -E
vs_link_exe --intdir=src\arrow\CMakeFiles\arrow-misc-test.dir
--rc=C:\PROGRA~2\WI3CF2~1\8.1\bin\x64\rc.exe
--mt=C:\PROGRA~2\WI3CF2~1\8.1\bin\x64\mt.exe --manifests --
C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\link.exe /nologo
src\arrow\CMakeFiles\arrow-misc-test.dir\memory_pool_test.cc.obj
src\arrow\CMakeFiles\arrow-misc-test.dir\result_test.cc.obj
src\arrow\CMakeFiles\arrow-misc-test.dir\pretty_print_test.cc.obj
src\arrow\CMakeFiles\arrow-misc-test.dir\status_test.cc.obj
/out:release\arrow-misc-test.exe /implib:release\arrow-misc-test.lib
/pdb:release\arrow-misc-test.pdb /version:0.0 /machine:x64
/NODEFAULTLIB:LIBCMT /INCREMENTAL:NO /subsystem:console
release\arrow_testing.lib release\arrow.lib
googletest_ep-prefix\src\googletest_ep\lib\gtest_main.lib
googletest_ep-prefix\src\googletest_ep\lib\gtest.lib
googletest_ep-prefix\src\googletest_ep\lib\gmock.lib
C:\Users\yibcai01\Miniconda3\envs\arrow-dev\Library\lib\boost_filesystem.lib
C:\Users\yibcai01\Miniconda3\envs\arrow-dev\Library\lib\boost_system.lib
Ws2_32.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib
ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
LINK: command "C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\link.exe /nologo
src\arrow\CMakeFiles\arrow-misc-test.dir\memory_pool_test.cc.obj
src\arrow\CMakeFiles\arrow-misc-test.dir\result_test.cc.obj
src\arrow\CMakeFiles\arrow-misc-test.dir\pretty_print_test.cc.obj
src\arrow\CMakeFiles\arrow-misc-test.dir\status_test.cc.obj
/out:release\arrow-misc-test.exe /implib:release\arrow-misc-test.lib
/pdb:release\arrow-misc-test.pdb /version:0.0 /machine:x64 /NODEFAULTLIB:LIBCMT
/INCREMENTAL:NO /subsystem:console release\arrow_testing.lib release\arrow.lib
googletest_ep-prefix\src\googletest_ep\lib\gtest_main.lib
googletest_ep-prefix\src\googletest_ep\lib\gtest.lib
googletest_ep-prefix\src\googletest_ep\lib\gmock.lib
C:\Users\yibcai01\Miniconda3\envs\arrow-dev\Library\lib\boost_filesystem.lib
C:\Users\yibcai01\Miniconda3\envs\arrow-dev\Library\lib\boost_system.lib
Ws2_32.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib
oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST
/MANIFESTFILE:release\arrow-misc-test.exe.manifest" failed (exit code 1169)
with the following output:
arrow_testing.lib(arrow_testing.dll) : error LNK2005: "public: __cdecl
std::vector >::vector >(class std::initializer_list,class
std::allocator const &)"
(??0?$vector@HV?$allocator@H@std@@@std@@QEAA@V?$initializer_list@H@1@AEBV?$allocator@H@1@@Z)
already defined in result_test.cc.obj
arrow_testing.lib(arrow_testing.dll) : error LNK2005: "public: __cdecl
std::vector >::~vector >(void)" (??1?$vector@HV?$allocator@H@std@@@std@@QEAA@XZ)
already defined in result_test.cc.obj
arrow_testing.lib(arrow_testing.dll) : error LNK2005: "public: unsigned __int64
__cdecl std::vector >::size(void)const "
(?size@?$vector@HV?$allocator@H@std@@@std@@QEBA_KXZ) already defined in
result_test.cc.obj
release\arrow-misc-test.exe : fatal error LNK1169: one or more multiply defined
symbols found
[313/405] Building CXX object
src\arrow\CMakeFiles\arrow-table-test.dir\table_builder_test.cc.obj
ninja: build stopped: subcommand failed.
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-8227) [C++] Propose refining SIMD code framework
Yibo Cai created ARROW-8227: --- Summary: [C++] Propose refining SIMD code framework Key: ARROW-8227 URL: https://issues.apache.org/jira/browse/ARROW-8227 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai Arrow supports wide range of hardware(x86,arm,ppc?) + os(linux,windows,macos,others?) + compiler(gcc,clang,msvc,others?). Managing platform dependent code is non-trivial. This Jira aims to refine(or mess up) simd related code framework. Some goals: Move simd feature definition into one place, possibly in cmake, and reduce compiler based ifdef is source code. Manage simd code in one place, but leave non-simd default implementations where they are. Shouldn't introduce any performance penalty, prefer direct inline to runtime dispatcher. Code should be easy to maintain, expand, and hard to make mistakes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8129) [C++][Compute] Refine compare sorting kernel
Yibo Cai created ARROW-8129: --- Summary: [C++][Compute] Refine compare sorting kernel Key: ARROW-8129 URL: https://issues.apache.org/jira/browse/ARROW-8129 Project: Apache Arrow Issue Type: Improvement Reporter: Yibo Cai Assignee: Yibo Cai Sorting kernel implements two comparison functions, [CompareValues|https://github.com/apache/arrow/blob/ab21f0ee429c2a2c82e4dbc5d216ab1da74221a2/cpp/src/arrow/compute/kernels/sort_to_indices.cc#L67] use array.Value() for numeric data and [CompareViews|https://github.com/apache/arrow/blob/ab21f0ee429c2a2c82e4dbc5d216ab1da74221a2/cpp/src/arrow/compute/kernels/sort_to_indices.cc#L72] uses array.GetView() for non-numeric ones. It can be simplified by using GetView() only as all data types support GetView(). To my surprise, benchmark shows about 40% performance improvement after the change. After some digging, I find in current code, the [comparison callback|https://github.com/apache/arrow/blob/ab21f0ee429c2a2c82e4dbc5d216ab1da74221a2/cpp/src/arrow/compute/kernels/sort_to_indices.cc#L94] is not inlined (check disassembled code), it leads to a function call. It's very bad for this hot loop. Using only GetView() fixes this issue, code inlined okay. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8126) [C++][Compute] Add Top-K kernel benchmark
Yibo Cai created ARROW-8126: --- Summary: [C++][Compute] Add Top-K kernel benchmark Key: ARROW-8126 URL: https://issues.apache.org/jira/browse/ARROW-8126 Project: Apache Arrow Issue Type: Improvement Components: C++ - Compute Reporter: Yibo Cai Assignee: Yibo Cai -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7587) [C++][Compute] Add Top-k kernel
Yibo Cai created ARROW-7587: --- Summary: [C++][Compute] Add Top-k kernel Key: ARROW-7587 URL: https://issues.apache.org/jira/browse/ARROW-7587 Project: Apache Arrow Issue Type: New Feature Components: C++ - Compute Reporter: Yibo Cai Assignee: Yibo Cai Add a kernel to get top k smallest or largest elements (indices). std::paiital_sort should be a better solution than sorting everything then pick top k. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7557) [C++][Compute] Validate sorting stability in random test
Yibo Cai created ARROW-7557: --- Summary: [C++][Compute] Validate sorting stability in random test Key: ARROW-7557 URL: https://issues.apache.org/jira/browse/ARROW-7557 Project: Apache Arrow Issue Type: Improvement Components: C++ - Compute Reporter: Yibo Cai Assignee: Yibo Cai Sorting kernel unit test doesn't validate sorting stability in random test. [1] Should assert "lhs < rhs" when "array.Value(lhs) == array.Value(rhs)". [1] https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/sort_to_indices_test.cc#L112-L121 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7526) [C++][Compute]: Optimize small integer sorting
Yibo Cai created ARROW-7526: --- Summary: [C++][Compute]: Optimize small integer sorting Key: ARROW-7526 URL: https://issues.apache.org/jira/browse/ARROW-7526 Project: Apache Arrow Issue Type: Improvement Components: C++ - Compute Reporter: Yibo Cai Assignee: Yibo Cai Current sorting kernel handles all data types with stl stable_sort. It is suboptimal for small integers like Int8, in which case counting sort is more suitable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7464) [C++][util]: Refine CpuInfo singleton with std::call_once
Yibo Cai created ARROW-7464: --- Summary: [C++][util]: Refine CpuInfo singleton with std::call_once Key: ARROW-7464 URL: https://issues.apache.org/jira/browse/ARROW-7464 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai CpuInfo singleton is created and initialized on first invocation of [CpuInfo::GetInstance()|https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/cpu_info.cc#L188-L195]. All calls afterwards return reference to the same instance. Current code uses std::mutex to make sure that CpuInfo is created only once, but it introduces unnecessary overhead for later calls. Concurrent threads getting the created instance should not block each other. Replace std::mutex with std::call_once to fix this issue. References: [1] https://en.cppreference.com/w/cpp/thread/call_once [2] http://www.modernescpp.com/index.php/thread-safe-initialization-of-data -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7404) [C++][Gandiva] Fix utf8 char length error on Arm64
Yibo Cai created ARROW-7404: --- Summary: [C++][Gandiva] Fix utf8 char length error on Arm64 Key: ARROW-7404 URL: https://issues.apache.org/jira/browse/ARROW-7404 Project: Apache Arrow Issue Type: Bug Components: C++ - Gandiva Reporter: Yibo Cai Assignee: Yibo Cai Current code checks if a UTF-8 eight-bit code unit is within 0x00~0x7F by "if (c >= 0)", where c is defined as "char". This checking assumes char is always signed, which is not true[1]. On Arm64, char is unsigned by default and causes some Gandiva unit tests fail. Fix it by casting to "signed char" explicitly. [1] Cited from https://en.cppreference.com/w/cpp/language/types The signedness of char depends on the compiler and the target platform: the defaults for ARM and PowerPC are typically unsigned, the defaults for x86 and x64 are typically signed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7403) [C++][JSON] Enable Rapidjson on Arm64 Neon
Yibo Cai created ARROW-7403: --- Summary: [C++][JSON] Enable Rapidjson on Arm64 Neon Key: ARROW-7403 URL: https://issues.apache.org/jira/browse/ARROW-7403 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai Rapidjson support Arm64 Neon, but it's not enabled in Arrow now. We need to define macro RAPIDJSON_NEON to build Rapidjson with Neon support. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7397) [C++] Json white space length detection error
Yibo Cai created ARROW-7397: --- Summary: [C++] Json white space length detection error Key: ARROW-7397 URL: https://issues.apache.org/jira/browse/ARROW-7397 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Yibo Cai Commit 21ca13a5cd [1] introduces a bug in ConsumeWhitespace() function. When all chars in a string are white spaces, it should return string length. But current code returns 0. It's not noticed because x86 goes rapidjson simd code path which is okay. Arm64 now goes the buggy code path and triggers json unit test failure. [1] https://github.com/apache/arrow/commit/21ca13a5cd9c1478d64370732fcfae72d52350dd#diff-664e724274fbe0ff1e03745aa452b4d6R48 -- This message was sent by Atlassian Jira (v8.3.4#803005)
