[
https://issues.apache.org/jira/browse/IMPALA-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648723#comment-16648723
]
ASF subversion and git services commented on IMPALA-7704:
---------------------------------------------------------
Commit af76186e013607cb64baf151c039e4f6aaab4350 in impala's branch
refs/heads/master from [~joemcdonnell]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=af76186 ]
IMPALA-7704: Revert "IMPALA-7644: Hide Parquet page index writing with feature
flag"
The fix for IMPALA-7644 introduced ASAN issues detailed in
IMPALA-7704. Reverting for now.
This reverts commit 843683ed6c2ef41c7c25e9fa4af68801dbdd1a78.
Change-Id: Icf0a64d6ec747275e3ecd6e801e054f81095591a
Reviewed-on: http://gerrit.cloudera.org:8080/11671
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Michael Ho <[email protected]>
> ASAN tests failing in HdfsParquetTableWriter
> --------------------------------------------
>
> Key: IMPALA-7704
> URL: https://issues.apache.org/jira/browse/IMPALA-7704
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Joe McDonnell
> Assignee: Zoltán Borók-Nagy
> Priority: Blocker
> Labels: broken-build
>
> ASAN tests have been failing for the last few runs. Here is the output:
> {noformat}
> ==117268==ERROR: AddressSanitizer: use-after-poison on address 0x7ef9312e0ec0
> at pc 0x0000016a0c98 bp 0x7ef91e502540 sp 0x7ef91e501cf0
> READ of size 32681 at 0x7ef9312e0ec0 thread T82364
> #0 0x16a0c97 in __interceptor_memcpy
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:738
> #1 0x7f02e43c2dca in jni_SetByteArrayRegion
> (/usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so+0x6d3dca)
> #2 0x4ae29b7 in hdfsWrite
> /container.redhat6/build/cdh/hadoop/3.0.0-cdh6.x-SNAPSHOT/rpm/BUILD/hadoop-3.0.0-cdh6.x-SNAPSHOT/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c:1626
> #3 0x332ca67 in impala::HdfsTableWriter::Write(unsigned char const*, int)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-writer.cc:46:13
> #4 0x25ac037 in
> impala::HdfsParquetTableWriter::BaseColumnWriter::Flush(long*, long*, long*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:736:5
> #5 0x25b0e57 in impala::HdfsParquetTableWriter::FlushCurrentRowGroup()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:1195:5
> #6 0x25b4147 in impala::HdfsParquetTableWriter::Finalize()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:1161:3
> #7 0x251e424 in
> impala::HdfsTableSink::FinalizePartitionFile(impala::RuntimeState*,
> impala::OutputPartition*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:620:5
> #8 0x2523d1b in impala::HdfsTableSink::FlushFinal(impala::RuntimeState*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:660:5
> #9 0x1fc09f0 in impala::FragmentInstanceState::ExecInternal()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:324:3
> #10 0x1fbbd5c in impala::FragmentInstanceState::Exec()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14
> #11 0x1fd5f94 in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/query-state.cc:478:24
> #12 0x1cdef96 in boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
> #13 0x23ac27e in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/thread.cc:359:3
> #14 0x23b7708 in void boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
> #15 0x23b755b in boost::_bi::bind_t<void, void (*)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> #16 0x39a11e9 in thread_proxy
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a11e9)
> #17 0x7f02e19f4e24 in start_thread (/lib64/libpthread.so.0+0x7e24)
> #18 0x7f02e150b34c in __clone (/lib64/libc.so.6+0xf834c)
> 0x7ef9312e0ec0 is located 161472 bytes inside of 524288-byte region
> [0x7ef9312b9800,0x7ef931339800)
> allocated by thread T82364 here:
> #0 0x16fcd88 in malloc
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:67
> #1 0x1f2a5ec in impala::MemPool::FindChunk(long, bool)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/mem-pool.cc:140:45
> #2 0x1f44c37 in unsigned char* impala::MemPool::Allocate<false>(long, int)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/mem-pool.h:284:9
> #3 0x1f44c37 in impala::MemPool::Allocate(long)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/mem-pool.h:107
> #4 0x25ad32c in
> impala::HdfsParquetTableWriter::BaseColumnWriter::FinalizeCurrentPage()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:797:61
> #5 0x25bbdea in
> impala::HdfsParquetTableWriter::BaseColumnWriter::AppendRow(impala::TupleRow*)
>
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:590:5
> #6 0x25b36cc in impala::HdfsParquetTableWriter::AppendRows(impala::RowBatch*,
> std::vector<int, std::allocator<int> > const&, bool*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:1124:7
> #7 0x251ddec in
> impala::HdfsTableSink::WriteRowsToPartition(impala::RuntimeState*,
> impala::RowBatch*, std::pair<std::unique_ptr<impala::OutputPartition,
> std::default_delete<impala::OutputPartition> >, std::vector<int,
> std::allocator<int> > >*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:256:5
> #8 0x2522f01 in impala::HdfsTableSink::Send(impala::RuntimeState*,
> impala::RowBatch*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:590:5
> #9 0x1fc07ad in impala::FragmentInstanceState::ExecInternal()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:316:5
> #10 0x1fbbd5c in impala::FragmentInstanceState::Exec()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14
> #11 0x1fd5f94 in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/query-state.cc:478:24
> #12 0x1cdef96 in boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
> #13 0x23ac27e in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/thread.cc:359:3
> #14 0x23b7708 in void boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
> #15 0x23b755b in boost::_bi::bind_t<void, void (*)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> #16 0x39a11e9 in thread_proxy
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a11e9)
> Thread T82364 created by T82363 here:
> #0 0x164420d in __interceptor_pthread_create
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
> #1 0x39a05c9 in boost::thread::start_thread_noexcept()
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
> #2 0x45e0360d (<unknown module>)
> Thread T82363 created by T207 here:
> #0 0x164420d in __interceptor_pthread_create
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
> #1 0x39a05c9 in boost::thread::start_thread_noexcept()
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
> #2 0x45e0360d (<unknown module>)
> Thread T207 created by T199 here:
> #0 0x164420d in __interceptor_pthread_create
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
> #1 0x39a05c9 in boost::thread::start_thread_noexcept()
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
> #2 0x45e0360d (<unknown module>)
> Thread T199 created by T198 here:
> #0 0x164420d in __interceptor_pthread_create
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
> #1 0x39a05c9 in boost::thread::start_thread_noexcept()
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
> #2 0x45e0360d (<unknown module>)
> Thread T198 created by T0 here:
> #0 0x164420d in __interceptor_pthread_create
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
> #1 0x39a05c9 in boost::thread::start_thread_noexcept()
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
> #2 0x45e0360d (<unknown module>)
> SUMMARY: AddressSanitizer: use-after-poison
> /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:738
> in __interceptor_memcpy
> Shadow bytes around the buggy address:
> 0x0fdfa6254180: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa6254190: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa62541a0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa62541b0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa62541c0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> =>0x0fdfa62541d0: f7 f7 f7 f7 f7 f7 f7 f7[f7]f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa62541e0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa62541f0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa6254200: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa6254210: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 0x0fdfa6254220: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Container overflow: fc
> Array cookie: ac
> Intra object redzone: bb
> ASan internal: fe
> Left alloca redzone: ca
> Right alloca redzone: cb
> ==117268==ABORTING{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
