masaori335 opened a new issue, #9173: URL: https://github.com/apache/trafficserver/issues/9173
Last few days, I've been running ThreadSanitizer. Some reports has the same pattern of data race, atomic write v.s. normal read. The ink_atomic operations are used for writing variables but not for reading variables. It looks like this happens to almost all of the variables that are touched by ink_atomic operations. We need to change these variables with std::atomic to make reading safe. Also, replacing with std::atomic provides an advantage. We can control [memory order](https://en.cppreference.com/w/cpp/atomic/memory_order). This is not available with ink_atomic because it's using legacy gcc buit-in __sync functions, I think. - e.g. ``` WARNING: ThreadSanitizer: data race (pid=41543) Read of size 8 at 0x7b3000000390 by thread T21 (mutexes: write M0): #0 LogObjectManager::check_buffer_expiration(long) LogObject.cc:1146 (traffic_server:x86_64+0x28eeaf) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #1 Log::periodic_tasks(long) Log.cc:233 (traffic_server:x86_64+0x24a7e4) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #2 Log::flush_thread_main(void*) Log.cc:1390 (traffic_server:x86_64+0x255273) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #3 LoggingFlushContinuation::mainEvent(int, void*) Log.cc:282 (traffic_server:x86_64+0x255cf3) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #4 EThread::execute() UnixEThread.cc:333 (traffic_server:x86_64+0x520124) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #5 spawn_thread_internal(void*) Thread.cc:79 (traffic_server:x86_64+0x51d488) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) Previous atomic write of size 8 at 0x7b3000000390 by thread T11 (mutexes: write M1, write M2): #0 LogObject::_checkout_write(unsigned long*, unsigned long) LogObject.cc:417 (traffic_server:x86_64+0x28a54d) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #1 LogObject::log(LogAccess*, std::__1::basic_string_view<char, std::__1::char_traits<char> >) LogObject.cc:648 (traffic_server:x86_64+0x28afd4) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #2 LogObjectManager::log(LogAccess*) LogObject.cc:1350 (traffic_server:x86_64+0x292fee) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #3 Log::access(LogAccess*) Log.cc:1178 (traffic_server:x86_64+0x253683) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #4 HttpSM::kill_this() HttpSM.cc:7185 (traffic_server:x86_64+0xdc1b4) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #5 HttpSM::main_handler(int, void*) HttpSM.cc:2720 (traffic_server:x86_64+0xd712e) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #6 HttpTunnel::main_handler(int, void*) HttpTunnel.cc:1684 (traffic_server:x86_64+0x1a1437) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #7 CacheVC::calluser(int) P_CacheInternal.h:636 (traffic_server:x86_64+0x3b348a) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #8 CacheVC::openWriteMain(int, Event*) CacheWrite.cc:1441 (traffic_server:x86_64+0x3b2dc2) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #9 EThread::process_event(Event*, int) UnixEThread.cc:153 (traffic_server:x86_64+0x51e8fa) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #10 EThread::process_queue(Queue<Event, Event::Link_link>*, int*, int*) UnixEThread.cc:188 (traffic_server:x86_64+0x51f01f) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #11 EThread::execute_regular() UnixEThread.cc:244 (traffic_server:x86_64+0x51f831) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #12 EThread::execute() UnixEThread.cc:349 (traffic_server:x86_64+0x5202e9) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #13 spawn_thread_internal(void*) Thread.cc:79 (traffic_server:x86_64+0x51d488) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) Location is heap block of size 192 at 0x7b3000000300 allocated by main thread: #0 operator new(unsigned long) <null>:28420041 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x652ad) (BuildId: 065b6c81cd87343dafd65694f305ecb72400000010000000000a0a0000030c00) #1 YamlLogConfig::decodeLogObject(YAML::Node const&) YamlLogConfig.cc:225 (traffic_server:x86_64+0x29ff09) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #2 YamlLogConfig::loadLogConfig(char const*) YamlLogConfig.cc:100 (traffic_server:x86_64+0x29d4eb) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #3 YamlLogConfig::parse(char const*) YamlLogConfig.cc:39 (traffic_server:x86_64+0x29b8f0) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #4 LogConfig::evaluate_config() LogConfig.cc:856 (traffic_server:x86_64+0x26a72a) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #5 LogConfig::setup_log_objects() LogConfig.cc:427 (traffic_server:x86_64+0x269e7d) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #6 LogConfig::init(LogConfig*) LogConfig.cc:327 (traffic_server:x86_64+0x26711d) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #7 Log::init(int) Log.cc:1066 (traffic_server:x86_64+0x2525f3) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #8 main traffic_server.cc:2097 (traffic_server:x86_64+0x6d066) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) ``` ``` WARNING: ThreadSanitizer: data race (pid=41517) Atomic write of size 8 at 0x00000a05bb90 by main thread (mutexes: write M0, write M1, write M2, write M3, write M4, write M5, write M6, write M7, write M8, write M9, write M10, write M11, write M12, write M13): #0 ink_atomiclist_push ink_queue.cc:538 (libtscore.10.dylib:x86_64+0x4522d) (BuildId: ddb6086f2f64376aae6ba04a59161d1132000000200000000100000000000c00) #1 ProtectedQueue::enqueue(Event*) ProtectedQueue.cc:52 (traffic_server:x86_64+0x51c12d) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #2 EventProcessor::schedule(Event*, int) P_UnixEventProcessor.h:123 (traffic_server:x86_64+0x322d9d) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #3 RecProcessStart() RecProcess.cc:277 (traffic_server:x86_64+0x517e26) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #4 main traffic_server.cc:2072 (traffic_server:x86_64+0x6cef8) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) Previous read of size 8 at 0x00000a05bb90 by thread T18 (mutexes: write M14): #0 ProtectedQueue::wait(long long) ProtectedQueue.cc:94 (traffic_server:x86_64+0x51c435) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #1 EThread::DefaultTailHandler::waitForActivity(long long) I_EThread.h:369 (traffic_server:x86_64+0x521182) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #2 EThread::execute_regular() UnixEThread.cc:288 (traffic_server:x86_64+0x51fe17) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #3 EThread::execute() UnixEThread.cc:349 (traffic_server:x86_64+0x5202e9) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #4 spawn_thread_internal(void*) Thread.cc:79 (traffic_server:x86_64+0x51d488) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) Location is heap block of size 1101120 at 0x000009f5b000 allocated by main thread: #0 operator new(unsigned long) <null>:28420041 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x652ad) (BuildId: 065b6c81cd87343dafd65694f305ecb72400000010000000000a0a0000030c00) #1 EventProcessor::spawn_event_threads(int, int, unsigned long) UnixEventProcessor.cc:421 (traffic_server:x86_64+0x52494b) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #2 TasksProcessor::start(int, unsigned long) Tasks.cc:42 (traffic_server:x86_64+0x51cac0) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) #3 main traffic_server.cc:2070 (traffic_server:x86_64+0x6cef3) (BuildId: 279497f9a0ac37109783e3c337bc1aa632000000200000000100000000000c00) ``` Actually, Phil tried to replace ink_atomic with std::atomic by PR #2633 5 years ago. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
