nishant94 commented on code in PR #64667:
URL: https://github.com/apache/doris/pull/64667#discussion_r3803956951
##########
be/CMakeLists.txt:
##########
@@ -976,6 +996,49 @@ if (BUILD_META_TOOL OR BUILD_INDEX_TOOL)
add_subdirectory(${SRC_DIR}/tools)
endif()
+if (NOT MAKE_TEST)
+ # Offline generator: compiles the UTF-8 mecab-ipadic source into binary
files.
+ add_executable(kuromoji_build_dict EXCLUDE_FROM_ALL
${SRC_DIR}/tools/kuromoji_build_dict.cpp)
+ target_include_directories(kuromoji_build_dict PRIVATE
${PROJECT_SOURCE_DIR}/..)
+ pch_reuse(kuromoji_build_dict)
+ set_target_properties(kuromoji_build_dict PROPERTIES ENABLE_EXPORTS 1)
+ if (COMPILER_CLANG)
+ target_compile_options(kuromoji_build_dict PRIVATE
+ -Wno-implicit-int-conversion
+ -Wno-shorten-64-to-32)
+ endif()
+ target_link_libraries(kuromoji_build_dict ${DORIS_LINK_LIBS})
+
+ set(KUROMOJI_IPADIC_SRC
"${THIRDPARTY_DIR}/share/mecab-ipadic-2.7.0-20250920"
+ CACHE PATH "UTF-8 mecab-ipadic source directory used to generate the
kuromoji dictionary")
+ set(KUROMOJI_DICT_OUT "${BASE_DIR}/dict/kuromoji")
Review Comment:
KUROMOJI_DICT_OUT is the source-tree be/dict/kuromoji on purpose: the
real-dictionary UT kuromoji_real_dict_test locates the dict at the stable
${DORIS_HOME}/be/dict/kuromoji, and moving generation into a per-build-tree dir
would make that test unable to find it and always skip. Packaging is unaffected
(install copies from there into output/, which the runtime reads).
The cross-tree poisoning needs two build trees sharing one checkout with
different overridden KUROMOJI_IPADIC_SRC a rare workflow. default and
concurrent same-source builds produce identical bytes and already publish in
small parts. If multi-source-tree builds are a real need, I'll add a
provenance/completion stamp that regenerates on source mismatch as a follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
