Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3906: Materialize implicitly referenced IR functions ......................................................................
IMPALA-3906: Materialize implicitly referenced IR functions With lazy materialization of IR functions in the LLVM module, there is an assumption that functions not referenced by IR functions used in the query don't have to be materialized and can have their linkage types changed to being externally defined. These unmaterialized functions may be referenced by global variables in the LLVM module and LLVM resolves these references to their definitions in the native Impalad binary. These global variables are mostly arrays containing references to other global constants or boost and Impala functions included in the cross-compiled code. When compiling Impalad with optimization, gcc may actually inline some of the functions which the global variables in the LLVM modules reference and LLVM may have linking error if these referenced IR functions are not materialized as it can no longer find their definitions in the Impalad binary. This patch fixes the problem by parsing all the global variables in the LLVM module during Impalad's initialization and recording all the referenced functions which aren't defined in the Impalad binary and make sure they are always materialized. A second problem which this patch fixes is that linking in external LLVM module (e.g. UDF IR created by a user) may implicitly materialize some functions in the external module. Normally, we would expect functions to be materialized through the GetFunction() interface and LinkModule() seems to be an exception. This patch updates LinkModule() to also parse the functions from the external module just like we do for unmaterialized functions in GetFunction(). Change-Id: I147eb332adc3f272b7a7a4fb4415842432c05f77 Reviewed-on: http://gerrit.cloudera.org:8080/3792 Reviewed-by: Michael Ho <[email protected]> Tested-by: Internal Jenkins --- M be/src/codegen/llvm-codegen.cc M be/src/codegen/llvm-codegen.h M be/src/util/symbols-util-test.cc 3 files changed, 152 insertions(+), 27 deletions(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/3792 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I147eb332adc3f272b7a7a4fb4415842432c05f77 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho <[email protected]>
