Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/9134 )
Change subject: IMPALA-5717: Support for reading ORC data files ...................................................................... Patch Set 5: (2 comments) Answered your questions (sorry it took so long) and working through the whole change. http://gerrit.cloudera.org:8080/#/c/9134/5/be/src/exec/hdfs-orc-scanner.cc File be/src/exec/hdfs-orc-scanner.cc: http://gerrit.cloudera.org:8080/#/c/9134/5/be/src/exec/hdfs-orc-scanner.cc@567 PS5, Line 567: // TODO: combine this with the Parquet implementation > Not quite familiar with the LLVM Codegen codes. I've simply tried extractin We need to make sure that any functions that need to be used by codegen are in an -ir.cc file and compiled into the correct places. Easiest way to see is where a current file is included: $ git grep hdfs-parquet-scanner-ir.cc be/src/codegen/impala-ir.cc:#include "exec/hdfs-parquet-scanner-ir.cc" be/src/exec/CMakeLists.txt: hdfs-parquet-scanner-ir.cc If we needed to move this to hdfs-scanner, we could add it to hdfs-scanner-ir.cc, which is already cross-compiled to LLVM IR. Then functions that are loaded by codegen need to have the symbol name added to be/src/codegen/gen_ir_descriptions.py so that it can be found. Easiest way to figure out the symbol name is dump symbols in a binary and find the appropraite mangled name, e.g.: $ nm -g be/build/latest/service/impalad | grep EvalRuntimeFilter 0000000001d47982 T _ZN6impala18HdfsParquetScanner17EvalRuntimeFilterEiPNS_8TupleRowE 0000000001d2f9de T _ZN6impala18HdfsParquetScanner18EvalRuntimeFiltersEPNS_8TupleRowE http://gerrit.cloudera.org:8080/#/c/9134/5/be/src/exec/hdfs-orc-scanner.cc@590 PS5, Line 590: ColumnVectorBatch > To change this, we have to change the type of scratch_batch_ to unique_ptr< Can't the caller do the cast though? My thought was that would make the function more self-documenting. -- To view, visit http://gerrit.cloudera.org:8080/9134 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4 Gerrit-Change-Number: 9134 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Wed, 28 Mar 2018 21:14:34 +0000 Gerrit-HasComments: Yes
