Greetings All. I am working on IMPALA-3504 right now.
https://issues.cloudera.org/browse/IMPALA-3504
This JIRA basically wants to implement a UDF timestamp function. Following is
my process when working on this JIRA:
1. Back-end source code:
I have added an function entry in the file timestamp-functions.h:
static TimestampVal UtcTimestamp(FunctionContext* context);
and added corresponding implementation in the file timestamp-functions.cc.
2. Build back-end:
And after that, I first build impalad binary using make_debug.sh script.
After the impalad binary is generated, I run Linux "strings" command to get the
binary signature of the function like:
_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE.
And this entry is also added to the
Impala/common/function-registry/impala_functions.py file like:
[['utc_timestamp'], 'TIMESTAMP', [],
'_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'],
3. Overall build:
Then I execute the buildall.sh script to initiate build all front-end, back-end
and third-party tools.
Issue:
When the build process is done, I use the command:"start-impala-cluster.py -r
--build_type=debug" to start the Impala cluster. However, the impalad process
fails to run due to the error which is copied from
Impala/logs/cluster/impalad.INFO:
I0512 20:35:46.621750 31842 init.cc:204] Physical Memory: 11.69 GB
I0512 20:35:46.621752 31842 init.cc:205] OS version: Linux version 4.5.3
(root@debian) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 SMP Tue May 10 09:19:36
EDT 2016
Clock: clocksource: 'tsc', clockid_t: CLOCK_MONOTONIC
I0512 20:35:46.621754 31842 init.cc:206] Process ID: 31842
I0512 20:35:47.307698 31842 status.cc:40] Kudu is not supported on this
operating system.
@ 0x10de550 impala::Status::Status()
@ 0x10e1984 impala::CheckKuduAvailability()
@ 0x10e18f7 impala::KuduIsAvailable()
@ 0x10d89c1 impala::InitCommonRuntime()
@ 0x1446db1 ImpaladMain()
@ 0x1088720 main
@ 0x7f214e93ab45 (unknown)
@ 0x108859c (unknown)
I0512 20:35:47.689296 31842 hbase-table-scanner.cc:161] Detected HBase version
>= 0.95.2
I0512 20:35:47.700866 31842 GlogAppender.java:123] Logging initialized. Impala:
VLOG, All other: INFO
I0512 20:35:47.702227 31842 JniFrontend.java:125] Authorization is 'DISABLED'.
I0512 20:35:47.702286 31842 JniFrontend.java:127] Java Version Info: Java(TM)
SE Runtime Environment (1.8.0_65-b17)
W0512 20:35:47.871737 31842 HiveConf.java:2721] HiveConf of name
hive.access.conf.url does not exist
I0512 20:35:47.958432 31842 fe-support.cc:201] Could not find symbol
'_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'
in:
Unable to find
_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE
dlerror: /root/Impala/be/build/debug/service/impalad: undefined symbol:
_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE
F0512 20:35:47.959112 31842 frontend.cc:107] java.lang.RuntimeException:
Builtin symbol
'_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'[]
not found!
at
com.cloudera.impala.catalog.ScalarFunction.createBuiltin(ScalarFunction.java:79)
at com.cloudera.impala.catalog.Db.addScalarBuiltin(Db.java:389)
at com.cloudera.impala.catalog.Db.addScalarBuiltin(Db.java:379)
at
com.cloudera.impala.builtins.ScalarBuiltins.initBuiltins(ScalarBuiltins.java:124)
at
com.cloudera.impala.catalog.BuiltinsDb.initBuiltins(BuiltinsDb.java:58)
at com.cloudera.impala.catalog.BuiltinsDb.<init>(BuiltinsDb.java:39)
at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:97)
at
com.cloudera.impala.catalog.ImpaladCatalog.<init>(ImpaladCatalog.java:92)
at com.cloudera.impala.service.Frontend.<init>(Frontend.java:170)
at
com.cloudera.impala.service.JniFrontend.<init>(JniFrontend.java:129)
Caused by: com.cloudera.impala.common.AnalysisException: Could not find symbol
'_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'
in:
at
com.cloudera.impala.catalog.Function.lookupSymbol(Function.java:423)
at
com.cloudera.impala.catalog.ScalarFunction.createBuiltin(ScalarFunction.java:75)
... 9 more
This is really weird since such symbol
"ZN6impala18TimestampFunctions12UtcTimestampEPN10impala" is indeed in the
impalad binary and I have double-checked that. Actually I have some successful
experience of developing some UDF before. So this issue really makes me feel
confused. I have removed all built binaries and replay the steps above. This
issue still remains there.
Thank you so much for any advice or hint. :)
