Hi Team, While working with NiFi 1.5.0, we observed that for few queries, PutHiveQL throws NullPointerException while parsing the query. However, the query does complete the execution and flow file is routes to success afterwards.
From the observation made till now, we found that the queries containing back-tick(s) (`) are not getting parsed. This was not the case in NiFi 1.4.0. Will it be possible for you to confirm whether this is the desired behavior of NiFi 1.5.0. I’ve attached a file which contains a portion of the nifi-app.log showing the WARN messages of parsing exception. Regards, Amit
## PutHiveQL gives un-necessary WARN log of NullPointerException ## However, the query succeeds ####### Drop database hive query ####### 2018-03-08 02:19:01,003 INFO [Timer-Driven Process Thread-10] hive.ql.parse.ParseDriver Parsing command: drop table if exists rdsp_production_production_mart_nifi_test.stg_red_rms_shop_mst_base 2018-03-08 02:19:01,006 INFO [Timer-Driven Process Thread-10] hive.ql.parse.ParseDriver Parse Completed ####### Creating hive table: This throws NullPointerException ####### 2018-03-08 02:19:01,542 INFO [Timer-Driven Process Thread-5] hive.ql.parse.ParseDriver Parsing command: create table rdsp_production_production_mart_nifi_test.stg_red_rms_shop_mst_base (etl_hash string, etl_source string, `shop_id` int,`url` string,`shop_name` string,`shop_name_kana` string,`shop_genre_cd` int,`shop_subgenre_cd` int,`shop_genre_name` string,`shop_subgenre_name` string,`shop_genre_name_e` string,`shop_subgenre_name_e` string,`ecc_group` int,`ecc_name` string,`sales_dev_group` int,`sales_dev_name` string,`oa_group` int,`oa_name` string,`consultant_group` int,`consultant_name` string,`plan_id` int,`plan_name` string,`gold_service_used_flg` int,`open_flg` int,`rs_flg` int,`ao_date` date,`open_date` date,`close_date` date,`display_close_date` date,`passchng_date` date,`close_trans_date` date,`topdelete_date` date,`fee_date` date,`end_date` date,`remain_day` int,`login_en_datetime` timestamp,`login_rs_datetime` timestamp,`logout_rs_datetime` timestamp,`login_rb_datetime` timestamp,`login_rt_datetime` timestamp,`company_name` string,`company_tel` string,`company_fax` string,`address_zip` string,`address_prefecture` string,`address_city` string,`address_street` string,`shop_staff_name` string,`shop_staff_post` string,`information_tel` string,`information_fax` string,`support_tel` string,`support_fax` string,`emergency_tel` string,`shop_staff_mail` string,`order_mail` string,`shop_sec_staff_name` string,`shop_sec_staff_mail` string,`affiliate_banner_char` string,`affiliate_rate` decimal(8,3),`additem_char` string,`rmail_char` string,`html_char` string,`allitem_char` string,`old_max_item` int,`new_max_item` int,`min_price` int,`millioncard_char` string,`l_nittsu_char` string,`l_seino_char` string,`p_711_char` string,`p_lawson_char` string,`p_ebank_char` string,`p_jpnet_char` string,`s_csvdl_char` string,`sf_csvul_char` string,`auth_flag_char` string,`mobile_char` string,`item_db` string,`category_db` string,`customer_id` int,`company_id` int,`time_stamp` timestamp ) partitioned by (etl_loaddatetime timestamp) row format delimited fields terminated by '\u0001' 2018-03-08 02:19:01,673 WARN [Timer-Driven Process Thread-5] o.apache.nifi.processors.hive.PutHiveQL PutHiveQL[id=211931d7-cbe0-16ee-9a95-4873c5899053] Failed to parse hiveQL: create table rdsp_production_production_mart_nifi_test.stg_red_rms_shop_mst_base (etl_hash string, etl_source string, `shop_id` int,`url` string,`shop_name` string,`shop_name_kana` string,`shop_genre_cd` int,`shop_subgenre_cd` int,`shop_genre_name` string,`shop_subgenre_name` string,`shop_genre_name_e` string,`shop_subgenre_name_e` string,`ecc_group` int,`ecc_name` string,`sales_dev_group` int,`sales_dev_name` string,`oa_group` int,`oa_name` string,`consultant_group` int,`consultant_name` string,`plan_id` int,`plan_name` string,`gold_service_used_flg` int,`open_flg` int,`rs_flg` int,`ao_date` date,`open_date` date,`close_date` date,`display_close_date` date,`passchng_date` date,`close_trans_date` date,`topdelete_date` date,`fee_date` date,`end_date` date,`remain_day` int,`login_en_datetime` timestamp,`login_rs_datetime` timestamp,`logout_rs_datetime` timestamp,`login_rb_datetime` timestamp,`login_rt_datetime` timestamp,`company_name` string,`company_tel` string,`company_fax` string,`address_zip` string,`address_prefecture` string,`address_city` string,`address_street` string,`shop_staff_name` string,`shop_staff_post` string,`information_tel` string,`information_fax` string,`support_tel` string,`support_fax` string,`emergency_tel` string,`shop_staff_mail` string,`order_mail` string,`shop_sec_staff_name` string,`shop_sec_staff_mail` string,`affiliate_banner_char` string,`affiliate_rate` decimal(8,3),`additem_char` string,`rmail_char` string,`html_char` string,`allitem_char` string,`old_max_item` int,`new_max_item` int,`min_price` int,`millioncard_char` string,`l_nittsu_char` string,`l_seino_char` string,`p_711_char` string,`p_lawson_char` string,`p_ebank_char` string,`p_jpnet_char` string,`s_csvdl_char` string,`sf_csvul_char` string,`auth_flag_char` string,`mobile_char` string,`item_db` string,`category_db` string,`customer_id` int,`company_id` int,`time_stamp` timestamp ) partitioned by (etl_loaddatetime timestamp) row format delimited fields terminated by '\u0001' due to java.lang.NullPointerException: ####### Loading data into the hive table ####### 2018-03-08 02:19:02,035 INFO [Timer-Driven Process Thread-6] hive.ql.parse.ParseDriver Parsing command: load data inpath '/user/datalake/rdsp_etl_work/nifi_test/mart/red_rms_shop_mst_base_1520475488687.txt' into table rdsp_production_production_mart_nifi_test.stg_red_rms_shop_mst_base partition (etl_loaddatetime = '2018-03-08 02:19:01') 2018-03-08 02:19:02,046 INFO [Timer-Driven Process Thread-6] hive.ql.parse.ParseDriver Parse Completed 2018-03-08 02:19:02,113 INFO [Process Cluster Protocol Request-3] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 95821d6a-0d4b-4426-95f7-2c842bd47d66 (type=HEARTBEAT, length=3645 bytes) from odin-jp2-nifi-poc003.dev.hnd2.bdd.local:9443 in 108 millis 2
