[
https://issues.apache.org/jira/browse/YARN-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin Chundatt resolved YARN-9847.
----------------------------------
Resolution: Invalid
As confirmed by [~suxingfate] the issue is already fixed by YARN-6967
Closing as invalid. Please do reopen if required.
> ZKRMStateStore will cause zk connection loss when writing huge data into znode
> ------------------------------------------------------------------------------
>
> Key: YARN-9847
> URL: https://issues.apache.org/jira/browse/YARN-9847
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Wang, Xinglong
> Assignee: Wang, Xinglong
> Priority: Minor
> Attachments: YARN-9847.001.patch, YARN-9847.002.patch
>
>
> Recently, we encountered RM ZK connection issue due to RM was trying to write
> huge data into znode. This behavior will zk report Len error and then cause
> zk session connection loss. And eventually RM would crash due to zk
> connection issue.
> *The fix*
> In order to protect ResouceManager from crash due to this.
> This fix is trying to limit the size of data for attemp by limiting the
> diagnostic info when writing ApplicationAttemptStateData into znode. The size
> will be regulated by -Djute.maxbuffer set in yarn-env.sh. The same value will
> be also used by zookeeper server.
> *The story*
> ResourceManager Log
> {code:java}
> 2019-07-29 02:14:59,638 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x36ab902369100a0 for serverabc-zk-5.vip.ebay.com/10.210.82.29:2181,
> unexpected error, closing socket connection and attempting reconnect
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
> at sun.nio.ch.IOUtil.write(IOUtil.java:65)
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
> at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:117)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
> 2019-07-29 04:27:35,459 INFO
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore:
> Exception while executing a ZK operation.
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
> = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:935)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:915)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$5.run(ZKRMStateStore.java:998)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$5.run(ZKRMStateStore.java:995)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1174)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1207)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doStoreMultiWithRetries(ZKRMStateStore.java:1001)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doStoreMultiWithRetries(ZKRMStateStore.java:1009)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.setDataWithRetries(ZKRMStateStore.java:1050)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.updateApplicationAttemptStateInternal(ZKRMStateStore.java:699)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppAttemptTransition.transition(RMStateStore.java:317)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppAttemptTransition.transition(RMStateStore.java:299)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:955)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:1036)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:1031)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:109)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> ResourceManager will retry to connect to zookeeper until it exhausted retry
> number and then give up.
> {code:java}
> 2019-07-29 02:25:06,404 INFO
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore:
> Retrying operation on ZK. Retry no. 999
> 2019-07-29 02:25:06,718 INFO org.apache.zookeeper.client.ZooKeeperSaslClient:
> Client will use GSSAPI as SASL mechanism.
> 2019-07-29 02:25:06,718 INFO org.apache.zookeeper.ClientCnxn: Opening socket
> connection to server 2019-07-29 02:25:06,404 INFO
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore:
> Retrying operation on ZK. Retry no. 999
> 2019-07-29 02:25:06,718 INFO org.apache.zookeeper.client.ZooKeeperSaslClient:
> Client will use GSSAPI as SASL mechanism.
> 2019-07-29 02:25:06,718 INFO org.apache.zookeeper.ClientCnxn: Opening socket
> connection to server serverabc-lvs-zk-5.vip.ebay.com/10.210.82.29:2181. Will
> attempt to SASL-authenticate using Login Context section 'Client'
> 2019-07-29 02:25:06,718 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to serverabc-lvs-zk-5.vip.ebay.com/10.210.82.29:2181,
> initiating session
> 2019-07-29 02:25:06,720 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server
> serverabc-lvs-zk-5.vip.ebay.com/10.210.82.29:2181, sessionid =
> 0x36ab902369100a0, negotiated timeout = 40000
> 2019-07-29 02:25:06,749 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x36ab902369100a0 for server
> serverabc-lvs-zk-5.vip.ebay.com/10.210.82.29:2181, unexpected error, closing
> socket connection and attempting reconnect
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
> at sun.nio.ch.IOUtil.write(IOUtil.java:65)
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:117)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
> 2019-07-29 02:25:06,850 INFO
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore: Maxed
> out ZK retries. Giving up!
> {code}
> The retry behavior is controlled by following config.
> {code:java}
> <property>
> <name>yarn.resourcemanager.zk-state-store.parent-path</name>
> <value>/rmstore</value>
> <source>yarn-default.xml</source>
> </property>
> <property>
> <name>yarn.resourcemanager.zk-acl</name>
> <value>sasl:yarn:rwcda</value>
> <source>yarn-site.xml</source>
> </property>
> <property>
> <name>yarn.resourcemanager.zk-num-retries</name>
> <value>1000</value>
> <source>yarn-default.xml</source>
> </property>
> <property>
> <name>yarn.resourcemanager.zk-retry-interval-ms</name>
> <value>1000</value>
> <source>yarn-default.xml</source>
> </property>
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore
> </value>
> <source>yarn-site.xml</source>
> </property>
> {code}
> While on zk side, Len error
> {code:java}
> 2019-07-29 02:14:45,809 [myid:5] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362] - Exception
> causing close of session 0x36ab902369100a0 due to java.io.IOException: Len
> error 9156576
> {code}
> After went through all zk nodes, the following large znode was found.
> Surprisingly the reason for the huge size of this znode is that, it was with
> very huge (18MB in our case) diagnostic info for ApplicationAttemptStateData.
> {code:java}
> /rmstore/ZKRMStateRoot/RMAppRoot/application_1568721711618_198456/appattempt_1568721711618_198456_000002
> dataLength:18913283
> {code}
> Just show a piece of the diagnostic info for this attemp.
> I believe in the original design no one would expect such a huge error
> message. However it happened in our cluster twice recently.
> {code:java}
> User class threw exception: org.apache.spark.sql.AnalysisException: resolved
> attribute(s) BMID#74532 missing from
> BMID#2643,LEAF_CATEG_ID#19326,letterkey#35970,item_site_id#30429,id_in_leaf#2640,ITEM_BRAND#8208
> in operator !Project [item_site_id#30429, LEAF_CATEG_ID#19326,
> letterkey#35970, ITEM_BRAND#8208, id_in_leaf#2640, BMID#74532];;
> Sort [item_id#160 ASC NULLS FIRST], false
> +- RepartitionByExpression [item_id#160]
> +- Project [item_id#160, item_vrtn_id#36412, item_site_id#531,
> BSNS_VRTCL_NAME#823, CATEG_LVL3_ID#803, LEAF_CATEG_ID#161, epid#36398,
> BMID#2643, ITEM_TITL#36413, ITEM_BRAND#36414, TITL#36394 AS PROD_TITLE#74530,
> BRAND#36395 AS PROD_BRAND#74531, SampleItem#36396]
> +- Filter isnotnull(bmid#2643)
> +- Join LeftOuter, (bmid#2643 = bmid#74532)
> :- SubqueryAlias a
> : +- SubqueryAlias cbt_bdtool_prodid_w
> : +- Project [item_id#160, CASE WHEN isnotnull(item_vrtn_id#1634) THEN
> item_vrtn_id#1634 ELSE cast(0 as decimal(18,0)) END AS item_vrtn_id#36412,
> item_site_id#531, BSNS_VRTCL_NAME#823, CATEG_LVL3_ID#803, LEAF_CATEG_ID#161,
> regexp_replace(regexp_replace(lower(auct_titl#36740),
> [^$₤£€#×^*-~@!?./&%()+=":_<>,0-9A-Za-zäöüßÄÖÜ ]+, ), s+, ) AS
> ITEM_TITL#36413, regexp_replace(regexp_replace(item_brand#1563,
> [^$₤£€#×^*-~@!?./&%()+=":_<>,0-9A-Za-zäöüßÄÖÜ ]+, ), s+, ) AS
> ITEM_BRAND#36414, epid#36398, BMID#2643]
> : +- Join LeftOuter, ((item_id#160 = item_id#36781) && (item_site_id#531 =
> item_site_id#53076))
> : :- Join LeftOuter, (item_id#160 = item_id#36778)
> : : :- Join LeftOuter, (item_id#160 = item_id#36674)
> : : : :- SubqueryAlias a
> : : : : +- SubqueryAlias cbt_bdtool_drv_epid_w
> : : : : +- Project [item_id#160, item_vrtn_id#1634, ITEM_SITE_ID#531,
> BSNS_VRTCL_NAME#823, CATEG_LVL3_ID#803, LEAF_CATEG_ID#161,
> coalesce(epid#1083, epid#1566) AS ePID#36398]
> : : : : +- Join LeftOuter, (item_id#160 = item_id#36399)
> : : : : :- SubqueryAlias drv
> : : : : : +- SubqueryAlias cbt_bdtool_drv_w
> : : : : : +- Project [item_id#160, ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823,
> CATEG_LVL3_ID#803, LEAF_CATEG_ID#161, coalesce(AT_PROD_REF_ID#1096,
> PROD_REF_ID#1097) AS ePID#1083]
> : : : : : +- Join LeftOuter, ((item_id#160 = item_id#1094) &&
> (ITEM_SITE_ID#531 = ITEM_SITE_ID#1264))
> : : : : : :- SubqueryAlias drv
> : : : : : : +- SubqueryAlias cbt_bdtool_drv3_w
> : : : : : : +- Distinct
> : : : : : : +- Project [ITEM_ID#160, ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823,
> CATEG_LVL3_ID#803, LEAF_CATEG_ID#161]
> : : : : : : +- SubqueryAlias cbt_bdtool_drv2_w
> : : : : : : +- Project [ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823,
> CATEG_LVL3_ID#803, LEAF_CATEG_ID#161, ITEM_ID#160, cum_gmv#1072,
> sec_gmv#1073, cum_pct#1074]
> : : : : : : +- Filter (cast(cum_pct#1074 as decimal(38,21)) <= cast(0.95 as
> decimal(38,21)))
> : : : : : : +- SubqueryAlias rk
> : : : : : : +- Project [ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823,
> CATEG_LVL3_ID#803, LEAF_CATEG_ID#161, ITEM_ID#160, cum_gmv#1072,
> sec_gmv#1073, cum_pct#1074]
> : : : : : : +- Project [ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823,
> CATEG_LVL3_ID#803, LEAF_CATEG_ID#161, ITEM_ID#160, GMV_PLAN_USD#162,
> cum_gmv#1072, sec_gmv#1073, _we2#1079, _we3#1080, cum_gmv#1072, sec_gmv#1073,
> CheckOverflow((promote_precision(cast(cast(_we2#1079 as decimal(18,4)) as
> decimal(18,4))) / promote_precision(cast(cast(_we3#1080 as decimal(18,4)) as
> decimal(18,4)))), DecimalType(38,21)) AS cum_pct#1074]
> : : : : : : +- Window [sum(GMV_PLAN_USD#162)
> windowspecdefinition(ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823, GMV_PLAN_USD#162
> DESC NULLS LAST, ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS
> cum_gmv#1072, sum(GMV_PLAN_USD#162) windowspecdefinition(ITEM_SITE_ID#531,
> BSNS_VRTCL_NAME#823, GMV_PLAN_USD#162 DESC NULLS LAST, ROWS BETWEEN UNBOUNDED
> PRECEDING AND UNBOUNDED FOLLOWING) AS sec_gmv#1073, sum(GMV_PLAN_USD#162)
> windowspecdefinition(ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823, GMV_PLAN_USD#162
> DESC NULLS LAST, ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS
> _we2#1079, sum(GMV_PLAN_USD#162) windowspecdefinition(ITEM_SITE_ID#531,
> BSNS_VRTCL_NAME#823, GMV_PLAN_USD#162 DESC NULLS LAST, ROWS BETWEEN UNBOUNDED
> PRECEDING AND UNBOUNDED FOLLOWING) AS _we3#1080], [ITEM_SITE_ID#531,
> BSNS_VRTCL_NAME#823], [GMV_PLAN_USD#162 DESC NULLS LAST]
> : : : : : : +- Project [ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823,
> CATEG_LVL3_ID#803, LEAF_CATEG_ID#161, ITEM_ID#160, GMV_PLAN_USD#162]
> : : : : : : +- SubqueryAlias cbt_bdtool_drv1_w
> : : : : : : +- Aggregate [LSTG_ID#310, ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823,
> CATEG_LVL3_ID#803, MOVE_TO#748], [LSTG_ID#310 AS ITEM_ID#160,
> ITEM_SITE_ID#531, BSNS_VRTCL_NAME#823, CATEG_LVL3_ID#803, MOVE_TO#748 AS
> LEAF_CATEG_ID#161,
> sum(cast(CheckOverflow((promote_precision(cast(CheckOverflow((promote_precision(cast(QTY#337
> as decimal(20,2))) * promote_precision(cast(ITEM_PRICE#336 as
> decimal(20,2)))), DecimalType(37,2)) as decimal(38,6))) *
> promote_precision(cast(CURNCY_PLAN_RATE#364 as decimal(38,6)))),
> DecimalType(38,8)) as decimal(18,2))) AS GMV_PLAN_USD#162]
> : : : : : : +- Filter ((((cast(rprtd_gmv_dt#348 as string) >= 2019-08-01) &&
> (cast(rprtd_gmv_dt#348 as string) <= 2019-08-31)) && (rprtd_wacko_yn#347 =
> N)) && (cast(LSTG_SITE_ID#321 as decimal(10,0)) IN (cast(0 as
> decimal(10,0)),cast(3 as decimal(10,0)),cast(15 as decimal(10,0)),cast(77 as
> decimal(10,0)),cast(100 as decimal(10,0))) && BSNS_VRTCL_NAME#823 IN (Parts &
> Accessories,Business & Industrial,Electronics,Lifestyle,Home & Garden)))
> : : : : : : +- Join LeftOuter, ((move_to#748 = LEAF_CATEG_ID#788) &&
> (SITE_ID#718 = SITE_ID#790))
> : : : : : : :- Join LeftOuter, ((leaf_categ_id#540 = LEAF_CATEG_ID#716) &&
> (cast(ITEM_SITE_ID#531 as decimal(9,0)) = cast(SITE_ID#718 as decimal(9,0))))
> : : : : : : : :- Join LeftOuter, ((lstg_id#310 = item_id#526) &&
> (LSTG_SITE_ID#321 = item_site_id#531))
> : : : : : : : : :- Join LeftOuter, (cast(LSTG_CURNCY_ID#323 as decimal(9,0))
> = cast(CURNCY_ID#363 as decimal(9,0)))
> : : : : : : : : : :- SubqueryAlias ck, `batch_views`.`dw_gem2_cmn_ck_i`
> : : : : : : : : : : +- Project [gen_attr_0#263 AS lstg_id#310, gen_attr_2#264
> AS ck_trans_id#311, gen_attr_4#265 AS ck_date#312, gen_attr_6#266 AS
> ck_ts#313, gen_attr_8#267 AS seller_type_cd#314, gen_attr_10#268 AS
> glbl_rprt_bsns_ctgry_group_cd#315, gen_attr_12#269 AS user_dsgntn_id#316,
> gen_attr_14#270 AS lstg_end_dt#317, gen_attr_16#271 AS byr_id#318,
> gen_attr_18#272 AS byr_cntry_id#319, gen_attr_20#273 AS leaf_categ_id#320,
> gen_attr_22#274 AS lstg_site_id#321, gen_attr_24#275 AS lstg_type_code#322,
> gen_attr_26#276 AS lstg_curncy_id#323, gen_attr_28#277 AS
> offrd_slng_chnl_grp_id#324, gen_attr_30#278 AS sold_slng_chnl_grp_id#325,
> gen_attr_32#279 AS slr_id#326, gen_attr_34#280 AS slr_cntry_id#327,
> gen_attr_36#281 AS lc_exchng_rate#328, gen_attr_38#282 AS
> slr_lc_exchng_rate#329, gen_attr_40#283 AS byr_lc_exchng_rate#330,
> gen_attr_42#284 AS lc_mnthly_exchng_rate#331, gen_attr_44#285 AS
> slr_lc_mnthly_exchng_rate#332, gen_attr_46#286 AS
> byr_lc_mnthly_exchng_rate#333, ... 23 more fields]
> : : : : : : : : : : +- SubqueryAlias gen_subquery_1
> : : : : : : : : : : +- Project [gen_attr_1#212 AS gen_attr_0#263,
> gen_attr_3#213 AS gen_attr_2#264, gen_attr_5#260 AS gen_attr_4#265,
> gen_attr_7#214 AS gen_attr_6#266, gen_attr_9#215 AS gen_attr_8#267,
> gen_attr_11#216 AS gen_attr_10#268, gen_attr_13#217 AS gen_attr_12#269,
> gen_attr_15#218 AS gen_attr_14#270, gen_attr_17#219 AS gen_attr_16#271,
> gen_attr_19#220 AS gen_attr_18#272, gen_attr_21#221 AS gen_attr_20#273,
> gen_attr_23#222 AS gen_attr_22#274, gen_attr_25#223 AS gen_attr_24#275,
> gen_attr_27#251 AS gen_attr_26#276, gen_attr_29#224 AS gen_attr_28#277,
> gen_attr_31#225 AS gen_attr_30#278, gen_attr_33#226 AS gen_attr_32#279,
> gen_attr_35#227 AS gen_attr_34#280, gen_attr_37#228 AS gen_attr_36#281,
> gen_attr_39#229 AS gen_attr_38#282, gen_attr_41#230 AS gen_attr_40#283,
> gen_attr_43#231 AS gen_attr_42#284, gen_attr_45#232 AS gen_attr_44#285,
> gen_attr_47#233 AS gen_attr_46#286, ... 23 more fields]
> : : : : : : : : : : +- SubqueryAlias dw_gem2_cmn_ck_i
> : : : : : : : : : : +- Project [gen_attr_1#212, gen_attr_3#213,
> gen_attr_5#260, gen_attr_7#214, gen_attr_9#215, gen_attr_11#216,
> gen_attr_13#217, gen_attr_15#218, gen_attr_17#219, gen_attr_19#220,
> gen_attr_21#221, gen_attr_23#222, gen_attr_25#223, gen_attr_27#251,
> gen_attr_29#224, gen_attr_31#225, gen_attr_33#226, gen_attr_35#227,
> gen_attr_37#228, gen_attr_39#229, gen_attr_41#230, gen_attr_43#231,
> gen_attr_45#232, gen_attr_47#233, ... 23 more fields]
> : : : : : : : : : : +- SubqueryAlias gen_subquery_0
> : : : : : : : : : : +- Project [gen_attr_96#163 AS gen_attr_1#212,
> gen_attr_97#164 AS gen_attr_3#213, gen_attr_98#166 AS gen_attr_7#214,
> gen_attr_99#167 AS gen_attr_9#215, gen_attr_100#168 AS gen_attr_11#216,
> gen_attr_101#169 AS gen_attr_13#217, gen_attr_102#170 AS gen_attr_15#218,
> gen_attr_103#171 AS gen_attr_17#219, gen_attr_104#172 AS gen_attr_19#220,
> gen_attr_105#173 AS gen_attr_21#221, gen_attr_106#174 AS gen_attr_23#222,
> gen_attr_107#175 AS gen_attr_25#223, gen_attr_108#176 AS gen_attr_29#224,
> gen_attr_109#177 AS gen_attr_31#225, gen_attr_110#178 AS gen_attr_33#226,
> gen_attr_111#179 AS gen_attr_35#227, gen_attr_112#180 AS gen_attr_37#228,
> gen_attr_113#181 AS gen_attr_39#229, gen_attr_114#182 AS gen_attr_41#230,
> gen_attr_115#183 AS gen_attr_43#231, gen_attr_116#184 AS gen_attr_45#232,
> gen_attr_117#185 AS gen_attr_47#233, gen_attr_118#186 AS gen_attr_49#234,
> gen_attr_119#187 AS gen_attr_51#235, ... 25 more fields]
> : : : : : : : : : : +- SubqueryAlias gen_subquery_0
> : : : : : : : : : : +- Project [lstg_id#824 AS gen_attr_96#163,
> ck_trans_id#825 AS gen_attr_97#164, ck_date#826 AS gen_attr_146#165,
> ck_ts#827 AS gen_attr_98#166, seller_type_cd#828 AS gen_attr_99#167,
> glbl_rprt_bsns_ctgry_group_cd#829 AS gen_attr_100#168, user_dsgntn_id#830 AS
> gen_attr_101#169, lstg_end_dt#831 AS gen_attr_102#170, byr_id#832 AS
> gen_attr_103#171, byr_cntry_id#833 AS gen_attr_104#172, leaf_categ_id#834 AS
> gen_attr_105#173, lstg_site_id#835 AS gen_attr_106#174, lstg_type_code#836 AS
> gen_attr_107#175, offrd_slng_chnl_grp_id#837 AS gen_attr_108#176,
> sold_slng_chnl_grp_id#838 AS gen_attr_109#177, slr_id#839 AS
> gen_attr_110#178, slr_cntry_id#840 AS gen_attr_111#179, lc_exchng_rate#841 AS
> gen_attr_112#180, slr_lc_exchng_rate#842 AS gen_attr_113#181,
> byr_lc_exchng_rate#843 AS gen_attr_114#182, lc_mnthly_exchng_rate#844 AS
> gen_attr_115#183, slr_lc_mnthly_exchng_rate#845 AS gen_attr_116#184,
> byr_lc_mnthly_exchng_rate#846 AS gen_attr_117#185, bin_lstg_yn_id#847 AS
> gen_attr_118#186, ... 25 more fields]
> : : : : : : : : : : +- SubqueryAlias dw_gem2_cmn_ck_i_tdcopy
> : : : : : : : : : : +-
> Relation[lstg_id#824,ck_trans_id#825,ck_date#826,ck_ts#827,seller_type_cd#828,glbl_rprt_bsns_ctgry_group_cd#829,user_dsgntn_id#830,lstg_end_dt#831,byr_id#832,byr_cntry_id#833,leaf_categ_id#834,lstg_site_id#835,lstg_type_code#836,offrd_slng_chnl_grp_id#837,sold_slng_chnl_grp_id#838,slr_id#839,slr_cntry_id#840,lc_exchng_rate#841,slr_lc_exchng_rate#842,byr_lc_exchng_rate#843,lc_mnthly_exchng_rate#844,slr_lc_mnthly_exchng_rate#845,byr_lc_mnthly_exchng_rate#846,bin_lstg_yn_id#847,...
> 25 more fields] parquet
> : : : : : : : : : +- SubqueryAlias fx,
> `batch_views`.`ssa_curncy_plan_rate_dim`
> : : : : : : : : : +- Project [gen_attr_0#357 AS curncy_id#363, gen_attr_1#358
> AS curncy_plan_rate#364, gen_attr_2#359 AS cre_date#365, gen_attr_3#360 AS
> cre_user#366, gen_attr_4#361 AS upd_date#367, gen_attr_5#362 AS upd_user#368]
> : : : : : : : : : +- SubqueryAlias ssa_curncy_plan_rate_dim
> : : : : : : : : : +- Project [gen_attr_0#357, gen_attr_1#358, gen_attr_2#359,
> gen_attr_3#360, gen_attr_4#361, gen_attr_5#362]
> : : : : : : : : : +- SubqueryAlias gen_subquery_0
> : : : : : : : : : +- Project [curncy_id#873 AS gen_attr_0#357,
> curncy_plan_rate#874 AS gen_attr_1#358, cre_date#875 AS gen_attr_2#359,
> cre_user#876 AS gen_attr_3#360, upd_date#877 AS gen_attr_4#361, upd_user#878
> AS gen_attr_5#362]
> : : : : : : : : : +- SubqueryAlias ssa_curncy_plan_rate_dim
> : : : : : : : : : +-
> Relation[CURNCY_ID#873,CURNCY_PLAN_RATE#874,CRE_DATE#875,CRE_USER#876,UPD_DATE#877,UPD_USER#878]
> parquet
> : : : : : : : : +- SubqueryAlias hot, `batch_views`.`dw_lstg_item`
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
