Izek Greenfield created SPARK-30332:
---------------------------------------
Summary: When running sql query with limit catalyst throw
StackOverFlow exception
Key: SPARK-30332
URL: https://issues.apache.org/jira/browse/SPARK-30332
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.0
Environment: spark version 3.0.0-preview
Reporter: Izek Greenfield
Running that SQL:
{code:sql}
SELECT BT_capital.asof_date,
BT_capital.run_id,
BT_capital.v,
BT_capital.id,
BT_capital.entity,
BT_capital.level_1,
BT_capital.level_2,
BT_capital.level_3,
BT_capital.level_4,
BT_capital.level_5,
BT_capital.level_6,
BT_capital.path_bt_capital,
BT_capital.line_item,
t0.target_line_item,
t0.line_description,
BT_capital.col_item,
BT_capital.rep_amount,
root.orgUnitId,
root.cptyId,
root.instId,
root.startDate,
root.maturityDate,
root.amount,
root.nominalAmount,
root.quantity,
root.lkupAssetLiability,
root.lkupCurrency,
root.lkupProdType,
root.interestResetDate,
root.interestResetTerm,
root.noticePeriod,
root.historicCostAmount,
root.dueDate,
root.lkupResidence,
root.lkupCountryOfUltimateRisk,
root.lkupSector,
root.lkupIndustry,
root.lkupAccountingPortfolioType,
root.lkupLoanDepositTerm,
root.lkupFixedFloating,
root.lkupCollateralType,
root.lkupRiskType,
root.lkupEligibleRefinancing,
root.lkupHedging,
root.lkupIsOwnIssued,
root.lkupIsSubordinated,
root.lkupIsQuoted,
root.lkupIsSecuritised,
root.lkupIsSecuritisedServiced,
root.lkupIsSyndicated,
root.lkupIsDeRecognised,
root.lkupIsRenegotiated,
root.lkupIsTransferable,
root.lkupIsNewBusiness,
root.lkupIsFiduciary,
root.lkupIsNonPerforming,
root.lkupIsInterGroup,
root.lkupIsIntraGroup,
root.lkupIsRediscounted,
root.lkupIsCollateral,
root.lkupIsExercised,
root.lkupIsImpaired,
root.facilityId,
root.lkupIsOTC,
root.lkupIsDefaulted,
root.lkupIsSavingsPosition,
root.lkupIsForborne,
root.lkupIsDebtRestructuringLoan,
root.interestRateAAR,
root.interestRateAPRC,
root.custom1,
root.custom2,
root.custom3,
root.lkupSecuritisationType,
root.lkupIsCashPooling,
root.lkupIsEquityParticipationGTE10,
root.lkupIsConvertible,
root.lkupEconomicHedge,
root.lkupIsNonCurrHeldForSale,
root.lkupIsEmbeddedDerivative,
root.lkupLoanPurpose,
root.lkupRegulated,
root.lkupRepaymentType,
root.glAccount,
root.lkupIsRecourse,
root.lkupIsNotFullyGuaranteed,
root.lkupImpairmentStage,
root.lkupIsEntireAmountWrittenOff,
root.lkupIsLowCreditRisk,
root.lkupIsOBSWithinIFRS9,
root.lkupIsUnderSpecialSurveillance,
root.lkupProtection,
root.lkupIsGeneralAllowance,
root.lkupSectorUltimateRisk,
root.cptyOrgUnitId,
root.name,
root.lkupNationality,
root.lkupSize,
root.lkupIsSPV,
root.lkupIsCentralCounterparty,
root.lkupIsMMRMFI,
root.lkupIsKeyManagement,
root.lkupIsOtherRelatedParty,
root.lkupResidenceProvince,
root.lkupIsTradingBook,
root.entityHierarchy_entityId,
root.entityHierarchy_Residence,
root.lkupLocalCurrency,
root.cpty_entityhierarchy_entityId,
root.lkupRelationship,
root.cpty_lkupRelationship,
root.entityNationality,
root.lkupRepCurrency,
root.startDateFinancialYear,
root.numEmployees,
root.numEmployeesTotal,
root.collateralAmount,
root.guaranteeAmount,
root.impairmentSpecificIndividual,
root.impairmentSpecificCollective,
root.impairmentGeneral,
root.creditRiskAmount,
root.provisionSpecificIndividual,
root.provisionSpecificCollective,
root.provisionGeneral,
root.writeOffAmount,
root.interest,
root.fairValueAmount,
root.grossCarryingAmount,
root.carryingAmount,
root.code,
root.lkupInstrumentType,
root.price,
root.amountAtIssue,
root.yield,
root.totalFacilityAmount,
root.facility_rate,
root.spec_indiv_est,
root.spec_coll_est,
root.coll_inc_loss,
root.impairment_amount,
root.provision_amount,
root.accumulated_impairment,
root.exclusionFlag,
root.lkupIsHoldingCompany,
root.instrument_startDate,
root.entityResidence,
fxRate.enumerator,
fxRate.lkupFromCurrency,
fxRate.rate,
fxRate.custom1,
fxRate.custom2,
fxRate.custom3,
GB_position.lkupIsECGDGuaranteed,
GB_position.lkupIsMultiAcctOffsetMortgage,
GB_position.lkupIsIndexLinked,
GB_position.lkupIsRetail,
GB_position.lkupCollateralLocation,
GB_position.percentAboveBBR,
GB_position.lkupIsMoreInArrears,
GB_position.lkupIsArrearsCapitalised,
GB_position.lkupCollateralPossession,
GB_position.lkupIsLifetimeMortgage,
GB_position.lkupLoanConcessionType,
GB_position.lkupIsMultiCurrency,
GB_position.lkupIsJointIncomeBasis,
GB_position.ratioIncomeMultiple,
GB_position.interestRate,
GB_position.exclusionFlag,
GB_position.lkupFDIDirection,
GB_position.lkupIsRTGS,
GB_positionExtended.nonRecourseFinanceAmount,
GB_positionExtended.arrearsAmount,
GB_Counterparty.lkupIsClearingFirm,
GB_Counterparty.lkupIsIntermediateFinCorp,
GB_Counterparty.lkupIsImpairedCreditHistory,
GB_Counterparty.lkupFDIRelationship FROM portfolio_41446 BT_capital
JOIN aggr_41390 root ON (root.id = BT_capital.id AND root.entity =
BT_capital.entity AND (root.instance_id =
'e3b82807-9371-44f4-9c97-d63cde21dbf2'))
JOIN aggr_41380 fxRate ON (fxRate.lkupToCurrency = root.lkupRepCurrency AND
(root.lkupCurrency = fxRate.lkupFromCurrency) AND (fxRate.instance_id =
'641f3e44-8d0a-4040-8f26-1d459c5a47a3'))
LEFT OUTER JOIN aggr_41418 GB_position ON (GB_position.id = root.id AND
(GB_position.instance_id = 'a8eb71eb-906b-4b76-a977-589618f5f1b5'))
LEFT OUTER JOIN aggr_41406 GB_positionExtended ON (GB_positionExtended.id =
GB_position.id AND (GB_positionExtended.instance_id =
'bc7415ca-8cda-4c64-a323-fdfe72b00d54'))
LEFT OUTER JOIN aggr_41410 GB_Counterparty ON (GB_Counterparty.id = root.cptyId
AND (GB_Counterparty.instance_id = '6e0d1229-c142-4411-81e2-c1cc5f636843'))
LEFT OUTER JOIN t_41233 t0 ON (t0.line_item = BT_capital.line_item)
WHERE ((BT_capital.instance_id = '4fa9a3a5-67e2-4c78-9708-76849d424123')) AND
(root.exclusionFlag!='yes'
) AND (GB_position.exclusionFlag!='yes')
AND
((((NVL(t0.target_line_item,BT_capital.line_item) = '10') AND
(BT_capital.col_item = 'Euro') AND
(BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '10') AND
(BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '10') AND
(BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '11A') AND
(BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '11A') AND
(BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '11A') AND
(BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '11B') AND
(BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '11B') AND
(BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '11B') AND
(BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19A') AND
(BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19A') AND
(BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19A') AND
(BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19B') AND
(BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19B') AND
(BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19B') AND
(BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19BL') AND
(BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19BL') AND
(BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19BL') AND
(BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19CCA') AND
(BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19CCA') AND
(BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
((NVL(t0.target_line_item,BT_capital.line_item) = '19CCA') AND
(BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002'))) AND
(BT_capital.level_1 != 'Orphans')) limit 101
{code}
I get a StackOverflow exception. remove the LIMIT at the end and all is working.
with spark version 2.4.4 it does not happen
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
