Hello Experts
During a join operation, I see this error below (spark 3.0.2)
Any suggestions on how to debug?
Error:
java.lang.AssertionError: assertion failed: Found duplicate rewrite attribute
Source code:
val dfFilteredFinal=dfFiltered
.join(dfScenarioSite, Seq("tid","site"), "left_outer")
dfFiltered column names below
+--------------------+-------------------+------------+--------------+--------------------+--------------------+-----+-------------+---------------+-----------+--------------+---------------+----------+-------------+----------------------+------------------+-------------------+---------------+-----------------+-------------------------+---------------------+---------------------+------------------------+--------------------+-------------------+------------------------------+--------------------------+--------------------------+-------------------------+-------------------+-----------------------+
| tid| site| ap_mac| svrip|
swarm_id| ap_name|model|
ts|samplesmeasured|anomalycntr|
firmware|scenario_number|date_month|
date_hour|tid_ap_samplesmeasured|tid_ap_anomalycntr|tid_samplesmeasured|tid_anomalycntr|scenario_ap_array|tid_svrip_samplesmeasured|tid_svrip_anomalycntr|scenario_server_array|tid_site_samplesmeasured|tid_site_anomalycntr|scenario_site_array|tid_site_svrip_samplesmeasured|tid_site_svrip_anomalycntr|site_scenario_server_array|site_scenario_apmac_array|
scenario_array|impacted_scenario_count|
+--------------------+-------------------+------------+--------------+--------------------+--------------------+-----+-------------+---------------+-----------+--------------+---------------+----------+-------------+----------------------+------------------+-------------------+---------------+-----------------+-------------------------+---------------------+---------------------+------------------------+--------------------+-------------------+------------------------------+--------------------------+--------------------------+-------------------------+-------------------+-----------------------+
dfScenarioSite column names below
| tid|
site|site_scenario_array|impacted_site_scenario_count|
