The reason of no response is that this feature is not available yet.
You can vote and following this JIRA https://issues.apache.org/jira/browse/SPARK-13721, if you really need this feature. Yong ________________________________ From: Don Drake <[email protected]> Sent: Monday, July 25, 2016 9:12 PM To: [email protected] Subject: Fwd: Outer Explode needed No response on the Users list, I thought I would repost here. See below. -Don ---------- Forwarded message ---------- From: Don Drake <[email protected]<mailto:[email protected]>> Date: Sun, Jul 24, 2016 at 2:18 PM Subject: Outer Explode needed To: user <[email protected]<mailto:[email protected]>> I have a nested data structure (array of structures) that I'm using the DSL df.explode() API to flatten the data. However, when the array is empty, I'm not getting the rest of the row in my output as it is skipped. This is the intended behavior, and Hive supports a SQL "OUTER explode()" to generate the row when the explode would not yield any output. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LateralView Can we get this same outer explode in the DSL? I have to jump through some outer join hoops to get the rows where the array is empty. Thanks. -Don -- Donald Drake Drake Consulting http://www.drakeconsulting.com/ https://twitter.com/dondrake<http://www.MailLaunder.com/> 800-733-2143<tel:800-733-2143> -- Donald Drake Drake Consulting http://www.drakeconsulting.com/ https://twitter.com/dondrake<http://www.MailLaunder.com/> 800-733-2143
