I’m writing this email to reach out to the community to demisty the py-files 
parameter when working with spark-submit and python projects.

Currently I have a project, say


  *   Main.py
  *   Modules/module1.py

When I zip up the src directory and submit it to spark via emr add step , the 
namespacing is lost.

Main.py example:
From Modules.module1 import SomeClass

My code returns and error that it cannot find this class, now this works if I 
goto the instance download my project, and submit it to spark from within the 
EMR instance via spark-submit , but not when adding it as a step in emr from 
external call.


This message is confidential, intended only for the named recipient(s) and may 
contain information that is privileged or exempt from disclosure under 
applicable law. If you are not the intended recipient(s), you are notified that 
the dissemination, distribution, or copying of this message is strictly 
prohibited. If you receive this message in error or are not the named 
recipient(s), please notify the sender by return email and delete this message. 
Thank you.

Reply via email to