[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301134#comment-15301134 ] Satish Subhashrao Saley commented on OOZIE-2482: Hi [~rkanter] and [~gezapeti], Setting

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300990#comment-15300990 ] Hadoop QA commented on OOZIE-2482: -- Testing JIRA OOZIE-2482 Cleaning local git workspace

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300765#comment-15300765 ] Peter Cseh commented on OOZIE-2482: --- Fixed up the documentation to explain that py files should be

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300706#comment-15300706 ] Robert Kanter commented on OOZIE-2482: -- One last thing on the -5 patch: A dependency on commons-io is

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299883#comment-15299883 ] Hadoop QA commented on OOZIE-2482: -- Testing JIRA OOZIE-2482 Cleaning local git workspace

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299627#comment-15299627 ] Peter Cseh commented on OOZIE-2482: --- Thank you for the review [~rkanter]. 1) changed to: These files

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299448#comment-15299448 ] Hadoop QA commented on OOZIE-2482: -- Testing JIRA OOZIE-2482 Cleaning local git workspace

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299376#comment-15299376 ] Robert Kanter commented on OOZIE-2482: -- Thanks [~satishsaley] and [~gezapeti] for working on this.

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299351#comment-15299351 ] Robert Kanter commented on OOZIE-2482: -- Now that OOZIE-2532 is in, the patch should apply, so I've

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297418#comment-15297418 ] Hadoop QA commented on OOZIE-2482: -- Testing JIRA OOZIE-2482 Cleaning local git workspace

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297354#comment-15297354 ] Peter Cseh commented on OOZIE-2482: --- Thank you for the great work [~satishsaley]! I managed to modify

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296530#comment-15296530 ] Hadoop QA commented on OOZIE-2482: -- Testing JIRA OOZIE-2482 Cleaning local git workspace

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293604#comment-15293604 ] Alexandre Linte commented on OOZIE-2482: Hi [~satishsaley], I won't be able to give you the logs

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292628#comment-15292628 ] Satish Subhashrao Saley commented on OOZIE-2482: Thank you for review Robert. I set

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291537#comment-15291537 ] Satish Subhashrao Saley commented on OOZIE-2482: Could please share the logs for

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290655#comment-15290655 ] Alexandre Linte commented on OOZIE-2482: Hi [~satishsaley], sorry for the delay. I tried your

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290255#comment-15290255 ] Robert Kanter commented on OOZIE-2482: -- Here's some feedback on the 2 patch: # Rename TestPyspark to

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287015#comment-15287015 ] Marcelo Vanzin commented on OOZIE-2482: --- `PYSPARK_ARCHIVES_PATH` won't be removed unless there's a

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286997#comment-15286997 ] Peter Cseh commented on OOZIE-2482: --- It seems like the patch command does not handle binary files. I've

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285851#comment-15285851 ] Robert Kanter commented on OOZIE-2482: -- By the way, [~satishsaley] and [~gezapeti], if you use the

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285831#comment-15285831 ] Peter Cseh commented on OOZIE-2482: --- Regarding environment variables: I think the solution with

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285778#comment-15285778 ] Hadoop QA commented on OOZIE-2482: -- Testing JIRA OOZIE-2482 Cleaning local git workspace

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285702#comment-15285702 ] Satish Subhashrao Saley commented on OOZIE-2482: Setting up SPARK_HOME=. will work as

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285594#comment-15285594 ] Robert Kanter commented on OOZIE-2482: -- I suppose the other option is to make it so that the user has

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285505#comment-15285505 ] Peter Cseh commented on OOZIE-2482: --- I have an ugly way to extract the py files and create the

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285023#comment-15285023 ] Satish Subhashrao Saley commented on OOZIE-2482: Tests are failing because jenkins is

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284965#comment-15284965 ] Robert Kanter commented on OOZIE-2482: -- Thanks [~satishsaley] for working on this. [~gezapeti] has

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283775#comment-15283775 ] Hadoop QA commented on OOZIE-2482: -- Testing JIRA OOZIE-2482 Cleaning local git workspace

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267198#comment-15267198 ] Satish Subhashrao Saley commented on OOZIE-2482: Earlier I tried pyspark with

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266919#comment-15266919 ] Robert Kanter commented on OOZIE-2482: -- The Spark Action actually will have the spark-default.conf

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250512#comment-15250512 ] Mike Grimes commented on OOZIE-2482: Based on some of the comments above it looks like the issue is

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245547#comment-15245547 ] Ferenc Denes commented on OOZIE-2482: - Please feel free to work on that, I'm not close to the solution

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15243898#comment-15243898 ] Satish Subhashrao Saley commented on OOZIE-2482: Hi [~BigDataOrange], I set SPARK_HOME

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242873#comment-15242873 ] Satish Subhashrao Saley commented on OOZIE-2482: [~fdenes] Have you resolved the issue

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242608#comment-15242608 ] Alexandre Linte commented on OOZIE-2482: Hi [~satishsaley], Thank you for the replay. My bad, the

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242102#comment-15242102 ] Satish Subhashrao Saley commented on OOZIE-2482: Hi [~BigDataOrange], Could you please

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223847#comment-15223847 ] Alexandre Linte commented on OOZIE-2482: Hi [~murali.msse], I tried your solution this morning. I

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15221150#comment-15221150 ] Murali Ramasami commented on OOZIE-2482: [~BigDataOrange] Please specify the SPARK_HOME in your

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209920#comment-15209920 ] Alexandre Linte commented on OOZIE-2482: Hi [~murali.msse], Thank you for your feedback, I'm

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204386#comment-15204386 ] Murali Ramasami commented on OOZIE-2482: Also, can you tell me which mode you have tried. I have

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

[ https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204383#comment-15204383 ] Murali Ramasami commented on OOZIE-2482: Can you try setting the SPARK_HOME in your hadoop-env.sh