Re: [discuss] ending support for Java 6?
+1. One issue with dropping Java 6: if we use Java 7 to build the assembly jar, it will use zip64. Could Python 2.x (or even 3.x) be able to load zip64 files on PYTHONPATH? -Xiangrui On Tue, May 5, 2015 at 3:25 PM, Reynold Xin r...@databricks.com wrote: OK I sent an email. On Tue, May 5, 2015 at 2:47 PM, shane knapp skn...@berkeley.edu wrote: +1 to an announce to user and dev. java6 is so old and sad. On Tue, May 5, 2015 at 2:24 PM, Tom Graves tgraves...@yahoo.com wrote: +1. I haven't seen major objections here so I would say send announcement and see if any users have objections Tom On Tuesday, May 5, 2015 5:09 AM, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
@tgraves can chime in, but I think this pr aims to fix it: https://github.com/apache/spark/pull/5580 We should probably get that in for 1.4. On Tue, May 5, 2015 at 11:46 PM, Xiangrui Meng men...@gmail.com wrote: +1. One issue with dropping Java 6: if we use Java 7 to build the assembly jar, it will use zip64. Could Python 2.x (or even 3.x) be able to load zip64 files on PYTHONPATH? -Xiangrui On Tue, May 5, 2015 at 3:25 PM, Reynold Xin r...@databricks.com wrote: OK I sent an email. On Tue, May 5, 2015 at 2:47 PM, shane knapp skn...@berkeley.edu wrote: +1 to an announce to user and dev. java6 is so old and sad. On Tue, May 5, 2015 at 2:24 PM, Tom Graves tgraves...@yahoo.com wrote: +1. I haven't seen major objections here so I would say send announcement and see if any users have objections Tom On Tuesday, May 5, 2015 5:09 AM, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
That is correct. I plan to try it out and review it today. Tom On Wednesday, May 6, 2015 1:48 AM, Reynold Xin r...@databricks.com wrote: @tgraves can chime in, but I think this pr aims to fix it: https://github.com/apache/spark/pull/5580 We should probably get that in for 1.4. On Tue, May 5, 2015 at 11:46 PM, Xiangrui Meng men...@gmail.com wrote: +1. One issue with dropping Java 6: if we use Java 7 to build the assembly jar, it will use zip64. Could Python 2.x (or even 3.x) be able to load zip64 files on PYTHONPATH? -Xiangrui On Tue, May 5, 2015 at 3:25 PM, Reynold Xin r...@databricks.com wrote: OK I sent an email. On Tue, May 5, 2015 at 2:47 PM, shane knapp skn...@berkeley.edu wrote: +1 to an announce to user and dev. java6 is so old and sad. On Tue, May 5, 2015 at 2:24 PM, Tom Graves tgraves...@yahoo.com wrote: +1. I haven't seen major objections here so I would say send announcement and see if any users have objections Tom On Tuesday, May 5, 2015 5:09 AM, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
OK I sent an email. On Tue, May 5, 2015 at 2:47 PM, shane knapp skn...@berkeley.edu wrote: +1 to an announce to user and dev. java6 is so old and sad. On Tue, May 5, 2015 at 2:24 PM, Tom Graves tgraves...@yahoo.com wrote: +1. I haven't seen major objections here so I would say send announcement and see if any users have objections Tom On Tuesday, May 5, 2015 5:09 AM, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
+1 in favor of dropping Java1.6 support. +1 in favor of doing a wide ANNOUNCE to the user and dev groups declaring which version of Spark (sounds like 1.5) will drop support and when (if it isn¹t already posted somewhere) Spark 1.5 will release. On 5/5/15, 3:08 AM, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer. - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
+1. I haven't seen major objections here so I would say send announcement and see if any users have objections Tom On Tuesday, May 5, 2015 5:09 AM, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
+1 to an announce to user and dev. java6 is so old and sad. On Tue, May 5, 2015 at 2:24 PM, Tom Graves tgraves...@yahoo.com wrote: +1. I haven't seen major objections here so I would say send announcement and see if any users have objections Tom On Tuesday, May 5, 2015 5:09 AM, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;)
Re: [discuss] ending support for Java 6?
If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com wrote: If we just set JAVA_HOME in dev/run-test-jenkins, I think it should work. On Mon, May 4, 2015 at 7:20 PM, shane knapp skn...@berkeley.edu wrote: ...and now the workers all have java6 installed. https://issues.apache.org/jira/browse/SPARK-1437 sadly, the built-in jenkins jdk management doesn't allow us to choose a JDK version within matrix projects... so we need to manage this stuff manually. On Sun, May 3, 2015 at 8:57 AM, shane knapp skn...@berkeley.edu wrote: that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;)
Re: [discuss] ending support for Java 6?
Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
that bug predates my time at the amplab... :) anyways, just to restate: jenkins currently only builds w/java 7. if you folks need 6, i can make it happen, but it will be a (smallish) bit of work. shane On Sun, May 3, 2015 at 2:14 AM, Sean Owen so...@cloudera.com wrote: Should be, but isn't what Jenkins does. https://issues.apache.org/jira/browse/SPARK-1437 At this point it might be simpler to just decide that 1.5 will require Java 7 and then the Jenkins setup is correct. (NB: you can also solve this by setting bootclasspath to JDK 6 libs even when using javac 7+ but I think this is overly complicated.) On Sun, May 3, 2015 at 5:52 AM, Mridul Muralidharan mri...@gmail.com wrote: Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;)
Re: [discuss] ending support for Java 6?
We could build on minimum jdk we support for testing pr's - which will automatically cause build failures in case code uses newer api ? Regards, Mridul On Fri, May 1, 2015 at 2:46 PM, Reynold Xin r...@databricks.com wrote: It's really hard to inspect API calls since none of us have the Java standard library in our brain. The only way we can enforce this is to have it in Jenkins, and Tom you are currently our mini-Jenkins server :) Joking aside, looks like we should support Java 6 in 1.4, and in the release notes include a message saying starting in 1.5 we will drop Java 6 support. On Fri, May 1, 2015 at 2:00 PM, Thomas Graves tgra...@yahoo-inc.com wrote: Hey folks, 2 more things that broke jdk6 got committed last night/today. Please watch the java api's being used until we choose to deprecate jdk6. Tom On Thursday, April 30, 2015 2:04 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
+1 On Sat, May 2, 2015 at 1:09 PM, Mridul Muralidharan mri...@gmail.com wrote: We could build on minimum jdk we support for testing pr's - which will automatically cause build failures in case code uses newer api ? Regards, Mridul On Fri, May 1, 2015 at 2:46 PM, Reynold Xin r...@databricks.com wrote: It's really hard to inspect API calls since none of us have the Java standard library in our brain. The only way we can enforce this is to have it in Jenkins, and Tom you are currently our mini-Jenkins server :) Joking aside, looks like we should support Java 6 in 1.4, and in the release notes include a message saying starting in 1.5 we will drop Java 6 support. On Fri, May 1, 2015 at 2:00 PM, Thomas Graves tgra...@yahoo-inc.com wrote: Hey folks, 2 more things that broke jdk6 got committed last night/today. Please watch the java api's being used until we choose to deprecate jdk6. Tom On Thursday, April 30, 2015 2:04 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
Hi Shane, Since we are still maintaining support for jdk6, jenkins should be using jdk6 [1] to ensure we do not inadvertently use jdk7 or higher api which breaks source level compat. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) On Sat, May 2, 2015 at 1:09 PM, Mridul Muralidharan mri...@gmail.com wrote: We could build on minimum jdk we support for testing pr's - which will automatically cause build failures in case code uses newer api ? Regards, Mridul On Fri, May 1, 2015 at 2:46 PM, Reynold Xin r...@databricks.com wrote: It's really hard to inspect API calls since none of us have the Java standard library in our brain. The only way we can enforce this is to have it in Jenkins, and Tom you are currently our mini-Jenkins server :) Joking aside, looks like we should support Java 6 in 1.4, and in the release notes include a message saying starting in 1.5 we will drop Java 6 support. On Fri, May 1, 2015 at 2:00 PM, Thomas Graves tgra...@yahoo-inc.com wrote: Hey folks, 2 more things that broke jdk6 got committed last night/today. Please watch the java api's being used until we choose to deprecate jdk6. Tom On Thursday, April 30, 2015 2:04 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
It's really hard to inspect API calls since none of us have the Java standard library in our brain. The only way we can enforce this is to have it in Jenkins, and Tom you are currently our mini-Jenkins server :) Joking aside, looks like we should support Java 6 in 1.4, and in the release notes include a message saying starting in 1.5 we will drop Java 6 support. On Fri, May 1, 2015 at 2:00 PM, Thomas Graves tgra...@yahoo-inc.com wrote: Hey folks, 2 more things that broke jdk6 got committed last night/today. Please watch the java api's being used until we choose to deprecate jdk6. Tom On Thursday, April 30, 2015 2:04 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) On Sat, May 2, 2015 at 1:09 PM, Mridul Muralidharan mri...@gmail.com wrote: We could build on minimum jdk we support for testing pr's - which will automatically cause build failures in case code uses newer api ? Regards, Mridul On Fri, May 1, 2015 at 2:46 PM, Reynold Xin r...@databricks.com wrote: It's really hard to inspect API calls since none of us have the Java standard library in our brain. The only way we can enforce this is to have it in Jenkins, and Tom you are currently our mini-Jenkins server :) Joking aside, looks like we should support Java 6 in 1.4, and in the release notes include a message saying starting in 1.5 we will drop Java 6 support. On Fri, May 1, 2015 at 2:00 PM, Thomas Graves tgra...@yahoo-inc.com wrote: Hey folks, 2 more things that broke jdk6 got committed last night/today. Please watch the java api's being used until we choose to deprecate jdk6. Tom On Thursday, April 30, 2015 2:04 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
i think i might be misunderstanding, but shouldnt java 6 currently be used in jenkins? On Sat, May 2, 2015 at 11:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) On Sat, May 2, 2015 at 1:09 PM, Mridul Muralidharan mri...@gmail.com wrote: We could build on minimum jdk we support for testing pr's - which will automatically cause build failures in case code uses newer api ? Regards, Mridul On Fri, May 1, 2015 at 2:46 PM, Reynold Xin r...@databricks.com wrote: It's really hard to inspect API calls since none of us have the Java standard library in our brain. The only way we can enforce this is to have it in Jenkins, and Tom you are currently our mini-Jenkins server :) Joking aside, looks like we should support Java 6 in 1.4, and in the release notes include a message saying starting in 1.5 we will drop Java 6 support. On Fri, May 1, 2015 at 2:00 PM, Thomas Graves tgra...@yahoo-inc.com wrote: Hey folks, 2 more things that broke jdk6 got committed last night/today. Please watch the java api's being used until we choose to deprecate jdk6. Tom On Thursday, April 30, 2015 2:04 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
+1 for this think it's high time. We should of course do it with enough warning for users. 1.4 May be too early (not for me though!). Perhaps we specify that 1.5 will officially move to JDK7? — Sent from Mailbox On Fri, May 1, 2015 at 12:16 AM, Ram Sriharsha harsh...@yahoo-inc.com.invalid wrote: +1 for end of support for Java 6 On Thursday, April 30, 2015 3:08 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: FYI, after enough consideration, we the Hadoop community dropped support for JDK 6 starting release Apache Hadoop 2.7.x. Thanks +Vinod On Apr 30, 2015, at 12:02 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
On 30 Apr 2015, at 21:40, Marcelo Vanzin van...@cloudera.com wrote: As for the idea, I'm +1. Spark is the only reason I still have jdk6 around - exactly because I don't want to cause the issue that started this discussion (inadvertently using JDK7 APIs). And as has been pointed out, even J7 is about to go EOL real soon. +1, perhaps with a roadmap for people to plan for Even Hadoop is moving away (I think 2.7 will be j7-only). Hive 1.1 is already j7-only. And when Hadoop moves away from something, it's an event worthy of headlines. The constraint here was that there were too many people stuck in Java 6, and java 7 wasn't compelling enough to pull people off a JVM they trusted to be stable at large scale. One problem with production hadoop is that across 5000 14-core servers, all race conditions will surface —leading to a reluctance to upgrade JVMs or even OS's. There was also the fact that for a long time Hadoop wouldn't build on OSX on Java 7 (HADOOP-9350). Even today, OS/X's JDK has better rendering than java7+, leaving it nice to have around for the IDEs. After Hadoop 2.5 shipped an announcement was made that 2.6 would be the last Java 1.6 release, with the switch taking place in November. Moving ASF Jenkins up was probably the hardest bit ( HADOOP-10530 ). Switching to JDK7 has enabled moving kerberos support to Java 8 (HADOOP-10786; some changes in the internal kerberos classes used directly for kerberos to work properly). See HADOOP-11090 for the JDK8 migration; Hadoop trunk will be switching to Java 8 before long They're still on Jetty 6! While moving off Jetty entirely wherever possible, leaving jetty 6 on the transitive-maven-classpath in the hope of not breaking code that expects it to be there. It's not that the project likes Jetty 6 (there are threads whose sole aim is to detect jetty startup failures), but that moving off it is felt to be better than upgrading. As for pyspark, https://github.com/apache/spark/pull/5580 should get rid of the last incompatibility with large assemblies, by keeping the python files in separate archives. If we remove support for Java 6, then we don't need to worry about the size of the assembly anymore. zzhang's patch drops to Java 6 just to rebuild the assembly Jar; you can still build Java7-only classes. So it will work even before the pyspark patch goes in.
Re: [discuss] ending support for Java 6?
On 1 May 2015 at 21:26, Dean Wampler deanwamp...@gmail.com wrote: FWIW, another reason to start planning for deprecation of Java 7, too, is that Scala 2.12 will require Java 8. Scala 2.12 will be released early next year. Will 2.12 be the release that based on dotty https://github.com/lampepfl/dotty? Cheers, Steve.
Re: [discuss] ending support for Java 6?
FWIW, another reason to start planning for deprecation of Java 7, too, is that Scala 2.12 will require Java 8. Scala 2.12 will be released early next year. Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition http://shop.oreilly.com/product/0636920033073.do (O'Reilly) Typesafe http://typesafe.com @deanwampler http://twitter.com/deanwampler http://polyglotprogramming.com On Thu, Apr 30, 2015 at 3:37 PM, Ted Yu yuzhih...@gmail.com wrote: +1 on ending support for Java 6. BTW from https://www.java.com/en/download/faq/java_7.xml : After April 2015, Oracle will no longer post updates of Java SE 7 to its public download sites. On Thu, Apr 30, 2015 at 1:34 PM, Punyashloka Biswal punya.bis...@gmail.com wrote: I'm in favor of ending support for Java 6. We should also articulate a policy on how long we want to support current and future versions of Java after Oracle declares them EOL (Java 7 will be in that bucket in a matter of days). Punya On Thu, Apr 30, 2015 at 1:18 PM shane knapp skn...@berkeley.edu wrote: something to keep in mind: we can easily support java 6 for the build environment, particularly if there's a definite EOL. i'd like to fix our java versioning 'problem', and this could be a big instigator... right now we're hackily setting java_home in test invocation on jenkins, which really isn't the best. if i decide, within jenkins, to reconfigure every build to 'do the right thing' WRT java version, then i will clean up the old mess and pay down on some technical debt. or i can just install java 6 and we use that as JAVA_HOME on a build-by-build basis. this will be a few days of prep and another morning-long downtime if i do the right thing (within jenkins), and only a couple of hours the hacky way (system level). either way, we can test on java 6. :) On Thu, Apr 30, 2015 at 1:00 PM, Koert Kuipers ko...@tresata.com wrote: nicholas started it! :) for java 6 i would have said the same thing about 1 year ago: it is foolish to drop it. but i think the time is right about now. about half our clients are on java 7 and the other half have active plans to migrate to it within 6 months. On Thu, Apr 30, 2015 at 3:57 PM, Reynold Xin r...@databricks.com wrote: Guys thanks for chiming in, but please focus on Java here. Python is an entirely separate issue. On Thu, Apr 30, 2015 at 12:53 PM, Koert Kuipers ko...@tresata.com wrote: i am not sure eol means much if it is still actively used. we have a lot of clients with centos 5 (for which we still support python 2.4 in some form or another, fun!). most of them are on centos 6, which means python 2.6. by cutting out python 2.6 you would cut out the majority of the actual clusters i am aware of. unless you intention is to truly make something academic i dont think that is wise. On Thu, Apr 30, 2015 at 3:48 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that’s for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
No. That will be 3.0 some day Sent from my rotary phone. On May 1, 2015, at 9:04 AM, Steven Shaw ste...@steshaw.org wrote: On 1 May 2015 at 21:26, Dean Wampler deanwamp...@gmail.com wrote: FWIW, another reason to start planning for deprecation of Java 7, too, is that Scala 2.12 will require Java 8. Scala 2.12 will be released early next year. Will 2.12 be the release that based on dotty? Cheers, Steve.
Re: [discuss] ending support for Java 6?
it seems spark is happy to upgrade scala, drop older java versions, upgrade incompatible library versions (akka), and all of this within spark 1.x does the 1.x mean anything in terms of compatibility of dependencies? or is that limited to its own api? what are the rules? On May 1, 2015 9:04 AM, Steven Shaw ste...@steshaw.org wrote: On 1 May 2015 at 21:26, Dean Wampler deanwamp...@gmail.com wrote: FWIW, another reason to start planning for deprecation of Java 7, too, is that Scala 2.12 will require Java 8. Scala 2.12 will be released early next year. Will 2.12 be the release that based on dotty https://github.com/lampepfl/dotty? Cheers, Steve.
Re: [discuss] ending support for Java 6?
i am not sure eol means much if it is still actively used. we have a lot of clients with centos 5 (for which we still support python 2.4 in some form or another, fun!). most of them are on centos 6, which means python 2.6. by cutting out python 2.6 you would cut out the majority of the actual clusters i am aware of. unless you intention is to truly make something academic i dont think that is wise. On Thu, Apr 30, 2015 at 3:48 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that’s for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
something to keep in mind: we can easily support java 6 for the build environment, particularly if there's a definite EOL. i'd like to fix our java versioning 'problem', and this could be a big instigator... right now we're hackily setting java_home in test invocation on jenkins, which really isn't the best. if i decide, within jenkins, to reconfigure every build to 'do the right thing' WRT java version, then i will clean up the old mess and pay down on some technical debt. or i can just install java 6 and we use that as JAVA_HOME on a build-by-build basis. this will be a few days of prep and another morning-long downtime if i do the right thing (within jenkins), and only a couple of hours the hacky way (system level). either way, we can test on java 6. :) On Thu, Apr 30, 2015 at 1:00 PM, Koert Kuipers ko...@tresata.com wrote: nicholas started it! :) for java 6 i would have said the same thing about 1 year ago: it is foolish to drop it. but i think the time is right about now. about half our clients are on java 7 and the other half have active plans to migrate to it within 6 months. On Thu, Apr 30, 2015 at 3:57 PM, Reynold Xin r...@databricks.com wrote: Guys thanks for chiming in, but please focus on Java here. Python is an entirely separate issue. On Thu, Apr 30, 2015 at 12:53 PM, Koert Kuipers ko...@tresata.com wrote: i am not sure eol means much if it is still actively used. we have a lot of clients with centos 5 (for which we still support python 2.4 in some form or another, fun!). most of them are on centos 6, which means python 2.6. by cutting out python 2.6 you would cut out the majority of the actual clusters i am aware of. unless you intention is to truly make something academic i dont think that is wise. On Thu, Apr 30, 2015 at 3:48 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that’s for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
I'd also support this. In general, I think it's good that we try to have Spark support different versions of things (Hadoop, Hive, etc). But at some point you need to weigh the costs of doing so against the number of users affected. In the case of Java 6, we are seeing increasing cost from this. Some of the newer unsafe code is not supported in Java 6 (and it's a pretty large internal initiative). And the ability to upgrade dependencies is starting to cause pain for users. Sean and I had to wontfix an important bug fix for users because the library requires JRE 7. On Thu, Apr 30, 2015 at 1:00 PM, Koert Kuipers ko...@tresata.com wrote: nicholas started it! :) for java 6 i would have said the same thing about 1 year ago: it is foolish to drop it. but i think the time is right about now. about half our clients are on java 7 and the other half have active plans to migrate to it within 6 months. On Thu, Apr 30, 2015 at 3:57 PM, Reynold Xin r...@databricks.com wrote: Guys thanks for chiming in, but please focus on Java here. Python is an entirely separate issue. On Thu, Apr 30, 2015 at 12:53 PM, Koert Kuipers ko...@tresata.com wrote: i am not sure eol means much if it is still actively used. we have a lot of clients with centos 5 (for which we still support python 2.4 in some form or another, fun!). most of them are on centos 6, which means python 2.6. by cutting out python 2.6 you would cut out the majority of the actual clusters i am aware of. unless you intention is to truly make something academic i dont think that is wise. On Thu, Apr 30, 2015 at 3:48 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
I'm in favor of ending support for Java 6. We should also articulate a policy on how long we want to support current and future versions of Java after Oracle declares them EOL (Java 7 will be in that bucket in a matter of days). Punya On Thu, Apr 30, 2015 at 1:18 PM shane knapp skn...@berkeley.edu wrote: something to keep in mind: we can easily support java 6 for the build environment, particularly if there's a definite EOL. i'd like to fix our java versioning 'problem', and this could be a big instigator... right now we're hackily setting java_home in test invocation on jenkins, which really isn't the best. if i decide, within jenkins, to reconfigure every build to 'do the right thing' WRT java version, then i will clean up the old mess and pay down on some technical debt. or i can just install java 6 and we use that as JAVA_HOME on a build-by-build basis. this will be a few days of prep and another morning-long downtime if i do the right thing (within jenkins), and only a couple of hours the hacky way (system level). either way, we can test on java 6. :) On Thu, Apr 30, 2015 at 1:00 PM, Koert Kuipers ko...@tresata.com wrote: nicholas started it! :) for java 6 i would have said the same thing about 1 year ago: it is foolish to drop it. but i think the time is right about now. about half our clients are on java 7 and the other half have active plans to migrate to it within 6 months. On Thu, Apr 30, 2015 at 3:57 PM, Reynold Xin r...@databricks.com wrote: Guys thanks for chiming in, but please focus on Java here. Python is an entirely separate issue. On Thu, Apr 30, 2015 at 12:53 PM, Koert Kuipers ko...@tresata.com wrote: i am not sure eol means much if it is still actively used. we have a lot of clients with centos 5 (for which we still support python 2.4 in some form or another, fun!). most of them are on centos 6, which means python 2.6. by cutting out python 2.6 you would cut out the majority of the actual clusters i am aware of. unless you intention is to truly make something academic i dont think that is wise. On Thu, Apr 30, 2015 at 3:48 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that’s for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
Hi Team, Should we take this opportunity to layout and evangelize a pattern for EOL of dependencies.I propose, we follow the official EOL of java, python, scala, .And add say 6-12-24 months depending on the popularity. Java 6 official EOL Feb 2013Add 6-12 monthsAug 2013 - Feb 2014 official End of Support for Java 6 in SparkAnnounce 3-6 months prior to EOS. Thanking you. With Regards Sree On Thursday, April 30, 2015 1:41 PM, Marcelo Vanzin van...@cloudera.com wrote: As for the idea, I'm +1. Spark is the only reason I still have jdk6 around - exactly because I don't want to cause the issue that started this discussion (inadvertently using JDK7 APIs). And as has been pointed out, even J7 is about to go EOL real soon. Even Hadoop is moving away (I think 2.7 will be j7-only). Hive 1.1 is already j7-only. And when Hadoop moves away from something, it's an event worthy of headlines. They're still on Jetty 6! As for pyspark, https://github.com/apache/spark/pull/5580 should get rid of the last incompatibility with large assemblies, by keeping the python files in separate archives. If we remove support for Java 6, then we don't need to worry about the size of the assembly anymore. On Thu, Apr 30, 2015 at 1:32 PM, Sean Owen so...@cloudera.com wrote: I'm firmly in favor of this. It would also fix https://issues.apache.org/jira/browse/SPARK-7009 and avoid any more of the long-standing 64K file limit thing that's still a problem for PySpark. -- Marcelo - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
I'm firmly in favor of this. It would also fix https://issues.apache.org/jira/browse/SPARK-7009 and avoid any more of the long-standing 64K file limit thing that's still a problem for PySpark. As a point of reference, CDH5 has never supported Java 6, and it was released over a year ago. On Thu, Apr 30, 2015 at 8:02 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
As for the idea, I'm +1. Spark is the only reason I still have jdk6 around - exactly because I don't want to cause the issue that started this discussion (inadvertently using JDK7 APIs). And as has been pointed out, even J7 is about to go EOL real soon. Even Hadoop is moving away (I think 2.7 will be j7-only). Hive 1.1 is already j7-only. And when Hadoop moves away from something, it's an event worthy of headlines. They're still on Jetty 6! As for pyspark, https://github.com/apache/spark/pull/5580 should get rid of the last incompatibility with large assemblies, by keeping the python files in separate archives. If we remove support for Java 6, then we don't need to worry about the size of the assembly anymore. On Thu, Apr 30, 2015 at 1:32 PM, Sean Owen so...@cloudera.com wrote: I'm firmly in favor of this. It would also fix https://issues.apache.org/jira/browse/SPARK-7009 and avoid any more of the long-standing 64K file limit thing that's still a problem for PySpark. -- Marcelo - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
[discuss] ending support for Java 6?
This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
nicholas started it! :) for java 6 i would have said the same thing about 1 year ago: it is foolish to drop it. but i think the time is right about now. about half our clients are on java 7 and the other half have active plans to migrate to it within 6 months. On Thu, Apr 30, 2015 at 3:57 PM, Reynold Xin r...@databricks.com wrote: Guys thanks for chiming in, but please focus on Java here. Python is an entirely separate issue. On Thu, Apr 30, 2015 at 12:53 PM, Koert Kuipers ko...@tresata.com wrote: i am not sure eol means much if it is still actively used. we have a lot of clients with centos 5 (for which we still support python 2.4 in some form or another, fun!). most of them are on centos 6, which means python 2.6. by cutting out python 2.6 you would cut out the majority of the actual clusters i am aware of. unless you intention is to truly make something academic i dont think that is wise. On Thu, Apr 30, 2015 at 3:48 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that’s for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
+1 on ending support for Java 6. BTW from https://www.java.com/en/download/faq/java_7.xml : After April 2015, Oracle will no longer post updates of Java SE 7 to its public download sites. On Thu, Apr 30, 2015 at 1:34 PM, Punyashloka Biswal punya.bis...@gmail.com wrote: I'm in favor of ending support for Java 6. We should also articulate a policy on how long we want to support current and future versions of Java after Oracle declares them EOL (Java 7 will be in that bucket in a matter of days). Punya On Thu, Apr 30, 2015 at 1:18 PM shane knapp skn...@berkeley.edu wrote: something to keep in mind: we can easily support java 6 for the build environment, particularly if there's a definite EOL. i'd like to fix our java versioning 'problem', and this could be a big instigator... right now we're hackily setting java_home in test invocation on jenkins, which really isn't the best. if i decide, within jenkins, to reconfigure every build to 'do the right thing' WRT java version, then i will clean up the old mess and pay down on some technical debt. or i can just install java 6 and we use that as JAVA_HOME on a build-by-build basis. this will be a few days of prep and another morning-long downtime if i do the right thing (within jenkins), and only a couple of hours the hacky way (system level). either way, we can test on java 6. :) On Thu, Apr 30, 2015 at 1:00 PM, Koert Kuipers ko...@tresata.com wrote: nicholas started it! :) for java 6 i would have said the same thing about 1 year ago: it is foolish to drop it. but i think the time is right about now. about half our clients are on java 7 and the other half have active plans to migrate to it within 6 months. On Thu, Apr 30, 2015 at 3:57 PM, Reynold Xin r...@databricks.com wrote: Guys thanks for chiming in, but please focus on Java here. Python is an entirely separate issue. On Thu, Apr 30, 2015 at 12:53 PM, Koert Kuipers ko...@tresata.com wrote: i am not sure eol means much if it is still actively used. we have a lot of clients with centos 5 (for which we still support python 2.4 in some form or another, fun!). most of them are on centos 6, which means python 2.6. by cutting out python 2.6 you would cut out the majority of the actual clusters i am aware of. unless you intention is to truly make something academic i dont think that is wise. On Thu, Apr 30, 2015 at 3:48 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that’s for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
(On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that’s for another thread.) (To continue the parenthetical, Python 2.6 was in fact EOL-ed in October of 2013. https://www.python.org/download/releases/2.6.9/) On Thu, Apr 30, 2015 at 3:18 PM Nicholas Chammas nicholas.cham...@gmail.com wrote: I understand the concern about cutting out users who still use Java 6, and I don't have numbers about how many people are still using Java 6. But I want to say at a high level that I support deprecating older versions of stuff to reduce our maintenance burden and let us use more modern patterns in our code. Maintenance always costs way more than initial development over the lifetime of a project, and for that reason anti-support is just as important as support. (On that note, I think Python 2.6 should be next on the chopping block sometime later this year, but that's for another thread.) Nick On Thu, Apr 30, 2015 at 3:03 PM Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920
Re: [discuss] ending support for Java 6?
FYI, after enough consideration, we the Hadoop community dropped support for JDK 6 starting release Apache Hadoop 2.7.x. Thanks +Vinod On Apr 30, 2015, at 12:02 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
But it is hard to know how long customers stay with their most recent download. Cheers On Thu, Apr 30, 2015 at 2:26 PM, Sree V sree_at_ch...@yahoo.com.invalid wrote: If there is any possibility of getting the download counts,then we can use it as EOS criteria as well.Say, if download counts are lower than 30% (or another number) of Life time highest,then it qualifies for EOS. Thanking you. With Regards Sree On Thursday, April 30, 2015 2:22 PM, Sree V sree_at_ch...@yahoo.com.INVALID wrote: Hi Team, Should we take this opportunity to layout and evangelize a pattern for EOL of dependencies.I propose, we follow the official EOL of java, python, scala, .And add say 6-12-24 months depending on the popularity. Java 6 official EOL Feb 2013Add 6-12 monthsAug 2013 - Feb 2014 official End of Support for Java 6 in SparkAnnounce 3-6 months prior to EOS. Thanking you. With Regards Sree On Thursday, April 30, 2015 1:41 PM, Marcelo Vanzin van...@cloudera.com wrote: As for the idea, I'm +1. Spark is the only reason I still have jdk6 around - exactly because I don't want to cause the issue that started this discussion (inadvertently using JDK7 APIs). And as has been pointed out, even J7 is about to go EOL real soon. Even Hadoop is moving away (I think 2.7 will be j7-only). Hive 1.1 is already j7-only. And when Hadoop moves away from something, it's an event worthy of headlines. They're still on Jetty 6! As for pyspark, https://github.com/apache/spark/pull/5580 should get rid of the last incompatibility with large assemblies, by keeping the python files in separate archives. If we remove support for Java 6, then we don't need to worry about the size of the assembly anymore. On Thu, Apr 30, 2015 at 1:32 PM, Sean Owen so...@cloudera.com wrote: I'm firmly in favor of this. It would also fix https://issues.apache.org/jira/browse/SPARK-7009 and avoid any more of the long-standing 64K file limit thing that's still a problem for PySpark. -- Marcelo - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
+1 for end of support for Java 6 On Thursday, April 30, 2015 3:08 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: FYI, after enough consideration, we the Hadoop community dropped support for JDK 6 starting release Apache Hadoop 2.7.x. Thanks +Vinod On Apr 30, 2015, at 12:02 PM, Reynold Xin r...@databricks.com wrote: This has been discussed a few times in the past, but now Oracle has ended support for Java 6 for over a year, I wonder if we should just drop Java 6 support. There is one outstanding issue Tom has brought to my attention: PySpark on YARN doesn't work well with Java 7/8, but we have an outstanding pull request to fix that. https://issues.apache.org/jira/browse/SPARK-6869 https://issues.apache.org/jira/browse/SPARK-1920 - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [discuss] ending support for Java 6?
If there is any possibility of getting the download counts,then we can use it as EOS criteria as well.Say, if download counts are lower than 30% (or another number) of Life time highest,then it qualifies for EOS. Thanking you. With Regards Sree On Thursday, April 30, 2015 2:22 PM, Sree V sree_at_ch...@yahoo.com.INVALID wrote: Hi Team, Should we take this opportunity to layout and evangelize a pattern for EOL of dependencies.I propose, we follow the official EOL of java, python, scala, .And add say 6-12-24 months depending on the popularity. Java 6 official EOL Feb 2013Add 6-12 monthsAug 2013 - Feb 2014 official End of Support for Java 6 in SparkAnnounce 3-6 months prior to EOS. Thanking you. With Regards Sree On Thursday, April 30, 2015 1:41 PM, Marcelo Vanzin van...@cloudera.com wrote: As for the idea, I'm +1. Spark is the only reason I still have jdk6 around - exactly because I don't want to cause the issue that started this discussion (inadvertently using JDK7 APIs). And as has been pointed out, even J7 is about to go EOL real soon. Even Hadoop is moving away (I think 2.7 will be j7-only). Hive 1.1 is already j7-only. And when Hadoop moves away from something, it's an event worthy of headlines. They're still on Jetty 6! As for pyspark, https://github.com/apache/spark/pull/5580 should get rid of the last incompatibility with large assemblies, by keeping the python files in separate archives. If we remove support for Java 6, then we don't need to worry about the size of the assembly anymore. On Thu, Apr 30, 2015 at 1:32 PM, Sean Owen so...@cloudera.com wrote: I'm firmly in favor of this. It would also fix https://issues.apache.org/jira/browse/SPARK-7009 and avoid any more of the long-standing 64K file limit thing that's still a problem for PySpark. -- Marcelo - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
