https://issues.apache.org/jira/browse/SPARK-19224
Maciej/Holden, If its ok for I can come up with a proposal for reorganization and add the proposal to the JIRA as next steps before we break up the work? Thanks ________________________________ From: Maciej Szymkiewicz <mszymkiew...@gmail.com> Sent: Thursday, January 12, 2017 10:44 AM To: dev@spark.apache.org Subject: Re: [PYSPARK] Python tests organization Sounds good, but it looks like JIRA is still down. Personally I can look at sql.tests and see what can be done there. Depending on the resolution of https://issues.apache.org/jira/browse/SPARK-19160 I may have to adjust some tests anyway. On 01/12/2017 07:36 PM, Saikat Kanjilal wrote: Maciej? LGTM, what do you think? I can create a JIRA and drive this. ________________________________ From: Holden Karau <hol...@pigscanfly.ca><mailto:hol...@pigscanfly.ca> Sent: Thursday, January 12, 2017 10:34 AM To: Saikat Kanjilal; dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: Re: [PYSPARK] Python tests organization I'd be happy to help with reviewing Python test improvements. Maybe make an umbrella JIRA and do one sub components at a time? On Thu, Jan 12, 2017 at 12:20 PM Saikat Kanjilal <sxk1...@hotmail.com<mailto:sxk1...@hotmail.com>> wrote: Following up, any thoughts on next steps for this? ________________________________ From: Maciej Szymkiewicz <mszymkiew...@gmail.com<mailto:mszymkiew...@gmail.com>> Sent: Wednesday, January 11, 2017 10:14 AM To: Saikat Kanjilal Subject: Re: [PYSPARK] Python tests organization Not yet, I want to see if there is any consensus about it. It is a lot of tedious work and I would be shame if someone started working on this just to get it dropped. On 01/11/2017 06:44 PM, Saikat Kanjilal wrote: Hello Maciej, If there's a jira available for this I'd like to help get this moving, let me know next steps. Thanks in advance. ________________________________ From: Maciej Szymkiewicz <mszymkiew...@gmail.com><mailto:mszymkiew...@gmail.com> Sent: Wednesday, January 11, 2017 4:18 AM To: dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: [PYSPARK] Python tests organization Hi, I can't help but wonder if there is any practical reason for keeping monolithic test modules. These things are already pretty large (1500 - 2200 LOCs) and can only grow. Development aside, I assume that many users use tests the same way as me, to check the intended behavior, and largish loosely coupled modules make it harder than it should be. If there's no rationale for that it could be a good time start thinking about moving tests to packages and separating into modules reflecting project structure. -- Best, Maciej --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org<mailto:dev-unsubscr...@spark.apache.org> -- Maciej Szymkiewicz -- Maciej Szymkiewicz
