damccorm commented on code in PR #24204:
URL: https://github.com/apache/beam/pull/24204#discussion_r1058456102
##########
.test-infra/jenkins/job_PreCommit_Python.groovy:
##########
@@ -22,6 +22,9 @@ PrecommitJobBuilder builder = new PrecommitJobBuilder(
scope: this,
nameBase: 'Python',
gradleTask: ':pythonPreCommit',
+ gradleSwitches: [
Review Comment:
Good catch - I did miss that.
> In general, I am thinking about an exclude list for this "remaining"
precommit may be more preferrable, so that we do not miss, and no need to
change here when new module being added, however not sure how to implement this
either... @tvalentyn any thought?
I agree that this would be cleaner and was considering a more significant
refactor to do this. You can accomplish this with the `ignore` or `ignore-glob`
options
(https://stackoverflow.com/questions/11117062/how-to-tell-py-test-to-skip-certain-directories),
but it requires a more significant refactor because that can't be passed as a
posarg and there's currently not a clean way to pass that through tox.ini. I
initially started with that approach before deciding it would make the PR too
big.
That's fixable, but I'd probably vote we scope this PR to the current change
and address that as a follow up so that we're splitting the "is this a
reasonable split of our suites?" question from the bigger refactor required to
make that happen. I'll add an issue for that before I merge this PR if we have
agreement, I'll wait until we have consensus to do so though. If we go with the
current approach, I'll take on the refactor next week.
##########
.test-infra/jenkins/job_PreCommit_Python_Dataframe.groovy:
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import PrecommitJobBuilder
+
+PrecommitJobBuilder builder = new PrecommitJobBuilder(
+ scope: this,
+ nameBase: 'Python_Dataframe',
+ gradleTask: ':pythonPreCommit',
+ gradleSwitches: [
+ '-Pposargs=apache_beam/dataframe/'
+ ],
+ timeoutMins: 180,
+ triggerPathPatterns: [
Review Comment:
Right - I actually don't plan on changing the trigger pattern for any of
these since the dependency logic isn't clear for any of them. For example, for
Dataframes a change to an IO could actually break some of the tests I think
since Dataframes can rely on specific IO behavior (I think). A change to most
directories still have potential to impact all of these suites unfortunately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
