[beam] branch release-2.19.0 updated: Add version guards to requirements file for integration tests.
This is an automated email from the ASF dual-hosted git repository. boyuanz pushed a commit to branch release-2.19.0 in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/release-2.19.0 by this push: new b87e306 Add version guards to requirements file for integration tests. new ee43260 Merge pull request #10619 from tvalentyn/cp_10568 b87e306 is described below commit b87e3060d62b3b2fd5254bb604be5a47bd5a8c6a Author: Valentyn Tymofieiev AuthorDate: Mon Jan 13 11:15:27 2020 -0800 Add version guards to requirements file for integration tests. --- sdks/python/scripts/run_integration_test.sh | 6 -- sdks/python/setup.py| 5 +++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/sdks/python/scripts/run_integration_test.sh b/sdks/python/scripts/run_integration_test.sh index 1133147..0804ace 100755 --- a/sdks/python/scripts/run_integration_test.sh +++ b/sdks/python/scripts/run_integration_test.sh @@ -194,8 +194,10 @@ if [[ -z $PIPELINE_OPTS ]]; then fi # Install test dependencies for ValidatesRunner tests. - echo "pyhamcrest" > postcommit_requirements.txt - echo "mock" >> postcommit_requirements.txt + # pyhamcrest==1.10.0 doesn't work on Py2. + # See: https://github.com/hamcrest/PyHamcrest/issues/131. + echo "pyhamcrest!=1.10.0,<2.0.0" > postcommit_requirements.txt + echo "mock<3.0.0" >> postcommit_requirements.txt # Options used to run testing pipeline on Cloud Dataflow Service. Also used for # running on DirectRunner (some options ignored). diff --git a/sdks/python/setup.py b/sdks/python/setup.py index 8599b7f..65bfdbd 100644 --- a/sdks/python/setup.py +++ b/sdks/python/setup.py @@ -181,8 +181,9 @@ REQUIRED_TEST_PACKAGES = [ 'nose_xunitmp>=0.4.1', 'pandas>=0.23.4,<0.25', 'parameterized>=0.6.0,<0.8.0', -# pyhamcrest==1.10.0 requires Py3. Beam still supports Py2. -'pyhamcrest>=1.9,<1.10.0', +# pyhamcrest==1.10.0 doesn't work on Py2. Beam still supports Py2. +# See: https://github.com/hamcrest/PyHamcrest/issues/131. +'pyhamcrest>=1.9,!=1.10.0,<2.0.0', 'pyyaml>=3.12,<6.0.0', 'requests_mock>=1.7,<2.0', 'tenacity>=5.0.2,<6.0',
[beam] branch release-2.19.0 updated: [BEAM-9123] HadoopResourceId returns wrong directoryName bugfix
This is an automated email from the ASF dual-hosted git repository.
boyuanz pushed a commit to branch release-2.19.0
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/release-2.19.0 by this push:
new 5ab8ead [BEAM-9123] HadoopResourceId returns wrong directoryName
bugfix
new 526158c Merge pull request #10626 from dmvk/BEAM-9123-backport-2.19
5ab8ead is described below
commit 5ab8ead726dea577205056cc03a417c3753a65f2
Author: marek.simunek
AuthorDate: Wed Jan 15 16:42:18 2020 +0100
[BEAM-9123] HadoopResourceId returns wrong directoryName bugfix
---
.../org/apache/beam/sdk/io/hdfs/HadoopResourceId.java | 4
.../org/apache/beam/sdk/io/hdfs/HadoopResourceIdTest.java | 15 +++
2 files changed, 19 insertions(+)
diff --git
a/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopResourceId.java
b/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopResourceId.java
index 4e5f6e2..b68839b 100644
---
a/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopResourceId.java
+++
b/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopResourceId.java
@@ -65,6 +65,10 @@ class HadoopResourceId implements ResourceId {
@Override
public String getFilename() {
+if (isDirectory()) {
+ Path parentPath = new Path(uri).getParent();
+ return parentPath == null ? null : parentPath.getName();
+}
return new Path(uri).getName();
}
diff --git
a/sdks/java/io/hadoop-file-system/src/test/java/org/apache/beam/sdk/io/hdfs/HadoopResourceIdTest.java
b/sdks/java/io/hadoop-file-system/src/test/java/org/apache/beam/sdk/io/hdfs/HadoopResourceIdTest.java
index 4d7fb8d..1726a3e 100644
---
a/sdks/java/io/hadoop-file-system/src/test/java/org/apache/beam/sdk/io/hdfs/HadoopResourceIdTest.java
+++
b/sdks/java/io/hadoop-file-system/src/test/java/org/apache/beam/sdk/io/hdfs/HadoopResourceIdTest.java
@@ -17,6 +17,9 @@
*/
package org.apache.beam.sdk.io.hdfs;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNull;
+
import java.net.URI;
import java.util.Collections;
import org.apache.beam.sdk.io.FileSystems;
@@ -65,4 +68,16 @@ public class HadoopResourceIdTest {
"hdfs://" + hdfsClusterBaseUri.getPath(), true /* isDirectory */);
ResourceIdTester.runResourceIdBattery(baseDirectory);
}
+
+ @Test
+ public void testGetFilename() {
+assertNull(toResourceIdentifier("").getFilename());
+assertEquals("abc", toResourceIdentifier("/dirA/abc").getFilename());
+assertEquals("abc", toResourceIdentifier("/dirA/abc/").getFilename());
+assertEquals("xyz.txt",
toResourceIdentifier("/dirA/abc/xyz.txt").getFilename());
+ }
+
+ private ResourceId toResourceIdentifier(String path) {
+return new HadoopResourceId(hdfsClusterBaseUri.resolve(path));
+ }
}
[beam] branch asf-site updated: Publishing website 2020/01/20 16:24:42 at commit 201cfc1
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 1e0a9d8 Publishing website 2020/01/20 16:24:42 at commit 201cfc1 1e0a9d8 is described below commit 1e0a9d87beca1be44c54a3dc8747e8192502fe02 Author: jenkins AuthorDate: Mon Jan 20 16:24:42 2020 + Publishing website 2020/01/20 16:24:42 at commit 201cfc1 --- website/generated-content/contribute/release-guide/index.html | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/website/generated-content/contribute/release-guide/index.html b/website/generated-content/contribute/release-guide/index.html index 53cfcbe..ff506e7 100644 --- a/website/generated-content/contribute/release-guide/index.html +++ b/website/generated-content/contribute/release-guide/index.html @@ -276,6 +276,7 @@ Deploy artifacts to Maven Central Repository Deploy Python artifacts to PyPI + Deploy source release to dist.apache.org Deploy SDK docker images to DockerHub Git tag Merge website pull request @@ -399,10 +400,8 @@ limitations under the License. 8. Finalize the release Deploy artifacts to Maven Central Repository - Deploy Python artifacts to PyPI - Deploy source release to dist.apache.org - - + Deploy Python artifacts to PyPI + Deploy source release to dist.apache.org Deploy SDK docker images to DockerHub Git tag Merge website pull request @@ -1834,7 +1833,7 @@ delete the .asc, twine upload * from the directory with the .zip and .whl files; -Deploy source release to dist.apache.org +Deploy source release to dist.apache.org Copy the source release from the dev repository to the release repository at dist.apache.org using Subversion.
[beam] branch master updated (4254673 -> 201cfc1)
This is an automated email from the ASF dual-hosted git repository. iemejia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 4254673 Merge pull request #10639: [BEAM-8939] Report status code 0 when no stale jobs are found add 61a588c [BEAM-9153] Fix release guide heading level add 201cfc1 Merge pull request #10638: [website][BEAM-9153] Fix release guide heading level No new revisions were added by this update. Summary of changes: website/src/contribute/release-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Error while running jenkins feature from .asf.yaml in beam!
An error occurred while running jenkins feature in .asf.yaml!: GitHub whitelist cannot be more than 10 people!
[beam] branch master updated (ebe8bdc -> 4254673)
This is an automated email from the ASF dual-hosted git repository. kkucharczyk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from ebe8bdc Merge pull request #10622: [BEAM=6857] Fix timermap test to not use TestStream add c188c15 Report status code 0 when no stale jobs are found add 4254673 Merge pull request #10639: [BEAM-8939] Report status code 0 when no stale jobs are found No new revisions were added by this update. Summary of changes: .test-infra/tools/stale_dataflow_jobs_cleaner.sh | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-)
Error while running jenkins feature from .asf.yaml in beam!
An error occurred while running jenkins feature in .asf.yaml!: GitHub whitelist cannot be more than 10 people!
