Jenkins build is back to stable : beam_PostCommit_Java_RunnableOnService_Flink #1979

[Apache Jenkins Server]

See 

Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Flink #1978

[Apache Jenkins Server]

See 

[GitHub] beam-site pull request #190: Simplify merge process section

[aviemzur]

GitHub user aviemzur opened a pull request:

https://github.com/apache/beam-site/pull/190

Simplify merge process section

R: @davorbonaci 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aviemzur/beam-site simplify-merge-process

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam-site/pull/190.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #190


commit cd5a9b4c832cb79c2536f2cd538b5b12f81cc1b6
Author: Zur, Aviem 
Date:   2017-03-19T04:16:49Z

Simplify merge process section




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[3/4] beam-site git commit: Regenerate website

[aviemzur]

Regenerate website


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/c6d92268
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/c6d92268
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/c6d92268

Branch: refs/heads/asf-site
Commit: c6d922685e79463fa7f41a8f9dcd86aee59923c0
Parents: b470eb5
Author: Zur, Aviem 
Authored: Sun Mar 19 05:55:02 2017 +0200
Committer: Zur, Aviem 
Committed: Sun Mar 19 05:55:02 2017 +0200

--
 content/contribute/contribution-guide/index.html | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam-site/blob/c6d92268/content/contribute/contribution-guide/index.html
--
diff --git a/content/contribute/contribution-guide/index.html 
b/content/contribute/contribution-guide/index.html
index 4d4a782..af492a9 100644
--- a/content/contribute/contribution-guide/index.html
+++ b/content/contribute/contribution-guide/index.html
@@ -214,7 +214,7 @@
   Website
   One-time Setup
   Working on your change
-  Committing website changes
+  Committing website 
changes (committers only)
 
   
 
@@ -736,7 +736,7 @@ $ git checkout -b  origin/asf-site
 
 While you are working on your pull request, you can test and develop live 
by running the following command in the root folder of the website:
 
-$ bundle exec 
jekyll serve
+$ bundle exec 
jekyll serve --incremental
 
 
 
@@ -758,11 +758,20 @@ $ git checkout -b  origin/asf-site
 
 During review, committers will patch in your PR, generate the static content/, and review the changes.
 
-Committing website changes
+Committing website changes 
(committers only)
 
 Follow the same committer process as above, but using repository apache/beam-site and branch asf-site.
 
-In addition, the committer is responsible for doing the final jekyll build to generate the static content, 
so follow the instructions above to install jekyll.
+In addition, the committer is responsible for doing the final bundle exec jekyll build to generate the 
static content, so follow the instructions above to install jekyll.
+
+This command generates the content/ 
directory. The committer should add and commit the content related to the 
PR.
+
+$ git add 
content/
+$ git commit -m "Regenerate website"
+
+
+
+Finally you should merge the changes into the asf-site branch and push them into the apache repository.
 
   
 

[1/4] beam-site git commit: Improve committing website changes section in the contribution guide

[aviemzur]

Repository: beam-site
Updated Branches:
  refs/heads/asf-site f9b4c6e7a -> 41018ed60


Improve committing website changes section in the contribution guide


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/653218f8
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/653218f8
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/653218f8

Branch: refs/heads/asf-site
Commit: 653218f89c0a6b50283bf502538f8b131aa54ea1
Parents: f9b4c6e
Author: Ismaël Mejía 
Authored: Sun Mar 19 01:06:26 2017 +0100
Committer: Ismaël Mejía 
Committed: Sun Mar 19 01:08:50 2017 +0100

--
 src/contribute/contribution-guide.md | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam-site/blob/653218f8/src/contribute/contribution-guide.md
--
diff --git a/src/contribute/contribution-guide.md 
b/src/contribute/contribution-guide.md
index b2af352..c1617af 100644
--- a/src/contribute/contribution-guide.md
+++ b/src/contribute/contribution-guide.md
@@ -442,7 +442,7 @@ The general guidelines for cloning a repository can be 
adjusted to use the `asf-
 
 While you are working on your pull request, you can test and develop live by 
running the following command in the root folder of the website:
 
-   $ bundle exec jekyll serve
+   $ bundle exec jekyll serve --incremental
 
 Jekyll will start a webserver on port 4000. As you make changes to the 
content, Jekyll will rebuild it automatically.
 
@@ -458,8 +458,15 @@ When you are ready, submit a pull request using the [Beam 
Site GitHub mirror](ht
 
 During review, committers will patch in your PR, generate the static 
`content/`, and review the changes.
 
- Committing website changes
+ Committing website changes (committers only)
 
 Follow the same committer process as above, but using repository 
`apache/beam-site` and branch `asf-site`.
 
-In addition, the committer is responsible for doing the final `jekyll build` 
to generate the static content, so follow the instructions above to install 
`jekyll`.
+In addition, the committer is responsible for doing the final `bundle exec 
jekyll build` to generate the static content, so follow the instructions above 
to install `jekyll`.
+
+This command generates the `content/` directory. The committer should add and 
commit the content related to the PR.
+
+   $ git add content/
+   $ git commit -m "Regenerate website"
+
+Finally you should merge the changes into the `asf-site` branch and push them 
into the `apache` repository.

[2/4] beam-site git commit: Fix timezone to America/Los_Angeles

[aviemzur]

Fix timezone to America/Los_Angeles


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/b470eb5b
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/b470eb5b
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/b470eb5b

Branch: refs/heads/asf-site
Commit: b470eb5b418949f19e862a5faeac06e0a4dc8bfb
Parents: 653218f
Author: Ismaël Mejía 
Authored: Sun Mar 19 01:09:00 2017 +0100
Committer: Ismaël Mejía 
Committed: Sun Mar 19 01:12:03 2017 +0100

--
 _config.yml | 3 +++
 1 file changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/beam-site/blob/b470eb5b/_config.yml
--
diff --git a/_config.yml b/_config.yml
index 7d36162..2e9084c 100644
--- a/_config.yml
+++ b/_config.yml
@@ -52,3 +52,6 @@ kramdown:
 release_latest: 0.6.0
 
 # Plugins are configured in the Gemfile.
+
+# Set the time zone for site generation, fixed to US Pacific Time
+timezone: America/Los_Angeles

[GitHub] beam-site pull request #189: Improve committing website changes section in t...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/beam-site/pull/189


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[4/4] beam-site git commit: This closes #189

[aviemzur]

This closes #189


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/41018ed6
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/41018ed6
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/41018ed6

Branch: refs/heads/asf-site
Commit: 41018ed6016c18fc0dac1a2740e510a9a2460bca
Parents: f9b4c6e c6d9226
Author: Zur, Aviem 
Authored: Sun Mar 19 05:58:40 2017 +0200
Committer: Zur, Aviem 
Committed: Sun Mar 19 05:58:40 2017 +0200

--
 _config.yml  |  3 +++
 content/contribute/contribution-guide/index.html | 17 +
 src/contribute/contribution-guide.md | 13 ++---
 3 files changed, 26 insertions(+), 7 deletions(-)
--

Build failed in Jenkins: beam_PostCommit_Python_Verify #1547

[Apache Jenkins Server]

See 


--
[...truncated 423.09 KB...]
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/setuptools-34.3.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/abc.txt...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/abc.txt
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.1.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.1.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/setuptools-34.0.3.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/setuptools-34.0.3.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.2.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.2.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.3.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/appdirs-1.4.3.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/six-1.10.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/six-1.10.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/pyparsing-2.2.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/pyparsing-2.2.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/setuptools-34.1.0.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/setuptools-34.1.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/setuptools-34.2.0.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/setuptools-34.2.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/pbr-2.0.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/pbr-2.0.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/packaging-16.8.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/packaging-16.8.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/mock-2.0.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0319030551-207004.1489892751.207201/mock-2.0.0.tar.gz
root: INFO: Starting GCS upload to 
gs:

Jenkins build is back to stable : beam_PostCommit_Java_RunnableOnService_Spark #1277

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostCommit_Python_Verify #1546

[Apache Jenkins Server]

See 


--
[...truncated 423.08 KB...]
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/setuptools-34.3.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/abc.txt...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/abc.txt
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.1.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.1.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/setuptools-34.0.3.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/setuptools-34.0.3.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.2.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.2.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.3.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/appdirs-1.4.3.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/six-1.10.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/six-1.10.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/pyparsing-2.2.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/pyparsing-2.2.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/setuptools-34.1.0.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/setuptools-34.1.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/setuptools-34.2.0.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/setuptools-34.2.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/pbr-2.0.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/pbr-2.0.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/packaging-16.8.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/packaging-16.8.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/mock-2.0.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318210550-037025.1489871150.037199/mock-2.0.0.tar.gz
root: INFO: Starting GCS upload to 
gs:

[jira] [Commented] (BEAM-1750) Refactor BigQueryIO.java into multiple files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931371#comment-15931371
 ] 

ASF GitHub Bot commented on BEAM-1750:
--

GitHub user reuvenlax opened a pull request:

https://github.com/apache/beam/pull/2271

[BEAM-1750] Split BigQueryIO.java into several files.

BigQueryIO.java has become almost unmanageably large. This pull request 
refactors it into multiple files which are easier to peruse. There should be no 
functionality changes introduced by this PR.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reuvenlax/incubator-beam 
bigquery_file_refactor

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2271.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2271


commit 43f4dd34ccbfc8f4ce89a462a2b9fce001e2769b
Author: Reuven Lax 
Date:   2017-03-13T22:25:25Z

Cleanup BigQueryIO
  * Input type T
  * Allow data-dependent tables, based on ValueInSingleWindow

commit e1d28c04edb14c5b4728864de6cd949c6233808d
Author: Reuven Lax 
Date:   2017-03-15T16:05:55Z

Move ValueInSingleWindow from testing package to values package

commit 8c05cd8be9a588dd0ca1a83cfc3a58e9de7f29f0
Author: Reuven Lax 
Date:   2017-03-17T22:15:16Z

Refactor BigQueryIO helper methods into a BigQueryHelpers class.
Move helper transforms for BigQueryIO.Read into individual files.

commit 67b1397270b39270d596770868c79c91ce14f034
Author: Reuven Lax 
Date:   2017-03-17T22:18:17Z

Change private to package in BigQueryHelpers.

commit 8cb884f16255ed427f4e64d665b955f07d86f205
Author: Reuven Lax 
Date:   2017-03-17T22:20:58Z

Fix CheckStyle errors

commit 31e2e484220319e4c742d1d8ee36da3ab4f226ae
Author: Reuven Lax 
Date:   2017-03-17T22:30:53Z

Refactor BigQueryIO.Write helper transforms into separate files.

commit cd73b1ee78148465a0469ea5f754c9e32d9eeccc
Author: Reuven Lax 
Date:   2017-03-18T19:58:23Z

Refactor streaming Write transforms as well as TableRowWriter into separate 
files.

commit 57bdfd017a3ff1cb2a3b0e79267ce6a85d9700d8
Author: Reuven Lax 
Date:   2017-03-18T20:03:25Z

Move a few more functions into BigQueryHelper

commit 2026413156513979d0689718a47184de6c2dc1ee
Author: Reuven Lax 
Date:   2017-03-18T20:14:54Z

Move batch-loading path into its own PTransform and a separate file.

commit d66e48101b85d7646bddbfd6b6030896b05e088c
Author: Reuven Lax 
Date:   2017-03-18T20:17:29Z

Add javadoc.




> Refactor BigQueryIO.java into multiple files
> 
>
> Key: BEAM-1750
> URL: https://issues.apache.org/jira/browse/BEAM-1750
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Reuven Lax
>
> BigQueryIO.java has become almost unmanageably large. We should refactor this 
> into several files.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] beam pull request #2271: [BEAM-1750] Split BigQueryIO.java into several file...

[reuvenlax]

GitHub user reuvenlax opened a pull request:

https://github.com/apache/beam/pull/2271

[BEAM-1750] Split BigQueryIO.java into several files.

BigQueryIO.java has become almost unmanageably large. This pull request 
refactors it into multiple files which are easier to peruse. There should be no 
functionality changes introduced by this PR.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reuvenlax/incubator-beam 
bigquery_file_refactor

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2271.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2271


commit 43f4dd34ccbfc8f4ce89a462a2b9fce001e2769b
Author: Reuven Lax 
Date:   2017-03-13T22:25:25Z

Cleanup BigQueryIO
  * Input type T
  * Allow data-dependent tables, based on ValueInSingleWindow

commit e1d28c04edb14c5b4728864de6cd949c6233808d
Author: Reuven Lax 
Date:   2017-03-15T16:05:55Z

Move ValueInSingleWindow from testing package to values package

commit 8c05cd8be9a588dd0ca1a83cfc3a58e9de7f29f0
Author: Reuven Lax 
Date:   2017-03-17T22:15:16Z

Refactor BigQueryIO helper methods into a BigQueryHelpers class.
Move helper transforms for BigQueryIO.Read into individual files.

commit 67b1397270b39270d596770868c79c91ce14f034
Author: Reuven Lax 
Date:   2017-03-17T22:18:17Z

Change private to package in BigQueryHelpers.

commit 8cb884f16255ed427f4e64d665b955f07d86f205
Author: Reuven Lax 
Date:   2017-03-17T22:20:58Z

Fix CheckStyle errors

commit 31e2e484220319e4c742d1d8ee36da3ab4f226ae
Author: Reuven Lax 
Date:   2017-03-17T22:30:53Z

Refactor BigQueryIO.Write helper transforms into separate files.

commit cd73b1ee78148465a0469ea5f754c9e32d9eeccc
Author: Reuven Lax 
Date:   2017-03-18T19:58:23Z

Refactor streaming Write transforms as well as TableRowWriter into separate 
files.

commit 57bdfd017a3ff1cb2a3b0e79267ce6a85d9700d8
Author: Reuven Lax 
Date:   2017-03-18T20:03:25Z

Move a few more functions into BigQueryHelper

commit 2026413156513979d0689718a47184de6c2dc1ee
Author: Reuven Lax 
Date:   2017-03-18T20:14:54Z

Move batch-loading path into its own PTransform and a separate file.

commit d66e48101b85d7646bddbfd6b6030896b05e088c
Author: Reuven Lax 
Date:   2017-03-18T20:17:29Z

Add javadoc.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-1750) Refactor BigQueryIO.java into multiple files

[Reuven Lax (JIRA)]

Reuven Lax created BEAM-1750:


 Summary: Refactor BigQueryIO.java into multiple files
 Key: BEAM-1750
 URL: https://issues.apache.org/jira/browse/BEAM-1750
 Project: Beam
  Issue Type: Improvement
  Components: sdk-java-core
Reporter: Reuven Lax
Assignee: Reuven Lax


BigQueryIO.java has become almost unmanageably large. We should refactor this 
into several files.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Spark #1276

[Apache Jenkins Server]

See 

Jenkins build is back to stable : beam_PostCommit_Java_RunnableOnService_Flink #1976

[Apache Jenkins Server]

See 

Jenkins build is back to normal : beam_PostCommit_Python_Verify #1545

[Apache Jenkins Server]

See 

[GitHub] beam-site pull request #189: Improve committing website changes section in t...

[iemejia]

GitHub user iemejia opened a pull request:

https://github.com/apache/beam-site/pull/189

Improve committing website changes section in the contribution guide

R: @aviemzur 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/iemejia/beam-site asf-site

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam-site/pull/189.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #189


commit b0705f22edf82d978dfa95c22154c96548c53c98
Author: Ismaël Mejía 
Date:   2017-03-18T14:37:27Z

Improve committing website changes section in the contribution guide




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/3] beam-site git commit: Add aviemzur as committer

[iemejia]

Repository: beam-site
Updated Branches:
  refs/heads/asf-site 046538547 -> f9b4c6e7a


Add aviemzur as committer


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/41b8245b
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/41b8245b
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/41b8245b

Branch: refs/heads/asf-site
Commit: 41b8245b94d5b02512c94b53a49cb7910edfbc1f
Parents: 0465385
Author: Zur, Aviem 
Authored: Sat Mar 18 06:09:19 2017 +0200
Committer: Zur, Aviem 
Committed: Sat Mar 18 06:09:19 2017 +0200

--
 src/_beam_team/team.md | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/beam-site/blob/41b8245b/src/_beam_team/team.md
--
diff --git a/src/_beam_team/team.md b/src/_beam_team/team.md
index 1f2ce1a..92ad2dc 100644
--- a/src/_beam_team/team.md
+++ b/src/_beam_team/team.md
@@ -140,4 +140,10 @@ members:
 organization: Slack
 roles: committer, PMC
 time_zone: "-8"
+  - name: Aviem Zur
+apache_id: aviemzur
+email: aviemzur [at] apache [dot] org
+organization: PayPal
+roles: committer
+time_zone: "+2"
 ---

[3/3] beam-site git commit: This closes #187

[iemejia]

This closes #187


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/f9b4c6e7
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/f9b4c6e7
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/f9b4c6e7

Branch: refs/heads/asf-site
Commit: f9b4c6e7a09ef689f842a6a0be7eba4d1782b3f1
Parents: 0465385 144599f
Author: Ismaël Mejía 
Authored: Sat Mar 18 15:04:45 2017 +0100
Committer: Ismaël Mejía 
Committed: Sat Mar 18 15:05:07 2017 +0100

--
 content/contribute/team/index.html | 9 +
 src/_beam_team/team.md | 6 ++
 2 files changed, 15 insertions(+)
--

[GitHub] beam-site pull request #187: Add aviemzur as committer

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/beam-site/pull/187


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/3] beam-site git commit: Regenerate website

[iemejia]

Regenerate website


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/144599fd
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/144599fd
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/144599fd

Branch: refs/heads/asf-site
Commit: 144599fd88ce871ead449cd13ab3418fec702c28
Parents: 41b8245
Author: Ismaël Mejía 
Authored: Sat Mar 18 14:56:30 2017 +0100
Committer: Ismaël Mejía 
Committed: Sat Mar 18 14:56:30 2017 +0100

--
 content/contribute/team/index.html | 9 +
 1 file changed, 9 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/beam-site/blob/144599fd/content/contribute/team/index.html
--
diff --git a/content/contribute/team/index.html 
b/content/contribute/team/index.html
index 8d58a55..4564b10 100644
--- a/content/contribute/team/index.html
+++ b/content/contribute/team/index.html
@@ -379,6 +379,15 @@
   -8
 
   
+
+  Aviem Zur
+  aviemzur
+  aviemzur [at] apache [dot] org
+  PayPal
+  committer
+  +2
+
+  
 
   
 

Jenkins build became unstable: beam_PostCommit_Java_RunnableOnService_Flink #1975

[Apache Jenkins Server]

See 

[jira] [Comment Edited] (BEAM-1718) Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() causes Overflow/Underflow

[Tobias Feldhaus (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931216#comment-15931216
 ] 

Tobias Feldhaus edited comment on BEAM-1718 at 3/18/17 1:17 PM:


When batch-processing, shifting back in time is normal and crucial for me 
(especially when using windowing, so that elements get into the right partition 
in BigQuery). When this method gets deprecated, how should such cases be 
handled in the future?


was (Author: james-woods):
When batch-processing, shifting back in time is normal and crucial for me 
(especially when windowing elements to get them into the right partition in 
BigQuery). When this method gets deprecated, how should such cases be handled 
in the future then?

> Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() 
> causes Overflow/Underflow
> -
>
> Key: BEAM-1718
> URL: https://issues.apache.org/jira/browse/BEAM-1718
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Affects Versions: 0.5.0
>Reporter: Tobias Feldhaus
>Assignee: Thomas Groh
>
> Overriding getAllowedTimestampSkew() in DoFn and returning 
> Duration.millis(Long.MAX_VALUE) (as suggested in the JavaDoc for allowing 
> infinite skew) causes an Overflow/Underflow



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1718) Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() causes Overflow/Underflow

[Tobias Feldhaus (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931216#comment-15931216
 ] 

Tobias Feldhaus commented on BEAM-1718:
---

When batch-processing, shifting back in time is normal and crucial for me 
(especially when windowing elements to get them into the right partition in 
BigQuery). When this method gets deprecated, how should such cases be handled 
in the future then?

> Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() 
> causes Overflow/Underflow
> -
>
> Key: BEAM-1718
> URL: https://issues.apache.org/jira/browse/BEAM-1718
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Affects Versions: 0.5.0
>Reporter: Tobias Feldhaus
>Assignee: Thomas Groh
>
> Overriding getAllowedTimestampSkew() in DoFn and returning 
> Duration.millis(Long.MAX_VALUE) (as suggested in the JavaDoc for allowing 
> infinite skew) causes an Overflow/Underflow



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (BEAM-1727) Add setForNowAlign(period, offset) to Timer

[Jingsong Lee (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingsong Lee reassigned BEAM-1727:
--

Assignee: Jingsong Lee

> Add setForNowAlign(period, offset) to Timer
> ---
>
> Key: BEAM-1727
> URL: https://issues.apache.org/jira/browse/BEAM-1727
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model
>Reporter: Jingsong Lee
>Assignee: Jingsong Lee
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

[Jingsong Lee (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931210#comment-15931210
 ] 

Jingsong Lee commented on BEAM-1612:


[~aljoscha] Yes, this will greatly improve performance in some cases.

> Support real Bundle in Flink runner
> ---
>
> Key: BEAM-1612
> URL: https://issues.apache.org/jira/browse/BEAM-1612
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Jingsong Lee
>Assignee: Jingsong Lee
>
> The Bundle is very important in the beam model. Users can use the bundle to 
> flush buffer, can reuse many heavyweight resources in a bundle. Most IO 
> plugins use the bundle to flush. 
> Moreover, FlinkRunner can also use Bundle to reduce access to the FlinkState, 
> such as first placed in JavaHeap, flush into RocksDbState when invoke 
> finishBundle , this can reduce the number of serialization.
> But now FlinkRunner calls the finishBundle every processElement. We need 
> support real Bundle.
> I think we can have the following implementations:
> 1.Invoke finishBundle and next startBundle in {{snapshot}} of Flink. But 
> sometimes this "Bundle" maybe too big. This depends on the user's checkpoint 
> configuration.
> 2.Manually control the size of the bundle. The half-bundle will be flushed to 
> a full-bundle by count or eventTime or processTime or {{snapshot}}. We do not 
> need to wait, just call the startBundle and finishBundle at the right time.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Jenkins build is back to stable : beam_PostCommit_Java_RunnableOnService_Apex #798

[Apache Jenkins Server]

See 

Jenkins build is back to normal : beam_PostCommit_Java_MavenInstall #2944

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostCommit_Python_Verify #1544

[Apache Jenkins Server]

See 


--
[...truncated 423.09 KB...]
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/setuptools-34.3.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/abc.txt...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/abc.txt
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.1.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.1.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/setuptools-34.0.3.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/setuptools-34.0.3.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.2.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.2.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.3.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/appdirs-1.4.3.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/six-1.10.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/six-1.10.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/pyparsing-2.2.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/pyparsing-2.2.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/setuptools-34.1.0.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/setuptools-34.1.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/setuptools-34.2.0.zip...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/setuptools-34.2.0.zip
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/pbr-2.0.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/pbr-2.0.0.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/packaging-16.8.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/packaging-16.8.tar.gz
root: INFO: Starting GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/mock-2.0.0.tar.gz...
root: INFO: Completed GCS upload to 
gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/beamapp-jenkins-0318090556-616776.1489827956.616966/mock-2.0.0.tar.gz
root: INFO: Starting GCS upload to 
gs:

Jenkins build is back to stable : beam_PostCommit_Java_RunnableOnService_Dataflow #2582

[Apache Jenkins Server]

See