[
https://issues.apache.org/jira/browse/BEAM-2640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093632#comment-16093632
]
ASF GitHub Bot commented on BEAM-2640:
--------------------------------------
GitHub user jkff opened a pull request:
https://github.com/apache/beam/pull/3598
[BEAM-2640, BEAM-2641] Introduces TextIO.read().withHintMatchesManyFiles()
In that case it expands to TextIO.readAll(). Implementing this when the
filepattern is a ValueProvider nudged me to also implement BEAM-2640 -
Create.ofProvider(ValueProvider).
Links:
https://issues.apache.org/jira/browse/BEAM-2640
https://issues.apache.org/jira/browse/BEAM-2641
R: @reuvenlax
CC: @sammcveety
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jkff/incubator-beam textio-readall
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3598.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3598
----
commit d26416d142808d8279c40cd57a659e103b296f3f
Author: Eugene Kirpichov <[email protected]>
Date: 2017-07-19T18:50:58Z
[BEAM-2640] Introduces Create.ofProvider(ValueProvider)
I also converted DatastoreV1 to use this overload, and, as an
exercise, added a withQuery(ValueProvider) overload to JdbcIO.
commit 872f0df5c227385bc95a42ecc02934a4082565ba
Author: Eugene Kirpichov <[email protected]>
Date: 2017-07-19T18:51:33Z
[BEAM-2641] Introduces TextIO.read().withHintMatchesManyFiles()
In that case it expands to TextIO.readAll().
----
> Introduce Create.ofProvider(ValueProvider)
> ------------------------------------------
>
> Key: BEAM-2640
> URL: https://issues.apache.org/jira/browse/BEAM-2640
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Eugene Kirpichov
> Assignee: Eugene Kirpichov
>
> When you have a ValueProvider<T> that may or may not be accessible at
> construction time, a common task is to wrap it into a single-element
> PCollection<T>. This is especially common when migrating an IO connector that
> used something like Create.of(query) followed by a ParDo, to having query be
> a ValueProvider.
> Currently this is done in an icky way (e.g.
> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java#L615)
> We should have a convenience helper for that.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
