[jira] [Created] (BEAM-1388) Update default configuration of retry decorator so that wait times are more practical

Chamikara Jayalath created BEAM-1388: Summary: Update default configuration of retry decorator so that wait times are more practical Key: BEAM-1388 URL: https://issues.apache.org/jira/browse/BEAM-1388

[jira] [Resolved] (BEAM-877) Allow disabling flattening of results when using BigQuery source

[ https://issues.apache.org/jira/browse/BEAM-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-877. - Resolution: Fixed > Allow disabling flattening of results when using BigQuery source >

[jira] [Resolved] (BEAM-852) Validate sources when they are created

[ https://issues.apache.org/jira/browse/BEAM-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-852. - Resolution: Fixed > Validate sources when they are created >

[jira] [Resolved] (BEAM-1406) Remove deprecated fileio.TextFileSink

[ https://issues.apache.org/jira/browse/BEAM-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1406. -- Resolution: Fixed Fix Version/s: Not applicable > Remove deprecated

[jira] [Resolved] (BEAM-1388) Update default configuration of retry decorator so that wait times are more practical

[ https://issues.apache.org/jira/browse/BEAM-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1388. -- Resolution: Fixed Fix Version/s: Not applicable > Update default configuration of

[jira] [Created] (BEAM-1441) Add an IOChannelFactory interface to Python SDK

Chamikara Jayalath created BEAM-1441: Summary: Add an IOChannelFactory interface to Python SDK Key: BEAM-1441 URL: https://issues.apache.org/jira/browse/BEAM-1441 Project: Beam Issue

[jira] [Created] (BEAM-1406) Remove deprecated fileio.TextFileSink

Chamikara Jayalath created BEAM-1406: Summary: Remove deprecated fileio.TextFileSink Key: BEAM-1406 URL: https://issues.apache.org/jira/browse/BEAM-1406 Project: Beam Issue Type:

[jira] [Commented] (BEAM-1251) Python 3 Support

[ https://issues.apache.org/jira/browse/BEAM-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848783#comment-15848783 ] Chamikara Jayalath commented on BEAM-1251: -- Thanks Sergio and Justin for the interest in this. As

[jira] [Resolved] (BEAM-1298) Increment major version used by Dataflow runner to 5

[ https://issues.apache.org/jira/browse/BEAM-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1298. -- Resolution: Fixed Fix Version/s: Not applicable > Increment major version used by

[jira] [Resolved] (BEAM-1239) Update examples to use Beam text source

[ https://issues.apache.org/jira/browse/BEAM-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1239. -- Resolution: Fixed Fix Version/s: Not applicable > Update examples to use Beam

[jira] [Resolved] (BEAM-1299) Remove native text source and sink

[ https://issues.apache.org/jira/browse/BEAM-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1299. -- Resolution: Fixed Fix Version/s: Not applicable > Remove native text source and

[jira] [Commented] (BEAM-1440) Create a BigQuery source (that implements iobase.BoundedSource) for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-1440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872818#comment-15872818 ] Chamikara Jayalath commented on BEAM-1440: -- How about

[jira] [Commented] (BEAM-778) Make fileio._CompressedFile seekable.

[ https://issues.apache.org/jira/browse/BEAM-778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883081#comment-15883081 ] Chamikara Jayalath commented on BEAM-778: - Thanks for looking into this issue. I couldn't find you

[jira] [Resolved] (BEAM-1463) Update BigQuery read transform to handle 'null' values properly for DirectRunner

[ https://issues.apache.org/jira/browse/BEAM-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1463. -- Resolution: Fixed Fix Version/s: Not applicable > Update BigQuery read transform

[jira] [Commented] (BEAM-1440) Create a BigQuery source (that implements iobase.BoundedSource) for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-1440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870400#comment-15870400 ] Chamikara Jayalath commented on BEAM-1440: -- Hi Ibrahim, Great to hear that you are interested in

[jira] [Updated] (BEAM-1440) Create a BigQuery source (that implements iobase.BoundedSource) for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-1440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-1440: - Labels: (was: gsoc2017 mentor python) > Create a BigQuery source (that implements

[jira] [Commented] (BEAM-1440) Create a BigQuery source (that implements iobase.BoundedSource) for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-1440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870547#comment-15870547 ] Chamikara Jayalath commented on BEAM-1440: -- I'll be happy to help you with this issue but I don't

[jira] [Commented] (BEAM-643) Allow users to specify a custom service account

[ https://issues.apache.org/jira/browse/BEAM-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812789#comment-15812789 ] Chamikara Jayalath commented on BEAM-643: - Java SDK was updated in

[jira] [Resolved] (BEAM-553) Add a custom text source

[ https://issues.apache.org/jira/browse/BEAM-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-553. - Resolution: Fixed Fix Version/s: Not applicable > Add a custom text source >

[jira] [Resolved] (BEAM-578) Update FileBasedSource so that implementations can control whether the source gets split into ranges

[ https://issues.apache.org/jira/browse/BEAM-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-578. - Resolution: Fixed Fix Version/s: Not applicable > Update FileBasedSource so that

[jira] [Commented] (BEAM-778) Make fileio._CompressedFile seekable.

[ https://issues.apache.org/jira/browse/BEAM-778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947718#comment-15947718 ] Chamikara Jayalath commented on BEAM-778: - Currently this is not an issue since Beam FileBasedSoure

[jira] [Resolved] (BEAM-1782) BigQuery read transform fails for DirectRunner when reading empty repeated fields

[ https://issues.apache.org/jira/browse/BEAM-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1782. -- Resolution: Fixed Fix Version/s: Not applicable > BigQuery read transform fails

[jira] [Resolved] (BEAM-564) Update source framework so that remaining and consumed number of split points can be reported

[ https://issues.apache.org/jira/browse/BEAM-564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-564. - Resolution: Fixed Fix Version/s: Not applicable > Update source framework so that

[jira] [Commented] (BEAM-503) FileBasedSource should take a list of files/globs

[ https://issues.apache.org/jira/browse/BEAM-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954154#comment-15954154 ] Chamikara Jayalath commented on BEAM-503: - We currently don't have any plans to do this. This

[jira] [Closed] (BEAM-503) FileBasedSource should take a list of files/globs

[ https://issues.apache.org/jira/browse/BEAM-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath closed BEAM-503. --- Resolution: Won't Fix Fix Version/s: Not applicable > FileBasedSource should take a list

[jira] [Commented] (BEAM-1925) Make DoFn invocation logic of Python SDK more extensible

[ https://issues.apache.org/jira/browse/BEAM-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963243#comment-15963243 ] Chamikara Jayalath commented on BEAM-1925: -- cc: [~sb2nov] [~robertwb] [~jkff] > Make DoFn

[jira] [Created] (BEAM-1925) Make DoFn invocation logic of Python SDK more extensible

Chamikara Jayalath created BEAM-1925: Summary: Make DoFn invocation logic of Python SDK more extensible Key: BEAM-1925 URL: https://issues.apache.org/jira/browse/BEAM-1925 Project: Beam

[jira] [Commented] (BEAM-1373) Update Python SDK code to support both Python 2 and 3

[ https://issues.apache.org/jira/browse/BEAM-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959462#comment-15959462 ] Chamikara Jayalath commented on BEAM-1373: -- We'll have to run 2to3 but without porting

[jira] [Assigned] (BEAM-1373) Update Python SDK code to support both Python 2 and 3

[ https://issues.apache.org/jira/browse/BEAM-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-1373: Assignee: Ahmet Altay (was: Chamikara Jayalath) > Update Python SDK code to

[jira] [Commented] (BEAM-1874) Google Cloud Storage TextIO read fails with gz-files having Content-Encoding: gzip header

[ https://issues.apache.org/jira/browse/BEAM-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959683#comment-15959683 ] Chamikara Jayalath commented on BEAM-1874: -- Python SDK currently doesn't support these files but I

[jira] [Assigned] (BEAM-1874) Google Cloud Storage TextIO read fails with gz-files having Content-Encoding: gzip header

[ https://issues.apache.org/jira/browse/BEAM-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-1874: Assignee: Charles Chen (was: Chamikara Jayalath) > Google Cloud Storage TextIO

[jira] [Commented] (BEAM-1874) Google Cloud Storage TextIO read fails with gz-files having Content-Encoding: gzip header

[ https://issues.apache.org/jira/browse/BEAM-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960439#comment-15960439 ] Chamikara Jayalath commented on BEAM-1874: -- Seems like when you set the Content-Type along with

[jira] [Created] (BEAM-1630) Add Splittable DoFn to Python SDK

Chamikara Jayalath created BEAM-1630: Summary: Add Splittable DoFn to Python SDK Key: BEAM-1630 URL: https://issues.apache.org/jira/browse/BEAM-1630 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-1373) Update Python SDK code to support both Python 2 and 3

[ https://issues.apache.org/jira/browse/BEAM-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959443#comment-15959443 ] Chamikara Jayalath commented on BEAM-1373: -- This is for 2to3 conversion which I think only a part

[jira] [Created] (BEAM-1909) BigQuery read transform fails for DirectRunner when querying non-US regions

Chamikara Jayalath created BEAM-1909: Summary: BigQuery read transform fails for DirectRunner when querying non-US regions Key: BEAM-1909 URL: https://issues.apache.org/jira/browse/BEAM-1909

[jira] [Resolved] (BEAM-1179) Update assertions of source_test_utils from camelcase to underscore-separated

[ https://issues.apache.org/jira/browse/BEAM-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1179. -- Resolution: Fixed > Update assertions of source_test_utils from camelcase to

[jira] [Commented] (BEAM-1272) Align the naming of "generateInitialSplits" and "splitIntoBundles" to better reflect their intention

[ https://issues.apache.org/jira/browse/BEAM-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973588#comment-15973588 ] Chamikara Jayalath commented on BEAM-1272: -- Corresponding method in Python SDK BoundedSource is

[jira] [Commented] (BEAM-2708) Decompressing bzip2 files with multiple "streams" only reads the first stream

[ https://issues.apache.org/jira/browse/BEAM-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112011#comment-16112011 ] Chamikara Jayalath commented on BEAM-2708: -- A similar issue exists in Python SDK as well but

[jira] [Reopened] (BEAM-2708) Decompressing bzip2 files with multiple "streams" only reads the first stream

[ https://issues.apache.org/jira/browse/BEAM-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reopened BEAM-2708: -- Assignee: Chamikara Jayalath (was: Ben Chambers) Have to check if this applies to

[jira] [Commented] (BEAM-2708) Decompressing bzip2 files with multiple "streams" only reads the first stream

[ https://issues.apache.org/jira/browse/BEAM-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111520#comment-16111520 ] Chamikara Jayalath commented on BEAM-2708: -- I'll check if this applies to Python SDK as well.

[jira] [Created] (BEAM-2711) ByteKeyRangeTracker.getFractionConsumed() fails when out of range positions are claimed

Chamikara Jayalath created BEAM-2711: Summary: ByteKeyRangeTracker.getFractionConsumed() fails when out of range positions are claimed Key: BEAM-2711 URL: https://issues.apache.org/jira/browse/BEAM-2711

[jira] [Assigned] (BEAM-2711) ByteKeyRangeTracker.getFractionConsumed() fails when out of range positions are claimed

[ https://issues.apache.org/jira/browse/BEAM-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-2711: Assignee: Chamikara Jayalath > ByteKeyRangeTracker.getFractionConsumed() fails when

[jira] [Resolved] (BEAM-2643) Add TextIO and AvroIO read transforms that can read a PCollection of files

[ https://issues.apache.org/jira/browse/BEAM-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-2643. -- Resolution: Fixed Fix Version/s: 2.2.0 > Add TextIO and AvroIO read transforms

[jira] [Assigned] (BEAM-1286) DataflowRunner handling of missing filesToStage

[ https://issues.apache.org/jira/browse/BEAM-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-1286: Assignee: Kamil Szewczyk > DataflowRunner handling of missing filesToStage >

[jira] [Commented] (BEAM-1286) DataflowRunner handling of missing filesToStage

[ https://issues.apache.org/jira/browse/BEAM-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086224#comment-16086224 ] Chamikara Jayalath commented on BEAM-1286: -- [~davor] [~kenn] could one of you assign this to Kamil

[jira] [Commented] (BEAM-1286) DataflowRunner handling of missing filesToStage

[ https://issues.apache.org/jira/browse/BEAM-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086258#comment-16086258 ] Chamikara Jayalath commented on BEAM-1286: -- You have to be a project administrator to add users to

[jira] [Commented] (BEAM-1286) DataflowRunner handling of missing filesToStage

[ https://issues.apache.org/jira/browse/BEAM-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086259#comment-16086259 ] Chamikara Jayalath commented on BEAM-1286: -- Assigned JIRA to Kamil BTW. Thanks. > DataflowRunner

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086467#comment-16086467 ] Chamikara Jayalath commented on BEAM-2572: -- 1: Please seem my previous comment. How the

[jira] [Commented] (BEAM-2573) Better filesystem discovery mechanism in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078916#comment-16078916 ] Chamikara Jayalath commented on BEAM-2573: -- Will it make sense to add a FileSystems.register()

[jira] [Assigned] (BEAM-2573) Better filesystem discovery mechanism in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-2573: Assignee: (was: Thomas Groh) > Better filesystem discovery mechanism in Python

[jira] [Commented] (BEAM-2573) Better filesystem discovery mechanism in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078949#comment-16078949 ] Chamikara Jayalath commented on BEAM-2573: -- Right. Usually you should not have to import

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085074#comment-16085074 ] Chamikara Jayalath commented on BEAM-2572: -- Hi Dmitry, I think it might be better reduce the

[jira] [Commented] (BEAM-2532) BigQueryIO source should avoid expensive JSON schema parsing for every record

[ https://issues.apache.org/jira/browse/BEAM-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090618#comment-16090618 ] Chamikara Jayalath commented on BEAM-2532: -- This is not planned for 2.1.0 (which is happening

[jira] [Commented] (BEAM-2656) Introduce AvroIO.readAll()

[ https://issues.apache.org/jira/browse/BEAM-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096903#comment-16096903 ] Chamikara Jayalath commented on BEAM-2656: -- I created

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100536#comment-16100536 ] Chamikara Jayalath commented on BEAM-2490: -- I suspect this was due to

[jira] [Assigned] (BEAM-2643) Add TextIO.read_all() to Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-2643: Assignee: Chamikara Jayalath > Add TextIO.read_all() to Python SDK >

[jira] [Commented] (BEAM-2673) BigQuery Sink should use the Load API

[ https://issues.apache.org/jira/browse/BEAM-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099077#comment-16099077 ] Chamikara Jayalath commented on BEAM-2673: -- I think the fix here is to add a new BQ sink (which

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103390#comment-16103390 ] Chamikara Jayalath commented on BEAM-2490: -- To clarify, you are saying that performance is too

[jira] [Resolved] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-2490. -- Resolution: Fixed Fix Version/s: (was: Not applicable)

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103642#comment-16103642 ] Chamikara Jayalath commented on BEAM-2490: -- Great. I'll go ahead and close this issue. It's great

[jira] [Updated] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-2490: - Fix Version/s: (was: 2.2.0) 2.1.0 > ReadFromText function is not

[jira] [Created] (BEAM-2683) Update GCSFileSystem to support glob patterns of the form {x,y,z}

Chamikara Jayalath created BEAM-2683: Summary: Update GCSFileSystem to support glob patterns of the form {x,y,z} Key: BEAM-2683 URL: https://issues.apache.org/jira/browse/BEAM-2683 Project: Beam

[jira] [Updated] (BEAM-2683) Update GCSFileSystem to support glob patterns of the form {x,y,z}

[ https://issues.apache.org/jira/browse/BEAM-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-2683: - Component/s: (was: sdk-java-core) sdk-java-extensions > Update

[jira] [Created] (BEAM-2643) Add TextIO.read_all() to Python SDK

Chamikara Jayalath created BEAM-2643: Summary: Add TextIO.read_all() to Python SDK Key: BEAM-2643 URL: https://issues.apache.org/jira/browse/BEAM-2643 Project: Beam Issue Type: New

[jira] [Created] (BEAM-2531) Improve efficiency of reading compressed text files

Chamikara Jayalath created BEAM-2531: Summary: Improve efficiency of reading compressed text files Key: BEAM-2531 URL: https://issues.apache.org/jira/browse/BEAM-2531 Project: Beam Issue

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066824#comment-16066824 ] Chamikara Jayalath commented on BEAM-2490: -- I filed a separate ticket regarding performance issues

[jira] [Commented] (BEAM-539) Error when writing to the root of a GCS location

[ https://issues.apache.org/jira/browse/BEAM-539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983788#comment-15983788 ] Chamikara Jayalath commented on BEAM-539: - That PR will not automatically fix this but we can use

[jira] [Commented] (BEAM-539) Error when writing to the root of a GCS location

[ https://issues.apache.org/jira/browse/BEAM-539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985701#comment-15985701 ] Chamikara Jayalath commented on BEAM-539: - We currently write temp files to a sibling of output

[jira] [Created] (BEAM-2731) Add a generic Reshuffle transform to Python SDK

Chamikara Jayalath created BEAM-2731: Summary: Add a generic Reshuffle transform to Python SDK Key: BEAM-2731 URL: https://issues.apache.org/jira/browse/BEAM-2731 Project: Beam Issue

[jira] [Updated] (BEAM-2643) Add TextIO and AvroIO read transform that can read a PCollection of files

[ https://issues.apache.org/jira/browse/BEAM-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-2643: - Summary: Add TextIO and AvroIO read transform that can read a PCollection of files (was:

[jira] [Updated] (BEAM-2643) Add TextIO and AvroIO read transforms that can read a PCollection of files

[ https://issues.apache.org/jira/browse/BEAM-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-2643: - Summary: Add TextIO and AvroIO read transforms that can read a PCollection of files (was:

[jira] [Commented] (BEAM-1669) Pipeline I/O - add content on how to unit test

[ https://issues.apache.org/jira/browse/BEAM-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111368#comment-16111368 ] Chamikara Jayalath commented on BEAM-1669: -- Closing since

[jira] [Resolved] (BEAM-1669) Pipeline I/O - add content on how to unit test

[ https://issues.apache.org/jira/browse/BEAM-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1669. -- Resolution: Fixed Fix Version/s: Not applicable > Pipeline I/O - add content on

[jira] [Commented] (BEAM-2622) I/O Testing docs - Integration Testing section

[ https://issues.apache.org/jira/browse/BEAM-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111373#comment-16111373 ] Chamikara Jayalath commented on BEAM-2622: -- Can this be closed since

[jira] [Assigned] (BEAM-2547) FireBase IO

[ https://issues.apache.org/jira/browse/BEAM-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-2547: Assignee: (was: Chamikara Jayalath) > FireBase IO > --- > >

[jira] [Updated] (BEAM-1872) implement Reshuffle transform

[ https://issues.apache.org/jira/browse/BEAM-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-1872: - Labels: sdk-consistency (was: newbie sdk-consistency starter) > implement Reshuffle

[jira] [Commented] (BEAM-1872) implement Reshuffle transform

[ https://issues.apache.org/jira/browse/BEAM-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120826#comment-16120826 ] Chamikara Jayalath commented on BEAM-1872: -- Due to the complexities mentioned in following links I

[jira] [Commented] (BEAM-1872) implement Reshuffle transform

[ https://issues.apache.org/jira/browse/BEAM-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120824#comment-16120824 ] Chamikara Jayalath commented on BEAM-1872: -- I think it's good to keep the JIRA around since

[jira] [Resolved] (BEAM-2711) ByteKeyRangeTracker.getFractionConsumed() fails when out of range positions are claimed

[ https://issues.apache.org/jira/browse/BEAM-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-2711. -- Resolution: Fixed Fix Version/s: 2.2.0 >

[jira] [Commented] (BEAM-2265) Python word count gets stuck during application termination on Windows

[ https://issues.apache.org/jira/browse/BEAM-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007442#comment-16007442 ] Chamikara Jayalath commented on BEAM-2265: -- I tried running word count on Windows as well. Word

[jira] [Created] (BEAM-2294) Initial size estimation fails for mobile gaming examples for DataflowRunner when run in Windows

Chamikara Jayalath created BEAM-2294: Summary: Initial size estimation fails for mobile gaming examples for DataflowRunner when run in Windows Key: BEAM-2294 URL:

[jira] [Created] (BEAM-2241) Correctly mark top level classes and functions as private

Chamikara Jayalath created BEAM-2241: Summary: Correctly mark top level classes and functions as private Key: BEAM-2241 URL: https://issues.apache.org/jira/browse/BEAM-2241 Project: Beam

[jira] [Assigned] (BEAM-2241) Correctly mark top level classes and functions as private

[ https://issues.apache.org/jira/browse/BEAM-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-2241: Assignee: Chamikara Jayalath (was: Ahmet Altay) > Correctly mark top level classes

[jira] [Commented] (BEAM-2265) Python word count with DirectRunner gets stuck during application termination on Windows

[ https://issues.apache.org/jira/browse/BEAM-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008713#comment-16008713 ] Chamikara Jayalath commented on BEAM-2265: -- I tried running this with extra logging that prints

[jira] [Resolved] (BEAM-2494) Remove 'GroupedShuffleRangeTracker' which is unused in the SDK

[ https://issues.apache.org/jira/browse/BEAM-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-2494. -- Resolution: Fixed Fix Version/s: 2.1.0 > Remove 'GroupedShuffleRangeTracker'

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058426#comment-16058426 ] Chamikara Jayalath commented on BEAM-2490: -- I ran the same pipeline with a similar but smaller (8

[jira] [Created] (BEAM-2494) Remove 'GroupedShuffleRangeTracker' which is unused in the SDK

Chamikara Jayalath created BEAM-2494: Summary: Remove 'GroupedShuffleRangeTracker' which is unused in the SDK Key: BEAM-2494 URL: https://issues.apache.org/jira/browse/BEAM-2494 Project: Beam

[jira] [Commented] (BEAM-2497) TextIO can't read concatenated gzip files

[ https://issues.apache.org/jira/browse/BEAM-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059647#comment-16059647 ] Chamikara Jayalath commented on BEAM-2497: -- cc: [~robertwb] [~katsia...@google.com] > TextIO

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060384#comment-16060384 ] Chamikara Jayalath commented on BEAM-2490: -- Thanks Guillermo and Ahmet. Guillermo, let us know if

[jira] [Resolved] (BEAM-2497) TextIO can't read concatenated gzip files

[ https://issues.apache.org/jira/browse/BEAM-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-2497. -- Resolution: Fixed > TextIO can't read concatenated gzip files >

[jira] [Updated] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-2490: - Fix Version/s: (was: 2.1.0) Not applicable > ReadFromText function

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061345#comment-16061345 ] Chamikara Jayalath commented on BEAM-2490: -- Sounds good. For now I'll remove this from the release

[jira] [Commented] (BEAM-2490) ReadFromText function is not taking all data with glob operator (*)

[ https://issues.apache.org/jira/browse/BEAM-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062040#comment-16062040 ] Chamikara Jayalath commented on BEAM-2490: -- Hi Guillermo, what is the OS and version of Beam you

[jira] [Commented] (BEAM-2337) BigQuery IO improvements

[ https://issues.apache.org/jira/browse/BEAM-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16018194#comment-16018194 ] Chamikara Jayalath commented on BEAM-2337: -- Currently BQ sink is a native sink. Load job happens

[jira] [Created] (BEAM-2340) Update contributors guide to include proper development instructions for python SDK

Chamikara Jayalath created BEAM-2340: Summary: Update contributors guide to include proper development instructions for python SDK Key: BEAM-2340 URL: https://issues.apache.org/jira/browse/BEAM-2340

[jira] [Assigned] (BEAM-2340) Update contribution guide to include proper development instructions for python SDK

[ https://issues.apache.org/jira/browse/BEAM-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-2340: Assignee: Chamikara Jayalath > Update contribution guide to include proper

[jira] [Commented] (BEAM-1909) BigQuery read transform fails for DirectRunner when querying non-US regions

[ https://issues.apache.org/jira/browse/BEAM-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020138#comment-16020138 ] Chamikara Jayalath commented on BEAM-1909: -- Hi Uwe, The issue here is not creating a

[jira] [Resolved] (BEAM-2338) GCS filepattern wildcard broken in Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-2338. -- Resolution: Fixed > GCS filepattern wildcard broken in Python SDK >

[jira] [Commented] (BEAM-1909) BigQuery read transform fails for DirectRunner when querying non-US regions

[ https://issues.apache.org/jira/browse/BEAM-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016359#comment-16016359 ] Chamikara Jayalath commented on BEAM-1909: -- This still has to be fixed for queries I believe. >

[jira] [Updated] (BEAM-2340) Update contribution guide to include proper development instructions for python SDK

[ https://issues.apache.org/jira/browse/BEAM-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-2340: - Summary: Update contribution guide to include proper development instructions for python

  1   2   3   4   5   >