[ https://issues.apache.org/jira/browse/BEAM-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864056#comment-15864056 ]
ASF GitHub Bot commented on BEAM-1040: -------------------------------------- GitHub user diptikul opened a pull request: https://github.com/apache/beam/pull/1994 BEAM-1040 Hadoop InputFormat - IO Transform for reads Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-<Jira issue #>] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/diptikul/incubator-beam HadoopInputFormatIO Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/1994.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1994 ---- commit 45cf382e14fceb32b2738b548c851472659f84d9 Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-22T13:49:38Z First Hadoop InputFormat IO project code check in commit 9c1a54d8d686ac396a1f500c9b69abe4d44e18a5 Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-23T08:15:07Z Merge rtest from original Take latest from original. remote-tracking branch 'upstream/master' commit 5f927add640a9888ca26662450ceb1ecfa92c08c Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2016-12-23T08:55:12Z erg end() method remote-tracking branch 'upstream/master' commit 6c7ec6a37286ae55e16f267f126342694eb02b0e Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-23T09:43:48Z Merge remote-tracking branch 'upstream/master' taking latest-23/12 commit e49f22fed8fe0a8798fa1fe729c33df105e54cb5 Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-23T10:20:45Z Updated Pom files commit 9e4fdba32171b86b2f67f0c73780a06f6ca1b187 Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-23T10:24:04Z Merge branch 'master' of https://github.com/diptikul/incubator-beam commit 6357bdf3c151c8c7575abad0f0e2f4bc4a424d7a Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2016-12-23T14:41:23Z Added code and tests to read from DummyInputFormat commit a2163e823d7de3a024913acbb6a4035b8d8b6dd3 Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2016-12-23T14:48:38Z Resolved conflictsMerge branch 'master' of https://github.com/diptikul/incubator-beam commit fb1f2de7e479920ac53de25ba18dd673046c69da Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-23T14:50:06Z Merge branch 'master' of https://github.com/diptikul/incubator-beam commit 571fcb40838a61148d0d49669ec18df35779b784 Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-26T05:01:37Z Added Hadoop Input Format module in io pom file and minor changes in pom file. commit a785180fc3b95f66a9b30a03c184fb7d637777ad Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-26T05:30:25Z Merge branch 'master' of https://github.com/diptikul/incubator-beam commit b29276ae953e9fff7ed6eb0c241d21e34122c5d2 Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-26T09:49:39Z Moved MyCassandraRow to test folder commit 70f05266328d62ba3a9127d54bc9f3769effbde4 Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-26T10:51:29Z Removed CassandraRowCoder and Custom coder unit test, added MyCassandraRow in test, used Avro coder for MyCassandraRow commit f129ead54b1b7047f4c5ff592bc4f2e7ecc7bc43 Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-26T11:56:58Z Merge branch 'master' of https://github.com/diptikul/incubator-beam commit 276c3c217df6c062b8dd4b5deda0e3425c775513 Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2016-12-27T06:02:14Z Added test cases for validate method, getFractionsConsumed method commit 020576283b10c7630bb38df8836387404a57b76e Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-27T07:03:24Z Merge branch 'master' of https://github.com/diptikul/incubator-beam commit c01544ffb61d6c129a932eef6d957e57c698b858 Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2016-12-28T10:46:54Z Added junits for splitIntoBundles(),computeSplits and added autovalue builder. commit 6e6819eec404ec131ec1f2ef98d985a2d51e4c9e Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-28T10:48:28Z Merge branch 'master' of https://github.com/diptikul/incubator-beam commit 8ebbd2b124b5f40e67be19ef9caba8f1f3b3c6bc Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-28T10:56:50Z Made tests compile with read() commit e0b291958e804e7c328be87ce860e63ea9253b20 Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-28T11:23:24Z New package for dummy input formats, and add reference of DisplayDataMatchers for unit test commit a1b092a67fd0456a835fb3c4115c0b14a3f2d094 Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2016-12-28T11:28:23Z Added Apache License header to all classes commit 049473e3350fe12a2b4c217cc1cfae1737872135 Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-30T07:44:52Z Merge remote-tracking branch 'upstream/master' commit 46b13e4a3851b262cb1a0f51bf791cacee2b6d8f Author: Radhika S Kulkarni <radhika_kulkar...@persistent.co.in> Date: 2016-12-30T07:55:01Z Added TypeDescriptors commit b04eb728120fe42442184eecae16ac987cfc4226 Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2016-12-30T09:55:21Z Added junits for testing immutability of HadoopInputFormatIO and junit for checking key/value classes set by user in configuration. commit 59b0f272a5432b2738416850161f0e61e5739d6f Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2017-01-03T06:29:21Z InputFormat changes, Exception thrown using annotation , testPipeline used for creation of pipeline commit 8f2a57071cfe83a8586328b3dd4c30d5c1596948 Author: Dipti Kulkarni <dipti_dkulka...@persistent.co.in> Date: 2017-01-03T07:26:06Z Added few comments, TestPipeline Usage commit 995bfa9b32ee8e5ded8225ef5fa1e15e3b4cb2df Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2017-01-03T14:00:09Z Added cassandra inegration test and modified junits as per test review comments. commit 6d5fc47d9a2fb74d5eec634fba4d63a6f87789f7 Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2017-01-04T13:06:13Z Added logging into HadoopInputFormatIO and modifications in Junits and cassandra integration test. commit a5db53f986e4a8e95713f9e7d8a1e1d0a22eab59 Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2017-01-05T12:56:49Z Modification in code according to code review comments and google code formatting. commit 8d6392b058b08cdc7bc9b21623e35d57e996a29a Author: Sheetal Tarodekar <sheetal_tarode...@persistent.co.in> Date: 2017-01-06T12:02:26Z Modifications as per code review comments. ---- > Hadoop InputFormat - IO Transform for reads > ------------------------------------------- > > Key: BEAM-1040 > URL: https://issues.apache.org/jira/browse/BEAM-1040 > Project: Beam > Issue Type: Bug > Components: sdk-java-extensions > Reporter: Stephen Sisk > Assignee: Dipti Kulkarni > > We should build a IO read transform that will read data from services > supporting the Hadoop InputFormat interface [1] > This will make it easy to connect to a variety of databases while still > providing some aspects of scalability since the InputFormat interface > provides for parallel reading. > [1] > https://hadoop.apache.org/docs/r2.7.3/api/org/apache/hadoop/mapred/InputFormat.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)