This is an automated email from the ASF dual-hosted git repository. lcwik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git.
from d086d87 Merge pull request #4677: Add a Beam Jenkins Runner Inventory Cron add e2aa9b3 Initial sketches of a Go SDK add 7197d5e Initial version of the direct style w/ direct runner. Incomplete. add e92e919 Add Data as UserFn context w/ immediate value. add 995b2ec Added no-I/O wordcount for profiling. add 900da2e Fleshed out possible approach to generic transformations. add 7bc0498 Add “dag” example that use multiplexing and side input. add 915416a Added a more complex DAG example. add fa8723a Add yatzy example with more complex construction-time setup add 60d17c1 Add proto for Fn API add b1aaac5 Add beam.Composite helper for the most common pattern to align with java add d08fdaa Move pipeline-construction time errors into an accumulator add ee71e83 Add Dataflow job and Fn API clients. Incomplete. add 61db2fb Add binary cross-compile and upload to Dataflow runner. Incomplete. add 5ce4d1c Add tentative runner indirection (default: local). add 01111a9 Made data flow runner detect user main for cross-compilation. add 08ba947 Remove error accumulation in favor of panic. add f66bd66 Improve Dataflow translation of coders, side input and composite names. add c579e85 Fix name for AsView. add 26a99d5 Use 2 grpc endpoints in harness add d5cf63d Add gRPC harness logging add 0b2fdbc Flesh out harness and serialization further. add 6f4af7f Made the dataflow runner wait for job termination by default add 3c8f986 beam: add 687cbf8 beam: add c47a546 combinefn.go: fix compilation issues add b9d03b2 Improve dataflow serialization and execution. Incomplete. add af65f9d Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging. add 38ddbbb Move the 30s sleep for logging to the harness instead of in WordCap. add e185f48 Post-review updates. add 8dcd0c4 Doc updates. add d64868c Flesh out coders. Incomplete. add ca49202 Added prototype implementation of more coders and the runner source. Incomplete. add 0444a45 dofn: illustrates how dofns are written. add 9e2a7cc beam: add viewfn and windowfn to side inputs match support Beam 1.0 add b767a95 dofn: timers add be7e061 Complete revamp: coders, graph and execution use element-wise processing. add 055c859 Fix coder encoding for Dataflow side input. Otherwise, the job is rejected. add 22536e3 Added more godoc comments to graph types. add 747930f Added more comments plus made local GBK use coder equality. add ce8c4df Added Flatten support and “forest” example that uses it. add 61d8932 Move bigqueryio to defunct add ca59228 Make forest example print less add dbafc8b Add external struct registry and serialization. add 174a2b0 Updated comments in node.go. add 72e3bbe Replace real type with 'full type' since that's the current term. add 58dda52 Refactor Fn API dependency. add bf6ff10 Added more comments to the runner/dataflow and runner/beamexec packages add bcc5001 Fix most go vet issues add 24e2078 Make core operations panic to cut down on the error propagation boilerplate add 2662fc8 Add more comments to the graph package. add 4eab543 Add DoFn wrapper to handle either function or (ptr to) struct add eca6fec Fix remaining go vet warnings. add f1953d3 Code review for beam/graph/coder package. add 5951d28 Code review of the runtime/graphx package. add 978b60e Remove Data options in favor of using a Fn struct add d56db10 Code review of the beam/graph/userfn package. add 9a74200 Code review for beam/graph package. add 15489ee godoc for runtime/graphx add af323c8 Add support for []T and Combine functions add b1d8df2 Add adapted documentation from the Java SDK to the beam package add 4f5126c Update snapshot of Fn API. add bf3cac1 Add experiments flag to the Dataflow runner add 3c848b6 Remove context arg from beamexec.Init add ca38035 Migration to Runner API. add 7b6ca59 Add support for creating DOT graphs. add 0dc6e2d Make pretty printing of types and coders more concise add a854fbc Add flexible Signature to aid type checking add c87e176 Adding unit testability to harness translation. add b3f65ea Fix crash due to initialization order add ea6c7c4 Add CreateValues and Impulse add 1637568 Add Runner API support for WindowingStrategy. add 65d0195 Run goimports on baseline. add dc23901 Fix encoding of global window strategy. add adc7869 Ensure the windowed value is atomically encoded. add ce880b7 Limit gRPC messages to max size. add 841520e Developer conveniences for running jobs. add 35ff36e Fix sends to not close the network channel. add 461c315 Add re-iterable side input add ea78889 Add per-key Combine add 121acd8 Add Min add cbd622e Reorganize non-user-facing code into core add c0962fb Make type register reject unnamed or predeclared types add 6b51242 Add type specialization tool add 0d439e9 Don't run grpc plugin in generate phase. add 40d8111 Fix import reference path for runner API proto. add 34d9647 Revamp runner registration as _ imports add a9136b3 Add stats.Max and Mean add 214c076 Add global pipeline options add 11f9c03 Unify global and per-key combiners add c0658c5 Add beam convenience wrapper for imports and runner selection add dca064e Add session recording and CPU profiling to harness. add d17c601 Add ptest and passert for testing pipelines add 9e575d6 Add GCS and glob support to textio add 4fa8db0 Add BigQuery IO and examples add f01e74a Adds a session runner for testing. add 1627288 Add Partition and dynamic functions add 9bf757f Adding example that returns 10 words that contain provided search string. add e95172b Remove duplicate LOG line add 5322ed6 Enable Combine Fns in Dataflow runner by modifying translation. add 5e26b7a Fixing type bug by dropping T and using underlying type of value in addInput(). Changing Count() to use Sum(). add dbbe214 Adding graph validation at build time. add fc5ab4f Import the Fn API changes. add f945954 Simple changes to support new Fn API coder changes. add bf6bc20 Update translator to work with new Fn API changes. add 5c9cba6 Use appropriate equality tests. add b897ea4 Fix test to not use path of package. add 330d037 Renaming directory to match package name. add 895b735 Fixing random nits in comments. add 308fee0 Modify build command to avoid bash. add c762062 Fixing selected golint issues. add f109f5e Addressing import review comments. add 0c89de4 Add coder specialization for bytes/strings. add 7ee4c51 Adding unit tests to stats. add 00fefa5 Fixing typo. add b7b8c31 Add beam.External add da1c077 Fix grpc.Dial calls to block properly. add 3246480 Creates a symtab verifier by running Sym2Addr and Addr2Sym in a binary. add c75f747 Add spec field to help interpretation of payload. add 7ac34c7 Use beam.T alias for typex.T etc with Go 1.9 add a1ed07c Move shared GCP options to a separate package add fb2c2a3 Update portability protos add 3fdf71e Remove old source/sink from beam package add e6e05f5 Add context-aware logging for both pipeline-submission time and runtime add 52fdf4b Fix coder inference for strings. add 5f38b6d Improve tornadoes example add 6187549 Fix beam.External to map arguments correctly. add e99cb8e Added comments to yatzy and forest add e0a5b71 Add comments to tornadoes from the java counterpart add 738415f Rename Pipeline Composite to Scope add b8616e8 Add 3 progressive wordcount examples add 10f9c28 Clarify comments in wordcount pipelines add 55ea735 Add apache 2.0 license to files add c665314 Updates to examples. add afc6097 Adding more godoc for the main beam package. add 00a3bbc Update to new proto structure add bc58989 Split Combine and fields in to global and per-key variants add 1899718 Refactor Flatten of a single []T into Explode add bd962f6 Rename local runner to direct runner add 663046e Fix argument index error in ParDo execution add 670b13c Add Apache copyright header to files that need it. add 2d5a991 Made debug.Head not just work per bundle add 47b2e31 Impose a total ordering on Fn parameters. add 0b0aeac Rename Dedup to Distinct for consistency with other SDKs add baa38c2 Add coder to model coder translation add a088674 Simplify harness coder translation add 663fad3 Split Pipeline into Pipeline and Scope add 5743a37 Relocate Go SDK code add 999985c Fix Go SDK maven build add 38df1b7 Move Go SKD to latest version of bigquery add 473d7a0 Merge pull request #4200 from herohde/go-sdk2 add 62de5ca Add Go SDK container image add 06f6411 Merge pull request #4215: [BEAM-3287] Add Go SDK container image add c2c152a Add Go SDK README add 688782c Merge pull request #4217: [BEAM-2083] Add Go SDK README add 7c958a8 Update version for Go Dataflow pipelines add e966d4e Make Scope a value type add 008ba51 Merge pull request #4230: [BEAM-2083] Make Scope a value type in Go SDK add fffaed1 Add Go graph/pipeline translation add f42ab80 Stage Go model pipeline for Dataflow add eb4367d Use pipeline unmarhaller in runtime harness add 722a7e1 CR: [BEAM-3287] Use model pipelines in Go SDK add 99f3997 CR: [BEAM-3287] Use model pipelines in Go SDK add 92883b2 Merge pull request #4213: [BEAM-3287] Use model pipelines in Go SDK add 0d5d495 Fix name of syscallx ErrUnsupported add 18a7647 Allow any named type to be registered and serialized as external add 4fa1d78 Add more package comments for core packages add 2d695e7 Make Go SDK External a graph primitive add 4a534b0 Merge pull request #4237: [BEAM-3294] Make Go SDK external transform a primitive add b258467 Cache Go runtime symbol lookups add 61558b9 Merge pull request #4256: [BEAM-3324] Cache Go runtime symbol lookups add 9bdae2d Merge pull request #4271: merge master into go-sdk add cb6b6ba Fix storagePath variable. add a4e4cb3 Merge pull request #4279: Fix storagePath variable. add 14eb2cd [BEAM-3356] Add Go SDK int and varint custom coders (#4276) add 610db85 BEAM-3361 Increase Go gRPC message size add e49cde3 Go SDK runtime revamp add 60216f1 Merge pull request #4265 from herohde/runtime2 add 6557632 Add a few function call overhead benchmarks add fc550df Add type-specialized emitters add a9dc174 BEAM-3324 improve symtab memory usage add d85a502 BEAM-3324 improve symtab memory usage add 03f96c6 BEAM-3324 improve symtab memory usage add 412de6f Store objects in pool so they can be reused. add e6f3af2 Add builtin varint coder add f68f05e Merge pull request #4350: Integrate master into go-sdk add d4b3624 Type-specialize custom decoders and encoders in Go SDK runtime add 7bdffd6 Type-specialize iterators and side input in the Go SDK add 9be0836 Add warnings if Go runtime registrations are overwritten add 0b43308 Add reusable element coders in Go SDK runtime add 2a0ba98 Merge pull request #4336 from herohde/runtime5 add 1379ad7 BEAM-3368 fix translation for external add b7168ce Updated translater to preserve payload and its URN. add 92cbda3 Merge pull request #4284 from wcn3/model_external_support add 0db4090 Initial version of type-specialized general reflective calls add 5ff88b0 Add general-purpose untyped callers in Go SDK runtime add 2dc8363 Use fast caller for filter transform predicate add 0f3ec80 CR: Clarified comment on caller template add acac64e Merge pull request #4373 from herohde/runtime6 add 599b746 Fix value encoding for Create add 334180c Merge pull request #4397 add 1fe55ca BEAM-3473: Fix GroupByKey iterators to be initialized. add 864b359 BEAM-3474 Include stacks in panic messages. add 0af2503 BEAM-3299: Add source reporting support. add a86c25e Remove GetId() call from under lock. add efa1c20 Add additional comments about concurrency invariants. add c5a3ce0 Add initialization of active plans map. add c67de93 Renamed Go runtime Caller to Func and added name add 09e98e4 Use reflectx.Func as the fundamental function representation add 8a9d916 CR: fix DynFn comments add e019f19 CR: fix comments add 92e3bbe Avoid reflect.Value conversions in Go runtime add 76cc0a6 Merge pull request #4452 from herohde/runtime8 add cebdbb7 Allow grpcx.Dial to support overrides. add b9b191f Merge pull request #4456: BEAM-3505: Allow grpcx.Dial to support overrides. add 927e46c Use a typeswitch instead of reflect.Convert when encoding strings or bytes. add da0d0df Update coder.go add 9aa8fe4 Merge pull request #4474 from lostluck/byteencode add 0f01201 Replace reflective convert to direct convert. add 8cbd440 Merge pull request #4509 add 5386431 Merge branch 'master' into go-sdk add 010272b Merge master into go-sdk add 1f33d31 Fix beam.Combine to combine globally add f7e4e41 Merge pull request #4556: Fix beam.Combine to combine globally add 6834287 Add optional function registration to Go SDK runtime add 4b69c36 Merge pull request #4582: [BEAM-3597] Add optional function registration to Go SDK runtime add bd97498 fixup! Remove reflection from varint codecs add 52a598d [BEAM-3388] Remove reflection from varint codecs add 07e534b Changed core GBK to CoGBK add 66ecd91 Add CoGBK support to direct runner and Join example add 79fb7e1 [BEAM-3316] Translate bundle descriptors directly to execution plans in Go SDK add 1cf50dd Translate CoGBK into GBK for Dataflow and model pipeline runners add b7dce98 CR: [BEAM-3302] Support CoGBK in the Go SDK add a328e44 Merge pull request #4540 [BEAM-3302] Support CoGBK in the Go SDK add 21bc8cf [BEAM-3579] Fix textio.Write add 6e71095 CR: fix Go SDK textio.Write add 8cb9500 Merge pull request #4624 add 0dcbb0b Improve rendering of DOT diagrams. add 5cb4ac6 Dot rendering improvement add 8e8d5fb Fixing filename. add e2d7408 Merge pull request #4654: CoGBK fixup - rename file add 48da05a Integrate master into go-sdk branch add ed922cb Update Go SDK coder constants add 5e6db92 Merge pull request #4682: [BEAM-3684] Integrate master to go-sdk and fixup coder urns add b7529d7 Merge branch 'master' into go-sdk add d960b53 Merge branch 'master' into go-sdk new 45ff873 Merge master into go-sdk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: pom.xml | 5 +- sdks/go/README.md | 98 + sdks/go/cmd/specialize/main.go | 240 + sdks/go/cmd/symtab/main.go | 101 + sdks/go/container/Dockerfile | 30 + sdks/go/container/boot.go | 127 + sdks/go/{ => container}/pom.xml | 87 +- sdks/go/data/haiku/old_pond.txt | 3 + sdks/go/data/shakespeare/hamlet.txt | 6045 ++++++ sdks/go/data/shakespeare/kinglear-hashtag.txt | 5525 +++++ sdks/go/data/shakespeare/kinglear.txt | 5525 +++++ sdks/go/data/shakespeare/romeoandjuliet.txt | 4766 +++++ .../examples/complete/autocomplete/autocomplete.go | 71 + sdks/go/examples/contains/contains.go | 99 + sdks/go/examples/cookbook/combine/combine.go | 106 + sdks/go/examples/cookbook/filter/filter.go | 106 + sdks/go/examples/cookbook/join/join.go | 111 + sdks/go/examples/cookbook/max/max.go | 88 + sdks/go/examples/cookbook/tornadoes/tornadoes.go | 119 + .../debugging_wordcount/debugging_wordcount.go | 163 + sdks/go/examples/forest/forest.go | 81 + sdks/go/examples/grades/grades.go | 105 + .../minimal_wordcount/minimal_wordcount.go | 101 + sdks/go/examples/pingpong/pingpong.go | 131 + sdks/go/examples/wordcap/wordcap.go | 75 + sdks/go/examples/wordcount/wordcount.go | 168 + sdks/go/examples/yatzy/yatzy.go | 139 + sdks/go/pkg/beam/coder.go | 180 + .../pkg/beam/{util/grpcx/dial.go => coder_test.go} | 34 +- sdks/go/pkg/beam/combine.go | 78 + .../syscall_default.go => core/funcx/doc.go} | 20 +- sdks/go/pkg/beam/core/funcx/fn.go | 427 + sdks/go/pkg/beam/core/funcx/fn_test.go | 224 + sdks/go/pkg/beam/core/funcx/output.go | 76 + sdks/go/pkg/beam/core/funcx/output_test.go | 48 + sdks/go/pkg/beam/core/funcx/sideinput.go | 100 + sdks/go/pkg/beam/core/funcx/sideinput_test.go | 70 + sdks/go/pkg/beam/core/funcx/signature.go | 212 + sdks/go/pkg/beam/core/funcx/signature_test.go | 212 + sdks/go/pkg/beam/core/graph/bind.go | 325 + sdks/go/pkg/beam/core/graph/bind_test.go | 207 + sdks/go/pkg/beam/core/graph/coder/coder.go | 298 + sdks/go/pkg/beam/core/graph/coder/int.go | 71 + sdks/go/pkg/beam/core/graph/coder/int_test.go | 88 + .../grpcx/dial.go => core/graph/coder/time.go} | 31 +- sdks/go/pkg/beam/core/graph/coder/varint.go | 98 + sdks/go/pkg/beam/core/graph/coder/varint_test.go | 94 + .../syscall_default.go => core/graph/doc.go} | 20 +- sdks/go/pkg/beam/core/graph/edge.go | 397 + sdks/go/pkg/beam/core/graph/fn.go | 304 + sdks/go/pkg/beam/core/graph/graph.go | 126 + sdks/go/pkg/beam/core/graph/graph_test.go | 68 + sdks/go/pkg/beam/core/graph/node.go | 70 + .../syscall_default.go => core/graph/scope.go} | 27 +- sdks/go/pkg/beam/core/graph/window/window.go | 71 + sdks/go/pkg/beam/core/runtime/coderx/int.go | 98 + sdks/go/pkg/beam/core/runtime/coderx/varint.go | 139 + .../go/pkg/beam/core/runtime/coderx/varint_test.go | 84 + sdks/go/pkg/beam/core/runtime/exec/coder.go | 319 + sdks/go/pkg/beam/core/runtime/exec/cogbk.go | 171 + sdks/go/pkg/beam/core/runtime/exec/combine.go | 257 + sdks/go/pkg/beam/core/runtime/exec/combine_test.go | 67 + sdks/go/pkg/beam/core/runtime/exec/datasink.go | 90 + sdks/go/pkg/beam/core/runtime/exec/datasource.go | 203 + sdks/go/pkg/beam/core/runtime/exec/decode.go | 102 + .../runtime/exec/discard.go} | 41 +- sdks/go/pkg/beam/core/runtime/exec/emit.go | 126 + sdks/go/pkg/beam/core/runtime/exec/encode.go | 102 + sdks/go/pkg/beam/core/runtime/exec/flatten.go | 75 + sdks/go/pkg/beam/core/runtime/exec/flatten_test.go | 49 + sdks/go/pkg/beam/core/runtime/exec/fn.go | 201 + sdks/go/pkg/beam/core/runtime/exec/fn_test.go | 256 + sdks/go/pkg/beam/core/runtime/exec/fullvalue.go | 145 + .../pkg/beam/core/runtime/exec/fullvalue_test.go | 72 + sdks/go/pkg/beam/core/runtime/exec/input.go | 188 + sdks/go/pkg/beam/core/runtime/exec/multiplex.go | 62 + .../pkg/beam/core/runtime/exec/multiplex_test.go | 55 + .../beam/core/runtime/exec/optimized/callers.go | 632 + .../beam/core/runtime/exec/optimized/callers.tmpl | 69 + .../beam/core/runtime/exec/optimized/decoders.go | 2407 +++ .../beam/core/runtime/exec/optimized/decoders.tmpl | 146 + .../beam/core/runtime/exec/optimized/emitters.go | 14215 +++++++++++++ .../beam/core/runtime/exec/optimized/emitters.tmpl | 112 + .../beam/core/runtime/exec/optimized/encoders.go | 2299 ++ .../beam/core/runtime/exec/optimized/encoders.tmpl | 146 + .../runtime/exec/optimized/gen.go} | 19 +- .../pkg/beam/core/runtime/exec/optimized/inputs.go | 21272 +++++++++++++++++++ .../beam/core/runtime/exec/optimized/inputs.tmpl | 181 + sdks/go/pkg/beam/core/runtime/exec/pardo.go | 193 + sdks/go/pkg/beam/core/runtime/exec/pardo_test.go | 97 + sdks/go/pkg/beam/core/runtime/exec/plan.go | 153 + .../runtime/exec/status.go} | 21 +- sdks/go/pkg/beam/core/runtime/exec/translate.go | 433 + sdks/go/pkg/beam/core/runtime/exec/unit.go | 75 + sdks/go/pkg/beam/core/runtime/exec/unit_test.go | 111 + sdks/go/pkg/beam/core/runtime/exec/util.go | 72 + sdks/go/pkg/beam/core/runtime/graphx/coder.go | 415 + sdks/go/pkg/beam/core/runtime/graphx/coder_test.go | 107 + sdks/go/pkg/beam/core/runtime/graphx/cogbk.go | 108 + .../runtime/graphx/doc.go} | 27 +- sdks/go/pkg/beam/core/runtime/graphx/serialize.go | 962 + sdks/go/pkg/beam/core/runtime/graphx/translate.go | 402 + .../pkg/beam/core/runtime/graphx/translate_test.go | 90 + sdks/go/pkg/beam/core/runtime/graphx/tree.go | 121 + sdks/go/pkg/beam/core/runtime/graphx/user.go | 110 + .../runtime/graphx/v1/gen.go} | 14 +- sdks/go/pkg/beam/core/runtime/graphx/v1/v1.pb.go | 821 + sdks/go/pkg/beam/core/runtime/graphx/v1/v1.proto | 232 + sdks/go/pkg/beam/core/runtime/harness/datamgr.go | 308 + .../runtime/harness/gen.go} | 14 +- sdks/go/pkg/beam/core/runtime/harness/harness.go | 291 + sdks/go/pkg/beam/core/runtime/harness/init/init.go | 116 + sdks/go/pkg/beam/core/runtime/harness/logging.go | 151 + sdks/go/pkg/beam/core/runtime/harness/session.go | 215 + .../go/pkg/beam/core/runtime/harness/session.proto | 69 + .../core/runtime/harness/session/session.pb.go | 444 + sdks/go/pkg/beam/core/runtime/init.go | 44 + sdks/go/pkg/beam/core/runtime/options.go | 91 + .../grpcx/dial.go => core/runtime/options_test.go} | 44 +- sdks/go/pkg/beam/core/runtime/symbols.go | 93 + sdks/go/pkg/beam/core/runtime/types.go | 62 + sdks/go/pkg/beam/core/runtime/types_test.go | 74 + sdks/go/pkg/beam/core/typex/class.go | 174 + sdks/go/pkg/beam/core/typex/class_test.go | 104 + sdks/go/pkg/beam/core/typex/fulltype.go | 400 + sdks/go/pkg/beam/core/typex/fulltype_test.go | 125 + sdks/go/pkg/beam/core/typex/special.go | 74 + sdks/go/pkg/beam/core/util/dot/dot.go | 130 + .../syscall.go => core/util/ioutilx/read.go} | 29 +- sdks/go/pkg/beam/core/util/protox/any.go | 101 + sdks/go/pkg/beam/core/util/protox/any_test.go | 83 + .../grpcx/dial.go => core/util/protox/base64.go} | 37 +- .../util/protox/protox.go} | 20 +- .../grpcx/dial.go => core/util/protox/query.go} | 34 +- sdks/go/pkg/beam/core/util/reflectx/call.go | 120 + sdks/go/pkg/beam/core/util/reflectx/calls.go | 1341 ++ sdks/go/pkg/beam/core/util/reflectx/calls.tmpl | 72 + sdks/go/pkg/beam/core/util/reflectx/functions.go | 45 + .../syscall.go => core/util/reflectx/json.go} | 21 +- sdks/go/pkg/beam/core/util/reflectx/tags.go | 74 + sdks/go/pkg/beam/core/util/reflectx/types.go | 111 + sdks/go/pkg/beam/core/util/symtab/symtab.go | 127 + sdks/go/pkg/beam/create.go | 108 + .../beam/{util/grpcx/dial.go => create_test.go} | 38 +- sdks/go/pkg/beam/doc.go | 67 + sdks/go/pkg/beam/encoding.go | 148 + sdks/go/pkg/beam/external.go | 64 + sdks/go/pkg/beam/flatten.go | 72 + sdks/go/pkg/beam/forward.go | 94 + sdks/go/pkg/beam/gbk.go | 104 + sdks/go/pkg/beam/impulse.go | 46 + sdks/go/pkg/beam/io/bigqueryio/bigquery.go | 241 + .../dial.go => io/bigqueryio/bigquery_test.go} | 35 +- sdks/go/pkg/beam/io/textio/filesystem.go | 49 + sdks/go/pkg/beam/io/textio/gcs/gcs.go | 130 + .../grpcx/dial.go => io/textio/local/local.go} | 44 +- sdks/go/pkg/beam/io/textio/textio.go | 207 + sdks/go/pkg/beam/log/log.go | 183 + .../{util/syscallx/syscall.go => log/standard.go} | 26 +- sdks/go/pkg/beam/option.go | 72 + .../grpcx/dial.go => options/gcpopts/options.go} | 27 +- sdks/go/pkg/beam/pardo.go | 314 + sdks/go/pkg/beam/partition.go | 124 + sdks/go/pkg/beam/partition_test.go | 155 + sdks/go/pkg/beam/pcollection.go | 90 + sdks/go/pkg/beam/pipeline.go | 87 + sdks/go/pkg/beam/runner.go | 51 + sdks/go/pkg/beam/runners/dataflow/dataflow.go | 367 + sdks/go/pkg/beam/runners/dataflow/messages.go | 174 + sdks/go/pkg/beam/runners/dataflow/translate.go | 448 + sdks/go/pkg/beam/runners/direct/buffer.go | 165 + sdks/go/pkg/beam/runners/direct/direct.go | 274 + sdks/go/pkg/beam/runners/direct/gbk.go | 135 + sdks/go/pkg/beam/runners/direct/impulse.go | 61 + sdks/go/pkg/beam/runners/dot/dot.go | 55 + sdks/go/pkg/beam/runners/session/session.go | 337 + sdks/go/pkg/beam/testing/passert/passert.go | 214 + sdks/go/pkg/beam/testing/ptest/ptest.go | 59 + .../dial.go => transforms/filter/distinct.go} | 39 +- .../go/pkg/beam/transforms/filter/distinct_test.go | 70 + sdks/go/pkg/beam/transforms/filter/filter.go | 95 + sdks/go/pkg/beam/transforms/filter/filter_test.go | 90 + .../syscall.go => transforms/stats/count.go} | 23 +- sdks/go/pkg/beam/transforms/stats/count_test.go | 69 + sdks/go/pkg/beam/transforms/stats/max.go | 44 + sdks/go/pkg/beam/transforms/stats/max_switch.go | 138 + .../stats/max_switch.tmpl} | 30 +- sdks/go/pkg/beam/transforms/stats/max_test.go | 134 + sdks/go/pkg/beam/transforms/stats/mean.go | 91 + sdks/go/pkg/beam/transforms/stats/mean_test.go | 121 + sdks/go/pkg/beam/transforms/stats/min.go | 44 + sdks/go/pkg/beam/transforms/stats/min_switch.go | 138 + .../stats/min_switch.tmpl} | 30 +- sdks/go/pkg/beam/transforms/stats/min_test.go | 121 + sdks/go/pkg/beam/transforms/stats/sum.go | 44 + sdks/go/pkg/beam/transforms/stats/sum_switch.go | 102 + .../stats/sum_switch.tmpl} | 27 +- sdks/go/pkg/beam/transforms/stats/sum_test.go | 117 + sdks/go/pkg/beam/transforms/stats/util.go | 48 + sdks/go/pkg/beam/transforms/top/top.go | 180 + sdks/go/pkg/beam/transforms/top/top_test.go | 126 + sdks/go/pkg/beam/util.go | 117 + sdks/go/pkg/beam/util/gcsx/gcs.go | 40 + sdks/go/pkg/beam/util/grpcx/dial.go | 12 +- sdks/go/pkg/beam/util/syscallx/syscall.go | 6 +- sdks/go/pkg/beam/util/syscallx/syscall_default.go | 4 +- sdks/go/pkg/beam/validate.go | 79 + sdks/go/pkg/beam/x/beamx/run.go | 41 + sdks/go/pkg/beam/x/debug/head.go | 68 + sdks/go/pkg/beam/x/debug/print.go | 97 + sdks/go/pom.xml | 8 + 211 files changed, 89285 insertions(+), 366 deletions(-) create mode 100644 sdks/go/README.md create mode 100644 sdks/go/cmd/specialize/main.go create mode 100644 sdks/go/cmd/symtab/main.go create mode 100644 sdks/go/container/Dockerfile create mode 100644 sdks/go/container/boot.go copy sdks/go/{ => container}/pom.xml (69%) create mode 100644 sdks/go/data/haiku/old_pond.txt create mode 100644 sdks/go/data/shakespeare/hamlet.txt create mode 100644 sdks/go/data/shakespeare/kinglear-hashtag.txt create mode 100644 sdks/go/data/shakespeare/kinglear.txt create mode 100644 sdks/go/data/shakespeare/romeoandjuliet.txt create mode 100644 sdks/go/examples/complete/autocomplete/autocomplete.go create mode 100644 sdks/go/examples/contains/contains.go create mode 100644 sdks/go/examples/cookbook/combine/combine.go create mode 100644 sdks/go/examples/cookbook/filter/filter.go create mode 100644 sdks/go/examples/cookbook/join/join.go create mode 100644 sdks/go/examples/cookbook/max/max.go create mode 100644 sdks/go/examples/cookbook/tornadoes/tornadoes.go create mode 100644 sdks/go/examples/debugging_wordcount/debugging_wordcount.go create mode 100644 sdks/go/examples/forest/forest.go create mode 100644 sdks/go/examples/grades/grades.go create mode 100644 sdks/go/examples/minimal_wordcount/minimal_wordcount.go create mode 100644 sdks/go/examples/pingpong/pingpong.go create mode 100644 sdks/go/examples/wordcap/wordcap.go create mode 100644 sdks/go/examples/wordcount/wordcount.go create mode 100644 sdks/go/examples/yatzy/yatzy.go create mode 100644 sdks/go/pkg/beam/coder.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => coder_test.go} (58%) create mode 100644 sdks/go/pkg/beam/combine.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/funcx/doc.go} (64%) create mode 100644 sdks/go/pkg/beam/core/funcx/fn.go create mode 100644 sdks/go/pkg/beam/core/funcx/fn_test.go create mode 100644 sdks/go/pkg/beam/core/funcx/output.go create mode 100644 sdks/go/pkg/beam/core/funcx/output_test.go create mode 100644 sdks/go/pkg/beam/core/funcx/sideinput.go create mode 100644 sdks/go/pkg/beam/core/funcx/sideinput_test.go create mode 100644 sdks/go/pkg/beam/core/funcx/signature.go create mode 100644 sdks/go/pkg/beam/core/funcx/signature_test.go create mode 100644 sdks/go/pkg/beam/core/graph/bind.go create mode 100644 sdks/go/pkg/beam/core/graph/bind_test.go create mode 100644 sdks/go/pkg/beam/core/graph/coder/coder.go create mode 100644 sdks/go/pkg/beam/core/graph/coder/int.go create mode 100644 sdks/go/pkg/beam/core/graph/coder/int_test.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => core/graph/coder/time.go} (50%) create mode 100644 sdks/go/pkg/beam/core/graph/coder/varint.go create mode 100644 sdks/go/pkg/beam/core/graph/coder/varint_test.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/graph/doc.go} (63%) create mode 100644 sdks/go/pkg/beam/core/graph/edge.go create mode 100644 sdks/go/pkg/beam/core/graph/fn.go create mode 100644 sdks/go/pkg/beam/core/graph/graph.go create mode 100644 sdks/go/pkg/beam/core/graph/graph_test.go create mode 100644 sdks/go/pkg/beam/core/graph/node.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/graph/scope.go} (60%) create mode 100644 sdks/go/pkg/beam/core/graph/window/window.go create mode 100644 sdks/go/pkg/beam/core/runtime/coderx/int.go create mode 100644 sdks/go/pkg/beam/core/runtime/coderx/varint.go create mode 100644 sdks/go/pkg/beam/core/runtime/coderx/varint_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/coder.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/cogbk.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/combine.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/combine_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/datasink.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/datasource.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/decode.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/runtime/exec/discard.go} (52%) create mode 100644 sdks/go/pkg/beam/core/runtime/exec/emit.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/encode.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/flatten.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/flatten_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/fn.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/fn_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/fullvalue.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/fullvalue_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/input.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/multiplex.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/multiplex_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/callers.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/callers.tmpl create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/decoders.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/decoders.tmpl create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/emitters.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/emitters.tmpl create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/encoders.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/encoders.tmpl copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/runtime/exec/optimized/gen.go} (51%) create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/inputs.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/optimized/inputs.tmpl create mode 100644 sdks/go/pkg/beam/core/runtime/exec/pardo.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/pardo_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/plan.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/runtime/exec/status.go} (71%) create mode 100644 sdks/go/pkg/beam/core/runtime/exec/translate.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/unit.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/unit_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/exec/util.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/coder.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/coder_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/cogbk.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/runtime/graphx/doc.go} (50%) create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/serialize.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/translate.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/translate_test.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/tree.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/user.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/runtime/graphx/v1/gen.go} (71%) create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/v1/v1.pb.go create mode 100644 sdks/go/pkg/beam/core/runtime/graphx/v1/v1.proto create mode 100644 sdks/go/pkg/beam/core/runtime/harness/datamgr.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/runtime/harness/gen.go} (71%) create mode 100644 sdks/go/pkg/beam/core/runtime/harness/harness.go create mode 100644 sdks/go/pkg/beam/core/runtime/harness/init/init.go create mode 100644 sdks/go/pkg/beam/core/runtime/harness/logging.go create mode 100644 sdks/go/pkg/beam/core/runtime/harness/session.go create mode 100644 sdks/go/pkg/beam/core/runtime/harness/session.proto create mode 100644 sdks/go/pkg/beam/core/runtime/harness/session/session.pb.go create mode 100644 sdks/go/pkg/beam/core/runtime/init.go create mode 100644 sdks/go/pkg/beam/core/runtime/options.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => core/runtime/options_test.go} (57%) create mode 100644 sdks/go/pkg/beam/core/runtime/symbols.go create mode 100644 sdks/go/pkg/beam/core/runtime/types.go create mode 100644 sdks/go/pkg/beam/core/runtime/types_test.go create mode 100644 sdks/go/pkg/beam/core/typex/class.go create mode 100644 sdks/go/pkg/beam/core/typex/class_test.go create mode 100644 sdks/go/pkg/beam/core/typex/fulltype.go create mode 100644 sdks/go/pkg/beam/core/typex/fulltype_test.go create mode 100644 sdks/go/pkg/beam/core/typex/special.go create mode 100644 sdks/go/pkg/beam/core/util/dot/dot.go copy sdks/go/pkg/beam/{util/syscallx/syscall.go => core/util/ioutilx/read.go} (67%) create mode 100644 sdks/go/pkg/beam/core/util/protox/any.go create mode 100644 sdks/go/pkg/beam/core/util/protox/any_test.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => core/util/protox/base64.go} (52%) copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => core/util/protox/protox.go} (72%) copy sdks/go/pkg/beam/{util/grpcx/dial.go => core/util/protox/query.go} (52%) create mode 100644 sdks/go/pkg/beam/core/util/reflectx/call.go create mode 100644 sdks/go/pkg/beam/core/util/reflectx/calls.go create mode 100644 sdks/go/pkg/beam/core/util/reflectx/calls.tmpl create mode 100644 sdks/go/pkg/beam/core/util/reflectx/functions.go copy sdks/go/pkg/beam/{util/syscallx/syscall.go => core/util/reflectx/json.go} (63%) create mode 100644 sdks/go/pkg/beam/core/util/reflectx/tags.go create mode 100644 sdks/go/pkg/beam/core/util/reflectx/types.go create mode 100644 sdks/go/pkg/beam/core/util/symtab/symtab.go create mode 100644 sdks/go/pkg/beam/create.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => create_test.go} (57%) create mode 100644 sdks/go/pkg/beam/doc.go create mode 100644 sdks/go/pkg/beam/encoding.go create mode 100644 sdks/go/pkg/beam/external.go create mode 100644 sdks/go/pkg/beam/flatten.go create mode 100644 sdks/go/pkg/beam/forward.go create mode 100644 sdks/go/pkg/beam/gbk.go create mode 100644 sdks/go/pkg/beam/impulse.go create mode 100644 sdks/go/pkg/beam/io/bigqueryio/bigquery.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => io/bigqueryio/bigquery_test.go} (57%) create mode 100644 sdks/go/pkg/beam/io/textio/filesystem.go create mode 100644 sdks/go/pkg/beam/io/textio/gcs/gcs.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => io/textio/local/local.go} (51%) create mode 100644 sdks/go/pkg/beam/io/textio/textio.go create mode 100644 sdks/go/pkg/beam/log/log.go copy sdks/go/pkg/beam/{util/syscallx/syscall.go => log/standard.go} (61%) create mode 100644 sdks/go/pkg/beam/option.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => options/gcpopts/options.go} (60%) create mode 100644 sdks/go/pkg/beam/pardo.go create mode 100644 sdks/go/pkg/beam/partition.go create mode 100644 sdks/go/pkg/beam/partition_test.go create mode 100644 sdks/go/pkg/beam/pcollection.go create mode 100644 sdks/go/pkg/beam/pipeline.go create mode 100644 sdks/go/pkg/beam/runner.go create mode 100644 sdks/go/pkg/beam/runners/dataflow/dataflow.go create mode 100644 sdks/go/pkg/beam/runners/dataflow/messages.go create mode 100644 sdks/go/pkg/beam/runners/dataflow/translate.go create mode 100644 sdks/go/pkg/beam/runners/direct/buffer.go create mode 100644 sdks/go/pkg/beam/runners/direct/direct.go create mode 100644 sdks/go/pkg/beam/runners/direct/gbk.go create mode 100644 sdks/go/pkg/beam/runners/direct/impulse.go create mode 100644 sdks/go/pkg/beam/runners/dot/dot.go create mode 100644 sdks/go/pkg/beam/runners/session/session.go create mode 100644 sdks/go/pkg/beam/testing/passert/passert.go create mode 100644 sdks/go/pkg/beam/testing/ptest/ptest.go copy sdks/go/pkg/beam/{util/grpcx/dial.go => transforms/filter/distinct.go} (56%) create mode 100644 sdks/go/pkg/beam/transforms/filter/distinct_test.go create mode 100644 sdks/go/pkg/beam/transforms/filter/filter.go create mode 100644 sdks/go/pkg/beam/transforms/filter/filter_test.go copy sdks/go/pkg/beam/{util/syscallx/syscall.go => transforms/stats/count.go} (63%) create mode 100644 sdks/go/pkg/beam/transforms/stats/count_test.go create mode 100644 sdks/go/pkg/beam/transforms/stats/max.go create mode 100644 sdks/go/pkg/beam/transforms/stats/max_switch.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => transforms/stats/max_switch.tmpl} (68%) create mode 100644 sdks/go/pkg/beam/transforms/stats/max_test.go create mode 100644 sdks/go/pkg/beam/transforms/stats/mean.go create mode 100644 sdks/go/pkg/beam/transforms/stats/mean_test.go create mode 100644 sdks/go/pkg/beam/transforms/stats/min.go create mode 100644 sdks/go/pkg/beam/transforms/stats/min_switch.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => transforms/stats/min_switch.tmpl} (68%) create mode 100644 sdks/go/pkg/beam/transforms/stats/min_test.go create mode 100644 sdks/go/pkg/beam/transforms/stats/sum.go create mode 100644 sdks/go/pkg/beam/transforms/stats/sum_switch.go copy sdks/go/pkg/beam/{util/syscallx/syscall_default.go => transforms/stats/sum_switch.tmpl} (70%) create mode 100644 sdks/go/pkg/beam/transforms/stats/sum_test.go create mode 100644 sdks/go/pkg/beam/transforms/stats/util.go create mode 100644 sdks/go/pkg/beam/transforms/top/top.go create mode 100644 sdks/go/pkg/beam/transforms/top/top_test.go create mode 100644 sdks/go/pkg/beam/util.go create mode 100644 sdks/go/pkg/beam/validate.go create mode 100644 sdks/go/pkg/beam/x/beamx/run.go create mode 100644 sdks/go/pkg/beam/x/debug/head.go create mode 100644 sdks/go/pkg/beam/x/debug/print.go -- To stop receiving notification emails like this one, please contact lc...@apache.org.