This is an automated email from the ASF dual-hosted git repository.
echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.
discard 46ed555 Remove Encoders based on kryo now that we call Beam coders in
the runner
discard 25d0401 Fix: Remove generic hack of using object. Use actual Coder
encodedType in Encoders
discard ebc53fd Remove unneeded cast
discard ad29daf Use beam encoders also in the output of the source translation
discard 507bbd8 Fix javadoc
discard c980d4c Fix typo
discard fb3aa34 Add missing windowedValue Encoder call in Pardo
discard ee2c0e6 Apply spotless
discard 31c91a9 Apply new Encoders to Pardo. Replace Tuple2Coder with
MultiOutputCoder to deal with multiple output to use in Spark Encoder for
DoFnRunner
discard 868204f Apply new Encoders to GroupByKey
discard 30c662a Create a Tuple2Coder to encode scale tuple2
discard d093ffe Apply spotless
discard 6edcfa2 Apply new Encoders to AggregatorCombiner
discard 5beb435 Apply new Encoders to Window assign translation
discard ab7d24c Ignore long time failing test: SparkMetricsSinkTest
discard 3ac3c71 Improve performance of source: the mapper already calls
windowedValueCoder.decode, no need to call it also in the Spark encoder
discard bcbb697 Apply new Encoders to Read source
discard aa25e85 Apply new Encoders to CombinePerKey
discard f0f2078 Catch Exception instead of IOException because some coders to
not throw Exceptions at all (e.g.VoidCoder)
discard 3a333fb Put Encoders expressions serializable
discard cfdf4a4 Improve exceptions catching
discard b879123 Apply spotless and checkstyle and add javadocs
discard 0fe6f9b Add an assert of equality in the encoders test
discard d8b8b42 Fix generated code: uniform exceptions catching, fix
parenthesis and variable declarations
discard ef69410 Fix equal and hashcode
discard 4351304 Remove example code
discard c4a4464 Remove lazy init of beam coder because there is no generic
way on instanciating a beam coder
discard 91e923c Cast coder instanciated by reflection
discard 723c004 Add try catch around reflexion call in lazy init of beam coder
discard 8bbf991 Fix beam coder lazy init using reflexion
discard 959664f Fix getting the output value in code generation
discard 668227b Fix ExpressionEncoder generated code: typos, try catch, fqcn
discard cbd7c2b Fix warning in coder construction by reflexion
discard 2c94eef Fix call to scala Fucntion1 in coder lazy init
discard a758985 Lazy init coder because coder instance cannot be interpolated
by catalyst
discard b11e100 Fix code generation in Beam coder wrapper
discard 2bf4cd9 Add a simple spark native test to test Beam coders wrapping
into Spark Encoders
discard e96af88 Fix visibility of serializer and deserializer
discard 23735e4 Conform to spark ExpressionEncoders: pass classTags,
implement scala Product, pass children from within the ExpressionEncoder
discard a5d49f5 Fix scala Product in Encoders to avoid StackEverflow
discard 95fd272 type erasure: spark encoders require a Class<T>, pass Object
and cast to Class<T>
discard 84f2cbd9 Fix EncoderHelpers.fromBeamCoder() visibility
discard d613d6b Wrap Beam Coders into Spark Encoders using ExpressionEncoder:
deserialization part
discard 031754c Wrap Beam Coders into Spark Encoders using ExpressionEncoder:
serialization part
discard c350188 Use "sparkMaster" in local mode to obtain number of shuffle
partitions + spotless apply
discard a524036 Improve Pardo translation performance: avoid calling a filter
transform when there is only one output tag
discard 0cedc7a Add a TODO on perf improvement of Pardo translation
new 22d6466 Improve Pardo translation performance: avoid calling a filter
transform when there is only one output tag
new 20d5bbd Use "sparkMaster" in local mode to obtain number of shuffle
partitions + spotless apply
new a5c7da3 Wrap Beam Coders into Spark Encoders using ExpressionEncoder:
serialization part
new 5fa6331 Wrap Beam Coders into Spark Encoders using ExpressionEncoder:
deserialization part
new c9e3534 type erasure: spark encoders require a Class<T>, pass Object
and cast to Class<T>
new fff5092 Fix scala Product in Encoders to avoid StackEverflow
new 2aaf07a Conform to spark ExpressionEncoders: pass classTags,
implement scala Product, pass children from within the ExpressionEncoder, fix
visibilities
new e4478ff Add a simple spark native test to test Beam coders wrapping
into Spark Encoders
new d5645ff Fix code generation in Beam coder wrapper
new e6b68a8 Lazy init coder because coder instance cannot be interpolated
by catalyst
new fdba22d Fix warning in coder construction by reflexion
new 8b07ec8 Fix ExpressionEncoder generated code: typos, try catch, fqcn
new d7c9a4a Fix getting the output value in code generation
new 0cf2c87 Fix beam coder lazy init using reflexion: use .clas + try
catch + cast
new 50060a8 Remove lazy init of beam coder because there is no generic
way on instanciating a beam coder
new ca01777 Remove example code
new f48067b Fix equal and hashcode
new 34e8aa8 Fix generated code: uniform exceptions catching, fix
parenthesis and variable declarations
new 78b2d22 Add an assert of equality in the encoders test
new c6f2ac9 Apply spotless and checkstyle and add javadocs
new 72c267c Wrap exceptions in UserCoderExceptions
new c8bfcf3 Put Encoders expressions serializable
new c33fdda Catch Exception instead of IOException because some coders to
not throw Exceptions at all (e.g.VoidCoder)
new 7d456b4 Apply new Encoders to CombinePerKey
new 3cc256e Apply new Encoders to Read source
new 68d3d67 Improve performance of source: the mapper already calls
windowedValueCoder.decode, no need to call it also in the Spark encoder
new c48d032 Ignore long time failing test: SparkMetricsSinkTest
new 7f1060a Apply new Encoders to Window assign translation
new 29f7e93 Apply new Encoders to AggregatorCombiner
new 21accab Create a Tuple2Coder to encode scala tuple2
new 039f58a Apply new Encoders to GroupByKey
new c5e78a0 Apply new Encoders to Pardo. Replace Tuple2Coder with
MultiOutputCoder to deal with multiple output to use in Spark Encoder for
DoFnRunner
new 62a87b6 Apply spotless, fix typo and javadoc
new 6a27839 Use beam encoders also in the output of the source translation
new 27ef6de Remove unneeded cast
new 824b344 Fix: Remove generic hack of using object. Use actual Coder
encodedType in Encoders
new 620a27a Remove Encoders based on kryo now that we call Beam coders in
the runner
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (46ed555)
\
N -- N -- N refs/heads/spark-runner_structured-streaming
(620a27a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 37 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
