Hi all, We just released Scio 0.8.0. This is based on the most recent Beam 2.17.0 release and includes a lot of new features & bug fixes over the past 10 months.
Cheers, Neville https://github.com/spotify/scio/releases/tag/v0.8.0 *"Amato Animo Animato Animagus"* Breaking changes & deprecations - See v0.8.0 Migration Guide <https://spotify.github.io/scio/migrations/v0.8.0-Migration-Guide.html> for detailed instructions - Remove @experimental from transform #2537 <https://github.com/spotify/scio/pull/2537> - Deprecate scio-elasticsearch2 and scio-cassandra2 #2414 <https://github.com/spotify/scio/pull/2414> #2421 <https://github.com/spotify/scio/pull/2421> - Deprecate hashFilter #2442 <https://github.com/spotify/scio/pull/2442> - Deprecate legacy components in scio-extras #2533 <https://github.com/spotify/scio/pull/2533> Features - Bump Beam to 2.17.0 #2577 <https://github.com/spotify/scio/pull/2577> - Add sharded Sparkey support. #2336 <https://github.com/spotify/scio/pull/2336> - Rework side input cache #2363 <https://github.com/spotify/scio/pull/2363> - Cleanup Side Inputs API, introduce Singleton Set SideInputs #2424 <https://github.com/spotify/scio/pull/2424> - Add schema support for GenericRecord #2514 <https://github.com/spotify/scio/pull/2514> Bug fixes & improvements - Add file:linenum only to outer transform #2405 <https://github.com/spotify/scio/pull/2405> - Fix join transform names #2444 <https://github.com/spotify/scio/pull/2444> - Remove Coder context bound for partitionByKey #2451 <https://github.com/spotify/scio/pull/2451> - Rename that method argument in join functions to rhs #2466 <https://github.com/spotify/scio/pull/2466> - Replace custom ClosureCleaner with chill's #2423Use chill's Externalizer to serialize predicates in SCollectionMatchers #2410 <https://github.com/spotify/scio/pull/2410> - Add errmsg when beamOpts == null in JobTest, fix #2430 <https://github.com/spotify/scio/issues/2430> #2545 <https://github.com/spotify/scio/pull/2545> - Add bigQuerySelect() method with default flattenResults value #2500 <https://github.com/spotify/scio/pull/2500> - Better consistency around BigQuery API #2412Fail early on malformed BigQuery spec #2345 <https://github.com/spotify/scio/pull/2345> - Rewrite typedBigQueryStorage #2434 <https://github.com/spotify/scio/pull/2434> - Add DML query support to bigquery client #2418 <https://github.com/spotify/scio/pull/2418> - Treat Avro array as java List in BigQuery read, fix #2068 <https://github.com/spotify/scio/issues/2068> #2415 <https://github.com/spotify/scio/pull/2415> - Fix NPE in scio-bigtable's ChannelPoolCreator when credentials aren't set #2317Fix bigtable scollection ops return type #2486 <https://github.com/spotify/scio/pull/2486> - Refactor PubsubIO for more typesafety #2457 <https://github.com/spotify/scio/pull/2457> - Avoid Mutation coder fallback for Spanner #2478 <https://github.com/spotify/scio/pull/2478> - Fix Parquet sink suffix #2367Improve iterable equality #2483 <https://github.com/spotify/scio/pull/2483> - Improve back compat with Scio 0.7 #2401 <https://github.com/spotify/scio/pull/2401> - Improve coder gen by checking companion implicits #2522 <https://github.com/spotify/scio/pull/2522> - Make recursive coders serializable #2404 <https://github.com/spotify/scio/pull/2404> - Remove kryo coder override in intermediate steps #2422Fix fallback warning when implicit is in scope #2511 <https://github.com/spotify/scio/pull/2511> - Improve the schema compatibility error message #2366 <https://github.com/spotify/scio/pull/2366> - Remove schema fallback #2489 <https://github.com/spotify/scio/pull/2489> - Add Schemas support for more types #2364 <https://github.com/spotify/scio/pull/2364> - Assert FileStorage.isDone in MaterializeTap #2518 <https://github.com/spotify/scio/pull/2518> - Add support for cleaning up TF models on shutdown #2549 <https://github.com/spotify/scio/pull/2549> - Rework TensorFlow predict ops #2343 <https://github.com/spotify/scio/pull/2343> - Remove unused/deprecated TensorFlow graph DoFn #2339 <https://github.com/spotify/scio/pull/2339> - Mark some APIs in scio-extras as experimental #2517 <https://github.com/spotify/scio/issues/2517> #2572 <https://github.com/spotify/scio/pull/2572>
