Hi all, We just released Scio 0.9.3. This bumps Beam SDK to 2.23.0 and includes a lot of improvements & bug fixes.
Cheers, Neville https://github.com/spotify/scio/releases/tag/v0.9.3 *"Petrificus Totalus"* There are no breaking changes in this release, but some were introduced with v0.9.0: See v0.9.0 Migration Guide <https://spotify.github.io/scio/migrations/v0.9.0-Migration-Guide.html> for detailed instructions. Improvements - Allow user-supplied filename prefix for smb writes/reads (#3215 <https://github.com/spotify/scio/pull/3215>) - Refactor SortedBucketTransform into a BoundedSource + reuse merge logic (#3097 <https://github.com/spotify/scio/pull/3097>) - Add keyGroupFilter optimization to scio-smb (#3160 <https://github.com/spotify/scio/pull/3160>) - Add error message to BaseAsyncLookupDoFn preconditions check (#3176 <https://github.com/spotify/scio/pull/3176>) - Add Elasticsearch 5,6,7 add/update alias on multiple indices ops (#3134 <https://github.com/spotify/scio/pull/3134>) - Add initial update alias op to ES7(#2920 <https://github.com/spotify/scio/pull/2920>) - Add ScioContext#applyTransform (#3146 <https://github.com/spotify/scio/pull/3146>) - Allow SCollection#transform name override (#3142 <https://github.com/spotify/scio/pull/3142>) - Allow setting default name through SCollection#applyTransform (#3144 <https://github.com/spotify/scio/pull/3144>) - Update 0.9 migration doc and add Bigquery Type read schema documentation(#3148 <https://github.com/spotify/scio/pull/3148>) Bug Fixes - AvroBucketMetadata should validate keyPath (fix #3038 <https://github.com/spotify/scio/issues/3038>) (#3140 <https://github.com/spotify/scio/pull/3140>) - Allow union types in non leaf field for key (#3187 <https://github.com/spotify/scio/pull/3187>) - Fix issue with union type as non-leaf field of smb key (#3193 <https://github.com/spotify/scio/pull/3193>) - Fix ContextAndArgs#typed overloading issue (#3199 <https://github.com/spotify/scio/pull/3199>) - Fix error propagation on Scala Future onSuccess callback (#3178 <https://github.com/spotify/scio/pull/3178>) - Fix ByteBuffer should be readOnly (#3220 <https://github.com/spotify/scio/pull/3220>) - Fix compiler warnings (#3183 <https://github.com/spotify/scio/pull/3183>) - Fix JdbcShardedReadOptions.fetchSize description (#3209 <https://github.com/spotify/scio/pull/3209>) - Fix FAQ typo (#3194 <https://github.com/spotify/scio/pull/3194>) - Fix scalafix error in SortMergeBucketScioContextSyntax (#3158 <https://github.com/spotify/scio/pull/3158>) - Add scalafix ExplicitReturnType and ProcedureSyntax rules (#3179 <https://github.com/spotify/scio/pull/3179>) - Cleanup a few more unused and unchecked params (#3223 <https://github.com/spotify/scio/pull/3223>) - Use GcpOptions#getWorkerZone instead of deprecated GcpOptions#getZone ( #3224 <https://github.com/spotify/scio/pull/3224>) - Use raw coder in SCollection#applyKvTransform (#3171 <https://github.com/spotify/scio/pull/3171>) - Add raw beam coder materializer (#3164 <https://github.com/spotify/scio/pull/3164>) - Avoid circular dep between SCollection and PCollectionWrapper (#3163 <https://github.com/spotify/scio/pull/3163>) - Remove unused param of internal partitionFn (#3166 <https://github.com/spotify/scio/pull/3166>) - Remove unused CoderRegistry (#3165 <https://github.com/spotify/scio/pull/3165>) - Remove defunct scio-bench (#3150 <https://github.com/spotify/scio/pull/3150>) - Reuse applyTransform (#3162 <https://github.com/spotify/scio/pull/3162> ) - Make multijoin.py python3 - Use TextIO#withCompression (#3145 <https://github.com/spotify/scio/pull/3145>) Dependency Updates - Update Beam SDK to 2.23.0 (#3197 <https://github.com/spotify/scio/pull/3197>) - Update dependencies to be inline with 2.23.0 (#3225 <https://github.com/spotify/scio/pull/3225>) - Update to scala 2.12.12 (#3157 <https://github.com/spotify/scio/pull/3157>) - Update auto-value to 1.7.4 (#3147 <https://github.com/spotify/scio/pull/3147>) - Update breeze to 1.1 (#3211 <https://github.com/spotify/scio/pull/3211> ) - Update cassandra-all to 3.11.7 (#3186 <https://github.com/spotify/scio/pull/3186>) - Update cassandra-driver-core to 3.10.0 (#3195 <https://github.com/spotify/scio/pull/3195>) - Update commons-lang3 to 3.11 (#3161 <https://github.com/spotify/scio/pull/3161>) - Update commons-text to 1.9 (#3185 <https://github.com/spotify/scio/pull/3185>) - Update contributing guidelines with current tools (#3149 <https://github.com/spotify/scio/pull/3149>) - Update elasticsearch-rest-client, ... to 7.8.1 (#3192 <https://github.com/spotify/scio/pull/3192>) - Update elasticsearch, ... to 6.8.11 (#3188 <https://github.com/spotify/scio/pull/3188>) - Update jackson-module-scala to 2.10.5 (#3169 <https://github.com/spotify/scio/pull/3169>) - Update jna to 5.6.0 (#3156 <https://github.com/spotify/scio/pull/3156>) - Update magnolify to 0.2.2 (#3154 <https://github.com/spotify/scio/pull/3154>) - Update mysql-connector-java to 8.0.21 (#3153 <https://github.com/spotify/scio/pull/3153>) - Update pprint to 0.6.0 (#3203 <https://github.com/spotify/scio/pull/3203>) - Update protobuf version to 3.11.4 (#3200 <https://github.com/spotify/scio/pull/3200>) - Update sbt-scalafix to 0.9.18 (#3138 <https://github.com/spotify/scio/pull/3138>) - Update sbt-sonatype to 3.9.4 (#3136 <https://github.com/spotify/scio/pull/3136>) - Update scalafmt-core to 2.6.2 (#3139 <https://github.com/spotify/scio/pull/3139>) - Update scalafmt-core to 2.6.3 (#3152 <https://github.com/spotify/scio/pull/3152>) - Update scalafmt-core to 2.6.4 (#3167 <https://github.com/spotify/scio/pull/3167>) - Update sparkey to 3.1.0 (#3204 <https://github.com/spotify/scio/pull/3204>) - Fix conflicting gcsio dependency (#3180 <https://github.com/spotify/scio/pull/3180>)
