@toelen Totally agree, we have IT for most of the Google IOs, for AWS we have only Kinesis but it is not part of the Beam CI yet, to put this in place we will need some company donating a bit to pay the AWS bills (we considered a docker with localstack at some moment but well it be still not a 'real' test).
In the meantime we need to prepare code for this (there is ongoing work on new AWS connectors e.g. SNS, SQS, Kinesis so this will matter). All contributions in this front are welcome :) I mentioned @lgajowy in the PR because he lead the work on file-based tests (gs://, hdfs://) so if you are interested in contributing in this area maybe it is worth to take a look at what he did, in principle it should be straightforward to add s3. [ Full content available at: https://github.com/apache/beam/pull/6232 ] This message was relayed via gitbox.apache.org for [email protected]
