Re: Some Thoughts on IO Integration Tests

2017-01-12 Thread Stephen Sisk
Thanks, that helps. I've been thinking of this mostly for connection options which I'm hoping should be pretty generic. Do we have known examples that we will want to use this with today, or is this a hypothetical? If it's hypothetical for now, would it make sense to defer making changes to

Re: Some Thoughts on IO Integration Tests

2017-01-12 Thread Lukasz Cwik
Imagine you had a KafkaReadIT and a KafkaWriteIT. Both ITs share the same Kafka configuration options (host, topic, ...) found in KafkaITPipelineOptions yet the KafkaReadIT also needs to have an additional parameter (like position to start reading kafka topic from) found in

Re: Some Thoughts on IO Integration Tests

2017-01-12 Thread Stephen Sisk
I see the need for/like KinesisIOTestPipelineOptions - it would allow all ITs/perf tests that need a kinesis connection to get one. What problem are we trying to solve with KinesisIOTest1PipelineOptions ? This presumes that there would also be KinesisIOTest2PipelineOptions ? What would be

Some Thoughts on IO Integration Tests

2017-01-10 Thread Jason Kuster
Hi all, Following up on some of the discussions already on-list, I wanted to solicit some more feedback about some implementation details regarding the IO Integration Tests. As it currently stands, we mostly have IO ITs for GCP-based IO, which our GCP-based Jenkins executors handle natively, but