siddharthteotia edited a comment on issue #4368: 3891: Check for validity of segment start/end time during segment generation URL: https://github.com/apache/incubator-pinot/pull/4368#issuecomment-506451732 @jackjlli , @mcvsubbu , I have been debugging the 31 unit test failures as reported here: https://travis-ci.org/apache/incubator-pinot/jobs/550998101 Some of them I have fixed like MultiplePinotSegmentRecordReaderTest. This test was generating its own test data and I tweaked it to account for generating time column values within acceptable range. Similarly, the unit test I added to test the newly added code in SegmentGenerationWithTimeColumnTest also piggy-backed on the random data generation in the test code. These tests first put the dummy data in a list of GenericRow and create a record reader with the schema and generic row set. Later SegmentIndexCreationDriver is initialized with the record reader for dummy data However, many others like IntArraysTest, ColumnMetadataTest, SegmentMetadataImplTest, SegmentPrepropcessorTest seem to be reading the test data from avro file and thus there is no control in the test code over the data. SegmentIndexCreationDriver is initialized just with the generation config that internally has details of path to avro test dataset. I am wondering how to fix these kinds of tests now that segment generation code does the time column validity check and the input data is in avro file? I am still going through rest of the test failures but wanted an opinion w.r.t fixing tests that use external dataset.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
