Hehe yes sorry I should have waited a bit. Thanks for taking care of it Brian.
On Thu, Feb 6, 2020 at 10:12 PM Brian Hulette <bhule...@google.com> wrote: > Seems to be green now. Just had to wait for another run after the schema > update. > > On Thu, Feb 6, 2020 at 9:47 AM Tomo Suzuki <suzt...@google.com> wrote: > >> Ismaël, >> >> The latest postcommit failure >> https://builds.apache.org/job/beam_PostCommit_SQL/3951/ was 4:15:45 PM >> Brian's successful case >> https://builds.apache.org/job/beam_PostCommit_SQL_PR/243/ started >> 4:29:57 PM. >> I hope the next SQL postcommit should succeed. >> >> On Thu, Feb 6, 2020 at 11:57 AM Ismaël Mejía <ieme...@gmail.com> wrote: >> > >> > This one is still broken, maybe there are two different data sources, >> one for the '_PR' version and the normal one, can you please confirm? >> > https://builds.apache.org/job/beam_PostCommit_SQL/ >> > >> > >> > On Thu, Feb 6, 2020 at 5:44 PM Brian Hulette <bhule...@google.com> >> wrote: >> >> >> >> Sorry for the delay. I had some issues updating the schema, I ended up >> having to drop it and re-create for some reason. Looks like SQL PostCommit >> is green on https://github.com/apache/beam/pull/10765 now. >> >> >> >> > setting up from scratch is a good idea. >> >> +1, I filed https://issues.apache.org/jira/browse/BEAM-9260 for this >> and assigned it to Kenn for now. >> >> >> >> > I’m still feeling it’s ok to read an Integer as Long. >> >> In this case the issue is that we were reading data with a schema (id: >> INT64), and then passing it to a PAssert that checked against Rows with a >> different schema (id: INT32). So I think it's a legitimate error because we >> were asserting that a PCollection contained rows with one schema but it >> actually had a different schema. The message should definitely be better in >> this case though. >> >> >> >> Brian >> >> >> >> On Wed, Feb 5, 2020 at 9:28 PM Tomo Suzuki <suzt...@google.com> wrote: >> >>> >> >>> (My understanding) >> >>> The test ensures the CSV data stored in GCS should be readable >> through Datacatalog. It fails because an Integer value in the CSV was read >> as Long as per Datacatalog. >> >>> >> >>> >> >>> > setting up from scratch is a good idea. >> >>> >> >>> I agree. Furthermore, it would be nice if it can test different >> type-cast behaviors. I’m still feeling it’s ok to read an Integer as Long. >> (If this is the case, how about Long to Integer? What if the long is small >> enough to fit in 32 bits? and so on) >> >>> >> >>> On Wed, Feb 5, 2020 at 23:15 Kenneth Knowles <k...@apache.org> wrote: >> >>>> >> >>>> I think that was me... sorry! >> >>>> >> >>>> Is this a test where it is important that the data is pre-existing? >> Otherwise I would say that setting up from scratch is a good idea. Does >> anyone have context on it? I am happy to take on the small bit of coding, >> since I broke it. >> >>>> >> >>>> Kenn >> >>>> >> >>>> On Wed, Feb 5, 2020 at 1:22 PM Brian Hulette <bhule...@google.com> >> wrote: >> >>>>> >> >>>>> So it looks like the schema for `integ_test_small_csv_test_1` was >> updated yesterday around the same time that PR#10563 went in, and it no >> longer matches the schema we expect in the test. >> >>>>> >> >>>>> I'm just going to change it back for now. I am curious who changed >> it and why, if the perpetrator is on this list please let us know :) >> >>>>> >> >>>>> >> >>>>> Note the updateTime: >> >>>>> ``` >> >>>>> ❯ gcloud beta data-catalog entries lookup >> '`datacatalog`.`entry`.`apache-beam-testing`.`us-central1`.`samples`.`integ_test_small_csv_test_1`'`` >> >>>>> gcsFilesetSpec: >> >>>>> filePatterns: >> >>>>> - gs://apache-beam-samples/integration_test_small_csv/test.csv >> >>>>> linkedResource: // >> datacatalog.googleapis.com/projects/apache-beam-testing/locations/us-central1/entryGroups/samples/entries/integ_test_small_csv_test_1 >> >>>>> name: >> projects/apache-beam-testing/locations/us-central1/entryGroups/samples/entries/integ_test_small_csv_test_1 >> >>>>> schema: >> >>>>> columns: >> >>>>> - column: id >> >>>>> mode: NULLABLE >> >>>>> type: INT64 >> >>>>> - column: name >> >>>>> mode: NULLABLE >> >>>>> type: STRING >> >>>>> - column: type >> >>>>> mode: NULLABLE >> >>>>> type: STRING >> >>>>> sourceSystemTimestamps: >> >>>>> createTime: '2019-08-16T01:49:06.235Z' >> >>>>> updateTime: '2020-02-04T17:18:17.671Z' >> >>>>> type: FILESET >> >>>>> ``` >> >>> >> >>> -- >> >>> Regards, >> >>> Tomo >> >> >> >> -- >> Regards, >> Tomo >> >
