Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125678 --- Ship it! - Bill Farner On March 28, 2016, 8:33 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 28, 2016, 8:33 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125657 --- Ship it! Master (83a078b) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On March 28, 2016, 3:33 p.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 28, 2016, 3:33 p.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125644 --- Bill - I'll take silence as consent and merge this today. - John Sirois On March 28, 2016, 9:33 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 28, 2016, 9:33 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/ --- (Updated March 28, 2016, 9:33 a.m.) Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. Changes --- Back task queries with `ITaskQuery`. This avoids special null checks for collection query criteria although it does require more care in struct handling code where `TaskQuery` <-> `ITaskQuery` are lossy for `isSet` query methods. src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 8 +++- src/main/java/org/apache/aurora/scheduler/base/Query.java | 19 +-- src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 15 +++ src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java | 2 +- src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java | 4 ++-- src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 14 +++--- src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java | 12 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java | 2 +- src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 10 +- src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java | 18 ++ src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java | 6 +++--- src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java | 20 ++-- 12 files changed, 66 insertions(+), 64 deletions(-) Repository: aurora Description --- Previously, `null` was handled differently from an empty collection in task queries. For the Go thrift bindings, this was problematic since zero values in Go are useful in almost all cases and in particular in the case of maps (used to represent sets). In these cases unset `TaskQuery` collection parameters are serialized as empty collections (empty maps) instead of `nil` (`null`), leading to the inability to use the query API in any natural way. src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +- src/main/java/org/apache/aurora/scheduler/base/Query.java| 2 +- src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +- src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 -- src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++--- src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +--- 6 files changed, 27 insertions(+), 11 deletions(-) Diffs (updated) - src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021 src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106 src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88 src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb Diff: https://reviews.apache.org/r/45193/diff/ Testing --- NB: This change was broken out of https://reviews.apache.org/r/42756/ since it stands on its own (although its slightly more awkward in the mutable thrift world) and the case of the Go Aurora API client forces the issue. Locally green: ``` ./build-support/jenkins/build.sh ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh ``` Thanks, John Sirois
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125643 --- Rebasing to pick up https://reviews.apache.org/r/45366/ - John Sirois On March 24, 2016, 8:52 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 24, 2016, 8:52 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125271 --- Bill - this has changed enough that you should re-assess your ship. - John Sirois On March 24, 2016, 8:52 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 24, 2016, 8:52 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125270 --- Ship it! Master (9927231) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On March 24, 2016, 2:52 p.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 24, 2016, 2:52 p.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125265 --- @ReviewBot retry - John Sirois On March 24, 2016, 8:52 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 24, 2016, 8:52 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125262 --- Ship it! The ripples into the scheduler code was a little bit more than I expected, but I think this is the better solution. I think whenever we get around to fixing AURORA-1650 and AURORA-1651 the code should end up a lot cleaner. - Zameer Manji On March 24, 2016, 7:52 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 24, 2016, 7:52 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java > 4bf40047e105389ac7139edc449857889d390106 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java > d326d24dd527d084bce1b300f1818d3b1d94c036 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > 5d246bee1a4dabc563a23c542384205537719f6a > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java > dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java > 3ba03429748448642571cfe0858278a50148745a > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125261 --- Master (9927231) is red with this patch. ./build-support/jenkins/build.sh [1m self._clock.converge(threads=[hct.threaded_health_checker])[0m [1m self._clock.assert_waiting(hct.threaded_health_checker, amount=1)[0m [1m[0m [1m assert hct._total_latency == 0[0m [1m assert hct.metrics.sample()['total_latency_secs'] == 0[0m [1m[0m [1m # start the health check (during health check it is still 0)[0m [1m epsilon = 0.001[0m [1m self._clock.tick(1.0 + epsilon)[0m [1m self._clock.converge(threads=[hct.threaded_health_checker])[0m [1m self._clock.assert_waiting(hct.threaded_health_checker, amount=0.5)[0m [1m assert hct._total_latency == 0[0m [1m assert hct.metrics.sample()['total_latency_secs'] == 0[0m [1m assert hct.metrics.sample()['checks'] == 0[0m [1m[0m [1m # finish the health check[0m [1m self._clock.tick(0.5 + epsilon)[0m [1m self._clock.converge(threads=[hct.threaded_health_checker])[0m [1m self._clock.assert_waiting(hct.threaded_health_checker, amount=1) # interval_secs[0m [1m> assert hct._total_latency == 0.5[0m [1m[31mE AssertionError: assert 0.5009 == 0.5[0m [1m[31mE + where 0.5009 = ._total_latency[0m src/test/python/apache/aurora/executor/common/test_health_checker.py:174: AssertionError -- Captured stderr call -- [] Time now: 0.0 [] Time now: 0.0 [] Time now: 1.0 [] Time now: 1.001 [] Time now: 1.001 [] Time now: 1.501 [] Time now: 1.502 generated xml file: /home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/415337499eb72578eab327a6487c1f5c9452b3d6.xml [1m[31m 1 failed, 663 passed, 5 skipped, 1 warnings in 313.43 seconds [0m FAILURE 15:26:21 07:04 [complete][31m FAILURE[0m I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On March 24, 2016, 2:52 p.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 24, 2016, 2:52 p.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java > 078dd8b63fdca192c735f9097edd030ee315a021 > src/main/java/org/apache/aur
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
> On March 23, 2016, 1:36 p.m., Zameer Manji wrote: > > I am in favor of making this change. However, it seems this patch could be > > improved because the storage layer has to now check for both `null` and > > empty collection. > > > > I think a better solution would be to change Query.Builder to be a Supplier > > of > > `ITaskQuery` instead of `TaskQuery`. The `I*` classes always return empty > > collections and never null. By doing this the storage layer can be > > simplified to > > only check for empty collections. This is also more future proof because the > > storage/query layer can never be given `null` which means the next time we > > add a > > new field to `TaskQuery` (such as image id?) we don't have to risk > > regressing on > > this behaviour. > > > > I have scanned the code and it doesn't seem to be a lot of work, it seems > > mostly > > places where there are entries like `query.getStatusesSize()` would need to > > be > > replaced with `query.getStatus().size()`. > > > > I don't feel super strong about my suggestion here so if you do object or > > if it > > does seem like a lot of work, I will just ship this change as is. > > John Sirois wrote: > I'll give it a spike and see how it works out. I will note though that > the storage layer (db) - was already doing this for 2/5 collection fields in > the TaskMapper.select. IOW: the scope here is expanding from making things > consistent to also paying down more fundamental existing debt. I'm all for > that, but I also want to call out I tried paying down that debt and then some > by eliminating I* alltogether and using a single consistent immutable model > where no collections were null, but instead empty. So this is make it nice - > but in a middling way. The `TaskMapper.xml` does of course get cleaner, but the scheduler code gets trickier due to some I* isSet issues (https://issues.apache.org/jira/browse/AURORA-1650 https://issues.apache.org/jira/browse/AURORA-1651). See what you think. - John --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125102 --- On March 23, 2016, 10:35 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 23, 2016, 10:35 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Siro
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/ --- (Updated March 24, 2016, 8:52 a.m.) Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. Changes --- Back task queries with `ITaskQuery`. This avoids special null checks for collection query criteria although it does require more care in struct handling code where `TaskQuery` <-> `ITaskQuery` are lossy for `isSet` query methods. src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 8 +++- src/main/java/org/apache/aurora/scheduler/base/Query.java | 19 +-- src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 15 +++ src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java | 2 +- src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java | 4 ++-- src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 14 +++--- src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java | 12 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java | 2 +- src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 10 +- src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java | 18 ++ src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java | 6 +++--- src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java | 20 ++-- 12 files changed, 66 insertions(+), 64 deletions(-) Repository: aurora Description --- Previously, `null` was handled differently from an empty collection in task queries. For the Go thrift bindings, this was problematic since zero values in Go are useful in almost all cases and in particular in the case of maps (used to represent sets). In these cases unset `TaskQuery` collection parameters are serialized as empty collections (empty maps) instead of `nil` (`null`), leading to the inability to use the query API in any natural way. src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +- src/main/java/org/apache/aurora/scheduler/base/Query.java| 2 +- src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +- src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 -- src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++--- src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +--- 6 files changed, 27 insertions(+), 11 deletions(-) Diffs (updated) - src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021 src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106 src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88 src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb Diff: https://reviews.apache.org/r/45193/diff/ Testing --- NB: This change was broken out of https://reviews.apache.org/r/42756/ since it stands on its own (although its slightly more awkward in the mutable thrift world) and the case of the Go Aurora API client forces the issue. Locally green: ``` ./build-support/jenkins/build.sh ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh ``` Thanks, John Sirois
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
> On March 23, 2016, 1:36 p.m., Zameer Manji wrote: > > I am in favor of making this change. However, it seems this patch could be > > improved because the storage layer has to now check for both `null` and > > empty collection. > > > > I think a better solution would be to change Query.Builder to be a Supplier > > of > > `ITaskQuery` instead of `TaskQuery`. The `I*` classes always return empty > > collections and never null. By doing this the storage layer can be > > simplified to > > only check for empty collections. This is also more future proof because the > > storage/query layer can never be given `null` which means the next time we > > add a > > new field to `TaskQuery` (such as image id?) we don't have to risk > > regressing on > > this behaviour. > > > > I have scanned the code and it doesn't seem to be a lot of work, it seems > > mostly > > places where there are entries like `query.getStatusesSize()` would need to > > be > > replaced with `query.getStatus().size()`. > > > > I don't feel super strong about my suggestion here so if you do object or > > if it > > does seem like a lot of work, I will just ship this change as is. I'll give it a spike and see how it works out. I will note though that the storage layer (db) - was already doing this for 2/5 collection fields in the TaskMapper.select. IOW: the scope here is expanding from making things consistent to also paying down more fundamental existing debt. I'm all for that, but I also want to call out I tried paying down that debt and then some by eliminating I* alltogether and using a single consistent immutable model where no collections were null, but instead empty. So this is make it nice - but in a middling way. - John --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125102 --- On March 23, 2016, 10:35 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 23, 2016, 10:35 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review125102 --- I am in favor of making this change. However, it seems this patch could be improved because the storage layer has to now check for both `null` and empty collection. I think a better solution would be to change Query.Builder to be a Supplier of `ITaskQuery` instead of `TaskQuery`. The `I*` classes always return empty collections and never null. By doing this the storage layer can be simplified to only check for empty collections. This is also more future proof because the storage/query layer can never be given `null` which means the next time we add a new field to `TaskQuery` (such as image id?) we don't have to risk regressing on this behaviour. I have scanned the code and it doesn't seem to be a lot of work, it seems mostly places where there are entries like `query.getStatusesSize()` would need to be replaced with `query.getStatus().size()`. I don't feel super strong about my suggestion here so if you do object or if it does seem like a lot of work, I will just ship this change as is. - Zameer Manji On March 23, 2016, 9:35 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 23, 2016, 9:35 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of maps (used to represent sets). In these cases unset `TaskQuery` > collection parameters are serialized as empty collections (empty > maps) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
> On March 23, 2016, 8:35 a.m., John Sirois wrote:
> > I think this change stands on its own aside from the current state of the
> > generated Go thrift bindings, but there has been a good deal of discussion
> > about those bindings offline. Some homework below.
> >
> > For the case of the `TaskQuery` thrift struct, thrift 0.9.3 generates the
> > following Go struct:
> > ```go
> > type TaskQuery struct {
> > // unused field # 1
> > JobName string `thrift:"jobName,2" json:"jobName"`
> > // unused field # 3
> > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> > Statuses map[ScheduleStatus]bool `thrift:"statuses,5" json:"statuses"`
> > // unused field # 6
> > InstanceIds map[int32]bool `thrift:"instanceIds,7" json:"instanceIds"`
> > // unused field # 8
> > Environment string `thrift:"environment,9" json:"environment"`
> > SlaveHosts map[string]bool `thrift:"slaveHosts,10" json:"slaveHosts"`
> > JobKeys map[*JobKey]bool `thrift:"jobKeys,11" json:"jobKeys"`
> > Offset int32`thrift:"offset,12" json:"offset"`
> > Limit int32`thrift:"limit,13" json:"limit"`
> > Rolestring `thrift:"role,14" json:"role"`
> > }
> > ```
> >
> > This is reasonable since `api.TaskQuery{}.TaskIds == nil` is true; ie the
> > collections (maps represent sets here) zero to nil.
> > The issue comes in the serialization for these fields, `taskIds` is shown
> > below as an example:
> > ```go
> > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> > return thrift.PrependError(fmt.Sprintf("%T write field begin
> > error 4:taskIds: ", p), err)
> > }
> > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
> > nil {
> > return thrift.PrependError("error writing set begin: ", err)
> > }
> > for v, _ := range p.TaskIds {
> > if err := oprot.WriteString(string(v)); err != nil {
> > return thrift.PrependError(fmt.Sprintf("%T. (0) field
> > write error: ", p), err)
> > }
> > }
> > if err := oprot.WriteSetEnd(); err != nil {
> > return thrift.PrependError("error writing set end: ", err)
> > }
> > if err := oprot.WriteFieldEnd(); err != nil {
> > return thrift.PrependError(fmt.Sprintf("%T write field end
> > error 4:taskIds: ", p), err)
> > }
> > return err
> > }
> > ```
> >
> > So, since its safe to do so in Go (`len(p.TaskIds) == 0` and `for v, _ :=
> > range p.TaskIds {` loops 0 times for `p.TaskIds == nil`), the code always
> > emits the `taskIds` field, whether nil or not, which presents on the other
> > end of the wire as an empty set (as opposed to a null or un-set set). This
> > does seem like a clear bug in the thrift compiler.
> > https://issues.apache.org/jira/browse/THRIFT-3700 is similar, but on the
> > deserialization side of things so I've filed
> > https://issues.apache.org/jira/browse/THRIFT-3752.
Hrm, so this may not be a thrift compiler bug per-se. Marking all TaskQuery
thrift fields as optional yields:
```go
type TaskQuery struct {
// unused field # 1
JobName *string `thrift:"jobName,2" json:"jobName,omitempty"`
// unused field # 3
TaskIds map[string]bool `thrift:"taskIds,4"
json:"taskIds,omitempty"`
Statuses map[ScheduleStatus]bool `thrift:"statuses,5"
json:"statuses,omitempty"`
// unused field # 6
InstanceIds map[int32]bool `thrift:"instanceIds,7"
json:"instanceIds,omitempty"`
// unused field # 8
Environment *string `thrift:"environment,9"
json:"environment,omitempty"`
SlaveHosts map[string]bool `thrift:"slaveHosts,10"
json:"slaveHosts,omitempty"`
JobKeys map[*JobKey]bool `thrift:"jobKeys,11"
json:"jobKeys,omitempty"`
Offset *int32 `thrift:"offset,12"
json:"offset,omitempty"`
Limit *int32 `thrift:"limit,13" json:"limit,omitempty"`
Role*string `thrift:"role,14" json:"role,omitempty"`
}
```
So map (set) fields are unchanged (still `nil`able), but primitives - not
`nil`able before, are now represented
as `nil`able pointers. This also has the effect of emitting `IsSet*` methods
and respecting these methods as a
gate for field serialization:
```go
func (p *TaskQuery) IsSetTaskIds() bool {
return p.TaskIds != nil
}
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if p.IsSetTaskIds() {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err
!= nil {
return thrift.PrependError(fmt.Sprintf("%T write field
begin error 4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds));
err != nil {
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/ --- (Updated March 23, 2016, 10:35 a.m.) Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. Changes --- Update the description to reflect homework. Repository: aurora Description (updated) --- Previously, `null` was handled differently from an empty collection in task queries. For the Go thrift bindings, this was problematic since zero values in Go are useful in almost all cases and in particular in the case of maps (used to represent sets). In these cases unset `TaskQuery` collection parameters are serialized as empty collections (empty maps) instead of `nil` (`null`), leading to the inability to use the query API in any natural way. src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +- src/main/java/org/apache/aurora/scheduler/base/Query.java| 2 +- src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +- src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 -- src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++--- src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +--- 6 files changed, 27 insertions(+), 11 deletions(-) Diffs - src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9 src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88 src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 Diff: https://reviews.apache.org/r/45193/diff/ Testing --- NB: This change was broken out of https://reviews.apache.org/r/42756/ since it stands on its own (although its slightly more awkward in the mutable thrift world) and the case of the Go Aurora API client forces the issue. Locally green: ``` ./build-support/jenkins/build.sh ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh ``` Thanks, John Sirois
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125024
---
I think this change stands on its own aside from the current state of the
generated Go thrift bindings, but there has been a good deal of discussion
about those bindings offline. Some homework below.
For the case of the `TaskQuery` thrift struct, thrift 0.9.3 generates the
following Go struct:
```go
type TaskQuery struct {
// unused field # 1
JobName string `thrift:"jobName,2" json:"jobName"`
// unused field # 3
TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
Statuses map[ScheduleStatus]bool `thrift:"statuses,5" json:"statuses"`
// unused field # 6
InstanceIds map[int32]bool `thrift:"instanceIds,7" json:"instanceIds"`
// unused field # 8
Environment string `thrift:"environment,9" json:"environment"`
SlaveHosts map[string]bool `thrift:"slaveHosts,10" json:"slaveHosts"`
JobKeys map[*JobKey]bool `thrift:"jobKeys,11" json:"jobKeys"`
Offset int32`thrift:"offset,12" json:"offset"`
Limit int32`thrift:"limit,13" json:"limit"`
Rolestring `thrift:"role,14" json:"role"`
}
```
This is reasonable since `api.TaskQuery{}.TaskIds == nil` is true; ie the
collections (maps represent sets here) zero to nil.
The issue comes in the serialization for these fields, `taskIds` is shown below
as an example:
```go
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin
error 4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field
write error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end
error 4:taskIds: ", p), err)
}
return err
}
```
So, since its safe to do so in Go (`len(p.TaskIds) == 0` and `for v, _ := range
p.TaskIds {` loops 0 times for `p.TaskIds == nil`), the code always emits the
`taskIds` field, whether nil or not, which presents on the other end of the
wire as an empty set (as opposed to a null or un-set set). This does seem like
a clear bug in the thrift compiler.
https://issues.apache.org/jira/browse/THRIFT-3700 is similar, but on the
deserialization side of things so I've filed
https://issues.apache.org/jira/browse/THRIFT-3752.
- John Sirois
On March 22, 2016, 8:32 p.m., John Sirois wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> ---
>
> (Updated March 22, 2016, 8:32 p.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> ---
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of slices (collections). In these cases unset `TaskQuery`
> collection parameters are represented as empty collections (empty
> slices[]) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java
> | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java
> | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java
> | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java
> | 6 --
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml
> | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java
> | 20 +---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java
> 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java
> ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/m
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review124954 --- Ship it! Ship It! - Bill Farner On March 22, 2016, 7:32 p.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 22, 2016, 7:32 p.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of slices (collections). In these cases unset `TaskQuery` > collection parameters are represented as empty collections (empty > slices[]) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/#review124951 --- Ship it! Master (c66a9ee) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On March 23, 2016, 2:32 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45193/ > --- > > (Updated March 23, 2016, 2:32 a.m.) > > > Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. > > > Repository: aurora > > > Description > --- > > Previously, `null` was handled differently from an empty collection in > task queries. For the Go thrift bindings, this was problematic since > zero values in Go are useful in almost all cases and in particular in the > case of slices (collections). In these cases unset `TaskQuery` > collection parameters are represented as empty collections (empty > slices[]) instead of `nil` (`null`), leading to the inability to use the > query API in any natural way. > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/base/Query.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > | 2 +- > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > | 6 -- > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > | 6 +++--- > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > | 20 +--- > 6 files changed, 27 insertions(+), 11 deletions(-) > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/base/JobKeys.java > 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 > src/main/java/org/apache/aurora/scheduler/base/Query.java > ee01eaa4d0230d6bf0909b6460f27a74f03240db > src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java > ac0bb374842741d7ccb7a83c574a90ac156af0f9 > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > 231a55615abfbb483667f5f8ef71d2709fc16a88 > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml > 684614ffc42dd6778c7675a6c2f81cb72c106c0e > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 > > Diff: https://reviews.apache.org/r/45193/diff/ > > > Testing > --- > > NB: This change was broken out of https://reviews.apache.org/r/42756/ > since it stands on its own (although its slightly more awkward in the > mutable thrift world) and the case of the Go Aurora API client forces the > issue. > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 45193: Treat empty and null collections equivalently in task queries.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45193/ --- (Updated March 22, 2016, 8:32 p.m.) Review request for Aurora, David Chung, Bill Farner, and Zameer Manji. Repository: aurora Description (updated) --- Previously, `null` was handled differently from an empty collection in task queries. For the Go thrift bindings, this was problematic since zero values in Go are useful in almost all cases and in particular in the case of slices (collections). In these cases unset `TaskQuery` collection parameters are represented as empty collections (empty slices[]) instead of `nil` (`null`), leading to the inability to use the query API in any natural way. src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +- src/main/java/org/apache/aurora/scheduler/base/Query.java| 2 +- src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +- src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 -- src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++--- src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +--- 6 files changed, 27 insertions(+), 11 deletions(-) Diffs - src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9 src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88 src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 Diff: https://reviews.apache.org/r/45193/diff/ Testing --- NB: This change was broken out of https://reviews.apache.org/r/42756/ since it stands on its own (although its slightly more awkward in the mutable thrift world) and the case of the Go Aurora API client forces the issue. Locally green: ``` ./build-support/jenkins/build.sh ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh ``` Thanks, John Sirois
