-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125024
-----------------------------------------------------------
I think this change stands on its own aside from the current state of the
generated Go thrift bindings, but there has been a good deal of discussion
about those bindings offline. Some homework below.
For the case of the `TaskQuery` thrift struct, thrift 0.9.3 generates the
following Go struct:
```go
type TaskQuery struct {
// unused field # 1
JobName string `thrift:"jobName,2" json:"jobName"`
// unused field # 3
TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
Statuses map[ScheduleStatus]bool `thrift:"statuses,5" json:"statuses"`
// unused field # 6
InstanceIds map[int32]bool `thrift:"instanceIds,7" json:"instanceIds"`
// unused field # 8
Environment string `thrift:"environment,9" json:"environment"`
SlaveHosts map[string]bool `thrift:"slaveHosts,10" json:"slaveHosts"`
JobKeys map[*JobKey]bool `thrift:"jobKeys,11" json:"jobKeys"`
Offset int32 `thrift:"offset,12" json:"offset"`
Limit int32 `thrift:"limit,13" json:"limit"`
Role string `thrift:"role,14" json:"role"`
}
```
This is reasonable since `api.TaskQuery{}.TaskIds == nil` is true; ie the
collections (maps represent sets here) zero to nil.
The issue comes in the serialization for these fields, `taskIds` is shown below
as an example:
```go
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin
error 4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err !=
nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field
write error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end
error 4:taskIds: ", p), err)
}
return err
}
```
So, since its safe to do so in Go (`len(p.TaskIds) == 0` and `for v, _ := range
p.TaskIds {` loops 0 times for `p.TaskIds == nil`), the code always emits the
`taskIds` field, whether nil or not, which presents on the other end of the
wire as an empty set (as opposed to a null or un-set set). This does seem like
a clear bug in the thrift compiler.
https://issues.apache.org/jira/browse/THRIFT-3700 is similar, but on the
deserialization side of things so I've filed
https://issues.apache.org/jira/browse/THRIFT-3752.
- John Sirois
On March 22, 2016, 8:32 p.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 22, 2016, 8:32 p.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of slices (collections). In these cases unset `TaskQuery`
> collection parameters are represented as empty collections (empty
> slices[]) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java
> | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java
> | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java
> | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java
> | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml
> | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java
> | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java
> 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java
> ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java
> ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java
> 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml
> 684614ffc42dd6778c7675a6c2f81cb72c106c0e
>
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java
> e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
