badalprasadsingh commented on code in PR #2591:
URL: https://github.com/apache/iceberg-go/pull/2591#discussion_r4592368296
##########
table/evaluators_test.go:
##########
@@ -1311,6 +1312,65 @@ func (p *ProjectionTestSuite)
TestMixedSpecUnknownTransformProjection() {
p.True(expr.Equals(iceberg.EqualTo(iceberg.Reference("id_bucket"),
int32(3))))
}
+func (p *ProjectionTestSuite) TestInvalidBuiltInMultiSourceFieldDoesNotPrune()
{
+ schema := p.schema()
+ filter := iceberg.EqualTo(iceberg.Reference("data"), "val")
+ singleSourceSpec := iceberg.NewPartitionSpecID(1,
iceberg.PartitionField{
+ SourceIDs: []int{2}, FieldID: 1000, Name: "data_bucket",
+ Transform: iceberg.BucketTransform{NumBuckets: 16},
+ })
+ singleProjection, err := newInclusiveProjection(schema,
singleSourceSpec, true)(filter)
+ p.Require().NoError(err)
+ singleEvaluator, err := newManifestEvaluator(singleSourceSpec, schema,
singleProjection, true)
+ p.Require().NoError(err)
+
+ // Find a bucket summary that the ordinary one-source projection
excludes.
+ // Reusing that summary with the malformed historical spec proves that
the
+ // malformed field no longer produces the unsafe unary projection.
+ var excludedManifest iceberg.ManifestFile
+ for bucket := int32(0); bucket < 16; bucket++ {
Review Comment:
use:
```go
for bucket := range int32(16)
```
##########
website/src/releases.md:
##########
@@ -45,6 +45,7 @@ Per-release notes (highlights, breaking changes,
contributors) are published on
- **Data-file replacements now reject concurrent deletes.** Commits staged
through `ReplaceDataFiles`, `ReplaceDataFilesByPath`, `ReplaceFiles`, or
rewrite operations can return `table.ErrConflictingDeleteFiles` when a
concurrent position delete targets a replaced file or any concurrent equality
delete is added. Compaction on tables with streaming equality deletes may see
additional conflicts.
- **Orphan cleanup now rejects `gc.enabled=false`.** `DeleteOrphanFiles`,
`PlanOrphanFiles`, `ExecuteOrphanCleanup`, and dry runs now return an error
before filesystem access. The `iceberg clean-orphan-files` command is also
affected.
- **Partition specs reject source types a transform cannot handle.** Creating
or updating a spec with an incompatible source, such as `day(string)`, now
fails. Writes to existing incompatible specs return `ErrInvalidTransform`
instead of writing null partition values.
+- **New partition specs reject multiple sources for built-in transforms.**
Create-table and add-spec payloads with this shape return
`ErrInvalidPartitionSpec`. Historical metadata with such a field still loads
and round-trips, but the field is skipped during partition pruning and residual
simplification. Multi-source unknown transforms remain supported.
Review Comment:
rest CreateTable now checks the whole spec before sending it, so any invalid
spec, such as day(string), returns "ErrInvalidPartitionSpec" instead of the
server's "rest.ErrBadRequest"
can the note say that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]