github  

Messages by Thread

• Re: [PR] feat(dataframe): add column to a dataframe (#25298) [datafusion] via GitHub
• [PR] feat(executor): report executor memory usage in heartbeats [datafusion-ballista] via GitHub
• Re: [I] Separate JNI-free core logic and error classification from native JNI entry points [datafusion-comet] via GitHub
• [PR] test: migrate expect-test snapshots to insta [datafusion-iceberg] via GitHub
• Re: [PR] chore: add release infrastructure [datafusion-iceberg] via GitHub
  • Re: [PR] chore: add release infrastructure [datafusion-iceberg] via GitHub
• [PR] [branch-1.1] fix: restore Spark write execs when reverting transition-heavy stages [datafusion-comet] via GitHub
• [PR] fix: [branch-1.1] count the days partition transform in UTC (#6348) [datafusion-comet] via GitHub
• [PR] refactor: build IcebergDataSource with a builder [datafusion-iceberg] via GitHub
  • Re: [PR] refactor: build IcebergDataSource with a builder [datafusion-iceberg] via GitHub
  • Re: [PR] refactor: replace IcebergDataSource::new_with_predicate with a builder [datafusion-iceberg] via GitHub
• [PR] [branch-1.1] fix: normalize float operands in the native comparison builder [datafusion-comet] via GitHub
• Re: [I] `https://datafusion.apache.org/blog/` routes to a directory listing [datafusion-site] via GitHub
• [PR] feat(benchmarks): add plan diagnostics to SQL benchmark runners [datafusion] via GitHub
• [PR] fix: [branch-1.1] initialize dispatched kernels with the partition the native plan computes (#6700) [datafusion-comet] via GitHub
  • Re: [PR] fix: [branch-1.1] initialize dispatched kernels with the partition the native plan computes (#6700) [datafusion-comet] via GitHub
• [PR] deps: [branch-1.1] bump to datafusion 55.2.0 (#6766, #6862) [datafusion-comet] via GitHub
• [PR] fix: [branch-1.1] keep a Spark scan unconverted when the plan reads input_file_name (#6703) [datafusion-comet] via GitHub
• [PR] fix: [branch-1.1] make Iceberg delete-file reflection failures fatal (#5515) [datafusion-comet] via GitHub
• [PR] fix: [branch-1.1] match Spark percentile ordering for negative NaNs (#6829) [datafusion-comet] via GitHub
• [PR] Improve UI [datafusion-site] via GitHub
• Re: [PR] fix: support JVM columnar shuffle with spark.shuffle.checksum.enabled=false [datafusion-comet] via GitHub
• [PR] fix: [branch-1.1] retry a native acquire that Spark failed after dropping the task's entry (#6310) [datafusion-comet] via GitHub
• [PR] Fix local Pelican build [datafusion-site] via GitHub
• Re: [I] Crate name `datafusion-iceberg` is taken on crates.io; switch back to `iceberg-datafusion` [datafusion-iceberg] via GitHub
  • Re: [I] Crate name `datafusion-iceberg` is taken on crates.io; switch back to `iceberg-datafusion` [datafusion-iceberg] via GitHub
  • Re: [I] Crate name `datafusion-iceberg` is taken on crates.io; switch back to `iceberg-datafusion` [datafusion-iceberg] via GitHub
  • Re: [I] Crate name `datafusion-iceberg` is taken on crates.io; switch back to `iceberg-datafusion` [datafusion-iceberg] via GitHub
• [PR] chore: rename crate back to iceberg-datafusion and start at version 0.12.0 [datafusion-iceberg] via GitHub
  • Re: [PR] chore: rename crate back to iceberg-datafusion and start at version 0.12.0 [datafusion-iceberg] via GitHub
  • Re: [PR] chore: rename crate back to iceberg-datafusion and start at version 0.12.0 [datafusion-iceberg] via GitHub
  • Re: [PR] chore: rename crate back to iceberg-datafusion and start at version 0.12.0 [datafusion-iceberg] via GitHub
• Re: [PR] docs: add a scan contributor guide and a scan PR review skill [datafusion-comet] via GitHub
• Re: [PR] fix: serialize large-offset string and binary vectors with 32-bit offsets [datafusion-comet] via GitHub
• [PR] test: fail Comet tests when a native plan mislabels a timestamp's timezone [datafusion-comet] via GitHub
• [PR] ci: add Apache RAT license header check [datafusion-iceberg] via GitHub
  • Re: [PR] ci: add Apache RAT license header check [datafusion-iceberg] via GitHub
• [PR] refactor: replace MapOffset tuple with ProbeOffset enum [datafusion] via GitHub
  • Re: [PR] refactor: replace MapOffset tuple with ProbeOffset enum [datafusion] via GitHub
  • Re: [PR] refactor: replace MapOffset tuple with ProbeOffset enum [datafusion] via GitHub
• [PR] fix: keep one row per key in a native PartialMerge that runs short of memory [datafusion-comet] via GitHub
• [PR] fix: enable Criterion automatically when a benchmark namespace is set [datafusion] via GitHub
  • Re: [PR] fix: enable Criterion automatically when a benchmark namespace is set [datafusion] via GitHub
• [PR] refactor: stop normalizing float divisors now that percentile_approx orders NaN as Spark does [datafusion-comet] via GitHub
  • Re: [PR] refactor: stop normalizing float divisors now that percentile_approx orders NaN as Spark does [datafusion-comet] via GitHub
• [PR] fix: derive string function nullability from inputs [datafusion] via GitHub
  • Re: [PR] fix: derive string function nullability from inputs [datafusion] via GitHub
  • Re: [PR] fix: derive string function nullability from inputs [datafusion] via GitHub
• Re: [PR] fix: use Spark-compatible shuffle for wide decimal hash keys [datafusion-comet] via GitHub
• [I] Closing the shuffle direct-read stream early fetches every remaining block [datafusion-comet] via GitHub
• [PR] test: sample current-process RSS before query execution [datafusion] via GitHub
  • Re: [PR] test: sample current-process RSS before query execution [datafusion] via GitHub
  • Re: [PR] test: sample current-process RSS before query execution [datafusion] via GitHub
• [PR] chore: scope mutable-key lint exceptions for window expressions [datafusion] via GitHub
  • Re: [PR] chore: scope mutable-key lint exceptions for window expressions [datafusion] via GitHub
  • Re: [PR] chore: scope mutable-key lint exceptions for window expressions [datafusion] via GitHub
• [PR] fix: preserve filter semantics for casts and negation [datafusion-iceberg] via GitHub
• [I] Expose Parquet read-ahead controls for Iceberg SQL scans [datafusion-iceberg] via GitHub
  • Re: [I] Expose Parquet read-ahead controls for Iceberg SQL scans [datafusion-iceberg] via GitHub
• [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
  • Re: [PR] DRAFT Bench 26133 [datafusion] via GitHub
• Re: [PR] feat: opt in to join filter pushdown through deterministic residuals [datafusion-comet] via GitHub
• Re: [I] Field id matching misses a container id after INT96 coercion drops container metadata [datafusion-comet] via GitHub
• Re: [PR] SQLite: Support signed and decimal numbers in type modifiers [datafusion-sqlparser-rs] via GitHub
• Re: [PR] ClickHouse: Support GLOBAL IN / GLOBAL NOT IN [datafusion-sqlparser-rs] via GitHub
• [PR] docs: Add Datadog distributed DataFusion blog post to readings page [datafusion] via GitHub
  • Re: [PR] docs: Add Datadog distributed DataFusion blog post to readings page [datafusion] via GitHub
• [PR] fix(spark): derive concat_ws nullability from separator [datafusion] via GitHub
  • Re: [PR] fix(spark): derive concat_ws nullability from separator [datafusion] via GitHub
  • Re: [PR] fix(spark): derive concat_ws nullability from separator [datafusion] via GitHub
  • Re: [PR] fix(spark): derive concat_ws nullability from separator [datafusion] via GitHub
• Re: [PR] Feat/map sql extension types [datafusion] via GitHub
• [PR] feat: attach Diagnostic to duplicate table name errors [datafusion] via GitHub
• [PR] fix: preserve schema consistency across union and cast [datafusion] via GitHub
  • Re: [PR] fix: preserve schema consistency across union and cast [datafusion] via GitHub
  • Re: [PR] fix: preserve schema consistency across union and cast [datafusion] via GitHub
  • Re: [PR] fix: preserve schema consistency across union and cast [datafusion] via GitHub
  • [PR] fix: preserve schema consistency across union and cast [datafusion] via GitHub
• Re: [PR] fix: stop gating columnar shuffle on native-serde checks it never uses [datafusion-comet] via GitHub
  • Re: [PR] fix: stop gating columnar shuffle on native-serde checks it never uses [datafusion-comet] via GitHub
  • Re: [PR] fix: stop gating columnar shuffle on native-serde checks it never uses [datafusion-comet] via GitHub
• Re: [PR] feat: improve Spark from_utc_timestamp compatibility [datafusion] via GitHub
  • Re: [PR] feat: improve Spark from_utc_timestamp compatibility [datafusion] via GitHub
  • Re: [PR] feat: improve Spark from_utc_timestamp compatibility [datafusion] via GitHub
• [PR] test: migrate required-child hash regressions to SQL fixtures [datafusion-comet] via GitHub
• [PR] test: migrate Java RLIKE semantics and batch coverage to SQL fixtures [datafusion-comet] via GitHub
• [PR] fix: coerce map lookup keys to the map key type in get_field [datafusion] via GitHub
• [PR] fix: validate join_integer_prefilter_min_pruning_ratio at SET time [datafusion] via GitHub
• Re: [PR] feat: support AtLeastNNonNulls natively [datafusion-comet] via GitHub
• [I] `map[key]` fails when the key literal's type differs from the map key type (e.g. `MAP(VARCHAR, ...)` with `m['a']`) [datafusion] via GitHub
  • Re: [I] `map[key]` fails when the key literal's type differs from the map key type (e.g. `MAP(VARCHAR, ...)` with `m['a']`) [datafusion] via GitHub
• Re: [I] Eliminate more outer joins by supporting more expressions [datafusion] via GitHub
• [PR] feat: mark null-propagating string functions as strict [datafusion] via GitHub
  • Re: [PR] feat: mark null-propagating scalar functions as strict [datafusion] via GitHub
• [PR] fix: infer missing nested Parquet fields as nullable [datafusion] via GitHub
  • Re: [PR] fix: infer missing nested Parquet fields as nullable [datafusion] via GitHub
  • Re: [PR] fix: infer missing nested Parquet fields as nullable [datafusion] via GitHub
• [PR] perf: speed up PrimitiveArray hashing for nullable inputs [datafusion] via GitHub
  • Re: [PR] perf: speed up PrimitiveArray hashing for nullable inputs [datafusion] via GitHub
• Re: [PR] fix(proto): preserve EmptyRelation schema across logical plan round trip [datafusion] via GitHub
  • Re: [PR] fix(proto): preserve EmptyRelation schema across logical plan round trip [datafusion] via GitHub
• [PR] test: cover container field ids next to INT96 timestamps in the native Parquet scan [datafusion-comet] via GitHub
  • Re: [PR] test: cover container field ids next to INT96 timestamps in the native Parquet scan [datafusion-comet] via GitHub
  • Re: [PR] test: cover container field ids next to INT96 timestamps in the native Parquet scan [datafusion-comet] via GitHub
• Re: [I] Clean up dead ANSI-related plumbing (CometEvalMode helpers, inert allow_incompat on casts) [datafusion-comet] via GitHub
• Re: [PR] feat: expose bloom filter, page index and sorting metadata in parquet… [datafusion] via GitHub
• Re: [PR] fix(proto): logical plan deserialization stack overflow protection [datafusion] via GitHub
• Re: [PR] feat: attach Diagnostic to "invalid function argument types" error [datafusion] via GitHub
• Re: [PR] perf(spark): fold concat_ws plans with literal-NULL separator via simplify [datafusion] via GitHub
• Re: [PR] fix: reject duplicate projection output names [datafusion] via GitHub
• Re: [PR] fix(substrait): Preserve pushed-down table scan offsets and limits [datafusion] via GitHub
  • Re: [PR] fix(substrait): Preserve pushed-down table scan offsets and limits [datafusion] via GitHub
• Re: [I] perf: use aligned slice access in SparkUnsafeArray bulk append [datafusion-comet] via GitHub
• [I] INSERT fails after a column is added to the table, until the provider is rebuilt [datafusion-iceberg] via GitHub
• [PR] fix: check for nix flake support before trying to load flake [datafusion] via GitHub
  • Re: [PR] fix: check for nix flake support before trying to load flake [datafusion] via GitHub
• [PR] DataFrame API serialize columns to a JSON string column (#26185) [datafusion] via GitHub
  • Re: [PR] DataFrame API serialize columns to a JSON string column (#26185) [datafusion] via GitHub
  • Re: [PR] DataFrame API serialize columns to a JSON string column (#26185) [datafusion] via GitHub
• Re: [PR] perf: skip DecimalPrecision.promote's rewrite when there is no decimal arithmetic [datafusion-comet] via GitHub
  • Re: [PR] perf: skip DecimalPrecision.promote's rewrite when there is no decimal arithmetic [datafusion-comet] via GitHub
• [I] Wrong results and panics after a column is dropped and added back under the same name [datafusion-iceberg] via GitHub
• [I] [EPIC] Support Hive tables for users migrating from Spark [datafusion-ballista] via GitHub
• [PR] fix: preserve columnar output when reverting a cached AQE result stage [datafusion-comet] via GitHub
  • Re: [PR] fix: preserve columnar output when reverting a cached AQE result stage [datafusion-comet] via GitHub
• Re: [I] Null-aware LeftMark joins miss logical NULLs in dictionary keys [datafusion] via GitHub
• [I] `sum` of a decimal returns a wider type than the plan schema declares when the query has a `DISTINCT` aggregate [datafusion] via GitHub
• [PR] perf: presize collections and reuse buffers in native loops [datafusion-comet] via GitHub
  • Re: [PR] perf: presize collections and reuse buffers in native loops [datafusion-comet] via GitHub
  • Re: [PR] perf: presize collections and reuse buffers in native loops [datafusion-comet] via GitHub
• [PR] perf: preserve dictionary encoding through regexp_replace [datafusion] via GitHub
  • Re: [PR] perf: preserve dictionary encoding through regexp_replace [datafusion] via GitHub
• [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
  • Re: [PR] perf: pre-size collections and reuse per-iteration buffers [datafusion] via GitHub
• Re: [I] Check `PiecewiseMergeJoin` performance for large tables [datafusion] via GitHub
• [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
  • Re: [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
  • Re: [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
  • Re: [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
  • Re: [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
  • Re: [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
  • Re: [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
  • Re: [PR] perf: unroll PrimitiveArray hashing for the no-null path [datafusion] via GitHub
• [PR] perf(substrait): avoid quadratic field lookup when consuming ReadRel [datafusion] via GitHub
• [PR] chore: allow clippy mutable_key_type on Expr hash sets in PushDownFilter (#26181) [datafusion] via GitHub
  • Re: [PR] chore: allow clippy mutable_key_type on Expr hash sets in PushDownFilter (#26181) [datafusion] via GitHub
• [PR] refactor(core): drain and size the sort-shuffle buffer through BufferedBatches [datafusion-ballista] via GitHub
• [PR] fix: do not extract common subexpressions from the arguments of an aggregate with a FILTER [datafusion] via GitHub
  • Re: [PR] fix: do not extract common subexpressions from the arguments of an aggregate with a FILTER [datafusion] via GitHub
  • Re: [PR] fix: do not extract common subexpressions from the arguments of an aggregate with a FILTER [datafusion] via GitHub
  • Re: [PR] fix: do not extract common subexpressions from the arguments of an aggregate with a FILTER [datafusion] via GitHub
• [I] DataFrame API: serialize columns to a JSON string column [datafusion] via GitHub
  • Re: [I] DataFrame API: serialize columns to a JSON string column [datafusion] via GitHub
  • Re: [I] DataFrame API: serialize columns to a JSON string column [datafusion] via GitHub
  • Re: [I] DataFrame API: serialize columns to a JSON string column [datafusion] via GitHub
  • Re: [I] DataFrame API: serialize columns to a JSON string column [datafusion] via GitHub
  • Re: [I] DataFrame API: serialize columns to a JSON string column [datafusion] via GitHub
  • Re: [I] DataFrame API: serialize columns to a JSON string column [datafusion] via GitHub
• [I] Common subexpression elimination evaluates the arguments of an aggregate with `FILTER` for rows that the filter rejects [datafusion] via GitHub
• [PR] perf: generate the in-memory cache reader's class once per executor [datafusion-comet] via GitHub
  • Re: [PR] perf: generate the in-memory cache reader's class once per executor [datafusion-comet] via GitHub
• [PR] feat: add lz4 as an opt-in codec for Comet's in-memory cache [datafusion-comet] via GitHub
  • Re: [PR] feat: add lz4 as an opt-in codec for Comet's in-memory cache [datafusion-comet] via GitHub
• Re: [PR] perf: add opt-in delta encoding for cached longs [datafusion-comet] via GitHub
  • Re: [PR] perf: add opt-in delta encoding for cached longs [datafusion-comet] via GitHub
• Re: [PR] fix: align nested collection buffer nullability before spilling [datafusion-comet] via GitHub
• [I] Follow-ups to native NULL short-circuiting of array and map functions [datafusion-comet] via GitHub
• [I] [EPIC] Nested-type conversion performance: C2R, R2C and JVM shuffle slower than Spark [datafusion-comet] via GitHub
  • Re: [I] [EPIC] Nested-type conversion performance: C2R, R2C and JVM shuffle slower than Spark [datafusion-comet] via GitHub
  • Re: [I] [EPIC] Nested-type conversion performance: C2R, R2C and JVM shuffle slower than Spark [datafusion-comet] via GitHub
• [PR] perf: generate the columnar-to-row projection once per executor instead of per partition [datafusion-comet] via GitHub
  • Re: [PR] perf: generate the columnar-to-row projection once per executor instead of per partition [datafusion-comet] via GitHub
  • Re: [PR] perf: generate the columnar-to-row projection once per executor instead of per partition [datafusion-comet] via GitHub
  • Re: [PR] perf: generate the columnar-to-row projection once per executor instead of per partition [datafusion-comet] via GitHub
  • Re: [PR] perf: generate the columnar-to-row projection once per executor instead of per partition [datafusion-comet] via GitHub
• [I] Internal error: CAST over an expression simplified to a metadata-carrying column, under UNION ALL + aggregate (physical vs logical field metadata) [datafusion] via GitHub
• [PR] feat: bump iceberg-rust and write float and double identity partitions natively [datafusion-comet] via GitHub

• Earlier messages