rangareddy opened a new pull request, #19795: URL: https://github.com/apache/hudi/pull/19795
### Describe the issue this Pull Request addresses The blog index has no entry for [Beyond Offset Lag: Computing Time in Queue for Apache Hudi Data Lake Pipelines at Petabyte Scale](https://www.infoq.com/articles/beyond-offset-lag-kafka-apache-hudi/), published on InfoQ on 26 August 2026 by Srikanth Mamidala. It is a production case study worth surfacing from hudi.apache.org. Twilio's data lake ingests five trillion records a month through Hudi Streamer, and the article is about why Kafka offset lag is the wrong freshness signal for those pipelines: offset lag says how far behind a consumer is, not how old the data is. It derives a time-in-queue metric instead, by reading the Kafka checkpoint out of the latest `.hoodie` commit file, seeking to that offset in the topic, and measuring the timestamp delta against now, so it works off the timeline Hudi Streamer already writes with no instrumentation change to the pipeline. It also covers the edge cases that showed up in production: commits carrying no checkpoint metadata (walk backward through the commit history), clock skew, multi-partition topics, multiple writers to one table, and epoch timestamp defaults. That makes it directly useful to anyone running Hudi Streamer against Kafka, and it is the kind of real-world adoption story the `case-study` category exists for. ### Summary and Changelog - `website/blog/2026-08-26-beyond-offset-lag-computing-time-in-queue-for-apache-hudi-pipelines.mdx` — a redirect stub in the established shape for externally hosted articles: frontmatter plus `import Redirect from '@site/src/components/Redirect';` and a `<Redirect url="...">`, the same as `2024-06-18-how-to-use-apache-hudi-with-databricks.mdx` and the other external posts. The author is declared inline (`authors:` / `- name: Srikanth Mamidala`), which is what the other external-author posts do rather than adding an `authors.yml` entry for a one-off contributor. The filename date matches the article's publication date. - `website/static/assets/images/blog/2026-08-26-beyond-offset-lag-computing-time-in-queue-for-apache-hudi-pipelines.png` — a purpose-drawn hero at the required 1200x600, in the site's own brand palette (`--ifm-color-primary-text` `#0DB1F9` on `--ifm-color-primary-darkest`). It is not a screenshot of the article: it draws the distinction the article is built on, with one Kafka partition, the checkpoint recorded in the latest `.hoodie` commit, and the same span measured twice — as an offset count above and as elapsed time below. 266 KB, which sits between the median (180 KB) and p90 (662 KB) of the existing blog images. - `.github/scripts/validate-blog.py` — adds `twilio` to `ALLOWED_TAGS`. ### One thing for the reviewer **The new `twilio` tag needs your approval.** `validate-blog.py` rejects it otherwise, and its own error message says to add it to the allowlist in the same PR and ask the reviewer. It follows the existing company-tag convention: `uber`, `walmart`, `robinhood`, `peloton`, `upstox`, `metica`, `penn interactive`, `southwest airlines`, `zupee`, `yuno` and `grab` are already in that set. Happy to drop the tag and the allowlist line if you would rather not grow it — the other three tags (`apache kafka`, `observability`, `hudi streamer`) carry the post on their own. Unrelated to this post, but noticed while writing it: README's blog section asks for a `source:` prefixed tag (`source:medium`, `source:linkedin`, `source:onehouse`). No post in `website/blog/` uses one and `ALLOWED_TAGS` contains none, so `source:infoq` would fail CI. Left it out. The README and the validator have drifted apart there and it is worth reconciling in a separate PR. The same README says `image` is required, and `validate-blog.py` does list it in `REQUIRED_FIELDS`, but its missing-field branch only ever raises for `author`, so a post with no `image` passes silently today. ### Verification ``` python3 .github/scripts/validate-blog.py website/blog/2026-08-26-beyond-offset-lag-...-pipelines.mdx -> All 1 blog(s) passed validation cd website && npm run build -> [SUCCESS] Generated static files in "build" curl -I https://www.infoq.com/articles/beyond-offset-lag-kafka-apache-hudi/ -> HTTP 200 ``` The build emits `/blog/2026/08/26/beyond-offset-lag-computing-time-in-queue-for-apache-hudi-pipelines/` with the redirect target present in the rendered HTML. `content/` is untouched, per the `asf-site.ci.yml` guard. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
