kparisa opened a new pull request, #4394:
URL: https://github.com/apache/iggy/pull/4394

   ## Which issue does this PR address?
   Closes #4393
   
   ## Rationale
   Streams Iggy messages into BigQuery through the Storage Write API _default 
stream with Arrow-encoded rows. Supports mapped and raw modes, Iggy metadata 
columns, per-row rejection with offset logging, request splitting below the 
AppendRows limit, and retries for transient gRPC failures. Verified against a 
live BigQuery table; CI coverage uses an in-process fake of tables.get and 
BigQueryWrite.
   
   
   ## What changed?
   New sink connector under /core/connectors/sinks
   
   ## Local Execution
   - Passed 
   - Pre-commit hooks ran
   
   ## AI Usage
   - Leveraged ChatGPT, Claude for planning, test scenarios, implementation.
   - Ran local tests, publishing payloads to iggy, configured GCP test project, 
created the BigQuery (BQ) dataset "iggy" and events table. 
   - Verified the records in the BQ table.
   - Tested failure scenarios as well for mis-configs and missing table 
definitions etc
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to