waltage commented on issue #21476: URL: https://github.com/apache/beam/issues/21476#issuecomment-1159157292
I added in levels for `method` and for `schema` (table below, and you might need to scroll to the right more). The last two columns are errors that are eventually returned by BigQuery, whereas SCHEMA AUTO ERR and WRITE DISPO ERR are exceptions raised by Beam itself. I'll make another comment with my thoughts. | ID| WTBQ(table)| WTBQ(method)| WTBQ(schema)| WTBQ(create)| WTBQ(write)| STATUS| Beam SCHEMA AUTO ERR| Beam WRITE DISPO ERR| BQ Schema ERR| BQ Table Missing ERR |--|--|--|--|--|--|--|--|--|--|--| 041|() → TableRef|DEFAULT|None|IF_NEEDED|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 045|() → TableRef|DEFAULT|None|IF_NEEDED|TRUNCATE| 🛑 | ○ | ○ | 🛑 | ○ 037|() → TableRef|DEFAULT|None|NEVER|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 005|() → str|DEFAULT|None|IF_NEEDED|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 009|() → str|DEFAULT|None|IF_NEEDED|TRUNCATE| 🛑 | ○ | ○ | 🛑 | ○ 001|() → str|DEFAULT|None|NEVER|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 053|() → TableRef|FILE_LOADS|None|IF_NEEDED|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 057|() → TableRef|FILE_LOADS|None|IF_NEEDED|TRUNCATE| 🛑 | ○ | ○ | 🛑 | ○ 049|() → TableRef|FILE_LOADS|None|NEVER|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 017|() → str|FILE_LOADS|None|IF_NEEDED|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 021|() → str|FILE_LOADS|None|IF_NEEDED|TRUNCATE| 🛑 | ○ | ○ | 🛑 | ○ 013|() → str|FILE_LOADS|None|NEVER|APPEND| 🛑 | ○ | ○ | 🛑 | ○ 065|() → TableRef|STREAMING_INSERTS|None|IF_NEEDED|APPEND| 🟡 | ○ | ○ | ○ | 🛑 069|() → TableRef|STREAMING_INSERTS|None|IF_NEEDED|TRUNCATE| 🛑 | ○ | 🛑 | ○ | ○ 061|() → TableRef|STREAMING_INSERTS|None|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 029|() → str|STREAMING_INSERTS|None|IF_NEEDED|APPEND| 🟡 | ○ | ○ | ○ | 🛑 033|() → str|STREAMING_INSERTS|None|IF_NEEDED|TRUNCATE| 🛑 | ○ | 🛑 | ○ | ○ 025|() → str|STREAMING_INSERTS|None|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 040|() → TableRef|DEFAULT|SCHEMA_AUTODETECT|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 044|() → TableRef|DEFAULT|SCHEMA_AUTODETECT|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 036|() → TableRef|DEFAULT|SCHEMA_AUTODETECT|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 004|() → str|DEFAULT|SCHEMA_AUTODETECT|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 008|() → str|DEFAULT|SCHEMA_AUTODETECT|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 000|() → str|DEFAULT|SCHEMA_AUTODETECT|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 052|() → TableRef|FILE_LOADS|SCHEMA_AUTODETECT|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 056|() → TableRef|FILE_LOADS|SCHEMA_AUTODETECT|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 048|() → TableRef|FILE_LOADS|SCHEMA_AUTODETECT|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 016|() → str|FILE_LOADS|SCHEMA_AUTODETECT|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 020|() → str|FILE_LOADS|SCHEMA_AUTODETECT|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 012|() → str|FILE_LOADS|SCHEMA_AUTODETECT|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 064|() → TableRef|STREAMING_INSERTS|SCHEMA_AUTODETECT|IF_NEEDED|APPEND| 🛑 | 🛑 | ○ | ○ | ○ 068|() → TableRef|STREAMING_INSERTS|SCHEMA_AUTODETECT|IF_NEEDED|TRUNCATE| 🛑 | 🛑 | ○ | ○ | ○ 060|() → TableRef|STREAMING_INSERTS|SCHEMA_AUTODETECT|NEVER|APPEND| 🛑 | 🛑 | ○ | ○ | ○ 028|() → str|STREAMING_INSERTS|SCHEMA_AUTODETECT|IF_NEEDED|APPEND| 🛑 | 🛑 | ○ | ○ | ○ 032|() → str|STREAMING_INSERTS|SCHEMA_AUTODETECT|IF_NEEDED|TRUNCATE| 🛑 | 🛑 | ○ | ○ | ○ 024|() → str|STREAMING_INSERTS|SCHEMA_AUTODETECT|NEVER|APPEND| 🛑 | 🛑 | ○ | ○ | ○ 043|() → TableRef|DEFAULT|TableSchema|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 047|() → TableRef|DEFAULT|TableSchema|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 039|() → TableRef|DEFAULT|TableSchema|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 007|() → str|DEFAULT|TableSchema|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 011|() → str|DEFAULT|TableSchema|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 003|() → str|DEFAULT|TableSchema|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 055|() → TableRef|FILE_LOADS|TableSchema|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 059|() → TableRef|FILE_LOADS|TableSchema|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 051|() → TableRef|FILE_LOADS|TableSchema|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 019|() → str|FILE_LOADS|TableSchema|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 023|() → str|FILE_LOADS|TableSchema|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 015|() → str|FILE_LOADS|TableSchema|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 067|() → TableRef|STREAMING_INSERTS|TableSchema|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 071|() → TableRef|STREAMING_INSERTS|TableSchema|IF_NEEDED|TRUNCATE| 🛑 | ○ | 🛑 | ○ | ○ 063|() → TableRef|STREAMING_INSERTS|TableSchema|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 031|() → str|STREAMING_INSERTS|TableSchema|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 035|() → str|STREAMING_INSERTS|TableSchema|IF_NEEDED|TRUNCATE| 🛑 | ○ | 🛑 | ○ | ○ 027|() → str|STREAMING_INSERTS|TableSchema|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 042|() → TableRef|DEFAULT|str|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 046|() → TableRef|DEFAULT|str|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 038|() → TableRef|DEFAULT|str|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 006|() → str|DEFAULT|str|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 010|() → str|DEFAULT|str|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 002|() → str|DEFAULT|str|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 054|() → TableRef|FILE_LOADS|str|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 058|() → TableRef|FILE_LOADS|str|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 050|() → TableRef|FILE_LOADS|str|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 018|() → str|FILE_LOADS|str|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 022|() → str|FILE_LOADS|str|IF_NEEDED|TRUNCATE| ✅ | ✅ | ✅ | ✅ | ✅ 014|() → str|FILE_LOADS|str|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 066|() → TableRef|STREAMING_INSERTS|str|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 070|() → TableRef|STREAMING_INSERTS|str|IF_NEEDED|TRUNCATE| 🛑 | ○ | 🛑 | ○ | ○ 062|() → TableRef|STREAMING_INSERTS|str|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 030|() → str|STREAMING_INSERTS|str|IF_NEEDED|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ 034|() → str|STREAMING_INSERTS|str|IF_NEEDED|TRUNCATE| 🛑 | ○ | 🛑 | ○ | ○ 026|() → str|STREAMING_INSERTS|str|NEVER|APPEND| ✅ | ✅ | ✅ | ✅ | ✅ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
