Doris-Breakwater commented on issue #67546: URL: https://github.com/apache/doris/issues/67546#issuecomment-5556520047
Breakwater-GitHub-Analysis-Slot: slot_a9654223c21b ## Initial maintainer triage **Assessment:** likely valid and actionable as a `dbt-doris`/Connector-Python compatibility problem. The cross-platform boundary scan and the same-process `use_pure=True` control are strong evidence that selecting the C-extension path is necessary to trigger the failure. However, the evidence currently does **not** prove the stronger claim that the C extension corrupts bytes in transit; the failing component could still be the C client library, TLS/packet framing interoperability, or Doris's handling of packets emitted by that path. I recommend keeping this open and adding the project's bug and dbt/extension component labels (the issue currently has no labels). ### Verified from code and upstream documentation - At local master commit [`82646c38c00`](https://github.com/apache/doris/blob/82646c38c0069eee4b1704eafef416f969bd4b93/extension/dbt-doris/dbt/adapters/doris/connections.py#L77-L89), the adapter does omit `use_pure` from both connection attempts. - The released package metadata permits any [`mysql-connector-python>=8.0.0`](https://github.com/apache/doris/blob/82646c38c0069eee4b1704eafef416f969bd4b93/extension/dbt-doris/setup.py#L39-L42), so version 26.7.0 can be selected. In contrast, the development requirements still constrain the connector to [`>=8.0.33,<8.3`](https://github.com/apache/doris/blob/82646c38c0069eee4b1704eafef416f969bd4b93/extension/dbt-doris/dev-requirements.txt#L40), which means the normal adapter test environment cannot exercise the reported version. - Oracle documents that `use_pure=False` is the default when the C extension is available, while `use_pure=True` selects the independent Python implementation: https://dev.mysql.com/doc/connector-python/en/connector-python-cext-development.html - In Connector/Python 26.7.0 source, the C path encodes the query and ultimately calls `mysql_real_query`; the pure path constructs a MySQL command packet and uses Python socket `sendall`. They are genuinely different protocol paths, so the A/B result is meaningful. - In the exact Doris `3.1.4-rc02` code, [`MysqlChannel.fetchOnePacket()`](https://github.com/apache/doris/blob/7f5ba43de6c7f148e1dacdb7cb838394d2ec1537/fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlChannel.java#L318-L437) reads the MySQL packet length and repeatedly fills the body buffer. The plain-TCP receive path has no 8 KiB cutoff. Repository inspection therefore does not reveal a Doris-side 8188-byte statement limit, but it also cannot prove what bytes reached the FE. ### Reproducer correction The posted `n` is not the actual statement length. The prefix is 30 ASCII bytes and the suffix is 14, so `30 + (n - 42) + 14 == n + 2`; with `n=8188`, the generated statement is 8190 bytes. This does not negate the failure, but the boundary claim and the “8192 minus header” explanation need a corrected scanner that asserts `len(stmt.encode("utf-8")) == target_length`. Please also include table cleanup or a unique table name for every size/run so an existing table cannot affect the matrix. ### Evidence still needed to locate the root cause 1. Attach the complete corrected synthetic scanner and its raw output for at least 8184–8194 bytes. Print `mysql.connector.__version__`, `mysql.connector.HAVE_CEXT`, and `type(conn)` for each mode; no credentials are needed. 2. State whether TLS is negotiated. Repeat the synthetic scan with TLS explicitly disabled, if the environment permits it. Doris 3.1 has a separate SSL packet-unwrapping path, so this is an important branch to isolate. 3. Provide the full Python exception/traceback and the matching FE log lines for the first failing request and the following request on the unusable connection. The `2013` result and a parser error may represent different failure paths. 4. For the synthetic query, capture the client-to-FE MySQL packet with TLS disabled and compare the declared payload length plus the SQL payload against `stmt.encode("utf-8")`. A packet capture or a small TCP proxy that records SHA-256 and the first differing byte would establish whether corruption occurs before or inside Doris. Avoid capturing or publishing authentication traffic/credentials. 5. Run the same C-extension scanner against a supported MySQL Server using the same wheel. If it fails there too, this is directly actionable upstream; if it only fails against Doris, treat it as protocol interoperability until the first byte-level divergence is located. A small connector-version matrix (for example the currently tested 8.0.x range versus 26.7.0) would also show whether this is a recent regression exposed by the unbounded runtime dependency. The linked upstream reports are useful background but are not direct root-cause evidence here: MySQL #77789 is about pure-Python `COM_STMT_SEND_LONG_DATA` performance, #96588 is prepared-result decoding on Windows, and #80238 shows an old pure-Python `network.py` socket-send failure. CONC-771 is in a different MariaDB connector codebase. ### Recommended next step Reproduce the corrected boundary test once in a maintainer-controlled Doris 3.1.4 environment. If the `use_pure` A/B result holds, adding `use_pure=True` is a reasonable narrow **adapter mitigation** because dbt-doris targets Doris and the pure implementation is officially supported. The change should include (a) a unit test asserting both connection attempts receive `use_pure=True`, (b) an end-to-end CTAS test whose SQL is safely above the boundary and whose sentinel is byte-checked, and (c) alignment of the development connector range with the published runtime range so CI covers the selected major version. Track the byte-level/upstream root cause separately rather than presenting the historical bugs as confirmation. Until that change ships, documenting `use_pure=True` as a verified workaround is appropriate, while noting that dbt-doris does not currently expose it through connection credentials and therefore users may need a patched adapter/environment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
