Hello,
I hope that I am sending this inquiry to the right place. We are
evaluating OLAP solutions for composable HTAP with distributed
PostgreSQL databases such as YugabyteDB and Citus. For this, we have
identified some specific features that would be required, similar to
what exists in purpose-built HTAP databases (e.g. OceanBase and TiDB),
and I would like to ask about the existence of each one in Apache Doris,
either now or in the near future, below:
1) Sub-Second replication from PostgreSQL to Apache Doris (without any
transformations). An example of what I mean is something like
ClickHouse's recently released "WALShadow" feature.
2) Consistent views of state, similar to what is achieved in HTAP
databases such as OceanBase and TiDB. So that OLAP queries in Apache
Doris include ALL data from the source database, meaning both what has
already been stored in Apache Doris AND what has yet to be replicated
into Apache Doris from the source database.
3) Automatic routing of queries between Apache Doris and PostgreSQL
based on which engine would handle it best. For example, if the query
is a DML or point-lookup operation then it should be routed to
PostgreSQL, and if the query is an OLAP query it should be processed by
Apache Doris. If this is not available within Apache Doris itself, then
compatibility with proxy tools such as ProxySQL and QueryFlux would be
required. And if Apache Doris's point-lookup performance is on par with
PostgreSQL, then perhaps a simple read-write splitting through ProxySQL
can be used. Are there Apache Doris users that are using such
architectures in production?
4) PostgreSQL interface compatibility and/or SQLGlot/QueryFlux
integration. I previously read an article on Dev.to describing SQLGlot
integration within Apache Doris for handling multiple SQL dialects. But
I could not find reference to this capability in the documentation. Has
this feature been removed? Also, are there plans to add PostgreSQL wire
protocol support to Apache Doris? If not, then the QueryFlux proxy can
be used for conversion of SQL dialect and wire protocol. QueryFlux is
relatively new, but looks very promising, and it lists compatibility
with StarRocks. Are you familiar with this project, and have you
used/tested QueryFlux with Apache Doris (especially in an HTAP
architecture)?
5) Are there any plans to create an Apache Doris extension for
PostgreSQL? I mean something similar to pg_clickhouse, pg_duckdb, and
pg_lake.
I look forward to your kind clarifications on the points above,
including both what exists today and what is planned. And if there is a
better place for me to send such inquiries, please do let me know. For
example, I don't know if Apache Doris has a dedicated users mailing
list.
Thank You and Best Regards,
Saleh
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
