Hello Rayner,
Thank you very much for the clarifications. Glad to know I'm in the
right place. Just two follow-up questions regarding Apache Doris
Streaming Jobs for CDC.
1) Approximately how fast is it? I mean for each transaction that takes
place in PostgreSQL/Citus/YugabyteDB, how much time does it take to be
reflected inside Apache Doris? Is the lag sub-second, or within 3
seconds, etc.? To be clear, I am talking about just direct
synchronization from the OLTP source with no transformations whatsoever.
2) For BSON type data from PostgreSQL DocumentDB or from MongoDB itself,
what data type does Apache Doris have? Is it just JSON? Or does it
have other types similar to JSONB or BSON?
Thank you very much for your kind clarifications. Looking forward to
seeing all the great things that come in Apache Doris 5.0!
Best Regards,
Saleh
On 14.11.2026 03:25, Mingyu Chen via dev wrote:
Hi Saleh,
The dev list is a good place for this question. For a faster, more
interactive discussion, I’d also suggest joining the Apache Doris
Slack community.
Doris doesn’t currently offer the same integrated HTAP capabilities as
TiDB. We support PostgreSQL CDC through Doris Streaming Jobs
(https://doris.apache.org/docs/dev/data-operate/import/import-way/streaming-job/continuous-load-overview),
so you can replicate changes into Doris in near real time and run
analytical queries there. Those queries see the data that has already
been applied in Doris. They don’t automatically include changes that
have committed in PostgreSQL but haven’t been replicated yet.
In practice, we often see OLTP and OLAP owned by different teams and
run on separate clusters, even when the database technology supports
both. That keeps analytical workloads from competing with
transactional traffic and lets each team scale independently. For your
setup, I’d start by keeping writes and point lookups on the
transactional side and sending analytical queries to Doris explicitly.
We don’t currently provide a PostgreSQL extension like pg_clickhouse
or pg_duckdb that lets you query Doris from a PostgreSQL session.
If you share which queries need to see a write immediately and how
much lag the rest can tolerate, we can discuss what would be practical
for your setup on Slack.
Best,
Rayner
On Fri, 13 Nov 2026 at 13:57,
<dev-reject-1794549464.2725509.pagamicmfbckjbjhn...@doris.apache.org>
wrote:
To approve:
dev-accept-1794549464.2725509.pagamicmfbckjbjhn...@doris.apache.org
To reject:
dev-reject-1794549464.2725509.pagamicmfbckjbjhn...@doris.apache.org
To give a reason to reject:
%%% Start comment
%%% End comment
---------- Forwarded message ----------
From: [email protected]
To: [email protected]
Cc:
Bcc:
Date: Fri, 13 Nov 2026 05:57:36 +0000
Subject: Composable HTAP with Apache Doris
Hello,
I apologize for the previous message I sent. I accidentally sent
it
before completing the message.
I hope that I am sending this inquiry to the right place. We are
evaluating OLAP solutions for composable HTAP with distributed
PostgreSQL databases such as YugabyteDB and Citus. For this, we have
identified some specific features that would be required, similar to
what exists in purpose-built HTAP databases (e.g. OceanBase and TiDB),
and I would like to ask about the existence of each one in Apache
Doris,
either now or in the near future, below:
1) Sub-Second replication from PostgreSQL to Apache Doris (without any
transformations). An example of what I mean is something like to
ClickHouse's recently released "WALShadow" feature.
2) Consistent views of state, similar to what is achieved in HTAP
databases such as OceanBase and TiDB. So that OLAP queries in Apache
Doris include ALL data from the source database, meaning both what has
already been stored in Apache Doris AND what has yet to be replicated
into Apache Doris from the source database.
3) Automatic routing of queries between Apache Doris and PostgreSQL
based on which engine would handle it best. For example, if the query
is a DML or point-lookup operation then it should be routed to
PostgreSQL, and if the query is an OLAP query it should be processed
by
Apache Doris. If this is not available within Apache Doris itself,
then
compatibility with a proxy tools such as ProxySQL and QueryFlux would
be
required.
4) PostgreSQL interface compatibility and/or SQLGlot/QueryFlux
integration. I previously read an article on Dev.to describing
SQLGlot
integration within Apache Doris for handling multiple SQL dialects.
But
I could not find reference to this capability in the documentation.
Has
this feature been removed? Also, are there plans to add PostgreSQL
wire
protocol support to Apache Doris? If not, then the QueryFlux proxy
can
be used for conversion of SQL dialect and wire protocol. QueryFlux is
relatively new, but looks very promising, and it lists compatibility
with StarRocks. Are you familiar with this project, and have you
used/tested QueryFlux with Apache Doris (especially in an HTAP
architecture)?
I look forward to your kind clarifications on the points above,
including both what exists today and what is planned. And if there is
a
better place for me to send such inquiries, please do let me know.
Thank You and Best Regards,
Saleh
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]