[
https://issues.apache.org/jira/browse/FLINK-30445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junyao Huang updated FLINK-30445:
---------------------------------
Description:
Web3 is very hot. But you could search GitHub open source blockchain explorer,
the most stars project is blockscout,
[https://github.com/blockscout/blockscout,] which use Elixir as a parallel
engine to sync block from blockchain node into a file(CSV format). I think
Flink is the best solution of ingestion. Reason:
(1)blockchain needs to match different chain, including Ethereum, Bitcoin,
Solana, etc. through JSON RPC.
(2)Like EtherScan, the blockchain needs to fetch the latest block into storage
for the index to search.
(3)Also as a supplement to (2), we need a connector to fully sync all block
from Blockchain Node. I think Flink Stream/Batch alignment feature is suit for
this scenarios.
(4)According to FLIP-27, we could use block number as SourceSplit to read. It
is very natural.
(5)Flink Community could use web3 topic to get PR effects on web3 cycle.
was:
Web3 is very hot. But you could search GitHub open source blockchain explorer,
the most stars project is blockscout,
[https://github.com/blockscout/blockscout,] which use Elixir as a parallel
engine to sync block from blockchain node into a file(CSV format). I think
Flink is the best solution of ingestion. Reason:
(1)blockchain needs to match different chain, including Ethereum, Bitcoin,
Solana, etc. through JSON RPC.
(2)Like EtherScan, the blockchain needs to fetch the latest block into storage
for the index to search.
(3)Also as a supplement to (2), we need a connector to fully sync all block
from Blockchain Node. I think Flink Stream/Batch alignment feature is suit for
this scenarios.
(4)According to FLIP-27, we could use block number as SourceSplit to read. It
is very natural.
> Add Flink Web3 Connector
> ------------------------
>
> Key: FLINK-30445
> URL: https://issues.apache.org/jira/browse/FLINK-30445
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / ORC
> Reporter: Junyao Huang
> Priority: Major
>
> Web3 is very hot. But you could search GitHub open source blockchain
> explorer, the most stars project is blockscout,
> [https://github.com/blockscout/blockscout,] which use Elixir as a parallel
> engine to sync block from blockchain node into a file(CSV format). I think
> Flink is the best solution of ingestion. Reason:
> (1)blockchain needs to match different chain, including Ethereum, Bitcoin,
> Solana, etc. through JSON RPC.
> (2)Like EtherScan, the blockchain needs to fetch the latest block into
> storage for the index to search.
> (3)Also as a supplement to (2), we need a connector to fully sync all block
> from Blockchain Node. I think Flink Stream/Batch alignment feature is suit
> for this scenarios.
> (4)According to FLIP-27, we could use block number as SourceSplit to read. It
> is very natural.
> (5)Flink Community could use web3 topic to get PR effects on web3 cycle.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
