Hi Junyao, I don't see value in this for the Apache Flink community. You're stating this as a 'Web3 connector' but are only referring to one specific project. If that project needs a Flink connector, I think it should be built and maintained by that project, not by the Flink community.
Best regards, Martijn On Tue, Dec 20, 2022 at 6:11 AM Junyao Huang <junyao.hu...@microsoft.com.invalid> wrote: > Web3 is very hot. But you could search GitHub open source blockchain > explorer, the most stars project is blockscout, > https://github.com/blockscout/blockscout< > https://github.com/blockscout/blockscout,> which use Elixir as a parallel > engine to sync block from blockchain node into a file(CSV format). I think > Flink is the best solution of ingestion. Reason: > > (1)blockchain needs to match different chain, including Ethereum, Bitcoin, > Solana, etc. through JSON RPC. > > (2)Like EtherScan, the blockchain needs to fetch the latest block into > storage for the index to search. > > (3)Also as a supplement to (2), we need a connector to fully sync all > block from Blockchain Node. I think Flink Stream/Batch alignment feature is > suit for this scenarios. > > (4)According to FLIP-27, we could use block number as SourceSplit to read. > It is very natural. > > (5)Flink Community could use web3 topic to get PR effects on web3 cycle. > > > [1] > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-30445?filter=allissues > > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface > >
