Dear fellow lightning developers, last week I have started a new repository [0] where I maintain a python package that can be used to test (and more importantly simulate) the improvements to payment delivery that we have suggested over the last years (c.f. [1][2][3]). I kindly invite you to check it out.
Feedback & code review will be highly appreciated if you find the spare time to do so. Similarly I will be delighted if you can provide a patch or an extension if necessary. Note while this may already be very useful that this is work in progress and there are already quite some issues open [4] and I am supervising several Summer of Bitcoin projects [5] that I expect to contribute to the repository over the next couple of weeks in various ways. You can easily install the **pre-alpha** package (version 0.0.0) with the PythonPackageIndex via: :~$ pip install pickhardtpayments Example code that shows how to use this library can be found in the `Readme.md` file and the `example` folder [6]. Unfortunately my presentation at MIT Bitcoin Expo where I first talked about this is still buried in a long Livestream video (Day 2 Track A starts at 1:16:19) and is not yet available as a stand alone video but I will maintain a `Resources.md` file where I link to useful talks & articles [7] and other resources related to the topic. One final and very significant way to speed up the computation of the min cost flow solver that I have included since my last mail [3] was achieved by pruning the network to remove edges that are highly unlikely to be part of the payment flow. The current solution is pretty ad-hoc but I am working on doing this in a more automated / reliable way[8]. In this way I am very pleased to report that as of now and with the given size of the zeroBaseFee subnet... **...The min cost flow solver consistently takes less that 100ms to find close to optimal flows!** That being said: Please be aware that the entire runtime of the provided code is currently much higher. The reason is that for mere convenience I have used the very slow `networkx` library that is used to store the ChannelGraph, UncertaintyNetwork and OracleLightningNetwork with a lot of memory overhead to maintain those data structures and to copy the necessary data to the min cost flow solver before starting the actual solver (c.f. [9]). For safety reasons I deliberately did not provide an API to do mainnet tests. However to the fellow expert user - who knows what they are doing - it should be fairly straight forward to conduct mainnet experiments via the following two steps: 1.) Create your own wrapper to an Lightning Node that exposes the `send_onion` call [10] in the same way how it is currently being done in the `OracleLightingNetwok` class and bring your own wrapper to the `SyncSimulatedPaymentSession` class instead of the oracle that we have 2.) You may wish to make the payment loop [11] inside the `SyncSimulatedPaymentSession` async. Of course delivering production ready mainnet code will require more work as one needs to figure out other constraints like channel reserves, min/max htlc sizes, available HTLC slots, offline peers, hanging htlcs (stuck payments),... Also if you want to do simulations with non uniform distributions of liquidity or because your LSP has a crawled snapshot of actual Liquidity you can just bring your own `OracleLightningNetwork` that encodes your assumed or known ground truth about the network. I hope all of this makes sense. Feel free to ask me anything that you need and I am curious to see if this will be useful for you. Note that there is a notebook that has basically the same code but more documentation and in particular a glossary of terms that can currently be found at [12]. Thanks to all people that I have been collaborating with so far and to NTNU, BitMEX, various anonymous donors and Patreons who all helped to achieve this. with kind regards Rene Pickhardt [0]: https://github.com/renepickhardt/pickhardtpayments [1]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-March/002984.html [2]: https://arxiv.org/abs/2107.05322 [3]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-March/003510.html [4]: https://github.com/renepickhardt/pickhardtpayments/issues [5]: https://www.summerofbitcoin.org/project-ideas-details?recordId=recJchpFa9tqSZkQ4 [6]: https://github.com/renepickhardt/pickhardtpayments/tree/main/examples [7]: https://github.com/renepickhardt/pickhardtpayments/blob/main/Resources.md [8]: https://github.com/renepickhardt/pickhardtpayments/issues/1 [9]: https://github.com/renepickhardt/pickhardtpayments/issues/6 [10]: https://github.com/renepickhardt/pickhardtpayments/blob/1121b48ec6bf5fb2dee2b1793f87d489ce3149e3/pickhardtpayments/OracleLightningNetwork.py#L37 [11]: https://github.com/renepickhardt/pickhardtpayments/blob/1121b48ec6bf5fb2dee2b1793f87d489ce3149e3/pickhardtpayments/SyncSimulatedPaymentSession.py#L351 [12]: https://github.com/renepickhardt/mpp-splitter/blob/pickhardt-payments-simulation-dev/Pickhardt-Payments-Simulation.ipynb (includes a glossary of terms and many explainations) -- https://ln.rene-pickhardt.de <https://ln.rene-pickhardt.de>
_______________________________________________ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
