Re: [Lightning-dev] Improving the initial gossip sync

[Fabrice Drouin]

Hi,

On 5 February 2018 at 14:02, Christian Decker
 wrote:
> Hi everyone
>
> The feature bit is even, meaning that it is required from the peer,
> since we extend the `init` message itself, and a peer that does not
> support this feature would be unable to parse any future extensions to
> the `init` message. Alternatively we could create a new
> `set_gossip_timestamp` message that is only sent if both endpoints
> support this proposal, but that could result in duplicate messages being
> delivered between the `init` and the `set_gossip_timestamp` message and
> it'd require additional messages.

We chose the other aproach and propose to use an optional feature

> The reason I'm using timestamp and not the blockheight in the short
> channel ID is that we already use the timestamp for pruning. In the
> blockheight based timestamp we might ignore channels that were created,
> then not announced or forgotten, and then later came back and are now
> stable.

Just to be clear, you propose to use the timestamp of the most recent
channel updates to filter
the associated channel announcements ?

> I hope this rather simple proposal is sufficient to fix the short-term
> issues we are facing with the initial sync, while we wait for a real
> sync protocol. It is definitely not meant to allow perfect
> synchronization of the topology between peers, but then again I don't
> believe that is strictly necessary to make the routing successful.
>
> Please let me know what you think, and I'd love to discuss Pierre's
> proposal as well.
>
> Cheers,
> Christian

Our idea is to group channel announcements by "buckets", create a
filter for each bucket, exchange and use them to filter out channel
announcements.

We would add a new `use_channel_announcement_filters` optional feature
bit (7 for example), and a new `channel_announcement_filters` message.

When a node that supports channel announcement filters receives an
`init` message with the `use_channel_announcement_filters` bit set, it
sends back its channel filters.

When a node that supports channel announcement filters receives
a`channel_announcement_filters` message, it uses it to filter channel
announcements (and, implicitly ,channel updates) before sending them.

The filters we have in mind are simple:
- Sort announcements by short channel id
- Compute a marker height, which is `144 * ((now - 7 * 144) / 144)`
(we round to multiples of 144 to make sync easier)
- Group channel announcements that were created before this marker by
groups of 144 blocks
- Group channel announcements that were created after this marker by
groups of 1 block
- For each group, sort and concatenate all channel announcements short
channel ids and hash the result (we could use sha256, or the first 16
bytes of the sha256 hash)

The new `channel_announcement_filters` would then be a list of
(height, hash) pairs ordered by increasing heights.

This implies that implementation can easily sort announcements by
short channel id, which should not be very difficult.
An additional step could be to send all short channel ids for all
groups for which the group hash did not match. Alternatively we could
use smarter filters

The use case we have in mind is mobile nodes, or more generally nodes
which are often offline and need to resync very often.

Cheers,
Fabrice
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

Re: [Lightning-dev] An Idea to Improve Connectivity of the Graph

[Christian Decker]

I'd also like to point out that the way we do state invalidations in
Lightning is not really suited for multi-party negotiations beyond 2
parties. The number of potential reactions to a party cheating grows
exponentially in the number of parties in the contract, which is the
reason the Channel Factories paper relies on the Duplex Micropayment
Channel construction instead of the retaliation construction in LN.

Furthermore I'm also not exactly clear how we could retaliate
misbehavior on one channel in the other channel if they are logically
independent. Without this you could potentially re-allocate your funds
to another channel and then attempt to cheat, without it costing your
funds.

Cheers,
Christian

ZmnSCPxj via Lightning-dev 
writes:

> Good morning Abhishek Sharma,
>
> While the goal of the idea is good, can you provide more details on the 
> Bitcoin transactions?  Presumably the on-chain anchor is a 3-of-3 multisig 
> UTXO, what is the transaction that spends that?  What do Lightning commitment 
> transactions spend?  Can you draw a graph of transaction chains that ensure 
> correct operation of this idea?
>
> Have you seen Burchert-Decker-Wattenhofer Channel Factories? 
> https://www.tik.ee.ethz.ch/file/a20a865ce40d40c8f942cf206a7cba96/Scalable_Funding_Of_Blockchain_Micropayment_Networks%20(1).pdf
>  What is the difference between your idea and the Burchert-Decker-Wattenhofer 
> Channel Factories?
>
> Regards,
> ZmnSCPxj
>
> Sent with [ProtonMail](https://protonmail.com) Secure Email.
>
>  Original Message 
> On February 4, 2018 6:21 PM, Abhishek Sharma  wrote:
>
>> Hello all,
>> I am not sure if this is the right place for this, but I have been thinking 
>> about the lightning network and how it could be modified so that fewer total 
>> channels would need to be open. I had the idea for a specific kind of 
>> transaction, in which three parties commit their funds all at once, and are 
>> able to move their funds between the three open channels between them. I 
>> will give a rough overview of my idea and give an example that I think 
>> illustrates how it could improve users' ability to route their transactions.
>>
>> Say that three parties, A, B, and C, create a special commitment transaction 
>> on the network that creates three open channels between each of them with a 
>> pre-specified balance in each channel. Now, these channels would be 
>> lightning network channels, and so the three of them could transact with 
>> each other and modify balances in their individual channels at will. 
>> However, this special agreement between the three of them also has the 
>> property than they can move their funds between channels, provided they have 
>> the permission of the counterparty to the channel they move their funds 
>> from, and then presents this to the other counterparty to show that funds 
>> have been moved.
>>
>> 1.) A, B, and C each create a commitment transaction, committing .5 BTC (3 
>> BTC in total) on their end of each of their channels.
>> 2.) A, B, and C transact normally using the lightning protocol. After some 
>> amount of time, the channel balances are as follows:
>> channel AB: A - 0.75, B - 0.25
>> channel BC: B - 0.4, C - 0.6,
>> channel AC: A - 0, C: 1.0
>> 3.) A would like to send .5 BTC to C, however she does not have enough funds 
>> in that channel to do so. It's also not possible for her to route her 
>> transaction through B, as B only has .4 in his channel with C. However, she 
>> does have those funds in her channel with B, and so asks for B's permission 
>> (in the form of a signed balance state that includes the hash of the 
>> previous balance), to move those funds over to her account with C. She gets 
>> this signed slip from B, and then presents it to C.
>> 4.) A, B, and C continue trading on their update balances.
>> 5.) When they wish to close out their channels, they all post the last 
>> signed balance statements each of them has.
>> Say, for example, A and B were to collude and trade on their old balance (of 
>> .75 and .25) after Bsigning the statement that A was 'moving' funds to C. If 
>> A and C were trading on their new balances, C has proof of both A and B's 
>> collusion, and she can present the signed slip which said that A was moving 
>> funds to AC and so the total balance on A and B's channel should've summed 
>> to 0.5. In this event, All funds in all three channels are forfeited to C.
>>
>> I believe this works because, in virtue of being able to make inferences 
>> based on her own channel balances, C always knows (if she is following the 
>> protocol) exactly how much should be in channel AB. and can prove this. If 
>> there were 4 parties, C couldn't prove on her own that some set of parties 
>> colluded to trade on an old balance.
>>
>> Now, I'll show why such a mechanism can be useful.
>> Now, assume that there are parties A, B, C, D, and E, and the following 

Re: [Lightning-dev] Manual channel funding

[Christian Decker]

Hi Alex,

not sure what the context of your question. It doesn't appear to be
protocol related, but rather an issue with the interface that the
implementations expose. If that is the case, I'd suggest filing an issue
with the respective implementation.

Cheers,
Christian

Alex P  writes:
> Hello!
>
> At the moment there is no option to choose outputs to fund channel
> manually. Moreover, there is no way to fund channel with "all available
> funds". That's weird, I set up a channel and tried to use "all I ave",
> and got is a transaction on blockchain with the output for 980 SAT:
>
> https://chain.so/tx/BTC/bc144507a85900d0fc0318cc54a4bcb29542bfcd543e7acf9f00061f03c997e5
>
> To my opinions at least there should be an option "take fee from funding
> amount", and may be an option to choose exact outputs to spend.
>
> Any ideas?
>
>
> ___
> Lightning-dev mailing list
> Lightning-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] Improving the initial gossip sync

[Christian Decker]

Hi everyone

When we started drafting the specification we decided to postpone the
topology syncronization mechanism until we have a better picture of the
kind of loads that are to be expected in the network, e.g., churn and
update rate, and instead implement a trivial gossip protocol to
distribute the topology updates. This includes the dreaded initial
synchonization dump that has caused some issues lately to all
implementations, given that we dump several thousands of updates, that
may require block metadata (short channel ID to txid conversion) lookup
and a UTXO lookup (is this channel still active?).

During the last call we decided to go for an incremental improvement,
rather than a full synchronization mechanism (IBLT, rsync, ...). So
let's discuss how that improvement could look like.

In the following I'll describe a very simple extension based on a
highwater mark for updates, and I think Pierre has a good proposal of
his own, that I'll let him explain.

We already have the `initial_routing_sync` feature bit, which (if
implemented) allows disabling the initial gossip synchronization, and
onyl forwarding newly received gossip messages. I propose adding a new
feature bit (6, i.e., bitmask 0x40) indicating that the `init` message
is extended with a u32 `gossip_timestamp`, interpreted as a UNIX
timestamp. The `gossip_timestamp` is the lowest `channel_update` and
`node_announcement` timestamp the recipient is supposed to send, any
older update or announcement is to be skipped. This allows the `init`
sender to specify how far back the initial synchronization should go.

The logic to forward announcements thus follows this outline:

 - Set `gossip_timestamp` for this peer
 - Iterate through all `channel_update`s that have a timestamp that is
   newer than the `gossip_timestamp` (skipping replaced ones as per BOLT
   07)
 - For each `channel_update` fetch the corresponding
   `channel_announcement` and the endpoints `node_announcement`.
 - Forward the messages in the correct order, i.e.,
 - `channel_announcement`, then `channel_update`, and then `node_announcement`

The feature bit is even, meaning that it is required from the peer,
since we extend the `init` message itself, and a peer that does not
support this feature would be unable to parse any future extensions to
the `init` message. Alternatively we could create a new
`set_gossip_timestamp` message that is only sent if both endpoints
support this proposal, but that could result in duplicate messages being
delivered between the `init` and the `set_gossip_timestamp` message and
it'd require additional messages.

`gossip_timestamp` is rather flexible, since it allows the sender to
specify its most recent update if it believes it is completely caught
up, or send a slightly older timestamp to have some overlap for
currently broadcasting updates, or send the timestamp the node was last
connected with the network, in the case of prolonged downtime.

The reason I'm using timestamp and not the blockheight in the short
channel ID is that we already use the timestamp for pruning. In the
blockheight based timestamp we might ignore channels that were created,
then not announced or forgotten, and then later came back and are now
stable.

I hope this rather simple proposal is sufficient to fix the short-term
issues we are facing with the initial sync, while we wait for a real
sync protocol. It is definitely not meant to allow perfect
synchronization of the topology between peers, but then again I don't
believe that is strictly necessary to make the routing successful.

Please let me know what you think, and I'd love to discuss Pierre's
proposal as well.

Cheers,
Christian
___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

[Lightning-dev] Manual channel funding

[Alex P]

Hello!

At the moment there is no option to choose outputs to fund channel
manually. Moreover, there is no way to fund channel with "all available
funds". That's weird, I set up a channel and tried to use "all I ave",
and got is a transaction on blockchain with the output for 980 SAT:

https://chain.so/tx/BTC/bc144507a85900d0fc0318cc54a4bcb29542bfcd543e7acf9f00061f03c997e5

To my opinions at least there should be an option "take fee from funding
amount", and may be an option to choose exact outputs to spend.

Any ideas?


___
Lightning-dev mailing list
Lightning-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev