Privacy and robustness in Lightning
Speakers: Rusty Russell
Date: August 9, 2023
Transcript By: kouloumos via tstbtc v1.0.0 --needs-review
Tags: Lightning, Eltoo, Privacy enhancements
Category: Podcast
Speaker 0: 00:00:00
Simplicity is a virtue in itself, right? So you can try to increase fairness. If you reduce simplicity, you end up actually often stepping backwards. So I really, the thing that really appeals about element symmetry is it’s simple. And every scheme to fix things by introducing more fairness and try to reintroduce penalties and everything else ends up destroying the simplicity.
Speaker 1: 00:00:27
So soon again.
Speaker 2: 00:00:28
Yes. For those of you that aren’t tracking us in real time, we are getting a bunch recorded today. So had Rusty Russell in the office.
Speaker 1: 00:00:39
Yeah. So this is one of my favorite podcast guests to listen to, and I’m very happy that- Why? Because he has the capacity to cut through a lot of the noise and get to the crux of things, say them as they are. Ugly truth, sort of. Very nice for engineers to actually talk about the elephants in the room. So I’m expecting that we’ll hear a little bit about what’s still broken and lightning where the ship is sailing. Okay, here come the elephants. Welcome,
Speaker 2: 00:01:21
Rusty.
Speaker 0: 00:01:22
It’s great to have you here. Thank you. I am fresh off the boat. I literally arrived less than 24 hours before from the other side of the world. So I hope this will be coherent.
Speaker 2: 00:01:30
Our budget is for this podcast is incredible. We fly people from all over the world to this office.
Speaker 0: 00:01:36
I must say that private jet was lovely. Thank you for that.
Speaker 2: 00:01:39
It was delightful. You are here for the lightning summit. Let’s talk about lighting. Just sort of just get into it.
Speaker 0: 00:01:46
Let’s do that. Yeah. Okay.
Speaker 2: 00:01:47
Tell us what you think this group should be thinking about as you’re as you’re gathering together. There’s going to be specific conversations that happen about specific proposals. But I guess more generally, as lightning has grown up, What are the kinds of things that you’re thinking about and concerned about?
Speaker 0: 00:02:04
Yeah, okay. That’s a fair question. There are obviously specific things that we’re going to talk about at the summit. Definite concrete proposals and that’s all great. This is always a fantastic time for everyone to come together. I think if we step back a bit, This is like eight years now for the Lightning Network. So we’ve been doing this for a while. And there are a couple of things that I think we really need to make sure we keep focusing on because it’s easy to get in the weeds of specific things and I would love to go into those and we can totally go there. But I think robustness and privacy are the two things that are rarely actually on fire, but are always important. And they’re both areas where we can definitely make incremental improvements, and I think we should. Privacy in particular is something that everyone cares about until you need to put the engineering effort in and there’s other stuff. There’s bugs, there’s issues, there’s, you know, there are always other things that are more urgent, even if they’re not more important. And so I definitely try to keep the perspective of going, you know, if not now, when? And I think, you know, privacy is something that we definitely need to put more effort in.
Speaker 2: 00:03:07
Yeah, I imagine that privacy goes to the end of the line pretty much every time. But can we talk about robustness for a moment? Because robustness is definitely something that you think about in Bitcoin quite a bit. It’s different though, in terms of layer one versus layer two, because layer two sort of promises to be this move fast and break things approach, because it needs to move fast and it needs to grow and it needs to catch up. Now things have broken, but I guess not to the same extent that I think people are trusting it. I mean, there’s serious money on the line.
Speaker 0: 00:03:42
Oh yeah.
Speaker 2: 00:03:43
And I guess the question is, do you have to react to the amount of money that’s in the system? Or is it still, well, we’re still figuring this thing out. And that’s up to you if you want to put that much money in there. But we, you know, we still need to experiment. And again, it is a robustness speed trade off in terms of what you can do. Yep.
Speaker 0: 00:04:07
So, okay, so robustness on some extent is always something we need to think about. But also, I think like the recent fee spike showed us that there are definitely places we can improve. Robustness from the protocol level means we want stuff where we become more immune to fee issues. I think that’s something definitely, you know, we need longer term. The assumption is fees are going to go up, they’re going to get more volatile, we’re going to have these spikes, We need to handle that really well. And I certainly hope that we, you know, we have a number of tricks we can do. Some of which is like, well, we want some soft forks, right?
LN symmetry
Speaker 0: 00:04:39
If we can get L and symmetry, which used to be called L2, but L and symmetry is a much better name. Sorry to Christian who named L2. He blames me. I blame him. But that, for example, sidesteps a whole heap of, it reduces robustness by introducing simplicity. It does make it a lot simpler to maintain your Lightning channel. There’s no longer toxic waste that you have to worry about, old states and stuff like that. Watchtower has become now almost trivial. So that adds some simplicity.
Speaker 2: 00:05:05
Can we take a quick side route on that? Sure. Because we’ve talked about LN symmetry specifically with Instagibs.
Do we want LN Symmetry to be symmetrical?
Speaker 2: 00:05:12
But I’d be interested in getting your take on, given that asymmetry seems to be healthy for the network, do you want it to be exactly symmetrical?
Speaker 0: 00:05:21
Okay, so this is a good question. So at the moment, let’s step back. LN penalty, as it’s retronamed into, is the current scheme where basically if you screw up, I take your money.
Speaker 1: 00:05:31
Yeah,
Speaker 2: 00:05:31
Tadge is over there, too. We can pull him in.
Speaker 0: 00:05:33
Yeah, okay. We can do that. Right now, so what we found in practice is that usually what happens is you screw up, I take your money, and I feel really sorry about it, and I try to give it back to you. Right? So, the current scheme is actually a little bit too harsh for the world that we actually live in, because we rarely see people actually trying to cheat, and we often see them do it accidentally because they’re a store from backups and oops, that was bad, right? With the LN symmetry approach, simply if you screw up, I just fix up, right? There’s no massive penalty for you doing so. The danger is then, have I created an incentive problem where you might try to cheat because it doesn’t cost you anything? Now, this is interesting in a couple of ways. One is, is that actually always more expensive for you to close unilaterally, even under LN symmetry, than it is to do a mutual close. But you talk to me and we go forward and we produce a mutual close transaction that’s got a tailored fee rate and everything else and it’s smaller. So if you can talk to me, you’re going to want to do that. But if I was unreachable, well, I might as well try to spend in an older state. So the cost of an attack has dropped significantly because that risk issue. Well, you’re paying some fees for it, sure, but you’re not risking that if you screw up, you’re going to lose all your funds. But on the other side of the coin, my implementation is much simpler and watchtowers are now trivial. It’s so much so that we can have many watchtowers looking out for these kind of things. They only have to remember one transaction, which is the last one. They see it, they drop it to chain, we’re done. So the fact that watchtowers are much more scalable means that while your risk is reduced as an attacker, your probability of success is reduced even more. So there is less of an incentive problem than you might have originally thought with going for LN symmetry.
Speaker 1: 00:07:09
But it still costs you the same to broadcast in the old state or the newest state, so you don’t lose anything, right?
Speaker 0: 00:07:16
Yes. But on the simple protocol level, you go, well, you didn’t lose anything. But on a higher level, I’m not going to trust you again because you’re obviously doing weird because they restored from backup. But you know, if you start doing this a lot, people are gonna be like, no, no, I don’t trust them anymore. So, so if the potential benefit for you is so small, that that social convention that that, you know, if I want to become a big node, I’m not going to get there because I people have noticed that I keep double spending because it’s really clear on chain.
Speaker 2: 00:07:47
Yeah, but that also changes with high fee environments, right? I mean, you start making it more difficult for people to…
Speaker 0: 00:07:55
Ah, but now, remember, in a high fee environment, the cost for you to do a unilateral close is much higher. I see. Right? You want to do a mutual close. So the only case where it really comes out is when I’m unavailable you can’t do mutual close. You’re gonna use the last three close anyway and now you’re like well I might as well try to cheat. So you’re already kind of in a corner case because that’s already sending more money than if you just spoke to me and closed your channel.
Speaker 1: 00:08:16
So we’ve talked quite a bit about LN symmetry and how we want to retain the symmetric aspect of it. There was also a proposal where you would have some sort of penalty for the initiator of the closing transaction by having a larger output on the commitment transaction than input. So you would have to add a second input to provide additional funds. Could we talk about that proposal?
Speaker 0: 00:08:42
Simplicity is a virtue in itself, right? So you can try to increase fairness. If you reduce simplicity, you end up actually often stepping backwards. So the thing that really appeals about element symmetry is it’s simple. And every scheme to fix things by introducing more fairness and try to reintroduce penalties and everything else ends up destroying the simplicity that, for me, is like, well, it depends on the ratio of attacks, right? At some point, how much engineering, how much effort is going in to save a few cents statistically on your attack probability. And so I’ve come around to the idea that making watchtowers ubiquitous is enough by itself to then go for a very simple and dumb protocol that at the end of the day will be far simpler to implement and have fewer corner cases and those things than trying to get fancy and bring back everything. Because then we end up back at basically somewhere like LN Penalty and I’m like, well no, we didn’t like that. You know, there’s definitely like, I’m not completely ruling out the design space, but I do like the simplicity of going, well, you’ve got one transaction, you spend it, you’re done. It’s nice. You know, and whoever spends it ends up having to fund the close. That’s pretty nice too. The other argument for breaking symmetry is you can then tell who screwed up. If everything’s symmetrical, I can make it look like you screwed up by just spending the n minus one transaction the nth transaction and then go, you spent that one and actually it was me. There’s some cost to doing that, I’m going to have to pay fees on both of them. But no one can canonically assign blame for that kind of screw up. So you start to go, well, maybe we do make the mason metric but then you’re like yeah you’re reintroducing complexity and the real appeal is simplicity and simplicity is often underrated as far as going you know there’s enough moving parts in lightning as we’ve discovered without making the base protocol. So I find the simplicity more compelling I think now than I did when the first idea was introduced. So I’ve definitely come along. The good news is that we need a soft fork, we need to roll this thing out, it’s not going to happen overnight, we’re going to have both side by side, and we will find the engineering tradeoffs, to be honest. If I’m wrong and we end up going, no, actually, we want some hybrid scheme, or we prefer LN penalty or everything else, but we’ll get there as well. There’s no huge hurry in this process.
Speaker 1: 00:11:02
Of course, people that still want a penalty can just continue to use ln penalty.
Speaker 0: 00:11:07
No, we’ll rip the damn thing out as soon as we can. I mean, the simplicity argument only applies if you don’t have to support both. Now, to be fair, you can migrate from LN penalty to LN symmetry, but you will have to handle a case where the other person goes back and spends one of those ones that needs a penalty, because you can’t symmetry that afterwards. I can’t come up with any scheme where you can just retro in and attach on. It’s theoretically possible, but it’s not actually possible as far as I can tell. So you will need to still be able to have your penalty code, even if you no longer open new ones. If you’ve got existing channels that have not been spliced or otherwise had some kind of on-chain commitment, you will still need to have the code that handles, oh crap, they spent the old one, I now have to do a penalty thing or whatever else. It’s not a huge amount of code, it’s not as complicated as some of the other parts, but it will still be there until the point where you go, no. Everyone has all those channels are dead. We’ve got all new channels. At that point, you would be able to get rid of the whole concept entirely. But this is the benefit of having different implementations and a simple protocol where a new implementation come along and go, I want to write it in whatever the language flavor of the week is, there’s always somebody who’s prepared to rewrite it. So by having a simple protocol, they may go, actually, I’m not going to implement all that crap. We’re only going to implement the new one. And that may well be what carries us forward. I don’t know.
Speaker 1: 00:12:29
And it might actually, it makes it way easier to have more than two parties, right? So it might be the roads to channel factories or you can still share your schemes.
Speaker 0: 00:12:40
So yeah, your point about opening the door to channel factories and multi-party channels and all that is definitely a huge impetus to do LN Symmetry because we can go multiparty, we can do all these wild and wonderful things on top. So I’m like, oh, we simplify it and then we add all new complexity by building another layer on top. But, you know, that’s all good because I’m hoping that that won’t be my problem and somebody else can do the implementation on top.
Speaker 2: 00:13:04
So we’ve covered robustness. Are there other things? Yeah, I mean, what else do you have in mind when you’re talking about that principle?
Speaker 0: 00:13:12
So at the moment, we have a fairly complicated state machine with just the way that we deal with updates. And when Greg implemented L2, he did roll in the simplified update proposal that I made a couple years ago now, where basically instead of both of us proposing changes at the same time, So the basic way our peer-to-peer protocol works is that I go, okay, add this, add this, add this, resolve this, resolve this.
Peer to peer protocol
Speaker 0: 00:13:37
At the same time, you can be proposing changes and we commit to each other and we kind of roll this thing in and eventually it’s okay. That protocol was something that is optimal, but also introduces a significant amount of complexity. And it turns out that if you just take turns in adding updates, then it’s a lot easier to understand the protocol. It is theoretically slightly higher latency in the case where we both want to make changes because you end up having to wait for me and then it’s your turn. But in practice, while that adds some latency in that corner case, it also gives you more opportunities for batching. So if you’re really being hammered in both ways, you don’t lose. Right? The latency of individual payments may go up, but the latency of the whole system is better because we end up just you end up doing five commitments at once because you’ve been waiting. So with that in mind, I think simplifying the pure level protocol can help us somewhat. And in fact, that simplification leads to it’s a subset of the current state machine, in fact, so it turns out to be pretty nice. And once we’ve got that, we have then the ability to knack changes. So at the moment, I expressed to your node, there are certain things I will not allow. Here’s your maximum of HTLCs, here’s your maximum HTLC size, all these restrictions because there’s no way in the protocol for you to send an update and me go, no, no, no, I don’t like that one. We have to go through a whole round and then I have to close it out. At some point, I’m left holding the commitment transaction with that thing that you put in. I have to tell you what you can’t do. If we have a NAC protocol where you’re going to know I’m going to reject that and force you to go around again, then that means that I no longer have to tell you what my restrictions are. You can try everything and I just go no that doesn’t work and close it out. Right. It’s a fairly simple protocol change But if we tried to do it in the existing protocol where we can have change from both sides in flight, it’s a bit of a nightmare. Whereas when we’ve got this turn taking protocol, it’s much, much easier. The other thing that happens with the turn taking protocol is if we want to do some kind of upgrade to the channel, we want to splice or we want to upgrade to a new kind of channel. It turns out with the current protocol, we have to have a whole quiescence protocol to go, hold on, I want you to stop sending updates. You go, cool, I’m ready to stop sending updates. We clear all the updates and then we’re ready to go. With turn taking, it happens automatically. When it’s my turn, by definition, you haven’t got anything in flight. So I can say, hey, it’s time to upgrade. And it’s much simpler in that way too. So this is something that we just learned through experience that we over-designed the original protocol. And now whether this becomes a separate change or goes in with an L2 change or not, this is something we’re going to discuss at the summit. But from my point of view, we end up with a more robust and simple protocol at the end of this. And it turns out that supporting both is actually pretty easy.
Speaker 2: 00:16:10
So you’re talking about when we were talking about L2, you’re talking about changes and you’re sort of casually mentioning the soft fork piece of it. I mean, there’s not going well, there’s Greg Sanders will be there and he’s obviously thinking about this, but you don’t really have the people at the table that are actually doing like would be pulling the soft fork together that are that really understand the lightning case, I guess. Maybe AJ, maybe InstaGibs. The list gets very short after that. So, like, how are you able to communicate those needs down to the lower layer?
Speaker 0: 00:16:47
Well, I think the any prep out cases have been pretty strong for a long time. The question’s always been, do we want other soft fork as well?
CTV
Speaker 0: 00:16:54
And people are like, oh, if we had obstacle the check sick from stack, we can also do it. Sure. I can probably build a car out of matchsticks and snot, but I don’t want to. And don’t make me do it.
Speaker 2: 00:17:09
Could you do that?
Speaker 0: 00:17:12
A theoretical engineer probably could do that. I could not do that. I’m not crafty at all. But, okay. That is probably an aside. We don’t need to go down. But, you know, check sync from stack is like, oh, it’s cool that you can even make that work. But it is a terrible basis for just about anything. And it’s more like a game we could play as far as what’s the most obscure way we could do some kind of introspection. And the answer would be, you know, check-sig from stack. So any prep-out is a – and the proposals that are in that cluster, and there are a few, are pretty clean, pretty well understood. AJ would definitely be the person unfortunately he’s not here in New York he’s kind of you know done a fair bit of travel recently he was like no I’m not going. So I’m hoping that while we’re in New York he is in his mountain lair somewhere coming up with a scheme to activate any prep out. That’s if you’re listening, AJ, I hope you’re going to make that prophecy come true. I think, you know, we understand that we want this and we’ve wanted it for a fair period of time and nothing else has come up that’s gone, oh no, we want this instead. Anyprepout by itself is a fairly simple piece.
Speaker 2: 00:18:21
Now. Not a precluding piece to other things that we can do in the future.
Speaker 0: 00:18:24
Absolutely.
Speaker 2: 00:18:25
This is the two-way door, one-way door kind of thing.
Speaker 0: 00:18:28
In this case, we will want anyprepout even when we have full covenants and everything else. So I’m pretty confident to say this is something that we should enable. Right. Now that said the detail the fine details you know hand wave hand wave I know try not to spend too much time coming over details but I’m pretty happy that you know somebody will come up with a proposal. We will vet it and make sure that it works. And away we go. Greg Sanders, of course, will be the person to talk to. He’s been doing both sides. I know he and AJ are all over this. I trust them to weed through all the different possibilities, come out with a proposal, and then we’ll tear it apart and then go again and do it properly.
Speaker 1: 00:19:07
So Yeah, I believe it’s live in inquisition already.
Speaker 0: 00:19:11
Yeah, exactly. So, you know, I think we’re, you know, we’re well on the way to getting that. Now, just to set expectations correctly, even if we saw for tomorrow and it activated the week after, you know, there’s a leer latency involved here, right? We still need to then go through and implement it, roll it out, you know, it gets it gets spec’d out, it gets, you know, packed together, then it gets, then we ratify it, We get two different independent implementations both to implement it and interoperate and make sure that works. Then it’s spec final. Then everyone else can go ahead and implement it. Then it can roll out on the network. Then everyone can upgrade. Then once both your peers are upgraded, then now you can open a new channel. A new channel will be one of these cool, you know, L2 channels and everything else. So, you know, this is a long road, right?
Speaker 1: 00:19:57
Yeah, we were just talking with El and Oli, how one and a half years after Taproot’s been active, simple Taproot channels are close to being spec’d out.
Speaker 0: 00:20:07
Yes, exactly.
Speaker 1: 00:20:08
We know what you mean.
Speaker 0: 00:20:10
And the other thing is that, look, it enables simplicity, it makes things a lot nicer for us, and it makes it cleaner, but not immediately. At the moment, it’s just work. Because we don’t get to drop anything that we’re currently doing. Now, eventually, sure, we do. At the end of the road, there’s this beautiful rainbow and there’s unicorns dancing and everything else. But we have to trade through a lot of unicorn crap.
Pushing privacy and robustness to the front of the line
Speaker 2: 00:20:35
This goes back to the open, which is privacy and robustness. They always go to the end of the line. So if you’re never prioritizing them, you’re never prioritizing those things which never get attention. It’s sort of the equivalent of like, it’s not tech debt, it’s like philosophical debt. And so, yeah, how do you push those to the front of the line and put them front and center? Is it because they can come in the form of new features that it’s time for? Like, yeah, how do you push those forward?
Speaker 0: 00:21:07
How do you push those forward? You have that internal compass of, you need to spend a certain amount of your time on stuff that’s literally broken on fire and everything else. You need to spend a certain amount of time on other things. As an engineer, this is the eternal battle between, you know, so just look at me personally, right? So I lead the core lightning team. There’s a certain amount of just dealing with time sheets and just doing random managerial overhead that I have to do. There’s a certain amount of like peering on random podcasts to talk about stuff. There’s a certain amount of actual coding that I have to do on core lightning features that people want because, you know, scalability, whatever thing it is, right? Oh, well, this is a cool new feature. We should do that. And then there’s the protocol work as well right so there’s always more things to do well in any fun job like working on this there’s always more things to do than you necessarily have time for you have to prioritize so to answer your question how do you do that And the answer is that you have to reserve a certain amount of time for long-term work. You just have to. And you have to be aware through experience that if you do not explicitly reinvent some of your time, It’ll never be a good time. Right.
The dynamics of developing a spec with commercially associated implementations
Speaker 2: 00:22:18
So, but okay, I have two, I have two things to push back on. One is LDK is a little bit of a maybe a question mark for me, but the others are three commercially driven implementations.
Speaker 0: 00:22:30
I didn’t realize we were commercially driven. Okay, good. I like that.
Speaker 2: 00:22:35
And so, well, they have to be, I mean, they’re companies, if the companies don’t exist, do these implementations exist, I guess? So that is a very good philosophical question.
Speaker 0: 00:22:45
And hey, I work for a startup, right? And you should always be thinking what happens if my startup collapses because almost by definition right that’s the the endgame for most startups is like they fail right and so when you have an open source project and you’re like well I’m running this what what happens what happens after this to some extent the answer is you know I’m gonna keep working on this whatever happens unfortunately my wife has given explicit approval for me to take some time off and work. Should all of us fail? We’ve had this discussion, right? She’s been like, cool, that’s okay. You can go do that. At least for a while before you have to get a real job again. So I feel fairly confident to go, you know, we’re going to move forward. And maybe not as fast as we are at the moment. But, you know, there’s some future there.
Speaker 2: 00:23:24
Now, I guess my other bit of that was Because they’re commercially associated, maybe not driven, commercially associated. That means that features get pushed to the front of the line. Because I’ve worked for companies, I know how it works. So when you’re talking about, you know, ring fencing things and making sure that we’re, you know, thinking long term. These are different than just like building a product that’s, you know, in JavaScript and you know, can wow an investor. There is a philosophical side to what’s happening here. And there is a collaboration side of no implementation can do it by themselves.
Speaker 0: 00:24:07
Yes.
Speaker 2: 00:24:07
And so those dynamics just don’t seem to add up to a, a long term thinking and long term, you know, moving, moving the ball forward in these kinds of dimensions.
Speaker 0: 00:24:21
So the piece that you’re missing here is the pie is still growing, right? And so there is more emphasis on growing that pie and doing those bold new things because the world is still in front of us right light network as it is in 10 years time is much bigger than light network today with that assumption how do you lead how do you become the one everyone wants to be and the answer is you do the cool new things. And that’s not so much features. It is that focus on the next, basically the light in 2.0. What are the cool new things coming down the pipe? Is it L2? Is it L-unsymmetry? Have you already got those, right? Is it taproot channels? Like look at LNDs, you know, huge amount of resources they’re putting in to doing that. Now, you know, a lot of that is commercially driven. Why? Because it gives them cool points and it makes the rest of us sweat. And we have to then catch up, right? This is always the way. One of the implementations does something really cool. And I’m going, on the one hand, I’m going, wow, that’s amazing. On the hand, I’m going, shit. Now we have to do that. Right? So it is, there is definitely a commercial reason to do these things. Now, is it theoretically possible that you could avoid doing any real work and just concentrate on, you know, adding features to your own implementation. I think it’s too early. At the moment, you can’t do that because no one else is doing that and you will get left behind. There are four implementations out there and there is choice in what you can run. So everyone is sweating at the moment a little bit. There’s something of a treadmill effect.
Speaker 2: 00:25:51
My other side of that.
Speaker 1: 00:25:54
Isn’t it five, meanwhile, with Electrum also in the mix now?
Speaker 0: 00:25:58
Electrum has been in the mix for a while, but we haven’t really seen them pushing the boundaries of the spec process. They’re quite happy to kind of follow along and meet the requirements.
Speaker 1: 00:26:08
Okay, so you’re talking about four that are participating in the spec.
Speaker 0: 00:26:12
There are three other teams I can hit up when I want someone else to interoperate with on some new feature who are really going to drive it forward. And those are really reliable.
Expecting new implementations
Speaker 0: 00:26:21
So there’s certainly other implementations to come up. And I actually expect to some extent that LN symmetry may open some new opportunities because people go, actually, I can implement that now. That is a serious subset. If I just implement that, maybe I’ve got an advantage over the others who have legacy they’ve got to still handle whereas I can just have a clean implementation that just does this new thing so we’re gonna see these you know these new kids coming up with their you know I don’t know Kotlin implementation I’ve heard of those I know some some some some of them one of those languages you know you’re like you implemented in what I’ve heard of that kind of thing. That may be the next thing. And that’s exciting for me. I think, you know, that brings, you know, new blood, just a new design space, a new point of view to the implementations, and that’s always great.
Privacy revisited
Speaker 2: 00:27:02
Privacy. Privacy, yes. So we were talking about sort of the robustness dimension and that sort of that principle. What’s on your mind about privacy?
Speaker 0: 00:27:11
Well, privacy, I think, you know, privacy is a bottomless well, right? You can keep going on privacy for just about forever. There are some fairly obvious things you can know on privacy. I think some of them are just like a quality of implementation. So the Oakland Protocol, which was discussed at the last summit, which is this idea that you can shield exposure to where HTLCs have gone through because you can make it a lot harder for people to probe. Hold on, you used to have this much in channel, now you’ve got this much. I can tell therefore that that payment that went through me went out this channel for you. So we can actually shield a lot of that using the Oakland protocol. So that requires widespread implementation.
Speaker 1: 00:27:48
As a reminder, that was basically just under-reporting your channel capacity.
Speaker 0: 00:27:53
That’s right. Yeah. So basically you fake a couple of… You act as if the payment may have gone out multiple channels. You artificially restrict their capacity by some amount, in the case where your maximum HTLC size has hit the bounds of the capacity. So it doesn’t actually make a difference in a lot of cases, because for most implementations, you set the maximum HTLC size that they can go through to, say, 10% of your channel capacity anyway. So as long as you’ve got more than 10%, they can’t tell. They literally cannot tell. When you’re under 10 you know the question is like when you’re on the boundary you can you can start probing and so at that point you start the Oakland protocol and well you know we’re going to actually let you use slightly less capacity than we actually have because we don’t want to leak that we didn’t send the HTLC out through there.
Speaker 1: 00:28:34
Couldn’t prober still send multiple payments, lock up more funds and then probe like the 40 to 50% by having four other 10%?
Speaker 0: 00:28:45
Exactly. So you can do it with multiple probes, but the Oakland protocol basically restricts it because it gives you a fixed point where you go. And some implementations actually restrict the total HLC in flight. So they can’t, you can have like a, you have another thing that they can’t basically exhaust the whole channel. So there’s that, there’s low-hanging fruit like that. There’s splicing, which just messes up your graph a little bit, so it makes it harder to… Dual funding particularly allows you to combine multiple things into one transaction, just the common ownership heuristics starts to break down. This is useful. There’s a whole pile of different fun things you can do once you’ve got splicing and you can splice in and out and everything else And just mess people up which is kind of nice Particularly if we start to see things like page join of things coming out of splices and really really kind of a lot more So once your lightning channel is also used for unchain things through splicing, I think it becomes a lot more complicated to see exactly what’s going on, which is just a nice low-hanging fruit that we’re going to get.
Speaker 1: 00:29:40
Especially if we no longer announce which UTXO a channel is from.
Speaker 0: 00:29:46
The gossip question, right? Do we break the linkage between for anti-spam reasons? We say, hey, improve. I have this UTXO and this is the basis of our channel. We’re going to have to weaken that slightly anyway, because chatbot channels are going to have a different form. So we need a new gossip thing anyway. At that point, we’re going to have this debate this week as to whether we go all the way and say, hey, we don’t have to prove the funds for this particular channel. One proposal is that you prove that you have some funds and that allows you to announce a certain number of channels or a certain capacity of channels, but they don’t have to be related. Cool. Just prove that you have some funds. So you can’t claim, hey, I’ve got all these giant channels without some anti-spam requirement. But we could loosen it out and say, cool, you know, you proved you got a million sats. You can advertise eight million sats worth of channels. Right. To hand wave, there have to be some other restrictions in there. But that would potentially allow us to break this heuristic where you’re basically announcing your channels. Now, the gossip announcement, that’s my fault actually, because I put that in the protocol. And I was thinking, well, you’re public anyway, it’s fine. But it turns out, of course, that leaking that one piece of information then often leads to a lot of other information that you did not intend to be leaked, particularly about unannounced channels, which we’ve recently had improvements on, right, with with SCID aliases, you know, we’re no longer leaking those, but indirectly you can now start to tell what’s happening with private channels, which was, or unannounced channels, which was never the intention.
Speaker 1: 00:31:04
Just through taint on on-chain transactions and pedigree?
Speaker 0: 00:31:08
Yeah, well, just the fact that now you can see, okay, I know this channel came from you, and therefore I can see this other UTXO went to a Lightning channel, I’m pretty much sure that that is yours as well. And you know, you can start to join things together just doing things like that.
What broke when fee rate spiked?
Speaker 1: 00:31:22
So I would like to call back to you said a bunch of things broke when the fee rates spiked.
Speaker 0: 00:31:27
Yeah.
Speaker 1: 00:31:28
And I was curious what what broke for various lightning implementations.
Speaker 0: 00:31:32
Can we get a little bit into that?
Speaker 1: 00:31:33
Yeah, absolutely. Robustness for fees folks.
Speaker 0: 00:31:36
So yeah, so part of the problem at the moment is that you basically have to agree on what fee you’re going to pay. Now there’s there’s now only one side pays fees. It’s the person who originally proposed it. And they’re the only one who can propose fee changes. But fees are also charged on second stage HTLC transactions. And that comes out of the HTLC. So it may not be paid by the person. So as a result of this flow through, you actually care what the person normally paying the fees sets it to. So you have to have some agreement. We tend to be pretty broad. Now it turns out if you restart BitcoinD it seems that it often just drops a bundle of shit and starts to think that the floor for fees has dropped back down. So people will restart the BitcoinD, it would go, no, no, min relay fees, all good. We’re, you know, we’re back on like, you know, one set or whatever. And you would propose that to your peer who has not restarted and you fucking want no, and you could get forced closures that way.
Speaker 1: 00:32:29
Sure. Yeah, this is meant to run consistently, and we don’t get good fee estimates until a couple blocks have been found.
Speaker 0: 00:32:36
That’s right. So some of that is just a learning experience, right? The answer is that basically you shouldn’t let the min fee drop over some period of time. You just go, we’re going to ratchet it up to a certain amount. And then if it’s been there for an hour and we’re still getting, okay, then we start to let it down. This is a workaround for that. There’s another issue that the Eclair people spotted that I think actually Bastien spotted, which is where you go on chain, you go to redeem an HLC, you go, it’s actually not worth me paying the fee that would be involved to redeem this because fees are high, whatever else, so I’m not going to, you still have to close it upstream at that point. Normally you’d wait till it spans and then you go, whatever, but if you’re not going to spend or you’re not gonna push paying a fee to force it through, you now need to make the call to fail it upstream. You’re at risk at that point because you could lose the HTLC, but you’ve already decided to write that money off because it’s not worth. But if you don’t close it upstream, upstream then closes on you because you’ve left this HTLC and clock’s ticking. Upstream goes, right, I’m going to close the news. So you can end up with channel failure because of this issue.
Speaker 1: 00:33:37
Right, so you have an HTLC that’s so small that at a high fee, it’s not worth claiming. That’s right. You write it off, but you have to.
Speaker 0: 00:33:44
But you don’t actually close it out to the peer. So that’s a quality of implementation issue and you end up closing another channel, not just the first one, right? So you end up this cascade effect. So we had a little bit of that.
Speaker 1: 00:33:56
Could you clarify? So the fee in a commitment transaction is paid by the channel proposer? Yes. Like the person that started the channel. So it’s always the one that started the channel that has to cough up the fees.
Speaker 0: 00:34:09
Okay. Yes. But remember with modern anchor channels, your fee is actually pretty low ball. It’s just going to be enough to get you into the mempool, and you’re going to use child pays for parent on one of the anchor outputs to actually bump the fee where it is. So it reduces the problem that we have. Just like I have to guess what the fee rate is going to be in future whenever I want to spend this to, I have to guess what the min fee is going to be in future.
Speaker 1: 00:34:30
So I can spend this. The dynamic mempool minimum fee.
Speaker 0: 00:34:32
Yeah, that’s right.
Speaker 1: 00:34:34
Which can be higher in the last few months.
Speaker 0: 00:34:36
It’s not actually that much of an easier problem, but at least the number is probably less. So, you can lowball fees to some extent, whereas before we had to bump fees up quite a lot. We used a magnifying factor to go, well, I could go as high as this. The ultimate answer to this is to package relay, version 3, all those things. So we can actually have no fees here and have all the fees in the child. Well, the ultimate answer for this is L2 and everything else. But meanwhile, package relay lets us not have-
Speaker 1: 00:35:03
But even with L2, you still need to have a mechanism to be able to broadcast on-chain.
Speaker 0: 00:35:07
But in that case, we use a SIGHASH signal. Anyone can pay. You have to bring your own fees. And you bring your own fees through, right?
Speaker 2: 00:35:13
Yeah, sure.
Speaker 0: 00:35:14
And in a more efficient way than child pays for parent, you actually bring your own fees on the board, which is really cool.
Speaker 1: 00:35:18
Anyway,
Speaker 0: 00:35:22
so this means that then the person closing actually is the one who bumps it and then pays fees above some de minimis level at the moment. So that actually works a little bit better and is a bit more incentive compatible. So you want to close the channel, you’re paying the fees. Whereas the moment like, you know, sometimes you want to close the channel, it’s just like, I’m paying the fees. That’s not right. Right. So, so it does help to some extent to solve that. So yeah. Okay.
Speaker 1: 00:35:42
So, so basically the commitment transaction must at least with anchor output still meet the dynamic minimum of the mempool. We’ve seen that spike to above 10, I think 15 maybe in the last few months.
Speaker 0: 00:35:57
That’s right. Now you’ve got some time, you have a longer delay on your channel, you’ve got some time to get it in. So a transient spike doesn’t hurt you so much, but you do have to get in. And this is one of the reasons that we talk about, you know, Bitcoin Core traditionally has divided rules into like the hard rules of what can go in a block and then the software rules about what can propagate. It’s like when you’re using it at a layer two, you don’t care. Like if you, you don’t care why your transaction doesn’t go in, whether it’s because it’s illegal or whether it’s because it’s considered immoral by the network, right? Just these soft rules don’t actually make a difference to you. They’re both really good, really important rules. And so there’s been more focus than there perhaps has been in the past, where these soft rules about what’s allowed to propagate through the network have been seen as less important. Yes, to some extent they are because they’re changeable, but in a sense of you’re operating, building on top of it, they’re both really good. So there’s been a proposal to have a workaround where we actually start spraying things through the Lightning Network so that people will jam them into their local nodes. It may still happen at some point if we really need to. As long as we can get them to miners.
Speaker 1: 00:36:57
Oh, is that the third approach next to Nostra and- Yeah, that’s right.
Speaker 0: 00:37:00
Exactly. You know, we just all you separate them over all the networks as long as there’s a minor missing, you might pick them up. I know there was there was a proposal to have some kind of local package relay where you could kind of inject a package locally. And then we could just paper over the rest if we needed to. Obviously, we’d like to just go all the way to the you know, hey, do the whole peer-to-peer protocol for an upgrade, we’re all good. But you know, that’s something that we could potentially do if we had to. But you know, in the meantime, it’s interesting to see the phenomenon. This stuff happened because fees were so low for so long that it got pushed to the back of the line, right? Even we were one of the first people to implement the old anchor system, which had fees built in. And then when people implemented it, they went, actually, we want zero fee anchors. And so that became whatever else implemented, we lacked. Core Lightning did not implement that because it meant basically you always had to bring your own fees and you had to have a reserve for that and you had to basically be able to do a lot more dynamic fee stuff. And it just wasn’t on fire until it was. And that PR is sitting there. It’s going to get merged. And it is going to get merged because I’m release captain and it’s damn well going to get merged before the release. So we’re finally fixing that. The one good thing about volatility like this is it does shake your priorities up a bit and reminds you that you’ve got to address these things.
Speaker 1: 00:38:11
Pushes adoption of best practices, better fee estimation, better output type usage, your excel management, you know, all the things.
Speaker 0: 00:38:20
All these things that are not important until they suddenly are and you put the engineering work in. Look, you know, life’s all about trade-offs and I actually can’t, you can’t be too harsh on people for going well, you know, I decided to defer that. Like people give crap to Moon Wallet for example for using on-chain transactions. I’m like look it’s an engineering trade-off that’s fine you know as long as you go in with your eyes open. Now sure your timing isn’t gonna be perfect but hell if you’d asked me two years ago I would have said no no this stuff is critical you know fees are going up you’ve got to do it. I mean really? You actually had two years to get away with it. And you could kind of bumble your way through the recent fees spike and go, okay, for all I know, we’ll have another two years of low fees. And you go, oh, okay, maybe you panicked a bit too hard. Just for the record, I do not think that’s true. And you should totally get your shit together now. But I can’t blame anyone else who made the same kind of decision and went, well, actually, if you’re not on fire right now. But yeah, it is part of a robustness approach means addressing these things. I think we are slowly moving our way through the list, but there will always be things that you wish you had done earlier, but that’s engineering. You can’t do all the things.
Speaker 1: 00:39:19
Right. So the rainbows and unicorns in this case look like v3 transactions with ephemeral anchors, zero fee commitment transactions, package relay. Package relay. And L2 also solves some of these problems.
Speaker 0: 00:39:33
But we have similar incentive problems, and you need to have the package relay as well, and v3, I think, for it to get L2 to be secure as well. We currently have two anchor outputs, and that makes your transactions bigger. So in a low fee environment, you could argue that you shouldn’t implement anchors because you’re just making a bigger transaction for unilateral closes. And you’re spending, basically, your transaction is almost twice as big because it’s got these two extra outputs. With package relay and v3, I think we only need one anchor that we can share. We’re still in anchors. You know, you’re still a little bit bigger.
Speaker 1: 00:40:06
And the ephemeral anchor is smaller because it’s just an uptrue.
Speaker 0: 00:40:10
Yeah, that point, the ephemeral anchor, it shrinks right down. So it’s pretty good. And at that point, you start to go, yeah, OK, it’s a bit of a wash. And output’s still not free, but you know, it’s cheap nine bytes nine bytes Someone’s on the math. Cool. So yeah, yeah that look that that’s definitely the where we want to get to and again, it’s an Maybe no one will notice if it takes a while to get there, but definitely it’s one of those, the engineers look at these things and see problems. And this is one of the problems we see and we crossed off the list and then we’ll work on the next one.
Speaker 1: 00:40:38
Yeah, Gloria and I have been writing, waiting for confirmation series and uptake newsletter. Next week we’re going to talk about mempool policy as an interface for layer 2 protocols. Yeah. So, not preparing for that yet, but I’m thinking already.
Speaker 0: 00:40:56
Yeah, I look forward to reading it. It’s going to be interesting.
Speaker 2: 00:40:58
What else is on your mind, Rusty? Anything else to mention?
Speaker 0: 00:41:02
Okay. I think my mind is still somewhere over the Pacific Ocean right now. So I’m not sure that it’s all present. Look, as I said, we’ve been going eight years on this and it’s been a fantastic journey just to go through with this incredibly bright set of inspired engineers. I’m not a morning person, but I still get up for 5.30 a.m. Calls every two weeks on Tuesday, my time, Monday, everyone else’s time, to hang out and like, and collaborate on the SPAC and do everything else. And, you know, it was really interesting to go through those days when there was excitement about lightning in the early days, and then everyone was pretty much pivoted to, no, Bitcoin’s a store of value, payments aren’t important. And we’re like, but we’re working on payments. Like, we’re working really hard, you know. And you know, to see it kind of come back around, now people are like, wow, this is actually usable. Like, it’s really cool. I think it’s a little bit of, it’s a little bit gratifying to kind of, everyone to kind of come back to oh yeah this is this is actually pretty nice thank you that’s good but you know we still have so much more to do and I think everyone’s pretty aware of that to some extent you know this is a job that will never be finished, but it’s going to get harder and harder as the network grows to change things. And so there’s a lot of effort on trying to make sure that we’ve got things correct now, we’ve laid the foundation correctly. So that, you know, while, you know, unlike Bitcoin, which is completely, you know, ground into stone. We do have some ability to move on the protocol, but over time it will become less and less just because of the weight and the inertia of things. I look at the IPv4 to v6 transition, right? It’s going great. It’s going great. Any day now, right? Two weeks, TM. You know, so, so, you know, and there were things that we put in the protocol that were way overkill. The onion stuff, like the onion routing, but just with everyone was very clear that while nobody’s screaming for this today, it is a critical component that needs to be in the version zero. It needs to be in there before anything else. So I think we’re in, you know, all up. You step back and you were in reasonably good shape. And yeah, as I said, engineers tend to focus on the problem. We tend to focus on this needs fixing and this needs fixing. We need to do this and this. We don’t often look back and go look at, you know, all the things that we’ve achieved over this time. And so for me, this week is a little bit of both. A little bit of kind of going, yeah, look at all the stuff that we’ve done. Look at the excitement that has moved up. The people building things on top of what we’re doing. That’s really exciting. And that’s where more people will come into there and they’re going to come into the low level and go, you know what I want to do? I want to implement Lightning Protocols from scratch. You know, I welcome the crazies. If you’re listening to this and going, that’s what I want to do, I want to implement that. You’re my people. I love that. But I just accept that most people will be building on layers above us. And that’s where the excitement will move up to. But still, our job is going to be to just continue that incremental engineering and just making it better, little by little, and those wins will accumulate.
Speaker 1: 00:43:36
Saurabh Mishra What are the things that you’re excited to see being built on top of Lightning? Well, I’ll let you answer first, but I might have follow up.
Speaker 0: 00:43:43
Jason Tucker Oh, yeah. What is exciting? I always said that there will be some killer application of lightning that I will think is stupid because I’m old. And you know,
Speaker 1: 00:43:52
how I feel about inscriptions.
Speaker 0: 00:43:55
Yeah, yeah, exactly. You know, it’ll be you know, I don’t even know what it will be. And part of that is just the problem is that when you don’t have a technology, when you don’t have a way of making micro payments, what we would call real micro payments, not like making tiny payments, by definition, any business model or things or infrastructure that would require that do not exist because they can’t. And so there’s an inertia behind that, right? You create a technology, but then there’s no immediate use for it. There are a few fringe uses and people who can use it and, you know, they can kind of like enhance things they’re already doing. But the whole Greenfields thing of things that didn’t exist or didn’t make sense before that suddenly makes sense. And you know, I think at this point, Noster tipping is probably something where people think, oh, it didn’t make sense before. You couldn’t really do that with credit cards.
Speaker 1: 00:44:43
It didn’t. I must admit, like on Stacker News, when I want to tip a post, and the little lightning zooms across the screen when I tip. Yeah. It’s just like I started Lightning Wallet, but I put in a few thousand sets and I got some tips back, bigger account number now. It’s neat. If I ever want to get those sets back, I can just pull them back to my Lightning wallet. It’s pretty neat.
Speaker 0: 00:45:08
And look at Cashew and some of the other things, the Fetty Mint and stuff using Lightning as a glue, right?
Speaker 1: 00:45:12
Yeah. And just international payments. So simple. I traveled to Central America a while back and being able to just go and pay with my money there, everywhere, that made it very real, much more real than all this theoretical pondering on what the world will look like.
Speaker 0: 00:45:35
Yeah, absolutely. And which of these will be the, is it like slow accretion of all these things? I don’t know. I’m not smart enough to tell what’s happening, but it is exciting. It is fantastic to see people actually use this and explore things. And these people will build things that I could not even conceive of. I certainly couldn’t have built myself.
Speaker 1: 00:45:54
It’s hard to see the potential of the black box thing when you’re in the nitty gritty building the black box.
Speaker 0: 00:45:59
That’s right. Yeah, And I’m sure any guesses that I would make about what it would do will become horribly dated and will be wrong. Because you’re a product of the pre-existing conditions, right? It will take someone for whom someone lightning native will come along and do something that yeah, probably the shaggiest, dumbest thing I’ve ever heard of. It’ll turn out to be a huge success and it shows that I knew. That’s why I’m not a business guy. I’m just an engineer.
Speaker 2: 00:46:25
Where are we at? Thank you. Thank you for doing this. Thank you.
Speaker 0: 00:46:30
Thanks for having me.
Speaker 1: 00:46:31
Yeah, thanks.
Speaker 2: 00:46:39
So did that live up to your expectations?
Speaker 1: 00:46:42
Yeah, I think so. Looking back eight years in with people that got in really early that had a 20-year vision for it from the get-go. It’s refreshing. I think it’s really nice thinking back, writing so much, reading so much about it when Lightning just came out. Yes, there’s been a lot done. It’s getting used by people. It’s very cool.
Speaker 2: 00:47:05
So eight out of 20 means we’re 40 percent and are we 40 percent there?
Speaker 1: 00:47:09
I don’t know. Maybe it’ll be a little faster. It’s already usable. We’re seeing some projects being built on Lightning. I mean, last year I could buy pupusas in a place with stomped earth ground but lightning network payments and I don’t know that was my plan so it’s it’s it feels like we’re definitely some part there where you can use it, but some of the features that I’ve been hearing about for years still need to happen. So yeah, maybe 40% is a good value. I don’t know if 20 years is the right amount, but taking that sort of approach and having that in mind makes it easier to.
Speaker 2: 00:47:49
Is it always just 20 years out? Do you take this?
Speaker 1: 00:47:54
I don’t think it’s like block times. It’s more like building a new protocol and also getting that person that will be a lightning network that builds the killer use case just takes a generation maybe. Anything really big that changes stuff progresses in 10 year steps. So two steps. Yeah, I think so.
Speaker 2: 00:48:15
Cool. Well, hope you enjoyed listening to it and we’ll see you