Mastering Lightning & Using BTCPayServer
Speakers: Andreas Antonopoulos
Date: January 6, 2020
Transcript By: Stephan Livera
Category: Podcast
Media: https://stephanlivera.com/download-episode/1707/139.mp3
podcast: https://stephanlivera.com/episode/139/
Stephan Livera: Andreas, welcome back to the show.
Andreas: It’s a pleasure. Thank you so much for having me again.
Stephan Livera: Oh, it’s, yeah, I know you’re doing, you’ve got a lot on, I know a couple months ago, you announced recently alongside Laolu Osuntukun and Rene Pickhardt that you’re working on Mastering Lightning. I’m excited to see the result of that. Let’s talk a little bit about how that came about.
Andreas: So I’ve been involved in the lightning network, for a very long time, probably back in 2016, I started trying to learn about, it enough so I could write about it in Mastering Bitcoin. So in Mastering Bitcoin, second edition, there is a whole chapter about lightning network which many people haven’t even noticed, but it was very early days. And I wrote about it when it was still in the beta stage. I started running a node. In fact I wrote a little script that allowed me to take the graph outputs which was the machine description of what nodes existed on the network and how they were connected and put it through some graphic software and created one of those little bubble diagrams. And turns out I was the first person to do that.
Andreas: And I produced a picture of the lightning network that had seven nodes on it and mine was one of them. Which I’m hoping one day to be in the Smithsonian. So I tweeted about that. That was January of 2017 with Testnet. I ran my nodes on the test net for a year and then in 2018 when the production network started I started running my node there. So I have one of the longest running nodes on the network. And it’s been a fascinating journey. I was always really interested in it. I think it’s unleashed speed of development and innovation that isn’t really possible in the Bitcoin base chain, which has to be much more conservative. And so many fascinating things are happening. So I just got involved in that and at some point I thought, well, this has to be my next technical book.
Andreas: And it took almost eight months of planning to get up to the announcement. And then another five months have passed since then. So, it’s been a year in the works and it surprisingly takes quite a lot of time to negotiate, plan and execute on a book. And we’re not really starting to write until this month. So this is when we’re really diving in. We’ve done some outlining, written a bit of chapters one, two, and three. It’s on GitHub and and it was announced August and it’s going to publish sometime in Q4 of 2020.
Stephan Livera: That’s awesome. And one thing that I see following lightning is that it’s just changing so quickly. And then the question then is how do you write a book about it when it’s moving so quickly?
Andreas: Well, the good news is I have plenty of experience and practice with Ethereum on writing a book about a moving target that was moving too fast for me to put down on paper. I think one of the things that I’ve been practicing and I think I’m getting better at this will be the third technical book effectively fourth cause Mastering Bitcoin second edition was almost a complete rewrite. And I’ve gotten good at striking this careful balance between writing about the things that are the essence the unchanging truths of something. That won’t change in a few months. And also just capturing a, snippet of all of the things that are in progress and the direction they’re going so that at least some of the book is still valid a year later or two when I have to do another edition. It’s a careful balance,
Stephan Livera: Right? Yeah. And a quick example off the top of my head would be something like, okay, right now one of the key concepts in lightning is this idea of HTLC routing. Right? And then potentially once we get Schnorr and taproot, then that might change to PTLC point time lock, you know, routing,
Andreas: Payment points are a very interesting concept. But you can’t really do, you can kind of do them with elliptic curve digital signature algorithms, but it’s very kludgey. Whereas with Schnorr you can do them very elegantly. And that’s one way that we’re going to see a big development. And, maybe if we have one more change, which is the I don’t know what they call it now. It was called a SIG HASH NO INPUT. So –
Stephan Livera: Any Prev Out.
Andreas: Any prev out, it’s changed names many times. But if we get that change, which is a bit more difficult to get into the protocol, then we’re looking at eltoo, which is another very big change. But both of these changes are changes in the inner mechanics of lightning. They’re not changes in the fundamental concepts or, the layout or the architecture or, the essence of the thing. The essence remains the same, which is transmitting, previously signed transactions, in a person to person or peer to peer model, and routing between multiple, of these bilateral commitments, to make a whole network where you can get security, and privacy. It’s a fantastic concept and it has so much more to go.
Stephan Livera: Right. Yeah. And can you tell us a little bit about your own process then of writing and researching around the lightning network and what goes into the book and what doesn’t?
Andreas: Right. So first of all, for any technology, my learning experience is trying to explain it to others. And in order to do that, I first have to use it. So in order to understand the technology, I use it and that means running the code and then running into the bugs and doing bug reports and rewriting and hacking through the code. And so that process really started more than two years ago when I started operating my own lightning node and ran into all of the problems that you would expect with an experimental technology. I then decided about four months ago to up the stakes a bit and I started running a shop on top of my lightning node so that I’d have some real skin in the game, right. To really creates an environment where uptime and availability mattered to where payments would flow, where inbound capacity mattered to really make it a functional node.
Andreas: And that’s been the case. So I play with it. I play with all of the different versions of the software. I run LND and C-lightning and I have all of the wallets on my, on my phone. I finally discovered what the limit of application groups is in Android, to where, when you click on Android, then it pops up a little application group and tiles, all the little apps. Did you know that it goes to a second page if you have too many in that group and then it puts a little scrolly dots at the bottom and you have to swipe to go to the next page.
Stephan Livera: Cause that’s just how many lightning apps there are.
Andreas: My, crypto wallets group in my Android phone is now two pages long. And I think it’s a five by five grid. So I think I passed. No, it’s a four by four grid. So once you go over 16, you go to the second page. Yeah, so I run all of the wallets now in terms of actually writing the book I really believe in using stories and taking a real life example. Otherwise, technology gets too dry unless you talk about specific applications and specific people. So I start with Alice and Bob. Alice buying a cup of coffee in Bob’s coffee shop, which ironically was obviously the first example I had in Mastering Bitcoin. And then got a ton of criticism from people when the fees got expensive. And that’s application was no longer quite as comfortable on the blockchain.
Andreas: So now I’m revisiting it. So now Alice has a lightning wallet, and Bob is running a lightning node. So we’re doing that. And then basically following a transaction or a few transactions like that through the network. And as it touches each layer of the technology, explain that layer and breakouts with real examples. The other thing is all of the examples are real, meaning that every transaction you see, just like in my book every command I type, every output you see, I do on actual computers that are set up in that environment. So the transactions are on the blockchain. You can follow the channel open and channel close. You can look at them with a block Explorer, you can explore them. Now, one thing I’m doing differently for this book which has been a progression is when I started Mastering Bitcoin, the first edition, I wrote probably 90% of that book by myself.
Andreas: By the second edition I wrote about 80%. With Mastering Ethereum I wrote about 60%. And the rest is contributions from people. So encouraging community contributions and collaboration. Why? why spend time trying to really dig into a concept that’s very difficult to understand and write it from scratch when someone has probably already written something that fits almost perfectly and I can get them to contribute it to the book. And then I can play the role of a Maestro of orchestra conductor, right? So I can then take a step back and look at the overall narrative. The what goes in and what stays out the, how do you link these things together? How do you explain them better and do the the editing and the language. With Mastering Lightning. I’m trying to take that even further. So my goal with every book is to write less and get more collaborators and do more of the high level orchestration and narrative. And I’ve got two fantastic collaborators right off the bat. So this is very much a book where all three of us are equals in this collaboration. We’re going into it with that mentality and it’s been a great collaboration so far. Yeah. Awesome. And
Stephan Livera: I suppose you can also crowdsource different answers to things. So for example, you might see a good explanation for something on Bitcoin Optech or on Quora or the, or one of those what’s the stack exchange, right? Bitcoin Stack Exchange. And you can sort of pull from there.
Andreas: Rene Pickhardt, my coauthor is the third most prolific contributor on stack exchange. So it would be, his answers anyway, he was also someone who had already started writing a lightning book. And he’s contributing excerpts of that wholeheartedly into this. But there’s also a lot of interest in the community. You know, the lightning community is a very vibrant community with a lot of developers but also a lot of non technical contributors and a lot of people who simply want to learn. And one of the best ways they can learn is by reading, fixing and trying out various examples and in the process contributing their own answers to it. So for Mastering Ethereum, the last book I did in this model, I think we had somewhere around 4,000 commits and pull requests from more than 180 contributors, all of whom were acknowledged in the preface.
Andreas: I’m hoping to take that up to 250 contributors and maybe five or 6,000 contributions for this book. It takes a village right, in the pure sense of the open source spirit and you can actually watch the book being written online. As it goes, it’s on github.com/LNbook. And it’s crap right now. And it will remain crap for a few months and gradually it will start taking shape. You know, it’s a work in progress. In writing a book like this, especially writing it’s open source we have to get comfortable with a lot of vulnerability and just put our best work, which at first is crap as I mentioned out there. And let people criticize it and then come back with the classic open source answer. It’s like, Oh, you think it’s wrong? Great. Make a pull request.
Stephan Livera: Right? And one concept that’s coming to my mind as well when you’re writing a book like Mastering Lightning is you’ve got different layers at which you can talk about things, right? Because you’ve got the specification, right? The, I think it’s 11 BOLTs and then you’ve got actual implementations, right? So c-lightning, LND, eclair and some others. And then you’ve also got on top of that specific software, right? Like zap or spark.
Andreas: The clients. But also even above that with the lapps the lightning applications, which are not a wallet interfaces but are front-ends to various applications, games, forms things you can do with lightning, not simply payment applications or not simple wallet applications.
Stephan Livera: That’s right. And so then how do you think about trying to explain that or weave that into a story? Might you do one story that has, okay, this is a story that’s involving, you know, c-lightning and spark and one particular application and then another story with LND. Like how do you, how do you think about that?
Andreas: That’s one of the things we do. One of the things is to take a single story and first run it at a very surface level, a, which is an experiential level. How does the user see an interaction? And that’s the Alice buys a cup of coffee from Bob. Then we can start talking about what is Alice’s wallet doing in the background and we can then dive in and then we can say, Oh, but if instead it was a different wallet that looks, here’s what it would do what is Bob’s store running on? How does Bob do a point of sale? What’s running underneath? And then we can talk about how Bob operates a lightning node and how Bob maintains inbound capacity and rebalances channels and things like that. And then we can also go and look at many of the behind the scenes things that are happening in terms of routing, pathfinding payment channels, et cetera, and go all the way down into the protocol.
Andreas: We’re not going to go all the way into rewriting the BOLTs because there’s no reason to do that. The BOLTs exist if you want to see what is the specific script that is used in an HTLC you can just go to the BOLTs. And the idea here is to give a developer the ability to learn how to read the BOLTs, which BOLTs to read, and how they all fit together in a big picture way because the BOLTs are not designed as an educational tool. They’re their reference document. So we’re not going to rewrite what’s in the BOLTs. That would be a waste of time. We’re going to go just above that level and then say, and for more details, see BOLT 2 see BOLT 3 et cetera. And there’s some topics we’re simply not going to go into. For example, I don’t think there’s much point in talking too much about the node to node protocol, the pure P2P message passing protocol because that’s not new technology. The onion routing part is, we’ll talk about that, but the simple message exchange and, you know, negotiating which features each node supports with that, fairly established technology. A developer can find other resources. Again, we want to write a book that isn’t 7,000 pages long which is another challenge. You’ve got to go deep enough without making it a ridiculously unreadable tome. My goal is for 300 pages. It will be a difficult task.
Stephan Livera: Excellent. and I guess just turning now to lightning more broadly, what are some of the more exciting developments that you are looking forward to over the next year, let’s say? Do you have any in particular lightning gaming or any you know routing or LNURL? Anything in particular?
Andreas: Yeah, so much. Let’s see. I think for me, one of the important developments that is now happening was amp, atomic multipath payments vast going to really, really improve the usability of lightning and make it a lot easier to manage a node. I think the other thing that I’ve been looking forward to and is beginning to emerge is more use of splice in, splice out, submarine swaps and things like that in order to make the difference between a lightning wallet and an on chain wallet disappear where you have one wallet and it can do payments to Bitcoin addresses or lightning invoices. It manages its own channels and channel capacity. And all of that detail is hidden from the user. We’re beginning to see the first of those emerge. And the ability to make sure that almost every transaction you do is both an on chain transaction and a channel management transaction.
Andreas: You should never miss the opportunity of an on chain transaction to do some channel work, right? Because we are going to be facing a world in which fees are gonna be higher. And I think that’s a desirable characteristic because otherwise the block subsidy will have to continue. You know, already in many other chains, they’re talking about what they call long tail distributions meaning endless inflation and continuing issuance forever so that they can subsidize miners. I don’t think that’s desirable in Bitcoin, which means we’re going to need to operate in a world of higher fees. And in that world, you want to make sure that every time you go on chain and pay the fee, you take as much advantage of that space as possible with channel management. So that’s another thing I’m looking forward to. I think we’re seeing the beginning of that. But, recently for example, the ACINQ team launched a new wallet called Phoenix. They have the eclair wallets, which is very popular. It’s very easy to use. Phoenix is a new breed of, of wallets, which is where I thought we were going. And glad to see where, you don’t see channels unless you want to where it manages inbound capacity for you. And where every transaction can be both on and off chain simultaneously.
Stephan Livera: Yeah, that really changes it a lot in terms of help when you’re helping with a beginner. I think Phoenix is probably a good example. If you want to get them into lightning, while still holding their own keys obviously. For me at this point, I’m still mostly when I’m working with a beginner, I’m personally just onboarding them more to just Bitcoin wallets like Samourai Wallet for example, because I think it’s just not quite easy enough yet for them to go into lightning on the first go. But what’s your view around that?
Andreas: I would agree. I would agree that we’re not ready to start onboarding onboarding users directly to lightning. And unless we take a shortcut, then that shortcut is to use custodial lightning wallets. Which a lot of people do, you know, they’ll, they’ll onboard users onto Blue wallet, for example. And so yeah, of course, if you do a custodial wallet, it removes a lot of the difficulties of the protocol just as it did, on chain transactions. You know, if you want it to onboard a user, it’s a hell of a lot easier to onboard them to a coinbase wallet than it is to make them control their own keys and have to write down 24 English words and all of that blather but you’re doing that person a disservice. You’re showing them the Venmo of Bitcoin, in which case, why don’t they just stay on Venmo. The whole point of Bitcoin is self custody, sovereignty, independence, empowerment. If you take those things away for the first demo then there is no point. Basically what you’re saying is that user wasn’t ready and you shouldn’t have introduced them yet. Anyhow. Yeah. So yeah, we’re not quite, we’re not quite there yet and that’s okay.
Stephan Livera: Yeah. And I think the other thing as well is there’s some discussion around this idea that, you know, obviously chain fees will rise longer term. I think that’s, I think we’re all agreed on that. Yeah. They just, they must do that. But the question then is how much further in the future and potentially what we saw in 2017 was like a unnatural amount of, chain usage and now that people, more people know about okay, using things like batching and using, you know, the latest versions of SegWit to save themselves in terms of the fees and so on. I wonder how much kind of runway of time is there that people can still use on chain, on normal transactions. So long as they’re not like for the very small value roughly how much time there is before it really, you basically have to use lightning or some kind of other means?
Andreas: Oh, I think we have many years ahead of us where for the right kinds of transactions where it really matters on chain is absolutely going to be the right choice. I don’t think we’re going to see the scenario where on chain is only used for channel management or in conjunction with channel management for quite a while. And, again, it depends on the kinds of uses you make. I do payroll every month. In Bitcoin, I run payroll for my employees. So every month I have to do a bunch of on chain transactions. I mean, I use Bitcoin quite a lot. I probably do two or three transactions a week on a regular basis. But the ones that really matter to me are the kinds of transactions where first I’m quite happy to wait. So one block, how about 10? 10 blocks is good enough when you’re running payroll, right?
Andreas: Usually it’s three business days if I do it through the banking system. So even if I have to wait to 150 blogs, I’m still beating the banks by a factor of three. I’m not worried.
Stephan Livera: Yeah.
Andreas: So that gives me a lot more latitude to to play hard with my fees, right, to bid low and wait patiently. I use RBF replaced by fee, and child pays for parent and other technologies to be able to bump fees when I need to. If something gets stuck, which is actually very, very rare. I haven’t paid more than a satoshi per byte since since August of 2017. And I’m kind of one of those curmudgeons who will be like, hell no, I’m not paying more than a satoshi per bite.
Andreas: Sometimes I’ll wait for half a day, not, not longer. And I use batching. And I also use native segwit addresses. In fact, I recently had my first complaint on the store because I’ve configured my BTCPayserver to only issue native SegWit addresses. And they, and someone complained and said, well, why you only issuing native segwit addresses my wallet doesn’t recognize them as a valid address. I’m like, dude, it’s 2019, get a better wallet. Your wallet sucks. And just for the value of me being able to explain to you why your wallet sucks is why I have native SegWit only forget the fees. This was a learning opportunity. And and in fact, that person changed wallets thereafter. You know, it’s been two years. It’s time, right?
Stephan Livera: And now I’m a big supporter of lightning. I, you know, I’m bullish on lightning, but it is also important to be realistic and say, well, what are the things that still need work? So, you know, a kind of a brief survey of the things in lightning that still need work, things like channel and liquidity management and making that easier. People could argue privacy as well. You know, you can still do probing. There’s still payment correlation and potentially stuck payments. UI, smoothing out the problems when things go awry, you know in your view, where are the main areas where lightning is falling down right now?
Andreas: I think the biggest difficulty is getting inbound capacity. And a lot of that has to do with user experience even more so than the raw technology. Lightning does require either better user experience or some steep learning curve learning around these little quirks that occur when you’re dealing with payment channels. And the biggest quirk of course is the payment channels are asymmetric things. They have a balance on one side and a balance on the other side. The metaphor I use is think of it as you have a bowl of nuts in front of you and I have a bowl of nuts and if I want to transfer nuts to you, I take some from my bowl and I put them in your bowl and now you have more nuts in your bowl and I have fewer nuts in my bowl.
Andreas: Well that works great. But if we start with my bowl being empty, obviously I can’t give you any nuts. And that’s hard to understand at first. It also is hard to understand together with the other aspect of it, which is there is also the nuts that are not in the bowls but are on chain. And so the idea that you have a different balance on chain and a different balance in channels or off chain, this makes it hard for users. And I think the idea that you set up a wallet, and then the first thing that you expect to do with the wallet, which is receive money from someone else is the one thing you can’t do can be very difficult for people to on the stand. Of course we can fix that. And one of the interesting ways to fix that is is the ability to rent inbound capacity.
Andreas: So we’re beginning to see some services that are emerging and I would like to see these become algorithmic services rather than manual services. Meaning that today you can pay someone to open a channel towards you with capacity so that you have inbound capacity and they will then charge you based on how long you want that channel to stay open and how big that channel is. Which is actually a very nice way to price the opportunity costs of, of money, if you like, that it takes to create inbound capacity. And once you have a bit of inbound capacity, you can actually build more easily. So those services today require you to close down your wallet, open a browser, or find a service. You don’t know if that service is trustworthy or an if it will deliver send them a payment get an inbound channel.
Andreas: I would like to see all of that automated, you know, there’s no reason why your wallets can’t do that by spending a few sats when you first start it up. And again, the other difficult thing is that the wallets take time to initialize, meaning that you can’t simply open a new lightning wallet and get started. At the very least, you’re looking at three confirmations until you have your first channel. So, and usually longer than that, it probably takes about a day before you can actually start using the wallet. A better way to get around that is if exchanges start offering the ability to withdraw to a lightning wallet because then you solve both of those problems simultaneously. And I think that’s where we’re going. I think eventually we’re going to see most withdrawals originate from exchanges that will also give you your first channels.
Stephan Livera: Right? And I think we’re seeing this now with Bitfinex for example, and I know another company, River Financial are coming and they are going to be very much more Bitcoin and lightning and allow you to dollar cost average and stack sats and withdrawal using lightning as well. Sure. I think there are a few examples there. In terms of, I guess the other part is, you know, it’s not just kind of the initial setup, but then the ongoing management, right. And then you might have to ongoing, So for example, lightning labs has loop in and loop out and you’ve got to continually, like, it’s about how much flow you can get through, not just the initial stock of that channel, let’s say. Right. So do you have any reflections on that and particularly your experience now being a merchant as well?
Andreas: I don’t think my experience being a merchant can be extrapolated. You know, how they say the plural of anecdotes is not data. But in my case, it’s even worse than that because listen, I’m running a lightning node that’s called ln.aantonop.com. And so I am very, very blatantly using my reputation to attract inbound channel capacity.
Stephan Livera: Influencer privilege!
Andreas: Right? Right. I got it easy from that perspective. Not everyone can do that. So from that perspective, I can’t really say because I have not run into any rebalancing. I actually have three times more inbound capacity than I have outbound capacity, because of that. And because when you have something to sell specifically on lightning people will open their first channel to you to make that payment. And then they’ll leave a little balance in there. So they’ll open the channel, usually that’s a bit bigger than the payment they want to make. So you end up constantly increasing your inbound capacity. I also do use a couple of techniques to rebalance, including making circular payments. So you know, paying where, my node is the destination and I go through a series of other nodes to rebalance.
Andreas: Of course, you know, we’ve seen mathematically, of course, that’s a zero sum game, meaning that if I do that, I’m rebalancing my channels at the expense of unbalancing somebody else’s right. So, you’ve got to look at this from a network wide perspective and that’s where technologies like Loop In and Loop Out come into play. I think in the long run, basically every single on chain transaction will also have a whole number of channel transactions in it to close and reopen channels, which is the best way to rebalance things. And I would like to see a lot more of that happening automatically. And it’s the kind of emergent service that can be priced. Which again is one of the interesting things about people at the moment are still thinking of lightning primarily as a way for people to pay people in small amounts.
Andreas: But what they’re not yet thinking about is machine to machine payments in small amounts in order to automate the economics of various services. Micro services, APIs and micro-payments for services to machines. And of course the first and most obvious application for that is microservices from lightning nodes to lightning nodes for lightning. Where you can have your machine can buy a lightning service from another machine without you even knowing that it’s doing so for a tiny, tiny payment and that payment can be priced very, very nicely. Because what your pricing is that opportunity cost of money. So you can say, okay, what is my rate of return? I want for the opportunity cost of, locking up this money into a channel for a period of time.
Stephan Livera: Right. I think a couple of things there. So one is typically companies have dealt with this more just by batching, right? So they would, in their sense batching to do a monthly subscription, right? They’d just bill you once a month instead of streaming the payment. So I suppose right now, I think because we’re still in this phase where a lot of people want to hold, obviously, so it’s not like there’s a huge, huge amount of commerce happening on lightning and it’s still developing. It’s still growing, obviously. But they might not really see it as a big deal that I can’t stream the payment. I’ll just pay you once a month. Right? And even on chain or do a lightning payment once a month. So yeah, but counterpoint would be some, a company like Sured Bits, right? So they are literally doing, as you were saying, I think they’re doing this idea of you pay for data and, you can pay with lightning. Right? So that’s a, that’s a cool,
Andreas: Well if you, if you think about that, if you batch these payments and pay one once a month what you’ve got there is you’re introducing an element of counterparty risk. It’s no longer trustless. So if you’re paying in advance, then you’re trusting the vendor or provider that they will continue to provide the service even though you’ve paid after you’ve paid for it, for the entire duration of that month or you’re basically giving out the service and hoping that at the end of the month, the customer will be true to their payment. Either way, someone’s taking a risk. And the whole point of streaming payments is basically to cut down the window of risk to the duration of that payment. So if instead of doing it monthly, you do it daily, now you’re taking one 30th of the risk 30 times a month for that same service.
Andreas: Now that risk carries with it a financial cost and that financial cost can be estimated. That’s what actuarial sciences are all about. So then you’ve got to start balancing that against the hassle of making micropayments. And there are certainly many industries in which that risk is absolutely worth paying or taking the hassle to do streaming money. And I think streaming money is going to become a huge, huge thing. One of the areas where it becomes an obvious advantage is games. So in the area of games we can do a lot more interesting things if we have the ability to make a small streaming payments without the counterparty risk of simply give all the money to the game company and hope they’re going to hold it well.
Stephan Livera: Mm, yeah, yeah. There’s, there’s some good points there. I’m also keen to talk about BTCPayServer. So as you mentioned, you’re using BTCPayServer. I’m a big fan. I use it myself as well. Can you tell us a little bit about your experience with setting up and your experience in, just evangelizing BTCpay?
Andreas: So first of all, I want to make it clear that I am very careful not to endorse companies or projects as a whole. So what I’m going to do is speak about my personal experience. This is not a suggestion that this is a project you should go out and, and do something with. But I will happily talk about the things that have made my life better. I think one of the most interesting things about BTCPayServer, is its origin story and its origin story is incorporated in the tweets by Nicolas Dorier where he said and I’m gonna paraphrase wildly, but he basically said, well, fuck these guys. Bitpay is a mess. I’m going to replace you with a script. Which is the classic motivation for open source software for decades. And has been a fantastic way to lodge projects where a company fails its own users and one of its users knows just enough coding to decide to reimplement the entire company.
Andreas: And at first it seems like the most audacious thing you can say and how can it possibly ever work, right? This is a company with resources and developers, never underestimate the raw audacity and determination of someone who wants to make something work better. Linus Torvalds and Linux is probably the best example. There was a big fuck you to AT&T and Cisco and IBM. The end result of this was what started out as a simple payment script that was there to do the event management and HTTP, challenge response required to support the BitPay API has now become a multiprotocol platform for retail payments. And it’s amazing how it’s evolved in that way. It’s become so much more. So BTCPayServer is a project written in C#, at its core is a Bitcoin node.
Andreas: But also now a lightning node and it has a series of APIs that allow it to do basic e-commerce functions, like generating an invoice for a shopping cart and then tracking whether that invoice has been paid and all of the nuances of what is the exchange rate for dollars or Canadian dollars or yen. And what if the exchange rate changes in the 15 minutes since you made the payments. And what if it doesn’t get into the first block but only makes it in the second block. And all of that crap that happens with real life payments and it does it beautifully. And it’s an open source volunteer project, which is based on donations. And, and by the way, please do donate. They fund themselves independently. They haven’t done a shitty ICO or anything like that.
Andreas: And kudos to them. I started running BTCPayServer about a year ago quietly in the background. I had a lot of trouble deploying it at first because I didn’t follow their method. So BTCPay is one of those things that you could deploy using a Docker image, which is a container, and you can do a one-click deployment on Azure or Amazon’s container service or any other Kubernetes docker type container service. But I already had a well-connected Bitcoin node or several, in fact, a fully synced, fully indexed full archive. And I already had several well connected properly run lightning nodes. So I was like, I want to run BTCPayServer, but with my nodes, I don’t want to bootstrap new ones. And so I tried to reverse engineer their installation script. That was painful. Eventually I managed to do it. And then after several instances I discovered that it was much easier to just let it deploy its own node and then ignore it. So yeah, I’ve been running it now for more than a year. It’s the basis I run a little store the Aantonop store at aantonop.com/shop where I sell mugs. You, you can see this one,
Stephan Livera: Not your keys, not your coins.
Andreas: Yeah. A little mug that says, not your keys, not your coins, and other slogans of those. It’s not, about really, it’s not about making money, it’s about spreading dank memes and conversation starters for people in the crypto space. And and also playing with technologies like BTCPay Server, and lightning and no fiat accepted. So the server takes Bitcoin and lightning payments through BTCPay Server and Ether and Litecoin payments through Coinbase commerce, which is the noncustodial service from Coinbase.
Stephan Livera: Yup. And how has the experience been were there any other hurdles you faced along the way? With setting up your BTC pay or getting lightning working? Even for me, it can still be a little experimental, right? You’ve still gotta manage the node and sometimes maybe if you know, the web service or the host service, if it hasn’t caught up to the chain or other little things can come up. Did you have anything like that? In terms of getting it going with, you know, Bitcoin and Lightning?
Andreas: Not from the BTC pay server side. And keep in mind on the BTC Pay Server I run four different stores. So I run one for my events company, which manages things like tickets to conferences and things like that and payments from conferences. One for my, sort of consulting and professional company where I do invoices for my clients. I run one for doing donations. So I used to have people send me donations to a vanity address 1Andreas and I took that away and now I have a BTC pay front-end for that and one for the store for the shop. Which means that I’m actually running a network of three different Bitcoin nodes in three different geographic zones that check to see that they are following the same chain.
Andreas: So they, I have a kind of a master process that compares between them and if one of them goes out of sync it shuts all three of them down. So that I can avoid problems where one of my nodes is being spoofed by network house isolation. So I check the hashes of the most recent block and make sure they’re all following the same chain and not diverging. I also run three different lightning nodes which are running different software including one that’s embedded in BTCPay with C-lightning one that’s LND inside BTCpay and one that’s LND outside of BTC pay. And then I run a few channels between them to keep them balanced. So it’s a more complicated infrastructure and most of the problems have come from the fact that it’s a more complicated infrastructure. I kinda did that to myself because the purpose was not to run e-commerce and sell things. The purpose was to run a learning experiment. But I have helped other people set up BTC pay to run actually commerce. And from their perspective, it’s pretty much a one click install. It’s very, very easy to do including doing it properly, which means putting a hardware wallet as the storage mechanism for your keys that never go online. Which is I think a very good idea.
Stephan Livera: Yeah, that’s awesome. And with BTC pay, you can set up the different stores and then inside each store you have a wallet. And inside that wallet you can put in an xPub or as a Zpub, et cetera. And that can be from your Coldcard or your Ledger or your Trezor. And then that is a good way to, even though the, you might be hosting it on a VPS, it’s still the keys are held on your hardware wallet.
Andreas: Yeah. You can’t do that with lightning. Of course, because lightning is a hot wallet infrastructure, but then again you shouldn’t keep too much on your lightning server. They are now integrating with the HWI hardware wallets interface specification so that you can plug in and initialize your hardware wallets directly with BTC Pay Server. That’s an interesting development. And they’ve also gradually had more and more contributions of various accessory software stacks. Whether that’s spark and charge, which are two c-lightning modules. Charge is an eCommerce capable server, similar to what BTC pay does for for the BitPay API and spark which is a very, very lightweight mobile wallet which is actually my preferred wallet, I’m showing you here on my Android. So I, run spark on my mobile and it does everything through the server which runs its own lightning and Bitcoin node.
Andreas: Which I really, really like. And then they’ve got a number of management tools like a ride the lightning or RTL as it’s known, a fantastic project that I really love and has made my life so much easier. Which is a web based graphical user interface for running LND and now I think C-lightning cause they’ve consolidate the API APIs,uand a whole bunch of other things. I mean I’m keep discovering, there’s a plug in that allows you to link,uBTC pay to QuickBooks so that when you generate an invoice on QuickBooks, it can automatically generate and link the URL to a crypto payment invoice on BTC pay. Haven’t done that integration yet. Intuit threatened and then cut me off of their credit card service because I had the word Bitcoin on my website. They asked me to remove the word Bitcoin from my website. So I removed Intuit from my life instead.
Stephan Livera: Can’t have a naughty word, like Bitcoin on there.
Andreas: That conversation was hilarious. And so yeah, they, they they won’t let me get paid with credit cards who Boohoo. So I’m, I’m a bit reluctant to try to integrate the BTC pay server with my accounting software just in case they freak out and cut off my accounting software, cause that that would cause quite a mess. But yeah, there’s all these goodies and what’s happened is BTC pay server has become a platform that can fairly easily integrate a whole number of different services similar to another project that I really like, which is called myNode. Do you know that one?
Stephan Livera: Yes. I hosted a Taylor on for an episode a little while ago. I really like it. It’s a really great one. And what’s your experience been?
Andreas: Oh, fantastic. I run all of those or, I mean I built and ran all of those services and manually built something very similar to myNode. It’s not a single nodes in my case. I run these kind of complex multi availability zones, on Amazon web services as well as some privately hosted computers. So I don’t put all of them on one computer instead, they’re all like little containers that can start up on their own. But what a great project. You know, making it easier for people to run some of the infrastructure of Bitcoin. I’m very much for that. As people say in this space, you know, if you don’t have your own keys, you don’t have your own coins. That’s the difference between a third class citizen and a second class citizen. But if you run your own node and have your own consensus rules, that’s when you’re truly a first class a citizen of this crypto space. So, you know, we’ve moved beyond just not your keys, not your coins. It’s now not your keys, not your consensus rules, not your nodes. Not your coins.
Stephan Livera: Right. Yeah. It’s a more, that’s the other thing is all for a long time we’ve been saying, okay, you’ve got to run your own node and hold your own keys, but how easy has that been made for the average person out there? And I think the different projects, yeah, like helping make it easy for people to do that.
Andreas: Absolutely. It’s up to us to make these things as easy as possible so that more and more people can do them. The trends are not very promising. I recently saw that Luke Jr’s stats on the total number of nodes in the Bitcoin network dropped by about 8% in the last year. So I think we went from about 60,000 to just over 50,000 nodes in total. That’s, you know, disappointing. But there’s all of these projects now where companies that are shipping ready built nodes you know, there’s two or three of those. And also a lot of these hobbyist projects where I’ve heard of people getting together at Bitcoin meetups and having parties where they all set up a Raspberry Pi 4’s and build nodes in a kind of group environment where when you get stuck you can get some help from a friend. I think those are great. Whether you intend to run that node longterm or not, whether you want to be involved in this or not, the learning experience will always stay with you.
Stephan Livera: Yeah. That’s fantastic. One other area I was keen to ask you ask about. I know you recently were a speaker at LaBit Conf.
Andreas: Yes.
Stephan Livera: there’s been, I think there’s potentially a different understanding when you’re coming from a country that has had these, you know, crazy monetary problems. And what’s been your experience interacting with people in South America and you know, for example, Argentina, what’s the difference in understanding, is it more about suggesting practical tools that they can use to get Bitcoin and use Bitcoin in their lives?
Andreas: That’s part of it for me. It’s, you know, it’s, it’s not what I can do for the South American community. It’s more what the South American community can and has done for me. And in this journey. My first La Bit Conf was the first La Bit Conf in December of 2013 in Buenos ares, Argentina. And that La Bit Conf changed my trajectory in the Bitcoin community. Before that La Bit Conf I was very, very focused on technology and the, the politics of money was a secondary interesting. But secondary to me, I went to that conference, those two flipped. I came back and I started talking about Bitcoin in a very different tone. It reminded me of my childhood in Greece and the financial crisis and currency crisis we had had there, the run on the banks, cause they were experiencing it in real time during the conference.
Andreas: And I came back and I started talking about why Bitcoin is for the other 6 billion. Why it’s about you know, preserving your financial freedom and with a much more defiant tone, with a much more loud tone with much less tolerance for playing nice with regulators playing nice with the banks helping them see the light. And I came back with a, well, fuck the banks attitudes. They’re never going to let us do this if we ask permission nicely. We’re going to seize the initiative and we’re going to do this despite whatever any regulator or government or bank says, because this is too important for too many people. So La Bit Conf gave me that gift. It gave me the gift of perspective. I walked into La Bit Conf, a super privileged, comfortable financially included North Americano.
Andreas: And I walked out an angry Greek with an attitude because I got reminded of what the stakes were. And, honestly that did wonders for my ability to connect with people because by, by changing my focus, I also changed my tone and I started connecting with people who really saw the importance of this technology. That was the first. There have been seven La Bit Confs and I’ve been to six of them every year they do it in a different city, in a different country, essentially touring central and South America in a way that brings this message to the places that need it the most. And it is, I do not want to miss any of them. The vibe the style of conversations that happen, the organizers are amazing. And, they’ve done an incredible job keeping it focused. No bullshit, no distractions, no marketing, no ICOs, no, none of that. It’s, it’s absolutely focused on financial inclusion and on teaching people how to build these things in their local community.
Stephan Livera: As you said, I think you make a great point that it’s stories that motivate people. So for those of us, and you know, many of my listeners are people who are, the Bitcoin guy and in their friend circle and they’re trying to help communicate the value of Bitcoin to people. Do you have any tips to close out with for them on how they can better communicate the value and you know, potentially using stories as well on how they can communicate the value of Bitcoin?
Andreas: So I often struggle with this because I think that for many of the people that we’re surrounded by Bitcoin is not yet unnecessary technology and it’s, it’s actually a technology that we shouldn’t hope becomes necessary anytime soon because for many of the people that surround us, the financial system and the institutions that operate the financial system and the institutions of governance and democracy kind of sorta work. Yes, they’re breaking down. Yes, they’re getting worse. There are all of these long term risks and you know quantitative easing, inflation stimulus and all of this crap that’s happening in the financial system. But there’s this let’s call it the privilege of freedom where you can go get on with your life without worrying about money. When money works in your life, when, when the system of money still operates in your country you can pretend it doesn’t exist.
Andreas: It’s invisible. It just goes into the background, right? Now Bitcoin is for the other 6 billion because for them it doesn’t work. And I think sometimes, it’s dangerous or not dangerous. Let’s put it this way. It can strain your friendships. If you go out and you try to push something or, you know, preach aggressively or sell hard on a technology that most people don’t really see much relevance into their life. So I go with a very soft approach for most people in my circle where they have functional banking I tell them about why this is a cool technology that they might want to learn something about. Just because it’s interesting to learn. And I also talk to them about why it might be risky to have all of your eggs in one basket, and right now all of their eggs are in one green basket printed by the federal reserve.
Andreas: And so, you know, as risky as Bitcoin is it’s a hell of a lot riskier to have your home, your job, your savings, your retirement, your company’s savings and all of the savings and assets of all of the neighbors and all of the people, you know, all in one thing. So from that perspective, a small amount of diversification Hey, I mean, that’s about as, as well as it can go. And let’s hope most of our neighbors don’t have a desperate need for Bitcoin in the next few years because that will mean something went really, really wrong with the economy. I think for many of us Bitcoiners we know that a system that’s as fragile and corrupted and broken as that will have problems in its future. But let’s hope they’re not as bad as they as they might be. Now, let’s hope we don’t turn into Argentina or Cyprus or Venezuela in order to promote Bitcoin.
Stephan Livera: Oh, of course not. And I think there’s certainly, a value of building a parallel system, right? We’re just building a better parallel system and people can opt in to that system if they so choose.
Andreas: Yeah. Let’s make it about having a choice and that choice being cool and futuristic rather than this is the lifeboat. It’s very small. It rocks very hard. But you’d better jump in because the big ship is sinking. Because no one wants to believe that story. Even if they, they kind of see it’s true. They’re gonna go for denial over their crazy friend who keeps pitching Bitcoin.
Stephan Livera: Excellent look, I don’t want to keep you too long Andreas. But make sure you shout out for the listeners, where can they find you and where can they follow your work?
Andreas: So aantonop.com is my website, aantonop is my Twitter and aantonop is my YouTube channel. All of my work is open source, free to use, share, and enjoy and build upon. My most recent book, the internet of money volume three just dropped in December. And my next book, Mastering the Lightning Network is coming out at the end of this year as a collaboration, as I mentioned before. And you know, our last conversation was about maximalism on the heels of me publishing, Mastering Ethereum. And so now we’re going to find out is Mastering lightning enough to appease the maximalists and bring me back into their good graces, or have I ever burned that bridge and, will never gain back their trust because of Mastering Ethereum. We’ll see. I’ll continue doing good books and good topics and we’ll see how it goes.
Stephan Livera: That’s right, we’ll see how it goes. And look, thank you very much for joining me, Andreas. It’s been a pleasure to chat with you.
Andreas: Absolutely a pleasure. As always, I hope we do this again soon.