Magical Bitcoin site: https://magicalbitcoin.org/

Magical Bitcoin wallet GitHub: https://github.com/MagicalBitcoin/magical-bitcoin-wallet

sipa’s Miniscript site: http://bitcoin.sipa.be/miniscript/

Andrew Poelstra Miniscript presentation at London Bitcoin Devs: https://diyhpl.us/wiki/transcripts/london-bitcoin-devs/2020-02-04-andrew-poelstra-miniscript/

Andrew Poelstra Miniscript workshop at Advancing Bitcoin: https://diyhpl.us/wiki/transcripts/advancing-bitcoin/2020/2020-02-07-andrew-poelstra-miniscript/

I am a developer, I work on Bitcoin. I came to Bitcoin pretty late compared to the people I work with. I started a couple of years ago working with Giacomo Zucco at BHB Network. I worked for a while at Blockstream and then I left. I didn’t have a theme when I left but then Magical Bitcoin became a big thing that I’m working on. This is my new project. I have been working on this for the last two months. I don’t like theoretical presentations, I like practical presentations where I show how things work. I am going to do an introduction to introduce some concepts.

I will talk about the goals of Magical Bitcoin. The main goal is to develop a set of tools and libraries for onchain wallets. There are tools and libraries to make wallets today like python-bitcoinlib and rust-bitcoin. These libraries are low level so you still need other layers on top of it. It is not something that a non-Bitcoin developer can take and use to make a wallet. A wallet gets integrated into other projects people might have. If you want to integrate Bitcoin into your project these tools are very low level and it is hard to use them so you maybe end up a centralized server with some APIs which is bad. I wanted to develop something that is fairly easy to use and concentrate the development effort of multiple people instead of having many people rebuilding the same thing over and over. Now there is this low level layer that is shared and people are repeating the same things, coin selection logic or transaction signing logic, these are being written and rewritten multiple times over multiple projects. I think this is a waste of resources. It would better to make it in one repository, make it well peer reviewed, modular and extensible and then we can all work on that one thing. Everybody can use it and it will probably be better than every single individual implementation made by single developers. This is the goal. Generalized, modular and extensible. If you make one library for everybody you have to make it generalized. If I make a single sig wallet library that would work but only for people who are interesting in single sig wallets. That is not everybody. If you support multisig and other scripting primitives then it can really become one library for everybody. The super very ambitious long term goal would be to become the de facto standard platform to build native wallets and to integrate wallets into other projects. Today if you want to install a Bitcoin full node you go with Bitcoin Core, you don’t question that because you have everything you need in there. That would be the very ambitious goal of Magical Bitcoin. You need to make some sort of wallet or integrate a wallet into your product you go with that because that is where everybody is working and everything is concentrated. I know it is very ambitious and it sounds crazy to say that but that is the long term goal of this project.

While I was thinking how to do that I also realized that today there is an adoption gap or lag. There are a lot of powerful scripting primitives that are pretty much unused today. If you think about relative and absolute timelocks, they exist in Bitcoin today and have been there for a while now but there are not many wallets that support them. There are few wallets that support them and they support them only in a specific way. Lightning uses these timelocks but it doesn’t support generalized scripting. It only supports the few scripts you need for Lightning. From the developer’s perspective there was value to write code to support these three specific cases. But if I want to make a fourth case? There are not generalized tools to use these powerful primitives. Today you use them in a specific way and it is not useful outside of these specific tools. You can obviously validate them but there are no options to generate a transaction with a timelock in the script. There is a good reason for this. Writing scripts manually is very hard so only a few expert developers can do that. It is not something a new developer could do. If you are making a product and you want to integrate Bitcoin into your product you don’t have time and it would be dangerous to write complex scripts. The result is that nobody uses them. The main goal of Magical Bitcoin is to provide something generalized but as a side effect it is filling this gap. When you try to make something generalized and support as many cases as possible you also need to start supporting these. It is not just a new wallet library but it is pushing the boundaries of what is being done in a wallet library like timelocks.

Q - Is that similar to Simplicity? Simplicity is going to be used to help write smart contracts. Is that the same thing that your software is trying to achieve?

A - Not really. Simplicity is a new scripting language. This uses Bitcoin Script that we have today. It is nothing new and doesn’t need a soft fork or anything to enable it. The current Bitcoin scripting system I believe is much more powerful than people think. There are a lot of hidden gems that nobody uses. It could support Simplicity when it is added to Bitcoin but I am more excited about Taproot because Taproot is something that could be added soon. The goal since it is modular and generalized is to support everything that works on Bitcoin. You could add the Simplicity module, a Taproot module or you can just use the normal scripting. It is not a new scripting language, it already exists in Bitcoin today.

In order to that you need to take advantage of things that we take for granted. One of them is a way to describe a wallet. Today we use a BIP 39 mnemonic and that is enough to describe a single sig wallet. You are describing just one key. A single sig wallet knows how to take that key, turn it into a Bitcoin script, turn it into an address, watch your addresses and see your balance. That works for a single sig wallet but if you want to move into more generalized wallets where you have multiple keys you need something that lets you express this, not only a way to encode single keys but also encode the structure of a script. The other very important primitive is PSBT, this is probably more popular. The idea is that if you are going to generalize scripts it is very likely that there are going to be multiple keys in the scripts. When you start having multiple keys you also have multiple parties that need to sign the transaction. PSBT becomes very useful.

Some details on these two primitives. Descriptors and Miniscript have a close relationship with each other. Descriptors are a way of formally describing how to construct a script. It my mind I see them as a one-to-many representation. You have one script to run from that descriptor. You can generate multiple single scripts by deriving every single key. In your descriptor you have xpubs that are extended keys. From these xpubs you can generate multiple single public keys. You can take a descriptor and generate every single address that makes up a wallet. This is more or less a standardized language. It is supported partially in Bitcoin Core. The main full implementation has been made with Miniscript. A descriptor encodes everything about your script. It encodes the script type, it can be a pay-to-script-hash (P2SH) or a pay-to-witness-script-hash (P2WSH), a nested script hash. It encodes the structure so the elements that make up the script, every single key, every single evaluation path, you have everything you need. You have your descriptor and you have your full wallet. It is normally used with a Miniscript compiler. There are a lot of misconceptions around that because there are two separate languages. One is the Miniscript language and one is the descriptor language. They can look pretty similar so people normally mistake them for one another. Miniscript is even more high level. The Miniscript language lets you express a policy in an abstract way. You can say “These two public keys need to sign in order to spend the funds.” That is all you say at the Miniscript level. Then Miniscript turns this into a descriptor. While doing that it also tries to make it as cheap as possible to spend from that script later on. You can say “I have two public keys. Either of them can spend the funds but I am more likely to use one of the two keys.” Maybe one of them is the backup key and I normally use one key over the other one. Miniscript takes that into account to produce a descriptor that has a structure that is going to be cheaper to spend from later on in terms of fees and transaction size. There are three layers. On the bottom you have the Bitcoin scripting, on top of it you have descriptors and on top of that you have Miniscript.

People are probably very familiar with PSBT because it is starting to be used everywhere. It is basically a way to standardize encoding of a transaction, the partial signatures of the transaction and also metadata about key derivation, inputs, change outputs etc. If you are making an air gapped wallet or something like that you need a lot of this metadata from who created the transaction. PSBT is very useful for that because they are self contained. If you need to sign a transaction a PSBT contains everything you need to do that. You know how to derive your keys, you know which inputs are being spent, you know how much is your change etc. It is very useful to have this thing go round multiple people and everybody can inspect and add their signatures. Then you can take all the partial signatures and build the full scriptSig or the full witness and spend the funds. These are the two main primitives of Magical Bitcoin. Magical Bitcoin is built using these.

There is this magical-bitcoin-wallet which is the main repository of the project. If you go on GitHub you will see this as well as some other little things. This is a fully working wallet library. You can take it and integrate it. It is written in Rust. It is not a very popular language but very cool for some security properties it has. If you have a Rust project you can integrate it into your project in 3 or 4 lines and you can do everything a regular wallet does. You can generate addresses, you can monitor the blockchain, you can create a transaction, sign a transaction. It is very modular so you have the choice. You can say “I want to use Electrum to monitor the blockchain” or Esplora. Esplora is a block explorer, blockstream.info. They are the two main methods today but it can very easily be extended. In the future we could also have the full node option where you connect directly to your full node or maybe compact block filters. There is also a very minimalistic command line. This is mostly meant to be something that you integrate into other projects. It is not a product by itself. Maybe in future there will be one but not today. There is no Magical Bitcoin wallet itself. But you can use it as a developer tool from the command line. This is a full wallet that you can install and use but not very user friendly as it is targeted mostly to developers who are going to integrate it into their own projects. It can be compiled to run in a browser. The library itself is in Rust but it would not be super hard to use it in other languages, Python or whatever. It can also be compiled into web assembly and then run in a browser. This is what I will be showing you. The full command line is probably a little more powerful than what runs in a browser. But what runs in a browser has a nicer UI so it is easier to follow.

https://youtu.be/QVhC2DOIl7I?t=1129

I’ll go to the demo. Something is probably going to go wrong because this is very early stage development. If you go to magicalbitcoin.org this is the main website for the project where the documentation is being written. On this website there is a playground section which is that magical-bitcoin-wallet repo compiled to run in a browser with a nice UI on top of it. The first screen we have up here is the Policy compiler. This is the Miniscript thing I was talking about. This is a one time thing. When you want to make a new wallet you have to write a Policy for this wallet and generate the descriptor for that wallet. From that point on you are not going to use the compiler anymore. It is a one time thing where you generate the wallet and you keep reusing the same descriptor. I will start with a fairly simple example, a 1-of-2 multisig. There are a few examples here. If you go to the Miniscript section there are the native individual blocks that you can use. I am just going to use the examples that are pre-buillt. It is easier to use them. This is an OR block. The spending policy that needs to be satisfied to move funds is an OR between these two items. These items can be keys or a timelock or something like that. There are two possible items and you need to satisfy at least one of them to spend from this wallet. This is the same thing in written form. This is the Miniscript language.

or(pk(Alice),pk(Bob))

This means either Alice or Bob should sign using their public key to spend from this wallet. You can choose the type. We can go with a native SegWit pay-to-script-hash. At the moment I use this website bip32.org to generate keys quickly. Obviously in the future it will be integrated. I am going to generate a random key that I can use. This is a random key that has been generated. I am going to take the public version of that to simulate a two party descriptor where a party only has one of the two private keys. I am going to take the public version of this key. I am going to say Alice has this key. I am also going to provide the derivation path, a very simple incremental index at the end. For Bob I am going to generate an extended key. This means we are also going to have the private part. This descriptor is going to have one public key which is Alice’s key. For Bob it is going to generate inside and also have the private part. This means take this script and replace every instance of the Alice public key with this key and every instance of the Bob key with the key that you generate. When I compile this is what I get. This is a descriptor. You can see it encodes the type of script which is a witness script hash (wsh). The condition is an OR between this side and this other item. You can see for this one we already have the public key. We can generate signatures for this one. But for this one, this is a private extended key so I also have the private part of that. This is my descriptor if I was doing that with someone else. I am Bob, I generated my own key and Alice provided her own public key. I put these together into this descriptor. Now this is the descriptor that describes my wallet. This is from my side and her side. She has the private key for her signatures and she has the public key for my item. Now I can take this descriptor and plug it in down here in the wallet section. This is the real wallet. Once you have the descriptor you can use it to build a full wallet. You plug this descriptor in. You can optionally have a different descriptor for change addresses. For instance when you create a transaction and you have to generate change. Instead of generating a new address from this descriptor you can generate from this other one. Bitcoin Core does that normally. Just to make it simpler I am not going to use that. You hit Start. What happens in the background is it creates a wallet and it also runs the sync command. This means synchronize with the blockchain with whatever method you choose. In this case from the browser Electrum doesn’t work so the only way is to use Esplora. In the background it is connecting to the API of blockstream.info and seeing what unspent outputs it has and what my balance is. It should be empty initially because I just generated it. get_balance outputs 0 satoshi. I can generate a new address for this wallet using the get_new_address command. This generates the first address from this descriptor. I can generate multiple addresses that are different because every address increments the index here at the end of the keys. Every time you generate a new address you are implementing the BIP 32 derivation for every key. I am going to deposit some funds. This is on testnet. I have a testnet wallet on my phone. I will deposit some satoshis using that. You will also be able to see that later on on the blockchain. This happens on the testnet blockchain obviously. I have signed the transaction and I am going to wait a second for propagation. I can run the sync command again. This time I will open the browser console so you can see what happens in the background. This is the network tab. You can see every network request that is being made in the background. You can see when I run the same command it starts making requests to blockstream.info. Unfortunately there are no batched APIs from blockstream.info so you have to individually query for every single address. You can also see this raw request. This is a raw transaction that has been downloaded. Presumably this is the transaction I just sent. Now it looks like it is done. When it sees 20 consecutive addresses that are not being used it is going to stop syncing. I sent something to the first address so it went to the 21st address and after that it stopped syncing. If I run get_balance I can see I have the satoshis in my balance.

Q - It is also interesting to note here that you compiled your code into Javascript from WASM. It is entirely running in the browser. You are not hitting the server right? It is all running in the browser sandbox?

Yes. I am connecting to a server to monitor the blockchain. I am connecting to blockstream.info to get information about the blockchain. All of this functionality like generating addresses it is all happening in the browser itself. It is also written in Rust which is an important point. It is not written in Javascript. It is written in Rust compiled to Javascript. I believe Rust as a language is much safer to use for many of these properties. I would be much more confident using a wallet made in Rust and compiled to work in a browser rather than a wallet made natively in Javascript.

While we are here I will show a few other commands. There is one very interesting command which is the policies command. This is complicated to see. I will open a JSON viewer online. This command is kind of an interpreter for a descriptor. It tells you the conditions encoded by the descriptor that is not really human readable but can be turned into a graphical thing very easily. If we run this command on our descriptor we can see that the top level, the root level we have a threshold (THRESH). We have a few items that are generalized. It could be some very complex conditions. We need to satisfy at least one of these, the threshold is 1 in order to be able to spend. These items in this case are the two signatures. Item 0 is signatures made from this key and Item 1 is signatures made from this other key. If we can satisfy at least one of these we can spend from this script. You can ignore the satisfaction thing because it is not implemented yet. The contribution field tells us how we can contribute to every single item. In this case the contribution to this signature is NONE because this is Alice’s signature from Bob’s perspective. I don’t have the key for this signature. I cannot contribute to that or make a signature for that. I can complete, fully satisfy this other block because this is my own key. The contribution for that is COMPLETE. When you combine these into the root level, this is the important thing, this tells you can you spend by yourself from this script. In this case yes because this is PARTIALCOMPLETE which means we can partially satisfy that which is enough to complete it. There are two items, we can satisfy one. It is going to tell us which item you can satisfy, item 1 because that is our signature right here. What this is telling us is that you by yourself can satisfy and fully spend from this script. I will now show how to spend. I can generate a new address and send some funds to this address (get_new_address). Then there is the create_tx --to [insert_address]:0 --send_all. send_all means send everything minus the fees to that address. I am going to run that and this gives me a PSBT. I can’t really show the console now so I will move to the next step. This can happen on watch only computer. There is this other public_descriptor command which returns the public version of the descriptor. I could take this public descriptor and place that on a computer that is connected to the internet and online. I can create a transaction from that computer and then I can move this PSBT to an air gapped computer where I have the keys which is safer. I can sign the PSBT like that. We get back another PSBT. This is a bit bigger than the one before because this contains the signatures. There are enough signatures in this PSBT to finalize and extract the final raw transaction. This is an older version of Magical Bitcoin and has since changed. With the new version you can broadcast that. Here I have to broadcast the PSBT by rerunning this finalization phase before broadcasting. I can enter broadcast --psbt [insert_psbt]. This is the TXID. I can look this up on the testnet block explorer. This is the transaction we just made. If you look at the witness you can see this is our script, this is the first public key and the second public key. This looks like a signature as it starts 3044, that normally happens with signatures. Then there is a dummy push which is for the other signatures that we are not making, something to make the script happy. Then this is the actual script that is encoded fully which is needed for P2SH.

Q - How do you usually generate these BIP 32 derivation paths? I saw that website but I tend to use Tails with Electrum. I saw a recent vulnerability in Tails so I was curious about other methods. I have been suggested HWI with Trezor but happy to hear any safe methods you think are good too.

A - That is being done in the browser. I don’t think there is one specific final answer, there are trade-offs in everything you do. It depends if you don’t know what you are doing with keys and might lose keys or expose them. I also like the idea of using an old computer air gapped. It depends on who you are trying to protect your keys from. If you are more worried about the classic Evil Maid attack, someone coming into your house and stealing your keys, in that case a hardware wallet is safer because of the secure element, especially Ledger with its full secure element. That is probably safer. But if you are more afraid about someone placing backdoors in hardware wallets at a state level, protecting your Bitcoin from big well funded attacks, it is probably better to use a general purpose computer. If you want to attack Bitcoiners it is more likely you will go attacking or placing backdoors in Ledger and Bitcoin companies. If you take a random computer it is probably more unlikely there is a backdoor there especially for Bitcoin. It is a trade-off. Maybe both them. With things like Magical Bitcoin it is very easy to create multisigs. You could use both of them and get the best of both worlds. It depends on the use case, there is no one answer for everybody.

Q - Is this using Miniscript?

A - Yes. This uses the Rust version of Miniscript. There are two implementations, one in C++ and one in Rust. It uses Miniscript for many things like compiling the policy, generating the descriptor and a few internal things. Like the part that generates the final witness, the final scriptSig is done using Miniscript. I am using the Miniscript library and providing the right parameters.

Q - It seems like your value that you also provide visualizations to help people create the script as well?

A - My main goal would be to just write developer tools. I don’t want to focus myself on graphical tools. I would just like to make tools that are very easy to integrate into graphical environments. The one I made here in the browser it is just an example. It took me an afternoon to build. I am mostly writing stuff in a way that is easy to integrate into user interfaces. What I showed would be fairly easy to integrate into a user interface to make a graphical thing that says “There are two items. You can satisfy Item number one and that is enough to satisfy everything.” You can make a nice UI around that. I am probably not going to make it myself because I am going to focus on the low level tools. But I am making them in a way that should allow other people to build actual full practical wallets on top of that.

Q - What node am I using if I am using your website to broadcast the transaction to testnet?

A - For this specifically you are using Blockstream’s node, so the blockstream.info block explorer. The problem there is that from a browser you cannot really do much. You are sandboxed there. The only thing you can do is HTTP requests so you need some kind of HTTP backend for Bitcoin. You could self host esplora I guess. You could install esplora on your node. It is pretty heavy, the full blockchain index and everything. You could do that. You have much more flexibility if you run that as a standalone command. In the docs it explains how to install Rust, how to compile it. If you install this on your own computer instead of running it in the browser you also have the Electrum support so you can connect to any Electrum server. Then in the future I will add the direct RPC full node, maybe compact block filters. I really want this to be modular. When you come as a developer you choose the things that you need for your own project. Maybe some developers would use the esplora interface and then host their own version of esplora. Maybe somebody else could use Electrum.

Q - Although you have compiled this for the web Rust is great for compiling to different platforms like mobile, Android and iOS. You have got these backends, which library for compact block filters?

A - I actually don’t know, I haven’t looked into that yet. I don’t think there is an implementation for that in rust-bitcoin so I will probably make my own in that case.

Q - I can send you a link. There is one in that project.

A - You mentioned mobile apps. There is an early prototype of an Android app to show how easy it is to compile to other platforms. There are JNI bindings for that. It only supports “show your balance.” It is very easy to port to multiple platforms.

Q - This reminds me of Unchained Capital’s Caravan project. They have a web interface for creating a multisig wallet. They wrote their project using Javascript. This makes more sense for that kind of application.

A - It is very easy to port if you want to make a native app or desktop app. You can compile the code there.

Q - Probably a lot fewer dependencies too.