Deterministic vs Non-Deterministic Wallets

2024-05-08 11:29 WIB - Hendrik Lie

This page is copied from my gitlab repository. You may find an updated version of the article there.

In this post, we are going to review two classes of crypto wallets. The first one we will discuss is how addresses used to be managed by a wallet software, and how such wallets are backed up. The second part will discuss on how modern wallets handle wallets, and how they are backed up. Then, the next part will discuss on ways you can secure and back up your wallets. At last, we will provide a quick review of popular wallet software available on the market.

Early Wallets

Prior to 2013,¹ wallets generate a buffer of fresh random private keys (typically 100 addresses) for receiving and change addresses for future use. Such wallets would also offer wallet backups, to backup all of those addresses. However, once the backup pool exhausted (the pre-generated 100 addresses are all used up), new addresses are generated.

As a result, the previous backup is invalidated, and new backup must be made. This, in effect, requires users to frequently back up their wallets. The result of losing the wallet without proper backups, or infrequent ones, results in the loss of funds stored in the non backed up parts of the wallet.

There are many ways to counter this issue, one of them is the use of paper wallets. Unlike its name, a paper wallet is not a wallet, but instead just a single keypair. Because of that, it promotes address reuse, false sense of security, but also requiring users to manually handle private keys. Details of its flaws could be found here.

Some other means to store your coins are the use of custodial wallets like Coinbase, web wallets (as a browser extension), cloud storage, removable media, and physical bitcoins. All of those are explored here under “Bad wallet ideas” sections.

The basic idea of backing up a traditional wallet is just by backing up every and all private keypairs. Personally, it means we are storing a lot of data that must be reproducible: not a single bit can change, or you might lose some if not all of your funds. However, the universe is very hostile to modern computers. With the presence of a Single Event Upset might actually upset your backups or devices storing your backups, no matter how hard or how well you store them.

Then, there must be a better way to secure and back up your wallets, right? Right?

The Era of Seed Phrases

According to this article, by 2019, all good wallets already adopt BIP32, a standard for hierarchial deterministic wallets. What this kind of wallet does to back up every and all possible keypairs you will ever use is by deriving all of your future keypairs from a single starting point known as a seed. That way, any modern wallet using the same seed and the same derivation path will always produce the same addresses.

The seed must have a sufficiently high entropy to be secure, and also convenient to store and back up. However, as discussed in the previous section, even by storing only the entropy, no computer is safe from a Single Event Upset. Meanwhile, storing the entropy manually by writing them down is prone to typo, material damage, hard to read handwritings, data leakage while printing, and many more attack vectors. Some ideas to store your coins are also discussed here, along with their weaknesses.

Now, how could we store the seed in a manner that is easy to use and handle, encapsulating a sufficiently high amount of entropy, and has a very good error correction? Enter the age of Seed Phrases², as a way to losslessly compress our initial entropy into a list of 12 to 24 words, and can optionally be further protected with seed extensions³. The way they accomplish it is by having a list of words, and have them represent a number (in BIP39, 0 to 2047). That way, our 12 to 24 words seed phrases store a sufficiently large number in an easily human readable series of natural language words.

The BIP39 wordlist has 2048 words, and for a phrase of 12 random words, the number of possible combinations would be 2048^12 = 2^132. Therefore, the phrase would have 132 bits of security. However, the last word in BIP39 standard is not random, but instead a checksum, hence the actual 12 words seed phrase has a security of only 128 bits, which is approximately the same strength of all Bitcoin private keys. It follows without saying that a 24-words seed phrase has a higher security compared to a 12-words seed phrase.

That way, for any good wallets that support the generation of and restoring from seed phrases, you can retrieve your funds back. That assuming all wallets use the same seed phrase standard and the same derivation paths, which is not necessarily true. Electrum, for example, uses a different seed phrase standard that tries to address the issue inherent to BIP39 standard.

Furthermore, not all wallets supports seed extension, which makes this matter more complicated. As mentioned before, a seed extension is a word, or a set of words used to extend the original entropy, and hence, a more secure way to store your coins. All and any seed extension combined with all and any seed phrases will produce valid seeds for any compliant deterministic clients. Therefore it offers possible deniality in the event where said seed phrases are compromised. For example, by having funds stored in keypairs derived from the seed phrases only, or from the seed phrases combined with fake seed extensions.

However, wallets such as Trust Wallet, Bitcoin.com Wallet, and Litewallet, do not support seed extension, and some can only accept 12-words seed phrases. Therefore, it is best for one to actually study each wallets first, before right away trust that they will support your backed up wallets.

For myself, I have found some android wallets that actually implement BIP39 properly, with standarized derivation paths. Those are Unstoppable Wallet, and Airgap. Unstoppable Wallet offers a large selection of coins (mostly ERC20, BEP20, or BEP2 tokens), while Airgap only offers a smaller selection of coins. However Airgap offers a more advanced security, and the Airgap Vault can function as a hardware wallet. Airgap is also very good on making new entropies, so far having the best options to create our own entropies. Not that Airgap is the best in making it, I just find it very interesting.

Backing Up Your Wallets

It is worth noting, that, as mentioned before, BIP39 has its own flaws: it is not true that all the user needs to restore their wallet is just their seed phrase. Some developers across the industry continue to build wallets that either:

• Don’t implement BIP standard(s).

• Implement a BIP standard, but inconsistently when compared with other wallets.

• Implement a BIP standard, but one that has not been widely adopted (and perhaps only by them).

• Don’t have clear documentation about their derivation paths, backup and recovery processes.

Wallets come and go, information gets lost, and users are left with tears. Responsible wallet developers document external recovery. Users should not have to dig through the source code to figure out the Derivation Paths or Redeem Scripts.

WalletsRecovery.org

Know Your Wallet

Therefore, before settling on any wallet, you have to understand:

1. Whether they’re a deterministic wallet or not, and if yes, of what type? Things to look out for is whether or not they have an option to recover from seed phrases. If they do, they are most likely a deterministic wallet. Most wallets that I have tested tend to derive your addresses with BIP44, BIP49, or BIP84 standards. However you still have to do your own research. This is a good starting place for you to read.

2. How they are going to derive your addresses from your seed phrases? One thing to lookout for is their derivation path. In the WalletsRecovery page, we could learn that via the Supported Paths column on their tables. You are required to study them yourselves, but as a general rule of thumbs:

   • BIP44 standard tend to use m/44'/0'/0' + Custom, with their generated addresses starts with a 1 and might looks like 1PC9aZC4hNX2rmmrt7uHTfYAS3hRbph4UN (Bitcoin donation addresses for the Free Software Foundation);

   • BIP49 standard tend to use m/49'/0'/0' + Custom, and their addresses starts with a 3, and might looks like 3MaFikrngEw8YK6T5sxVoRmmYL3awgh846 (don’t send anything to this address, I don’t know who owns it, and is generated only for demonstrative purpose!);

   • BIP84 standard tend to use m/84'/0'/0' + Custom, and their addresses starts with a bc1, and is not case sensitive. Their addresses look like bc1qm9ykrnyuknzsrrp6na6pysu7vzezf5yzlzj5hu (donation address for Xenomancy.id :) feel free to transfer some funds there lol).

3. Whether they are single address wallets or wallets that can generate new receive addresses for each transactions. Some wallets like Airgap Wallet, Trust Wallet, and many more wallets use a single address for all transactions.

   This is an expected behavior in some chains whose transaction is processed in an account model, for example in Ethereum chain (ETH), or in Binance Smart Chain (BSC). They do that because you need native funds (ETH for ERC20 tokens, BNB for BEP20 tokens) in those addresses to move tokens on that chain to another address.

   Bitcoin and coins forked from Bitcoin like Litecoin and Dogecoin uses UTXO model (UTXO stands for Unspent Transactions), where you are expected to never reuse addresses. Wallets that don’t reuse addresses include: Unstoppable Wallet and BlueWallet.

   It is worth noting that in Unstoppable Wallet, BEP2, BEP20, and ERC20 tokens (along with their native currencies ETH and BNB) uses only a single address for reasons discussed before.

To test them, you can use a randomly generated seed phrases from here, and check their generated addresses in any coins of your choosing, and compare those addresses with addresses produced by your wallet. For safety reasons, don’t actually use seed phrases generated online from that page as your actual wallet and store funds within. If, for whatever reasons, you need to use your seed phrase in that web tool, consider doing it offline by downloading the tool from source and run them in an offline device.

WARNING: YOU ARE RESPONSIBLE TO KEEP YOUR OWN SEED PHRASE SAFE AND SECURE!

To test whether or not they reuse addresses, it’s a bit trickier, and you might have to dig deeper on their documentations. I for example, learned it only by trying to actually send and receive funds from those wallet apps. That is when I learned that Trust Wallet does not generate new addresses after every transaction. The same is also true for Airgap Wallet.

However, I discovered a rather curious behavior of Airgap Wallet. Airgap Wallet did not generate new addresses after receiving a transaction, however they can detect transactions of any receive and change addresses derived from a seed phrase. How did I obtain other addresses derived from that seed phrase in Airgap Wallet? I couldn’t, but it is possible to look out other addresses from Airgap Vault, via a feature known as Address Explorer.

Some wallets didn’t directly provide information on how to export wallets externally, but we can get hints on their feature lists. For example, Unstoppable Wallet explicitly described in their “Feature List” that they supports BIP 44/49/84/69. BIP 44/49/84 refers to address standards discussed before.

Securing Your Secrets

Then, after understanding how your wallet derive your addresses, and what addresses they generate, the next step is to know how to secure your seed phrase and seed extension. In short, storing your coins boils down to backing up your seed phrase, and storing them in a safe place, or on multiple safe places for backup. According to the article, storing bitcoins consists of few independent goals. All of which can be explored below:

1. Protection against accidental loss. Modern day deterministic wallets enabling us to secure our entire wallet just by storing the seed phrase securely. Preferrably, in multiple copies stored in multiple geographic locations for redundancy. That seed phrase can be used to restore your wallet and recover all of your funds.⁴

   However, since a seed phrase can store any large amount of money⁵, it makes little sense to keep them unprotected. Therefore, you should also further secure your seed phrase with a seed extension. Note that not all wallets support storing seed extensions.

   I think, it is also a common sense not to store your seed phrases along with their seed extension. It will literally be the same as storing non-protected seed phrases. Not storing your seed extension at all is also dangerous, as losing access to that is essentially the same as losing your seed phrase.

   To conveniently store your seed phrases and seed extension, consider storing them with pencil and paper. Then that paper should be stored in the dark while avoiding extreme heat and moisture. Some people might decide to store them in a stamped or engraved metal. Whatever your favorite way to store your seed phrase, make sure to keep the paper (or metal plates) safe and secret like cash or jewelry.

2. Verification that the bitcoins are genuine, and privacy and protection against spying. Storing your seed phrases only store the seed required to derive private keys. To learn about how many coins are stored within, we need a crypto wallet. Ideally, we should have a private full node that would independently verify incoming and outgoing transactions from our addresses. However it is impractical to do it nowadays, as per February 2022, the size of Bitcoin blockchain is 324 gigabytes.

   There are lightweight wallets that do not store the entire blockchain history, and confirms transactions by communicating with external full nodes. Lightweight wallets have their weaknesses, especially as they do not validate the rules of bitcoin, while sending addresses to third parties and receive wallet balance and history, while also skipping several security steps that can make their users vulnerable.

   If you value privacy, consider setting up a full node. However, if you value speed and minimal storage use, consider using light wallets. Practically all mobile wallets are lightweight wallets, simply because it is impractical to store the entire blockchain history in a mobile device.

3. Protection against theft. As discussed before, anyone with access of your keys, has access to your coins. The same is true with seed phrases and seed extension, as they provides a mean to derive all of your keys. It is not even about a physical possession of the seed phrases and seed extension, but also extends on the security of devices you use to manage your wallets.

   Therefore, you have to ensure that the wallet software running on your device, your device’s operating system and its hardware are safe and secure. Also worth noting that generally desktop computers lack specific mobile features such as Secure Storage and Biometrics to secure your secrets. Generally, a linux-based system is less vulnerable to malwares.⁶

4. Easy access for spending or moving bitcoins. It really depends on whether or not you need to move your coins often or not. If you need to move your coins around often, consider having a separate wallet of which a reasonable amount of funds that are expected to be moved around are stored within. The rest of your funds that don’t need to be moved around that often should be stored in a more secure wallet. Consider setting up a cold storage device, or having a hardware wallet that would secure your private keys and would not expose them to your hot devices.⁷

   A compromise to all above that I find useful is to repurpose an old android device as your own airgapped hardware wallet, an approach that is done by Airgap with their dual app model.

As a summary, this page cleverly explains as follows:

bitcoin wallets should be backed up by writing down their seed phrase, this phrase must be kept safe and secret, and when sending or receiving transactions the wallet software should obtain information about the bitcoin network from your own full node.

However, it is worth noting that we should also understand the nature of our wallets of choosing (see Know Your Wallet section).