Storing unconfirmed transactions on disk

[arnoudk]

Some time ago, I read someone's doom scenario regarding a huge backlog of unconfirmed transactions.

It goes something like this.
During a long period of high transactions, nodes have two options: either delete some transaction from memory using whatever logic the node uses (random, lowest fee, whatever) - or crash and delete all transactions from memory. The mempool is, as the name suggests, kept only in memory. Thus, in both cases transactions are removed from memory.
Now, transactions that are valid, and SHOULD confirm, will never confirm, because they are forgotten. Even when the network becomes less busy, they will not confirm.

This scenario could happen during an extra busy period (some would automatically call that a spam attack or stress test, but I think any transaction that has a proper fee attached is valid). It could also happen if, during the halving, the hashpower is reduced significantly. (I actually think that will not be too great a worry. Most industrial scale miners (the vast majority at the moment) have already prepurchased their electricity. They are under obligation to use or lose it. They will keep mining, no matter what, at least until their contracts are up for renewal. Thus, the hash power will not suddenly crash through the floor).

But anyway. The scenario of forgetting transactions seemed plausible.

Yesterday evening I could not sleep. So I spent an hour or two coding an -inefficient but fully functional- script that queries the bitcoind and stores all transactions from its mempool to a MySQL database. (It is efficient enough to run comfortably on a Raspberry Pi 2 while also running bitcoind and a bunch of other programs that I have since added to it - it's quite powerful!).

Any transactions that are later confirmed in a block (I look back 6 blocks to not worry about competing chains) are removed again from the database.

So my Raspberry Pi is now storing all valid, but unconfirmed, transactions to disk. I guess after a week or so, I could rebroadcast (slowly) these transactions - and only if the mempool is below a certain level (ie, less than a few MB). I have built, but not activated, this because I have not thought through the possible impact. Of course, not resending transactions when a mempool is full, and of course not resending transactions at a ridiculous rate... but what is full and when is it ridiculous...

Any thoughts?