I'll answer questions by editing the top post so all the answers are bundled together.
How long does it take to create a project like Lighthouse, from the idea to the product? And how many people are needed?
Great question! It took me about 8-9 months of full time work to get Lighthouse to beta, working on my own. But this comes with several caveats. One is that I was working on other things during this time as well, like HD wallets. Another is that a big chunk of the Lighthouse code is actually the bitcoinj library, which I had been developing with many other people for years by that point. So I had a great understanding of the tools I was working with. If someone wanted to build an app like Lighthouse from scratch, it'd take them longer as they'd have to get up to speed, even if they used the same tools and frameworks I did.
That said, what I've seen is that most developers don't want to build downloadable P2P apps like Lighthouse or the original Bitcoin. They mostly want to make web apps. Once I launched Lighthouse, a couple of devs came along and built lightlist.io which is a web based version of Lighthouse, albeit with fewer features. It took them much less than 8 months, though partly that's because they were just reimplementing the Lighthouse design rather than doing something new, and because a more centralised solution is often easier to put together than something P2P.
Do you thing the difficulties of the blocksize debate will result in a more effective decision making process for future Bitcoin development and if so any thoughts on how that might work?
The only decision making process that should really matter is end users and miners picking what software to run. At the moment this mechanism has broken down completely due to a combination of censorship/DoS attacks and general propaganda of the form "if you run something other than Core the sky will fall". So the decision (non) making process used by Core has come under a lot of scrutiny because at the moment Bitcoin is a centralised system run by just one or two people (count differs depending on how you view things).
I don't know if Bitcoin will eventually recover its decentralisation. For that to happen users must feel empowered, and must learn to distrust even those who claim to be unbiased experts (including me). Instead they must learn to pick the software they use based on its merits rather than what the people writing it choose to say, and be willing to switch at any point if the software they use stops delivering.
One possible disadvantage is that faster confirmations may mean higher orphan rate ... do you think it is an impossible idea to make Bitcoin block confirmation times faster than 10 minutes to increase the scalability of Bitcoin?
Of course it's not technically impossible. 10 minutes is somewhat arbitrary. With faster block propagation, in theory the 10 minute number could be lowered. The questions are really:
1) Is it practically possible? Currently the Bitcoin protocol can only evolve when Blockstream want it to. So I'd say the answer is "no", especially if the justification is higher capacity, which they seem to be dead set against.
2) Would 5 minutes really be better than 10 minutes? I suspect the difference is too small to matter. 1 second is clearly a lot better than 10 minutes and 10 minutes is clearly better than 1 day as often the case in banking. But the difference between 5 and 10 minutes probably wouldn't change how people do business.
So I'm not sure how hard it would be to test BIP 101 on a test net (prefer the bitcoin test net!).
BIP 101 has been tested on private testnets. Gavin did this work months ago, with 20mb blocks. Things worked fine (which means, CPU/RAM/network usage increased linearly in line with block sizes).
Of course it's hard to simulate every detail of the network. In particular the interaction with the Great Firewall is especially problematic. But I'm not sure a bigger testnet would give us any more useful data at this point.
Besides, the block size debate has ceased to be about technological specifics. It's quite clearly become a fight over the arrangement of power and social philosophies. I don't think more testing would change anything.
What do you think an ordinary bitcoin user would want to know about XT?
If you mean practical details, it's on the website: https://bitcoinxt.software/
If you mean the higher level story, you can read my article "Why is Bitcoin forking?" https://medium.com/@octskyward/why-is-b ... 47312d22c1
Do you think we could possibly be reaching a point where a large enough group of industry leaders can come together and present a way forward and ostracise those who are attempting to hold it back?
At this point I think the best way for Bitcoin to survive and remain viable is if an industry alliance formed to get rid of Bitcoin Core. That alliance would have to include miners. I don't know if such an alliance might happen - what I've been hearing from industry and miners when I talk to them is that they're incredibly frustrated with Bitcoin Core. That project and its people are literally popular with nobody right now. But the miners in particular are very scared of switching to anything else, for a variety of reasons. I don't know how strong the 'fear factor' will end up being.
WRT Michael Marquardt's censorship on /r/Bitcoin, he doesn't seem able to accept that he might be wrong. The biggest problem that I think people are overlooking is that Marquardt/theymos also controls bitcoin.org .... a screwed up subreddit is probably survivable. A screwed up official website is potentially much more damaging.
If you had the opportunity to know who Satoshi was, would you take that opportunity?
If it were possible to get a quiet beer with him one day, I'd absolutely do it. I wouldn't care what his real name was, but I'm sure being able to talk to him in an informal setting would be fascinating.
lets assume that there will be a blocksize solution in december or january and XT will never gain enough traction. What are your plans in that scenario? Will you stay in the bitcoin ecosphere? Will you participape in the bitcoin development?
That depends on a lot of things, like the nature of the "solution" (I suspect if the limit is changed at all, it'll be something from blockstream that won't really solve the problem), and what work I'm doing professionally at the time.
I wouldn't participate in Bitcoin Core development again, if that's what you mean. It's important to understand that the issues are deeper than block sizes. Core has lost its way. They are just deeply uninterested in growth and mainstream adoption. For instance, they've turned their back on unconfirmed transactions, even though that's an important feature. They have spent a lot of time lately dumping on SPV wallets, even though they're the most popular kind of wallet on Android and becoming highly popular on iOS (via Breadwallet).
The differences of vision there are just too deep.
Hello mike,can bitcoin support a variable block size between 1 and 8 mb size ??
Remember the debate is about the upper limit, not the actual size. If BIP 101 activated tomorrow blocks wouldn't suddenly all become 8mb at once.
Could you please give a quick example of the differences and advantages of using a HD wallet for the average user.
HD wallets are where you get a 12 word phrase, and then the wallet gives you a constant stream of new addresses to use. The idea is you get privacy, but also easy backup-ability. Last year I implemented BIP 32 in bitcoinj so all wallets based on it could benefit.
Which digital currency do you think will be the biggest competitor to bitcoin in 5 years?
My intuition about the "blocksize debate" in general has been that there is a pretty concerted effort to ostracize you and Gavin from what Adam Back terms the "technical community ...... Am I viewing the situation with undue paranoia, or do you think there is some merit to perceiving the situation that way?
Yes to merit and yes to undue paranoia This isn't about Gavin or me specifically. Adam Back has also said listening to companies in general is a bad idea, because "it can give a misleading sense of consensus". I don't know what's up with Trace, he isn't very technical and seems to have been fed a garbled version of the March 2013 event. There's a full post-mortem document as a BIP if he wants to read it.
I'm not sure I ever mentioned this before, but a long time ago I studied psychology for a couple of years and got a qualification in it (a UK specific thing, not a degree). One of the topics we studied was group psychology. If you read the wikipedia page on groupthink, I think the first couple of paragraphs describe what's happened to Bitcoin Core quite clearly: a desire for harmony through unanimous consensus leading to irrational and dysfunctional decision making.
This is a well studied psychological phenomenon. In one famous early psychological experiment, since replicated, a test subject was put in a room and asked to fill out a pointless survey (they didn't know it was pointless). The experimenters then pumped smoke into the room. When the subject was alone 75% of people got up and left the room before the experiment was terminated. But when two accomplices were placed in the room as well and the accomplices ignored the smoke, that figure dropped to 10%. An apparent example of this effect in action was a fire in Manchester in 1979. Most people got out of the shopping centre in time, but those that died were almost all in the restaurant. The fire chief concluded they had been waiting to pay their bills because nobody wanted to be the first to get up and leave, then by the time they realised the danger they were in it was too late and they couldn't find the exits.
If the reason miners are not installing bitcoin xt now is that they are ddos'es how about first deprioritizing Bitcoin nodes running on the Tor network and then later when that is implemented support bigger block sizes
There have been different kinds of DoS against XT nodes. The most damaging are pure packet floods and cannot be fixed with code changes.
You said in this AMA that Core members "have spent a lot of time lately dumping on SPV wallets". Can you please say a little more about how they are doing that?
e.g. saying they are insecure, they suck, that actual existing wallets like Bitcoin Wallet for Android, MultiBit, BreadWallet etc aren't "real" SPV wallets, that the feature was badly designed etc. Recently they decided to make support for the protocol features they need optional, with the justification being DoS attacks, although someone did actually try this against XT nodes and it didn't work.
In general it's pretty clear they aren't going to do any work to improve this set of features, and I suspect they're quite likely at some point to just tell people to disable them.
What purpose (in your view) does the block size serve?
The purpose of the limit is to avoid DoS attacks by someone mining a massive block and then using it to explode nodes or break the block chain in some way. Same justification as why Satoshi put it in originally.
In particular, it isn't to try and bring about some sort of economic outcome, and it isn't to try and keep Bitcoin "decentralised", which in this context is a term frequently used but rarely defined.
Can you imagine hard forking XT before there is a super majority of miners for it and letting the market decide which coin they value more?]
In theory you only need 51%. But miners do have to be on board to at least some extent, obviously, as they're the ones making the blocks.
What are your thoughts about the asymmetry in the power structure between end users and miners? Is it something worth investing effort into balancing out and do you have any ideas about whether this would be technically possible and approaches?
It's hard to know what the solution is. Decentralising mining feels obvious, but would lots of smaller miners be better stewards of the network than a handful of big miners? At some level people have to want to vote, and they have to be informed voters.
To decentralise mining there are a bunch of technical things that could help. I wrote about them last year:
https://bitcoinfoundation.org/bitcoin/m ... ing-fruit/
Re: Adam Back
presuming a rough consensus on a scaling BIP including Wladimir, Greg, Pieter, Jeff & Gavin and many other experts and industry representatives etc with running code is standardised on will you collaborate with that? ..... I believe you are on record as saying you would support BIP100 also ...... Maybe you should come to the workshop and participate along with others ..... I think it's time for the community to pull together, what do you say?
OK, a whole bunch of things here.
If everyone except me ends up agreeing on a way to increase the block size, then I wouldn't have to collaborate on that, would I? It'd go ahead and happen anyway. Still, I think there's a risk is that Jeff and Gavin end up "agreeing" to a proposal that doesn't really make sense and they think is bad, just to try ensure something happens ..... but currently so little has happened that even this seems unlikely.
I don't recall saying I support BIP 100. I may have said if it happens anyway then XT would go along with it, obviously it'd have to, otherwise it wouldn't work properly anymore.
Workshop: right now neither me nor Gavin plan to attend Hong Kong.
Pulling together. I think the community has to get out of this notion that a vaguely defined community of experts hands decisions down from on high after a bunch of conferences or an IRC meeting or whatever. I realise why a lot of people like this idea but it's unhealthy. I wrote above about what can go wrong when groupthink takes hold. It's a very scary social failure mode, and yet isn't uncommon. Frankly right now I see a lot of the symptoms: the desire for social harmony through unanimous consensus, the attempts to minimise conflict at any cost, the 'illusion of invulnerability' etc.
Ironically, central banking has a lot of the same issues.
How can the dangers of groupthink be avoided? The wiki page has a few suggestions, but here are mine:
- For Core: replacing decision making through unanimity with a formal process for expressing and resolving disagreement, in a company this is usually a CEO, in a committee it can be a vote.
- Ensuring that for any decision that is made, the wider world has a clear and simple process for rejecting it. Any group must be able to be given "no" for an answer. In a country this is usually an election or a market. Gavin, myself and many others think a market for nodes which may cast different votes on changes to the block chain is the right way to do this.
- De-emphasising the role of (often self-proclaimed) intellectuals and experts, thus emphasising the judgement of work by results rather than the reputation of who was involved. Deference to expertise is a means to an end with the end being correct results, but in the Bitcoin community it's too often become the end itself.
Whilst I understand the rationale that went into proposing two separate conferences to discuss scalability, and the idea that everyone always gets along with each other is lovely, I don't think endless talk in the pursuit of 100% agreement is healthy or productive. The first conference didn't produce any concrete deliverable e.g. a requirements document, so I am skeptical the second will be more effective. And I've made these views clear many times before.
What do you think of Ethereum and Rootstock.io in relation to Bitcoin, Lighthouse and sidechains?
Huge question. I don't have space here to do a full Ethereum vs Bitcoin discussion, please remind me to write a blog post about it some time.
Bear in mind that the bulk of the work in Lighthouse was not actually the smart contracts bit. That isn't so complicated, actually. The hard work was in making it easy to use, in handling the P2P network, in handling the workflow, in defining a decentralised online update scheme etc.
I think some of these smart contract platforms tend to assume that all they need is a better scripting language and suddenly everything is easy. In reality that part is like 5% of the effort.
WRT side chains - the main problem I see with side chains is that they create a complexity explosion inside wallets, and side chains don't compose. That doesn't mean they're useless, they might be a kind of better testnet, but they aren't a replacement for putting their features directly into the Bitcoin block chain. By "don't compose" I mean consider what happens if you want to use features X and Y together but they exist only on separate side chains. You can't do it. With the features in the same block chain you can compose them, e.g. use a crowdfunding/assurance contract together with multisig.
Also, what are some of the coolest dentralized apps and projects you are excited about?
matrix.org seems cool. It's a bit tough to be excited by decentralised apps/projects right now because all the ones building on the Bitcoin block chain are being strangled by the block size problem, and the ones that built on altcoins/ethereum face the problem of lack of a currency.
Users are incentived to request the lowest fees and request UX features, and miners incentived to centralize to be more efficient and earn more money. Are they the best actors to take sound technical and non technical decisions ? Also, you forgot nodes that play an important role in the P2P network resiliency. Shouldn’t node maintainers also have something to say in the decision making process ?
Sorry, by "users" I meant literally anyone who uses the software. That includes node operators, merchants, etc. I discussed mining decentralisation above. Nodes: perhaps one day running nodes will be a business as well. Rewarding them with micropayments has been discussed on and off over the years.
My question is, will you be pursuing development on lighthouse as a more comercial project?
I'm not currently working on Lighthouse and have not been since around April. There's no point in developing Lighthouse further, or indeed any Bitcoin related product, unless the block size issue gets fixed .... otherwise the addressable market is too small to be able to justify the development costs.
In theory anyone could take it over though. It's open source. The online update system is multi-sig, so there's even potential to migrate existing users to the fork in a seamless manner (however, note that I own the Lighthouse logo).
Can you speak to XT's bandwidth requirements? Will it allow someone to run a node over TOR? Mining over TOR?
XT is just a patched Core, so its bandwidth requirements are the same. I posted a patch last week that makes XT download blocks as lists of hashes instead of duplicating the transaction data, so a full 1mb block would drop to about ~70 kilobytes, in theory, if you were online for the whole time since the last block. However, the patch needs more work and testing before it can be shipped to users.
Re: mining over Tor. Tor isn't actually very bandwidth constrained, if you check the graphs. The whole "omg mining over tor will break" thing doesn't make much sense if you scratch the surface. Gavin wrote about it here:
Generally, what thoughts do you have on the industry creating ecosystems that would naturally prefer/promote XT--perhaps transparently to the end users--and is the timing of this looking workable?
There's no requirement for the community to adopt XT specifically. Any node aligned with their interests would do, it could even be several different forks or reimplementations.
Industry has been pretty quiet over the past 7-8 months or so. Mostly I think they were hoping this whole nightmare would just go away. In recent days you saw Coinbase start to get more aggressive because they realised nothing was happening. I know there are other companies that would like to be more overt too but they're scared of theymos erasing them from bitcoin.org because they rely on referral traffic there.
Re: timing. The community is already out of time. At the start of the summer I predicted stagnant traffic levels until the northern hemisphere Autumn arrived which is what indeed happened, and then I went on to predict steady growth through the winter until the network was entirely out of capacity by the start of spring ..... with the caveat that sudden bubbles or price spikes could mean problems start earlier. That seems to have happened with the recent runup to $400. It's a bit hard to see because some miners aren't making blocks as big as they could, but we seem to be approaching some sort of soft traffic limit already.
What do you think of BIP 105 ?
The goal should be making it cheaper for people to process bigger blocks through software optimisation and better hardware. Not making it more expensive. This is just another convoluted attempt to make Bitcoin work badly for users.
A prominent Chinese mining operator (Marshall Long) recently stated that they not only won't switch to XT, they in fact do not update their software at all, under almost all circumstances (because it's "a waste of money") unless they "have to", i.e. forced by something overwhelming. Given that the Chinese control >51% of hashpower combined, it seems like this attitude can stagnate the code forever, XT or Core - and was probably what caused the BIP66 soft-fork fiasco, as they opted to simply flag their blocks instead of actually upgrading software to enforce anything. The only Chinese pool that seemed to actively care about Bitcoin, so far, seemed to be Antpool/Bitmain.
Do you think there's a way to overcome this? What can the rest of the community do to force their hands?
The general apathy and amount of handholding miners need has been a constant surprise to me. You'd expect people who fab their own ASICs and build their own datacenters would care a lot about the software they use, but apparently this often isn't true.
I do not have any solution to that problem. I think we'd have to understand it better first. Core to XT is just a node restart, which takes seconds. Perhaps they think it's something different: I've noticed a few miners seem to have a pretty garbled idea of what XT actually is.
I'm talking about a fork where you keep both sides of the split and blocks on either side are incompatible. You can do this with any percentage of miners. Not just 51% or more. If users decide to value one coin more, that coin will automatically attract more miners
You have to be a bit skeptical about this notion that a hard fork automatically results in two separate coins. Remember that transactions don't contain any information about which fork they're valid on. Only newly mined coins would be fork-specific, but over half of all bitcoins are already in circulation.
If you really wanted a block size hard fork to create two different currencies, you'd have to do a lot of complicated work to get that outcome, like by trying to take newly mined coins and send them as dust to literally every bitcoin user, to try and convince their wallets to spend them and make a chain-specific transaction. But wallets could easily be programmed to avoid that trick. Alternatively you could try and trigger a double spend that gets resolved in opposite ways on either side of the chain, if you didn't want to wait for new coins, but again ... same deal.
For unpolluted coins a payment for one side of the chain would automatically be valid on the other. Keeping them truly separate would be tough, so expecting there to be a market for trading across chains is ..... interesting.
That's why me and Gavin say that when there's a hard fork, the people on the minority side would just upgrade and continue on the majority side. Of course to some extent this is like saying, "if there is an election then the losing side will follow the laws passed by the new party instead of starting a civil war". It's an assumption about the basic level of civilisation people have. When people say "if we lose the vote we'll fight to the bitter end" then perhaps the community does need to split into two currencies. The logistics of that would be complicated.
What was your first physical item you bought with bitcoin?
I don't remember! But I suspect it was a beer. I definitely remember meeting someone in Zürich in a cafe and we traded bitcoins for cash using our laptops. This was before SPV or web wallets so we both had to use Bitcoin-Qt (before it was called that). It took about an hour!
Why has "thin blocks" never been implemented in Core?
Good question. I don't know. My guess: one of, the block chain has no future so why bother to improve it ... or ... dislike of the Bloom filtering protocol that thin blocks is based on ...... or, "this solution is insufficiently perfect so let's wait for something better" (but more complex).
Do you think it will be common in coffee shops/restaurants for users to just hand merchants signed transactions rather than broadcasting them to the Bitcoin network themselves? I was in a restaurant once with a poor internet connection and couldn't pay with Bitcoin
I already anticipated and implemented a solution to this problem years ago. BIP 70 which I authored with Gavin allows merchants to specify they want to receive transaction data directly and broadcast it themselves. Additionally I implemented (with Andreas Schildbach) Bluetooth support in Bitcoin Wallet for Android, meaning if the recipient uses the same app, the buyer sends signed transaction data to the recipient using local radio. The recipient then broadcasts it themselves. This can resolve many cases where the buyer does not have internet access.
To make this scheme work better it would need enhancement and then standardisation. At that point other wallets and POS vendors could implement it. However, this work was never done.
Re: John Nash.
Bitcoin is not economically useful as a coffee money, regardless of whether or not you COULD design it that way. What this world needs incredibly badly is a akin new gold standard. It would be silly for me to explain why, the argument has been perfectly and extensively laid out by Nash.
Nash sounds like a guy I'd agree with. I also would like the world to have a currency with a stable money supply and I'd agree with his analysis about the problems of money printing.
Where we disagree is the idea that you can/should have a money which is not actually usable as money by ordinary people. This is sort of like calling Treasury bonds "money". Sure, it is if you squint, but if I can't actually earn it and spend it in my ordinary life then to me, it's not money. Satoshi very clearly intended Bitcoin to be ordinary, every day money .... he talked about vending machines and things like that.
Still, I don't really understand the basis of your complaint. There is no need to choose between coffee-shop money and a Nash-style global stable currency. It's possible for one system to be both these things. Go review the calculations I did in 2011. Bitcoin can handle really huge amounts of traffic, even with today's hardware!