Big-O scaling - By Gavin Andresen

[kerneldump]

http://gavinandresen.svbtle.com/are-big ... -dangerous

[Melbustus]

Solid post. Gavin's ability to cut to the core *practical* ramifications of technical issues is a greatly needed skill in bitcoin dev, else we get the very narrow linear ultra-extrapolation thinking that's holding bitcoin core back.

And on a technical note regarding the first point that Gavin refutes - Metcalfe's Law's n^2 scaling - it may be more reasonable to treat Metcalfe's Law as (n)log(n) for large n: http://spectrum.ieee.org/computing/netw ... w-is-wrong .

[haq4good]

There are some problems with max block size scaling that are pretty technical and not easily discussed (because they describe vulnerabilities that could be exploited).

I am grateful that the "debate" is happening and that it is continuing to happen. The Mob rule of 'consensus' is just fine, except when it isn't. In this case I can pretty clearly see the result of what would have happened if a size increase was rushed and everyone was still on the same codebase.

Bigger block would be mined, and eventually one would be mined that would take >10 minutes to validate (not propagate, validate). Some miners begin building the next block before validation, others don't wait. Those that don't wait would have an advantage equal to the creator of the hard-to-validate block.
There is a pernicious effect here in that it encourages miners to not validate. There are also potential exploits based on the protocol encouraging miners to make blocks as hard to validate as possible as it gives that miner a head start on following blocks.

There is some work ongoing in core to mitigate these risks, and to improve on validation time processes to make reduce this risk.

It would have been a bit tragic to have rolled out a large block codebase if there were not another codebase to switch onto if there were a big problem. For this reason I am very grateful to Gavin for moving to XT for development. It reduces the codebase single point of failure and thus makes the protocol more robust.

I suspect that most any serious miner would be running nodes with both XT and Core, so that if a major problem surfaces, they can switch mining to the more functional codebase.


The increased complexity of multiple codebase switchings is one of the rare cases where complexity helps rather than hurts reliability.

[LiteCoinGuy]

There are some problems with max block size scaling that are pretty technical and not easily discussed (because they describe vulnerabilities that could be exploited).

I am grateful that the "debate" is happening and that it is continuing to happen. The Mob rule of 'consensus' is just fine, except when it isn't. In this case I can pretty clearly see the result of what would have happened if a size increase was rushed and everyone was still on the same codebase.

Bigger block would be mined, and eventually one would be mined that would take >10 minutes to validate (not propagate, validate). Some miners begin building the next block before validation, others don't wait. Those that don't wait would have an advantage equal to the creator of the hard-to-validate block.
There is a pernicious effect here in that it encourages miners to not validate. There are also potential exploits based on the protocol encouraging miners to make blocks as hard to validate as possible as it gives that miner a head start on following blocks.

would it be possibe to change that with code? i mean that all miners have to wait that the block is validated and then move to the next one together?

[haq4good]

would it be possibe to change that with code? i mean that all miners have to wait that the block is validated and then move to the next one together?

It is certainly possible to write any code we please, but unless the protocol were changed to limit this, there would need to be some really good reason to run such code and never deviate.

A modification of this type would be a soft fork, as it reduces the universe of possible blocks rather than increases.

I'd consider a better fix to be writing improvements in how blocks are validated into the code. This would give advantage to miners that run such code. Since it is open source, the developments will swiftly propagate into all codebases. I suspect that most developers would agree on this matter.

[LiteCoinGuy]

Thanks for the explanation. Good to have you here haq4good.

Are you a contributor to bitcoin qt or xt or something?

[rebuilder]

Bigger block would be mined, and eventually one would be mined that would take >10 minutes to validate (not propagate, validate).

I'm not familiar with this issue. Are we talking about validation taking over 10 minutes due to the amount or size of transactions included in the block? Is there an estimate on how big a block would have to be for this to happen?

[vanilladev]

Bigger block would be mined, and eventually one would be mined that would take >10 minutes to validate (not propagate, validate)

We can calculate the number of transactions to fill a big block and how long each one takes to verify and it doesn't come close to > 10 minutes. The Bitcoin wiki has this to say:

...to perform around 8000 signature verifications per second on an quad core Intel Core i7-2670QM 2.2Ghz processor. The average number of inputs per transaction is around 2, so we must halve the rate. This means 4000 tps is easily achievable CPU-wise with a single fairly mainstream CPU.

[gavinandresen]

RE: minutes-to-verify blocks: see the "Validation Sanity" talk I gave last week at DevCore San Mateo. PDF here:

https://bitcoincore.org/~gavin/ValidationSanity.pdf

... I believe video will be available soon.