User avatar
rogerver
Founder
Founder
Posts: 1868
Joined: Thu Sep 10, 2015 6:55 am

Donate BTC of your choice to 1PpmSbUghyhgbzsDevqv1cxxx8cB2kZCdP

Contact: Website Twitter

Time to sync a full node from scratch

Tue Jun 28, 2016 5:52 pm

I'm building a new system and I think it will be interesting to see how long it takes for it to sync the entire block chain from start to finish.
The system specs are:
  • Intel Core i7-5820K Haswell-E 6-Core 3.3GHz LGA 2011-v3 140W Desktop Processor $374.69
  • IKingston Technology HyperX Predator 32 GB RAM Kit, 4 x 8 GB 3000MHz DDR4 Non-ECC CL15 DIMM XMP PC4-19200 $216.94
  • IASUS ROG RAMPAGE V EXTREME/U3.1 LGA2011-V3 DDR4 M.2 ACWiFi USB 3.1 Type A Intel X99 eATX Motherboard $469
  • IMSI GAMING GeForce GTX 970 4GB OC DirectX 12 VR READY (GTX 970 GAMING 4G) Video Card $299
  • ICorsair Obsidian Series 750D Performance Full Tower Case $145.44
  • ICorsair Hydro Series H110 280 mm High Performance Liquid CPU Cooler $118.17
  • ISamsung XP941 512GB AHCI M.2 80mm PCIe 2.0 x4 SSD $249.90
  • ICorsair RM Series, RM1000, 1000 Watt (1000W), Fully Modular Power Supply, 80+ Gold Certified $188
Grand total, including the Purse.io 20% discount: $1,649.71

Who want's to guess how long it will take for it to sync the entire blockchain from start to finish?
I'll gladly let everyone know once all the parts show up. I'll even test out the time lapse video feature of my iphone so everyone can see for themselves exactly how long it will take. I'm guessing it will just be a few hours.
Help spread Bitcoin by linking to everything mentioned here:
topic7039.html


User avatar
rogerver
Founder
Founder
Posts: 1868
Joined: Thu Sep 10, 2015 6:55 am

Donate BTC of your choice to 1PpmSbUghyhgbzsDevqv1cxxx8cB2kZCdP

Contact: Website Twitter

Re: Time to sync a full node from scratch

Wed Jun 29, 2016 6:06 am

I'll go with 32 hours. Takes quite awhile for that last year of data.
If I can download 5MB / second, I think it will only take about 4.5 hours to download the whole block chain.
I'm looking forward to everything arriving by next week.
Help spread Bitcoin by linking to everything mentioned here:
topic7039.html

FUBAR-BDHR
Nickel Bitcoiner
Nickel Bitcoiner
Posts: 102
Joined: Fri Nov 06, 2015 9:56 pm

Donate BTC of your choice to 1KXcTNYKNfSpSBixvs52sB3C6y3PtKY8GR

Contact: Website Twitter

Re: Time to sync a full node from scratch

Wed Jun 29, 2016 6:47 am

Doesn't matter how fast your download speed is the bottleneck to initial sync is CPU/IO related and bootstrapping doesn't help anymore.

What needs to happen in the future is USB sticks or preloaded drives with the blockchain on them then is then verified after load so you can use it while verifying.

DOGed
Nickel Bitcoiner
Nickel Bitcoiner
Posts: 69
Joined: Sat Nov 21, 2015 6:25 am

Donate BTC of your choice to 136djiDtVHKq8KLnJQnCRJL2TydpGgnyyT

Re: Time to sync a full node from scratch

Wed Jun 29, 2016 8:44 am

This depends on if you plan on using a classic or a core node. If you are using a classic node then you might get DDoS'ed, causing it to take about 2-3 days or so to sync.

If I can download 5MB / second,
If you can avoid getting DDoS'ed then I would say within 12 hours.
Have fun, send me a tip 136djiDtVHKq8KLnJQnCRJL2TydpGgnyyT

User avatar
TomZ
Nickel Bitcoiner
Nickel Bitcoiner
Posts: 111
Joined: Thu Oct 29, 2015 5:28 pm
Contact: Website Twitter

Re: Time to sync a full node from scratch

Wed Jun 29, 2016 2:40 pm

This depends on if you plan on using a classic or a core node. If you are using a classic node then you might get DDoS'ed, causing it to take about 2-3 days or so to sync.

If I can download 5MB / second,
If you can avoid getting DDoS'ed then I would say within 12 hours.
DDoSed??

I haven't heard a single report for months...

User avatar
TomZ
Nickel Bitcoiner
Nickel Bitcoiner
Posts: 111
Joined: Thu Oct 29, 2015 5:28 pm
Contact: Website Twitter

Re: Time to sync a full node from scratch

Wed Jun 29, 2016 2:49 pm

I'm building a new system and I think it will be interesting to see how long it takes for it to sync the entire block chain from start to finish.
The system specs are:
{snip}

Who want's to guess how long it will take for it to sync the entire blockchain from start to finish?
I'll gladly let everyone know once all the parts show up. I'll even test out the time lapse video feature of my iphone so everyone can see for themselves exactly how long it will take. I'm guessing it will just be a few hours.

The downloading of the blockchain has gotten quite a bit faster in the last year. There are still some bottlenecks left, though.

Every block has a proof-of-work that gets checked. This is pretty cheap to check. But in many blocks every single transaction is also checked. This means that the transaction is checked for validity and that the signatures actually validate. This is much more expensive.

In Bitcoin there is something called 'checkpoints' which mark a date after which all validation is done. Unfortunately this hasn't been updated for ages, so expect lots of CPU to be used...

In Bitcoin Classic I added some code that automatically determines a checkpoint a week in the past, making validation much cheaper. But this change has not been released yet.

There are some other issues, for instance the effective utilization of the CPU is pretty poor. Making it just take longer, whatever hardware you buy.

Last; the UTXO database has a in-memory cache setting. It is set by default to a rather low value. For an initial sync I suggest a couple of GB, on your machine set it to the max of 16GB :) -dbcache=16384

My guess would be that you can then complete it in 9 hours.

User avatar
rogerver
Founder
Founder
Posts: 1868
Joined: Thu Sep 10, 2015 6:55 am

Donate BTC of your choice to 1PpmSbUghyhgbzsDevqv1cxxx8cB2kZCdP

Contact: Website Twitter

Re: Time to sync a full node from scratch

Sun Jul 10, 2016 8:09 am

The answer is in:
https://www.youtube.com/watch?v=vL8ZRYsLdoY
  • Ubuntu 16.04 LTS
  • Bitcoin Classic Version 1.1.0
I think the people that are so concerned over the block size really aren't seeing the bigger picture.
I didn't sync the full node with a super computer, I did it with a fairly modest home computer that costs about half as much as a single S9 Bitcoin mining rig from Bitmain.

For those that missed it, I previously wrote down my thoughts on why it's Time to End the Block-Size Blockade on the Foundation for Economic Education's website.

People who think that Bitcoin can't scale quickly today, aren't paying attention.
Help spread Bitcoin by linking to everything mentioned here:
topic7039.html


User avatar
LiteCoinGuy
Gold Bitcoiner
Gold Bitcoiner
Posts: 2505
Joined: Mon Sep 21, 2015 9:00 am

Donate BTC of your choice to 1Dbo5TtxG9cWoyw49GM8vbD7HgQhr1KVi6

Re: Time to sync a full node from scratch

Sun Jul 10, 2016 12:36 pm

Pretty good. Last time I synced it took almost a week but that was on a core2duo 2.4gh with 2gig of ram running xp. Since move to a better machine but I just copied the data directory so no resync required.
uhhh, i know that :oops: - i can take a week with a crappy pc.
********************************************
More informations about Bitcoin and scaling BTC on

bitcoin.org/en/

https://bitcoincore.org/en/2015/12/23/c ... reases-faq

&
reddit.com/r/Bitcoin/

User avatar
TomZ
Nickel Bitcoiner
Nickel Bitcoiner
Posts: 111
Joined: Thu Oct 29, 2015 5:28 pm
Contact: Website Twitter

Re: Time to sync a full node from scratch

Sun Jul 10, 2016 6:04 pm

Love the video, simple and to the point.

It should take less time even with a development version of bitcoin-classic, as I explained a couple of posts ago.

But the bottom like that Roger makes, and I fully agree with, is that a simple, not server, machine can catch up with bitcoins years of history in this short amount of time, then we really don't have to worry about scaling up for quite some time

FUBAR-BDHR
Nickel Bitcoiner
Nickel Bitcoiner
Posts: 102
Joined: Fri Nov 06, 2015 9:56 pm

Donate BTC of your choice to 1KXcTNYKNfSpSBixvs52sB3C6y3PtKY8GR

Contact: Website Twitter

Re: Time to sync a full node from scratch

Sun Jul 10, 2016 8:52 pm

It also shows that the technology is keeping up with if not staying ahead of the requirements to run a node.

Still I bet the time could be improved dramatically. Wonder if some kind of checkpoint system would be helpful. Say a monthly archive that is already verified, signed, and just verified against a transaction on the blockchain. Then you would only have to build a few weeks of data yourself at most. Thumb drives could be available with all but the most recent history on them. Advanced bootstrapping.

User avatar
TomZ
Nickel Bitcoiner
Nickel Bitcoiner
Posts: 111
Joined: Thu Oct 29, 2015 5:28 pm
Contact: Website Twitter

Re: Time to sync a full node from scratch

Mon Jul 11, 2016 3:38 pm

When you think about it, syncing 7 years of Bitcoin in 7 hours, thats pretty impressive.

I'm aware that this isn't the cheapest hardware, but is not extreme either.

I checked some small facts; the last checkpoint was at block 295000, which means that 125000 blocks were fully validated. I wrote a little app to count the amount of transactions and its more than 100 million of them. So, with that in mind you can calculate that this box would easily be able to download, store and validate 368 million transactions per day (in a 24 hours period). Makes you wonder what exactly we need to work on to make Bitcoin scale, doesn't it?

I wrote a bit more in depth here; http://zander.github.io/posts/Scaling%20Bitcoin/

User avatar
rogerver
Founder
Founder
Posts: 1868
Joined: Thu Sep 10, 2015 6:55 am

Donate BTC of your choice to 1PpmSbUghyhgbzsDevqv1cxxx8cB2kZCdP

Contact: Website Twitter

Re: Time to sync a full node from scratch

Mon Jul 11, 2016 4:27 pm

When you think about it, syncing 7 years of Bitcoin in 7 hours, thats pretty impressive.

I'm aware that this isn't the cheapest hardware, but is not extreme either.
Actually, if you look at what I bought, most of it wasn't needed for a node, and you could have had the same time with a system that costs about $800.
I didn't need the fancy video card, power supply, case, mother board, or memory kit.

Maybe tomorrow I will do a similar experiment with a computer I built when I first moved to Japan back in 2006.
It was state of the art at the time, but nearly garbage now. I suspect with the addition of a $100 SSD drive, it will be able to sync the entire blockchain in far less than a day.
Help spread Bitcoin by linking to everything mentioned here:
topic7039.html

iFixBTCmemoryIssues
Gold Bitcoiner
Gold Bitcoiner
Posts: 2682
Joined: Tue Nov 24, 2015 9:03 pm

Re: Time to sync a full node from scratch

Mon Jul 11, 2016 6:21 pm

When you think about it, syncing 7 years of Bitcoin in 7 hours, thats pretty impressive.

I'm aware that this isn't the cheapest hardware, but is not extreme either.
Actually, if you look at what I bought, most of it wasn't needed for a node, and you could have had the same time with a system that costs about $800.
I didn't need the fancy video card, power supply, case, mother board, or memory kit.

Maybe tomorrow I will do a similar experiment with a computer I built when I first moved to Japan back in 2006.
It was state of the art at the time, but nearly garbage now. I suspect with the addition of a $100 SSD drive, it will be able to sync the entire blockchain in far less than a day.
It's okay, you became a kid in a candy store, I have this same problem when shopping for cars :)
Image

If you are running a version of Bitcoin Core, stop using it. Upgrade to Bitcoin Unlimited or Classic immediately.

Fix Your Unconfirmed Transaction.

Vote for the future of our Bitcoin network!

Return to “Bitcoin Discussion”

Who is online

Users browsing this forum: No registered users and 11 guests