I think there isn't wide appreciation of how important Gavin's proposal is for the scalability of Bitcoin. It's the real deal, and will get us out of this sort of beta mode we've been in of a few transactions per second globally. I spent a few hours reviewing the papers referenced at the bottom of his excellent write-up and think I get it now.
If you already get it, then hang around and answer questions from me and others. If you don't get it yet, start by very carefully reading https://gist.github.com/gavinandresen/e20c3b5a1d4b97f79ac2
The big idea is twofold: fix the miner's incentives to align better with users wanting transactions to clear, and eliminate the sending of redundant data in the newblock message when a block is solved to save bandwidth.
I'll use (arbitrarily) a goal of 1 million tx per block, which is just over 1000 TPS. This seems pretty achievable, without a lot of uncertainty. Really! Read on.
Today, a miner really wants to propagate a solved block as soon as possible to not jeopardize their 25 BTC reward. It's not the cpu cost for handling the transactions on the miner's side that's the problem, it's the sending of a larger newblock message around the network that just might
cause her block to lose a race condition with another solution to the block.
So aside from transactions with fees of more than 0.0008 BTC that can make up for this penalty (https://gist.github.com/gavinandresen/5044482
), or simply the goodwill of benevolent pools to process transactions, there is today an incentive for miners not
to include transactions in a block. The problem is BTC price has grown so high so fast that 0.0008 BTC is about 50 cents, which is high for day-to-day transactions (and very high for third world transactions).
The whole idea centers around an old observation that since the network nodes (including miners) have already received transactions by the normal second-by-second operation of the p2p network, the newblock announcement message shouldn't have to repeat the transaction details. Instead, it can just tell people, hey, I approve these particular transactions called XYZ, and you can check me by taking your copy of those same transactions that you already have and running the hash to check that my header is correctly solved. Proof of work.
A basic way to do this would be to send around a Bloom filter in the newblock message. A receiving node would check all the messages they have, see which of them are in this solved block, and mark them out of their temporary memory pool. Using a BF calculator you can see that you need about 2MB in order to get an error rate of 10e-6 for 1 million entries. 2MB gives 16 million bits which is enough to almost always be able to tell if a tx that you know about is in the block or not.
There are two problems with this: there may be transactions in the solved block that you don't have, for whatever p2p network or policy reason. The BF can't tell you what those are. It can just tell you there were e.g. 1,000,000 tx in this solved block and you were able to find only 999,999 of them. The other glitch is that of those 999,999 it told you were there, a couple could be false positives. I think there are ways you could try to deal with this--send more types of request messages around the network to fill in your holes--but I'll dismiss this and flip back to Gavin's IBLT instead.
The IBLT works super well to mash a huge number of transactions together into one fixed-size (O(1)) data structure, to compare against another set of transactions that is really close, with just a few differences. The "few differences" part compared to the size of the IBLT is critical to this whole thing working. With too many differences, the decode just fails and the receiver wouldn't be able to understand this solved block.
Gavin suggests key size of 8B and data of 8B chunks. I don't understand his data size--there's a big key checksum you need in order to do full add and subtract of IBLTs (let's say 8B, although this might have to be 16B?) that I would rather amortize over more granular data chunks. The average tx is 250B anyway. So I'm going to discuss an 8B key and 64B data chunks. With a count field, this then gives 8 key + 64 data + 16 checksum + 4 count = 92B. Let's round to 100B per IBLT cell.
Let's say we want to fix our newblock message size to around 1MB, in order to not be too alarming for the change to this scheme from our existing 1MB block limit (that miners don't often fill anyway). This means we can have an IBLT with m=10K, or 10,000 cells, which with the 1.5d rule (see the papers) means we can tolerate about 6000 differences in cells, which because we are slicing transactions into multiple cells (4 on average), means we can handle about 1500 differences in transactions at the receiver vs the solver and have faith that we can decode the newblock message fully almost all the time (has to be some way to handle the occasional node that fails this and has to catch up).
So now the problem becomes, how can we define some conventions so that the different nodes can mostly
agree on which of the transactions flying around the network for the past N (~10) minutes should be included in the solved block. If the solver gets it wrong, her block doesn't get accepted by the rest of the network. Strong incentive! If the receiver gets it wrong (although she can try multiple times with different sets), she can't track the rest of the network's progress.
This is the genius part around this proposal. If we define the convention so that the set of transactions to be included in a block is essentially all of them, then the miners are strongly incentivized, not just by tx fees, but by the block reward itself to include all those transactions that happened since the last block. It still allows them to make their own decisions, up to 1500 tx could be added where convention would say not to, or not put in where convention says to. This preserves the notion of tx-approval freedom in the network for miners, and some later miner will probably pick up those straggler tx.
I think it might be important to provide as many guidelines for the solver as possible to describe what is in her block, in specific terms as possible without actually having to give tx ids, so that the receivers in their attempt to decode this block can build up as similar an IBLT on their side using the same rules. Something like the tx fee range, some framing of what tx are in the early part and what tx are near the end (time range I mean). Side note: I guess if you allow a tx fee range in this set of parameters, then the solver could put it real high and send an empty block after all, which works against the incentive I mentioned above, so maybe that particular specification is not beneficial.
for example, the propagation delay is about 30-40 seconds before almost all nodes have received any particular transaction, so it may be useful for the solver to include tx only up to a certain point in time, like 30 seconds ago. Any tx that is younger than this just waits until the next block, so it's not a big penalty. But some policy like this (and some way to communicate it in the absence of centralized time management among the nodes) will be important to keep the number of differences in the two sets small, below 1500 in my example. The receiver of the newblock message would know when trying to decode it, that they should build up an IBLT on their side also with tx only from up to 30 seconds ago.
I don't understand Gavin's requirement for canonical ordering. I see that it doesn't hurt, but I don't see the requirement for it. Can somebody elaborate? It seems that's his way to achieve the same framing that I am talking about in the previous paragraph, to obtain a minimum number of differences in the two sets. There is no need to clip the total number of tx in a block that I see, since you can keep shoving into the IBLT as much as you want, as long as the number of differences is bounded. So I don't see a canonical ordering being required for clipping the tx set. The XOR (or add-subtract) behavior of the IBLT doesn't require any ordering in the sets that I see, it's totally commutative. Maybe it's his way of allowing miners some control over what tx they approve, how many tx into this canonical order they want to get. But that would also allow them to send around solved empty blocks.
What is pretty neat about this from a consumer perspective is the tx fees could be driven real low, like down to the network propagation minimum which I think as of this spring per Mike Hearn is now 0.00001 BTC or 10 "bits" (1000 satoshis), half a US cent. Maybe that's a problem--the miners get the shaft without being able to bid on which transactions they approve. If they try to not approve too many tx their block won't be decoded by the rest of the network like all the non-mining nodes running the bitpay/coinbases of the world.
Edit: 10 bits is 1000 satoshis, not 10k satoshis
According to Mark T. Williams, as of 2014, bitcoin has volatility seven times greater than gold, eight times greater than the S&P 500, and 18 times greater than the U.S. dollar. Attempting to explain the high volatility, a group of Japanese scholars stated that there is no stabilization mechanism. The Bitcoin Foundation contends that high volatility is due to insufficient liquidity, while a ... Mike Hearn Bitcoin - Bitcoin Tax Software Mike Hearn Bitcoin Bitcoin Transaction Flow Where Can I Spend Bitcoins Online One of the Bitcoin Core’s developers, Mike Hearn, received an email one fateful day with Satoshi Nakamoto saying that the creator of Bitcoin has “moved on to other things” and felt that Bitcoin’s future was left “in good hands.” When Satoshi “retired,” there were a few notable things left behind: the premise of the workings of Bitcoin, a collection of notes and writings, and ... In the earliest days of Bitcoin, once the genesis block had been mined on Jan 3, 2009, Nakamoto sent messages back and forth with people such as Hal Finney, Mike Hearn, and Gavin Andresen ... Press question mark to learn the rest of the keyboard shortcuts. Log in sign up. User account menu . 2. Bitcoin profit/loss calculator. Close. 2. Posted by 2 years ago. Archived. Bitcoin profit/loss calculator. Hey everyone! I made a simple calculator to help beginners calculate how much money they will make when a specific BTC value is reached. Project is open source and code can be found at ...
The resolution of the Bitcoin experiment — Medium (2016-01-14 medium.com) Bitcoin Post by Mike Hearn ... Mark Yusko on How we got to QE Infinity from the Fed - Duration: 1:06:39. Anthony ... Maximum daily profits proved using my crypto trade calculator Net Profit: $671.88 ROI: 38.98% Must watch video: https://www.youtube.com/watch?v=Xc9TwWJZoWM&l... We are moving into spiritually dark times and the end is near, but many are unprepared. "We have never been closer to the end than today," says Pastor Greg. ... Reposted from Newfination. 2013 interview. Max Keiser and Stacy Herbert discuss Mike Hearn whiny rage quitting Bitcoin in order to join a banking system with life-or-death power but where no real auditing is possible. In the second half ...