Mathematics in finance and hiding lies in complexity
October 13, 2013 13 Comments
Few believe that Fermat actually had a correct proof, because the conjecture remained open for over 350 years and when Andrew Wiles resolved it in 1993, it used the deepest mathematical ideas and technologies from the 20th century. Wiles had encountered the ‘theorem’ as a ten year old, but quickly realized that he was not equipped to tackle it. Although the question stayed in the back of his mind, it was not until 19821986 that Gerhard Frey, JeanPierre Serre, and Ken Ribet built a bridge between Fermat’s conjecture and the mainstream study of elliptic curves — showing the mathematical community that the Taniyama–Shimura conjecture for semistable elliptic curves would imply Fermat’s last theorem — that Wiles had the audacity to change his research direction, skirt his teaching responsibilities at Princeton, and invest himself completely in finding a proof^{[3]}.
Had Wiles not resolved the conjecture in 1993, Fermat’s Last Theorem would have likely been made one of the Millennium Prize Problems. In 1998, Landon T. Clay — an armchair mathematician that majored in English at Harvard and went on to become a money manager, CEO of Eaten Vance Investment Managers, and one of the richest people in Boston — underwrote the million dollar prizes and several other awards when he founded the Clay Mathematics Institute. Unfortunately, the public largely knows the prize for the size of its purse, and not the fundamental importance of the problems^{[4]}.
Clay’s most recent philanthropy was a sizable contribution to the construction of the Andrew Wiles math building at the University of Oxford. At the opening ceremony, Wiles bemoaned the abuse of mathematics during the financial crisis, saying that “one has to be aware now that mathematics can be misused and that we have to protect its good name.”
This was probably a reference to the widespread use of complex derivatives, and the use of models like VaR to hide risk in the long tails of outcome distributions. Of course, no one can seriously blame just mathematics for this — although the financial firms have done their best to throw quants under the bus. Mathematicians provide a tool, and it is up to the users of said tools to turn them to good or evil. However, that also doesn’t excuse mathematicians of ethical consideration in building these tools. It is a matter of figuring out: are we building guns or screwdrivers? Sure, in the wrong hands either can be used to injury or kill a person, but it is clear that one of the tools is designed solely for that purpose while the other is intended as a constructive implement.
For defenders of economics and finance, the popular story is that complex derivatives like collateralized debt obligation, and credit default swaps allow participants to “complete the market” and reduce the effects of asymmetric information (DeMarzo, 2005). In particular, the informationempowered seller can find buyers for the informationinsensitive part of the asset’s cash flow and retain the informationsensitive part. Detractors of finance, point out that — in practice — pricing (or rating the risk of) a CDO is not robust even to very modest imprecisions in evaluating underlying risks (including systemic risk; see Coval et al., 2009). It is these mispricings of derivatives that most analyses place as central to the recent financial crisis (Brunnermeier, 2009; Coval et al., 2009).
So, are CDOs/CDSs a screwdriver turned murder weapon in the hands of greedy unethical bankers, or a loaded handgun in the hands of a unknowing child? The algorithmic lens suggests that it might be the latter. Arora et al. (2011) showed that for computationally bounded market participants DeMarzo’s (2005) perfect rationality analysis does not hold, and derivatives can actually amplify (instead of reducing) the cost of asymmetric information. The way common complex derivatives are set up^{[5]}, allows sellers to cherrypick the packaged assets in such a way that the buyer cannot detect the hidden risk^{[6]}. Think of this as analogous to how Amazon can find two large prime numbers to serve as a public key, but you or creditcard thieves can’t factor that key to crack the RSA encryption. Except in this case, the thieves are doing the encoding and the honest party is tasked with cracking, and the difficult problem is finding the densest subgraph instead of factoring. In other words, complex derivatives are set up in such a way that it is easy to hide dishonesty behind their complexity^{[7]}.
In this case, the blame is not on the side of buyers pricing models and algorithms, or of inherent market information asymmetry. The fundamental laws of computation stop them from being able to perform better. The blame is that the tool of complex derivatives is inherently unfair, or too easy to use for unfair ends. By using CDOs/CDSs, we are giving more power to people who have better access to information. Thus, we are widening an already existing power gap, something that many (but obviously not all) would find to be unethical. Without taking into account computational complexity, this is impossible to see, but even with the algorithmic lens to inform us, it is ultimately an ethical decision that needs to be made. Unfortunately, it seems like the ethics of bankers and mathematicians are fundamentally unaligned, and this makes it particularly difficult for wellmeaning mathematicians to imagine how their models will be misused.
Notes and References

it is impossible to separate a cube into two cubes, or a fourth power into two fourth powers, or in general, any power higher than the second, into two like powers. I have discovered a truly marvelous proof of this, which this margin is too narrow to contain.

As a testament to its difficulty, when David Hilbert was asked why he did not attempt to prove Fermat’s conjecture, he responded:
Before beginning I should have to put in three years of intensive study, and I haven’t that much time to squander on a probable failure.
However, that didn’t stop him from believing that all Diophantine equations can be solved by a mechanistic procedure, which he conjectured as his tenth problem at the 1900 International Congress of Mathematicians. By 1970, Yuri Matiyasevich resolved this conjecture in the negative showing that for an arbitrary Diophantine equation, asking if some integer solution exists is undecidable. This gave a computational justification for the difficulty of particular instances or families of equations like the ones in Fermat’s theorem.
 Andrew Wiles words summarizing his experience searching for the proof capture the fear, wonder, and locallypointless but globallynecessary meandering one is overwhelmed by when working on mathematics:
You enter the first room of the mansion and it’s completely dark. You stumble around bumping into the furniture but gradually you learn where each piece of furniture is. Finally, after six months or so, you find the light switch, you turn it on, and suddenly it’s all illuminated. You can see exactly where you were. Then you move into the next room and spend another six months in the dark. So each of these breakthroughs, while sometimes they’re momentary, sometimes over a period of a day or two, they are the culmination of, and couldn’t exist without, the many months of stumbling around in the dark that precede them.
 The Poincare conjecture — every simply connected, closed 3manifold is homeomorphic to the 3sphere — is the only millennium problem that has been solved to date. To highlight the difference between the public obsession with the monetary incentives and a mathematician’s drive, Grigori Perelman — the mathematician that solved the Poincare conjecture — declined both the Fields medal and millennium prize. For him, it was a matter of fairness to previous mathematicians (such as Richard Hamilton that developed the Ricci flow technique perfected by Perelman) that dedicated themselves to the problem. Unable to remain complacent with perceived ethical degradation of the mathematical community, Perelman left his academic job, severed all ties with former colleagues, and (some believe that he) spends his time practicing wall stacking in sleazy Riichi Majong dives in Saint Petersburg, where he lives with his mother.
 In their paper, Arora et al. (2011) analyze a simplified model of derivatives, but similar tricks would be even easier in the derivatives actually used on Wall St. One of their open problems is if we can use a more standard or difficult hard problem to improve the negative results by looking more realistic models of derivatives. See their FAQ for more information.
 Note that this ability to deceive is not purely a consequence of a lack of information on the part of the buyers. In particular, what DeMarzo (2005) showed is that a computationally unbounded buyer cannot be tricked even though they lack some information (just like publickey cryptography can’t work if the hackers are computationally unconstrained). Much like the evolutionary results of Livnat & Pippenger (2008), these systematic mistakes on the part of the buyer stem not from a lack of information but the inability to do arbitrary computations.
 I suspect that finance is not the only place where this is a concern. In most fields where a large portion of the participants are not mathematically literate is open to exploitation by the seeming objectivity of mathematical models. I see this all the time with computational models in the social sciences and (to a lesser extent) biology, where modelers known how to hide their opinions or biases in the research degrees of freedom to make their models seem robust to those less familiar with the techniques.
Arora, S., Barak, B., Brunnermeier, M., & Ge, R. (2011). Computational complexity and information asymmetry in financial products. Communications of the ACM, 54 (5), 101107 DOI: 10.1145/1941487.1941511
Brunnermeier, M. (2009). Deciphering the liquidity and credit crunch 200708. Journal of Economic Perspectives, 23(1): 77100.
Coval, J., Jurek, J., Stafford, E. (2009). The economics of structured finance. Journal of Economic Perspectives, 23(1): 325.
DeMarzo, P. (2005). The pooling and tranching of securities: A model of informed intermediation. Review of Financial Studies, 18(1): 135.
Livnat A, & Pippenger N (2008). Systematic mistakes are likely in bounded optimal decisionmaking systems. Journal of Theoretical Biology, 250(3): 41023
It is unclear to me if the post is questioning the use of mathematics in financial products or the creation of these new & complex financial products.
On the former topic: options were used in 300 BC and became widely traded in the 1600`s, but the BlackScholes optionpricing formula was not created until the 1970`s. MBS products were issued in the 1980`s, but the Gaussiancopula, widely blamed to underestimate the risks of MBS products, was not published until the year of 2000. In the analogy of `screwdrivers vs guns`, the screws were created long before screwdrivers in this case, and mathematics was simply used to create and finetune the screwdrivers. The buyerseller information gap does exist, as it does in any transaction. However, buyers of CDO & MBS are professional money managers (with their own quants) who hopefully know what they are doing. During the crisis, many buyers who did their research actually profited from the sellers of these complex products while those who didn’t lost. For regular people like myself, where the gap is the widest, I wouldn`t know how to buy a CDO or CDS even if I really want to. Yes math models in finance played a role in the crisis, but I believe that the overleveraging culture of American people are more to blame, aside from governments & banks. It’s just that no media would blame the ones who constituted their viewership and suffered the most. Andrew Wiles probably took what he read in the media at face value to make such obscure and (sadly for such brilliance of his work in mathematics) uneducated comments. (You mentioned VaR specifically, but it is just a riskmeasure imposed by the Basle Committee on Banking Supervision, a crossjurisdictional committee among the G10 countries. It was more of a regulatory requirement instead of a tool to hide risk.)
On the latter topic: it is an extremely complex issue and I believe that the creation of complex products (MBS, CDO) is simply a natural result of many economical, political, demographic, and geographic issues. Had these specific products not been created, other products would likely take their place and possibly create a different crisis. Also, these products have been bad in the last a few years, it does not mean they will continue to be bad, given that people have learnt their lessons and proper regulations will eventually catch up, provided US government is back to work. One would have to write a longer post than yours to explain the creation of these complex products in brief and an entire book to explain it in detail, so I wouldn`t attempt it in the comment section.
In hindsight, it is a little bit unclear to me what, if anything, I was questioning. My main goal (at least in the two paragraphs and three footnotes that aren’t just fluff) was to show that it isn’t always solely how financial instruments are used that is to blame for observed failings, sometimes it is the design of these instruments.
The computational bounds placed in ABBG11 are not so restrictive as to force us to compare individual investors versus finance firms. They are in fact generous enough to show that even these investment firms can be duped; much in the same way that even big organizations like the NSA can’t crack cryptographic protocols, the hardness of the problem that needs to be solved to uncover hidden risks and properly value an asset are beyond those of even big investments banks and their teams of quants. Of course, as we know with cryptography, sometimes the best way to overcome this is with social engineering:and just calling up your buddy and asking “what did you guys hide in these derivatives?”
You mention that:
I would be very interested to see some empirical data on this. My knowledge of actual finance (as you well know) is close to none, and I am going of Coval et al. (2009; cited in the post) for the sensitivity information. However, studies like Benardo & Cornell (1997) have shown that real investment banks using real valuation techniques can easily differ by 17% on their evaluations of the same CDO, and the evaluation within a single bank can differ enough on different sections of the same derivative as to be mutually inconsistent (Duffie, 2007). Of course, I am not very familiar with the literature, and so I am cherry picking results. Might be a good question for quant.SE.
Benardo, A. & Cornell, B. (1997). The valuation of complex derivatives by major investment firms: Empirical evidence. Journal of Finance, 52(2): 785798.
Duffie, D. (2007). Innovations in credit risk transfer: implications for financial stability. BIS Working Papers.
Would you be interested in expanding on your historic note in a full blog post:
I am very interested in the historic interaction of finance and mathematics, especially after reading Tim Johnson’s blog post on how finance spurred a number of the important developments in mathematics that we usually attribute to influence from physics. It would be nice to look at the historic roots and interactions of both fields.
If mathematical models cause financial destruction, then guns kill people and spoons make people fat, etc etc. Unless the development of the CDO/CDS model was precipitated on providing “unethical” parties tools with which to take advantage of lesserinformed parties, then the only blame the creators can rationally be saddled with falls potentially into two categories: the min representation of assumptions and predictive power of their models, and the naïveté needed to assume that any tool that can be used to take advantage of others will not be used for that purpose for any period of time.
Also agreed with Yunjun that empirically proper research on the buyers’ part during the financial crisis led to profits.
However I actually take issue with what seems to be an underlying assumption of of your post: that information asymmetry is something that you can, or maybe even should, eliminate from financial markets. What you claim is “unethical” behavior on a seller’s side is simply good business and proper reaction to incentives. In these markets the only reason a seller sells (and conversely, a buyer buys) is their belief that the price they are getting is above (below) fair value of the asset. As Yunjun said, the parties involved in such transactions are large institutions who have ample resources with which to develop pricing models for the assets they are ultimately transacting. In fact, many of them were money managers whose jobs literally involved only their understanding and expertise of the assets they transacted. Any trades they entered erroneously, either due to lack of information or inadequate models, speaks only of gross negligence or incompetence on their part.
Obviously there are regulations in place to protect Joe SixPack, who is not expected to understand derivative pricing, from entering into such a transaction, but that is not what we are talking about. The subsequent impact on the global economy as a result of CDO/CDS shenanigans are likewise a separate topic altogether (and one that is really not dependent on the ethics of bankers, since it would be very difficult to argue that bankers had any idea what dangerous wares they were peddling).
I guess my point is, whether you are buying a screwdriver, gun, or complex financial derivative… Caveat emptor.
As a pedestrian, I find misleading the uses of “only” in the argument ending in “…money managers whose jobs literally involved only their understanding and expertise of the assets they transacted. Any trades they entered erroneously, either due to lack of information or inadequate models, speaks only of gross negligence or incompetence on their part.”
The system has ample resources to renew a stock of yettobe “grossly negligent or incompetent” virgin cheerleaders between bursts or crashes. While this might apply less to complex derivatives – I dunno – isn’t bread and butter of speculative trading the matter of being early on rallies, which creates a systemic incentive to bring to the fore traders with backgrounds and culture similar enough that they can intuit each others reactions? The assets losses involved in a bursting bubble hide that the latter typically also disbands a generation of traders, don’t they?
That was the point of the post, John. I am not advocating that mathematical models are to blame for the financial crisis. I am asking how much ethical responsibility should the creators have, that of a gun maker or that of a screwdriver maker?
My assertion is that CDO/CDS make it much easier to take advantage of others than it was before. Hence, there is an ethical consequence in providing a tool that will make it easier to exploit buyers. Should regulators step in? It looks like these instruments are failing at their jobs of completing the market.
I’d like to see some statistics on this. In particular, was it research on fundamentals or research via oldboys connections on “what did you hide in there?” as Boris Borcic suggests.
I am not suggesting this, or at least I am not intending to suggest this. Information asymmetry can never be eliminated, and research should be rewarded. However, there are different measures of reward. With tools like this it is just too easy for the rich to get richer, and maintain their advantage not through doing more research (which would be fine) but by manipulating the CDOs they build to hide risk in a way that is hard to value. It is a matter of rewarding social useful work (i.e. fundamentals research and increasing market efficiency) versus defection and individualistic but socially destructive work (i.e. hiding risk, making assets harder to value, and decreasing market efficiency).
As in the previous question, it is good business if the seller/buyer disagree based on the effort they put into evaluating the companies. It is not good business if the disagreement is based on inverting cryptographicaly hard functions, or insider knowledge. I have a similar complain about the antisocial nature of bitcoin, but there it might be essential for it to function as a fair currency.
Again, the way that CDOs/CDSs are built, doing your job correctly (or knowing that you aren’t doing it correctly) can be as hard as cracking RSA (well, technically a different problem, but of similar complexity). That is like saying intelligence agencies are incompetent because they don’t have magical nonexistent computers. They do the best they can with what they have… usually through social engineering, and Boris Borcic suggests that this might be the case here (although it would be called something like insider trading), too.
Not wholly in tone, and since you moderate I’ll leave it to you to prefer it left private as the case may well occur, but I like how you, Artem Kaznatcheev, implicitly call here on a conjectural nonequation of guns with screwdrivers as it reminds me of a 20+ yo proof sketch for a related conjecture:
Theorem : Guns aren’t umbrellas
Proof (ca 1989, rev. 2013):
if guns are umbrellas, then wars are thunderstorms.
Meanwhile, despite knowing themselves capable of bringing damage to the atmosphere, mankind is incapable of damaging the atmosphere enough to make thunderstorms impossible. OTOH, mankind knows how to make wars ultimately impossible: just wipe themselves out by E=mc^2.
So here is a fundamental shape of potentially practical purpose – that of fighting adverse phenomena by removing their condition of possibility – according to which wars aren’t thunderstorms currently nor in any foreseeable future.
A fortiori, guns aren’t umbrellas!
QED
Historical note: salient is here that were contemporaries the end of the cold war and the ozone hole issue. Greenhouse gases provide a second life to the twist of the argument, but selfannihilation by E=mc^2 kind of faded (as a cultural theme).
Pingback: Liquidity hoarding and systemic failure in the ecology of banks  Theory, Evolution, and Games Group
Pingback: Mathematics in finance and hiding lies in compl...
Pingback: Cataloging a year of blogging: from behavior to society and mind  Theory, Evolution, and Games Group
Pingback: Big data, prediction, and scientism in the social sciences  Theory, Evolution, and Games Group
Pingback: Weapons of math destruction and the ethics of Big Data  Theory, Evolution, and Games Group
Pingback: Five motivations for theoretical computer science  Theory, Evolution, and Games Group
Pingback: Computational kindness and the revelation principle  Theory, Evolution, and Games Group