Software monocultures, imperialism, and weapons of math destruction

This past Friday, Facebook reported that they suffered a security breach that affected at least 50 million users. ‘Security breach’ is a bit of newspeak that is meant to hint at active malice and attribute fault outside the company. But as far as I understand it — and I am no expert on this — it was just a series of three bugs in Facebook’s “View As” feature that together allowed people to get the access tokens of whoever they searched for. This is, of course, bad for your Facebook account. The part of this story that really fascinated me, however, is how this affected other sites. Because that access token would let somebody access not only your Facebook account but also any other website where you use Facebook’s Single Sign On feature.

This means that a bug that some engineers missed at Facebook compromised the security of users on completely unrelated sites like, say, StackExchange (SE) or Disqus — or any site that you can log into using your Facebook account.

A case of software monoculture — a nice metaphor I was introduced to by Jonathan Zittrain.

This could easily have knock-on effects for security. For example, I am one of the moderators for the Theoretical Computer Science SE and also the Psychology and Neuroscience SE. Due to this, I have the potential to access certain non-public information of SE users like their IP addresses and hidden contact details. I can also send communications that look much more official, along-side expected abilities like bans, suspensions, etc. Obviously, part of my responsibility as a moderator is to only use these abilities for proper reasons. But if I had used Facebook — disclosure: I don’t use Facebook — for my SE login then a potential hacker could get access to these abilities and then attempt phishing or other attacks even on SE users that don’t use Facebook.

In other words, the people in charge of security at SE have to worry not only about their own code but also Facebook (and Google, Yahoo!, and other OpenIDs).

Of course, Facebook is not necessarily the worst case of software monoculture or knock-on effects that security experts have to worry about. Exploits in operating systems, browsers, serves, and standard software packages (especially security ones) can be even more devastating to the software ecology.

And exploits of aspects of social media other that login can have more subtle effects than security.

The underlying issue is a lack of diversity in tools and platforms. A case of having all our eggs in one basket. Of minimizing individual risk — by using the best available or most convenient system — at the cost of increasing systemic risk — because everyone else uses the same system.

We see the same issues in human projects outside of software. Compare this to the explanations of the 2008 financial crises that focused on individual vs systemic risk.

But my favourite example is the banana.

In this post, I’ll to sketch the analogy between software monoculture and agricultural monoculture. In particular, I want to focus on a common element between the two domains: the scale of imperial corporations. It is this scale that turns mathematical models into weapons of math destructions. Finally, I’ll close with some questions on if this analogy can be turned into tool transfer: can ecology and evolution help us understand and manage software monoculture?

Read more of this post

Overcoming folk-physics: the case of projectile motion for Aristotle, John Philoponus, Ibn-Sina & Galileo

A few years ago, I wrote about the importance of pairing tools and problems in science. Not selecting the best tool for the job, but adjusting both your problem and your method to form the best pair. There, I made the distinction between endogenous and exogenous questions. A question is endogenous to a field if it is motivated by the existing tools developed for the field or slight extensions of them. A question is exogenous if motivated by frameworks or concerns external to the field. Usually, such an external motivating framework is accepted uncritically with the most common culprits being the unarticulated ‘intuitive’ and ‘natural’ folk theories forced on us by our everyday experiences.

Sometimes a great amount of scientific or technological progress can be had from overcoming our reliance on a folk-theory. A classic examples of this would be the development of inertia and momentum in physics. In this post, I want to sketch a geneology of this transition to make the notion of endogenous vs exogenous questions a bit more precise.

How was the folk-physics of projectile motion abandoned?

In the process, I’ll get to touch briefly on two more recent threads on TheEGG: The elimination of the ontological division between artificial and natural motion (that was essential groundwork for Darwin’s later elimination of the division between artificial and natural processes) and the extraction and formalization of the tacit knowledge underlying a craft.
Read more of this post

Techne and Programming as Analytic Philosophy

This week, as I was assembling furniture — my closest approach to a traditional craft — I was listening to Peter Adamson interviewing his twin brother Glenn Adamson about craft and material intelligence. Given that this interview was on the history of philosophy (without any gaps) podcast, at some point, the brothers steered the conversation to Plato. In particular, to Plato’s high regard for craft or — in its Greek form — techne.

For Peter, Plato “treats techne, or craft, as a paradigm for knowledge. And a lot of the time in the Socratic dialogues, you get the impression that what Socrates is suggesting is that we need to find a craft or tekne for virtue or ethics — like living in the world — that is more or less like the tekne that say the carpenter has.” Through this, the Adamson twins proposed a view of craft and philosophy as two sides of the same coin.

Except, unlike the carpenter and her apprentice, Plato has Socrates trying to force his interlocutors to formulate their knowledge in propositional terms and not just live it. It is on this point that I differ from Peter Adamson.

The good person practices the craft of ethics: of shaping their own life and particular circumstances into the good life. Their wood is their own existence and their chair is the good life. The philosopher, however, aims to make the implicit (or semi-implicit) knowledge of the good person into explicit terms. To uncover and specify the underlying rules and regularities. And the modern philosopher applies these same principles to other domains, not just ethics. Thus, if I had to give an incomplete definition for this post: philosophy is the art of turning implicit knowledge into propositional form. Analytic philosophy aims for that propositional form to be formal.

But this is also what programmers do.

In this post, I want to convince you that it is fruitful to think of programming as analytic philosophy. In the process, we’ll have to discuss craft and the history of its decline. Of why people (wrongly) think that a professor is ‘better’ than a carpenter.
Read more of this post

Separating theory from nonsense via communication norms, not Truth

Earlier this week on twitter, Brian Skinner wrote an interesting thread on how to distinguish good theory from crackpottery. He started with a trait that both theorists and crackpots share: we have an “irrational self-confidence” — a belief that just by thinking we “can arrive at previously-unrealized truths about the world”. From this starting point, the two diverge in their use of evidence. A crackpot relies primarily on positive evidence: he thinks hard about a problem, arrives at a theory that feels right, and then publicizes the result.

A theorist, on the other prong, incorporates negative evidence: she ponders hard about a problem, arrives at a theory that feels right and then proceeds to try to disprove that theory. She reads the existing literature and looks at the competing theories, takes time to understand them and compare them against her own. If any disagree with hers then she figures out why those theories are wrong. She pushes her theory to the extremes, looks at its limiting cases and checks them for agreement with existing knowledge. Only after her theory comes out unscathed from all these challenges does she publicize it.

For Skinner, this second prong is the definition of scholarship. In practice, coming up with a correct theory is mostly a painful process of discarding many of your own wrong attempts. A good theorist is a thorough, methodical and skeptical of their own ideas.

The terminology of crackpottery vs scholarship is probably overly harsh, as Skinner acknowledges. And in practice, somebody might be a good theorist in one domain but a crackpot elsewhere. As Malkym Lesdrae points out, there are many accomplished accademics who are also crackpot theorists: “Most often it’s about things outside their field of specialty”. Thus, this ideal self-skepticism might be domain specific.

It is also a destructive ideal.

In other words, I disagreed with Skinner on the best way to separate good theory from nonsense. Mostly on the framing. Skinner crystalized our disagreement in a tweet: whereas he views self-skepticism as I an obligation to the Truth, I view a similar sort of self-reflective behavior as a social obligation. I am committed to this latter view because I want to make sense of things like heuristic models, where truth is secondary to other modelling concerns. Where truth is not the most useful yardstick for checking the usefulness of model. Where you hear Box’s slogan: “all models are wrong, but some are useful.

Given the brief summary of Skinner’s view above — and please, Brian, correct me in the comments if I misrepresented your position — I want to use the rest of this post to sketch what I mean by self-reflective behavior as a social obligation.
Read more of this post

On the Falsehood of Philosophy: a skeptic’s pastiche of Schopenhauer

Unless falsehood is the direct and immediate object of philosophy, our efforts must entirely fail of its aim.[1] It is absurd to look upon the enormous amount of wrong that abounds everywhere in philosophy, and originates in the words and writings of the greatest thinkers themselves, as serving no purpose at all and the result of mere error. Each separate mistake, as it topples an intricate system of thought, seems, no doubt to be something exceptional; but mistake in general is the rule.

I know of no greater absurdity than that propounded by the jury of Whig historians in declaring failure to be negative in its character. Failure is just what is positive; it feeds its own generating process. Plato is particularly concerned to defend failure as negative. To idealize a world for Forms and eternal Truths. Absurdly, he seeks to strengthen his position by dialogue with a man who knew but one things, he knew nothing. For Socrates recognized that it is success which is negative; in other words, truth and fact imply some discussion silenced, some process of inquiry brought to an end. If we have truth then there is no need for gadflies.

When the gadfly bites: the best consolation for mistake or wrong of any kind will be the thought of past great minds who erred still more than yourself. This is a form of consolation open for all time. But what an awful fate this means for philosophy as a whole!

Read more of this post