I have always enjoyed the way John Maynard Keynes had with words. He was responsible for many a bon mot, such as “in the long run we are dead” (skewering the idea of long-run equilibrium in economics), “It is better to be roughly right than precisely wrong”, “Capitalism is the astounding belief that the most wickedest of men will do the most wickedest of things for the greatest good of everyone” and (my favourite) “Markets can remain irrational a lot longer than you and I can remain solvent”.

Today I read a rather scathing piece on the performance of economists in the lead up to and throughout the financial crisis, which included this rather longer but nevertheless brilliant quotation from Keynes.

The completeness of the [orthodox] victory is something of a curiosity and a mystery. It must have been due to a complex of suitabilities in the doctrine to the environment into which it was projected. That it reached conclusions quite different from what the ordinary uninstructed person would expect, added, I suppose, to its intellectual prestige. That its teaching, translated into practice, was austere and often unpalatable, lent it virtue. That it was adapted to carry a vast and consistent logical superstructure, gave it beauty. That it could explain much social injustice and apparent cruelty as an inevitable incident in the scheme of progress, [with] the attempt to change such things as likely on the whole to do more harm than good, commended it to authority. That it afforded a measure of justification to the free activities of the individual capitalist, attracted to it the support of the dominant social force behind authority. But although the doctrine itself has remained unquestioned by orthodox economists up to a late date, its signal failure for purposes of scientific prediction has greatly impaired, in the course of time, the prestige of its practitioners. For professional economists…were apparently unmoved by the lack of correspondence between the results of their theory and the facts of observation—a discrepancy which the ordinary man has not failed to observe, with the result of his growing unwillingness to accord to economists that measure of respect which he gives to other groups of scientists…

John Maynard Keynes, The General Theory of Employment, Interest and Money (1936; London: Macmillan, 1964) 32–3

I have a copy of the General Theory, which I have only skimmed. One day I really should read the whole thing. In fact, there’s even a Kindle edition for $3, so maybe that day is not too far away…

Photo credit: Wikimedia Commons.

Design guru Edward Tufte famously lambasted pie charts in The Visual Display of Quantitative Information and went on to say

the only worse design than a pie chart is several of them

While pie charts do have their defenders, the basis for the contempt in which pie charts are held by Tufte and others is that the human eye is far better at differentiating position and length than angle and area.

So, I was a little disappointed when a correspondent drew my attention to this rather bubbly chart which appeared on an article by the excellent team at Pro-Publica (click on the chart to see a larger version).

Pro-Publica is an independent, not-for-profit newsroom that specialises in investigative journalism. They have collaborated with the team at Planet Money (one of my favourite podcasts), and have perhaps delved deeper than any other journalists into the arcane world of CDOs, a topic I have touched on a few times here on the Stubborn Mule.

The chart, attributed to Thetica Systems, was used to accompany an article by Pro-Publica exposing the fact that, in their words,

Over the last two years of the housing bubble, Wall Street bankers perpetrated one of the greatest episodes of self-dealing in financial history.

It is a fascinating story, but it would seem that Thetica’s graphics department was carried away with a visual pun on the title of Pro-Publica’s post “Circular CDOs” when they chose to use circles to depict the growth in CDO recycling from 2005 to 2007. It might look pretty, but the circles make it much harder to discern the trend and to compare the four banks. Pro-Publica’s article deserves better.

In the tradition of my junk chart posts, I have produced an alternative visualization of the same data. I am sure that graphic designers could improve on the colour-scheme, but this simple lattice of line charts makes for a much clearer view of the data.

CDO Self-Dealing by investment banks (2005-2007)

If this post has given you a taste for de-junking charts, you should also visit the Junk Charts blog for much, much more.

Yesterday’s post on informal votes generated a lot of questions, both on and off the blog. One commenter was interested in understanding why there was so much variability in informal votes in New South Wales. It is a good question, and one I do not have an answer to. Presumably demographic differences across electorates (such as varying facility with reading English among non-native speakers) would come into play. But this still leaves open the question as to why the swing in informal votes varies so much across New South Wales. I will have to leave it to you to explore: the table below has the informal vote in all 48 New South Wales seats for your perusal. Let me know if you have any theories!

An email correspondent asked whether it was in fact the 2007 election that was anomalous rather than the 2010 election, so I have also compared the 2010 informal vote to the 2004 election. Interestingly, the uptick in informal votes from 2004 to 2010 is indeed smaller. In fact, Western Australia had a lower rate of informal votes in this election than in 2004. New South Wales still shows significant increases in informal votes in a number of electorates, which helps drive a national trend. Overall, compared to 2004 there does still seem to be something going on with informal votes, but the effect is certainly less marked.

I also received various questions about whether correlations could be seen between informal votes and Green votes, whether the increase in informal votes was greater in more marginal seats and so on. Unfortunately, as yet my data mining has not revealed anything of substance. Here, for example, is the increase in the rate of informal votes versus the absolute two-party preferred margin. The regression lines show no simple relationship.

Informal Vote versus Two-Party Preferred Margin

Comparing Green votes to informal votes is just as unenlightening. That, at least, seems to make sense. While it is reasonable to consider some of the Green vote as a protest vote and some of the informal votes likewise as a protest vote, it may be that in some electorates more voters were inclined to protest by voting Green than informal, or vice versa. This would mean that there would be negligible correlation between the Green and informal swings at the division level.

So, despite my efforts, I am yet to squeeze further insight from the data. Of course I remain open to further suggestions! If you would like to do your own analysis, the current 2010 data is available from the AEC as is past data.

UPDATE: If you sort the table at the top by informal vote, you’ll see that the two electorates with the lowest rates of informal voting were New England and Lyne, the seats of the independents Tony Windsor and Rob Oakeshott respectively!

Also, here is a national table of informal votes (just to avoid being to NSW-centric).

While Australia still waits to see which party will manage to scrape into power, the Australian Electoral Commission (AEC) has announced an investigation into the unusually high rate of informal votes. Veteran ABC analyst Antony Green observed that the rate of informal votes was the highest since 1984. Some are attributing the rise to the “Latham effect” following the exhortation by former Labor leader now professional provocateur, Mark Latham, that voters should spoil their ballots to thumb their noses at both major parties.

It will be interesting to see what conclusions the AEC draws, but there is no doubt that the informal votes in this election were significant.There are more votes to be counted and the trends in postal votes may differ somewhat from votes cast in person, but enough of the votes are in to get a reasonable picture of what has been going on. The figures here are based on the AEC data for the House of Representatives as at 23 August 2010. Informal votes rose in every state from the rate seen in the 2007 election, increasing by a margin of between 1.0% and 2.4%.

Informal Votes by State (%)

One way to visualize the changes is to plot the informal vote rate in 2010 against that of 2007. The chart below does this at a state level and also adds in a 45 degree line. Points falling above this line (as they all do) show an increase from 2007 to 2010, while points below the line would indicate a decrease.

Aggregating to a state level hides a lot of the interesting detail and can be misleading. For example, the ACT shows the biggest increase in informal votes, but with only two electorates, these figures have less statistical value. A more interesting picture emerges when the changes are shown by division. The chart below groups the changes by state, but plots points for each division*. Once again, 45 degree lines provide a guide as to whether informal voting rates increased or decreased.

Leaping out from this picture is the extraordinarily high rate of informal votes in some divisions in New South Wales. It is also striking that the rate of informal votes has increased in almost every division. At this point, there are only 4 divisions in the whole country (one in Victoria and three in New South Wales) to see the rate of informal votes drop.

It is hard to escape the conclusion that the increase in informal votes reflects a protest vote arising from deep voter dissatisfaction with both major parties. The Greens are pleased with the “Greenslide” they have experienced, but some of their success is likely to amount to the same voter protest, only expressed another way, rather than a permanent shift in commitment to the Greens.

* For the purists, there were changes to electorates between elections, and the chart only shows divisions which existed in both 2007 and 2010. Given changes to boundaries, some of these electorates are, strictly speaking, no longer perfectly comparable, but they have been plotted regardless.

Last night I was watching the Chaser’s Yes We Canberra (only a day late), and jumped out of my chair when I saw Craig Reucassel corner Tony Abbott to challenge him about his obsession with reducing Government debt. Have a look at this to see why!

Here is the post referred to in the video.

UPDATE: here’s a tweet from Craig on the topic of attribution (or lack thereof): http://twitter.com/craigreucassel/status/21647798173

With Australia’s Federal election looming, the opposition has today proudly announced a new policy to fund infrastructure without actually increasing Government debt! What are we to make of this?

It’s hard to determine the details from a media announcement, but based on the text posted by Peter Martin on his blog, it would seem that the idea is to provide tax incentives for entities other than the Federal Government to borrow to fund infrastructure:

Private infrastructure operators and State and Local Governments will be eligible to apply for the concessional treatment.

The way the scheme would seem to work is that eligible projects could issue bonds and investors would receive a tax rebate amounting to 10% of the interest on the bond. So, if you received a $100 interest payment and your earning put you in the top marginal tax bracket, you would pay $45 in tax. Under this scheme, you would only pay $35 in tax.

So, the cost to the Federal Government would simply be forgone tax revenue (and this would be capped at $150 million per annum) and the Opposition believes that the program could support up to $20 billion in infrastructure financing. Presumably, investors currently buying plasma TVs would rush to buy these bonds instead.

Seems like a neat trick, but I have a number of reservations about the scheme.

First, I have argued in the past that the near-hysterical concern about Government debt is overdone. For a start, Government debt in Australia is far lower than in other developed countries around the world. More importantly, the facile analogy that compares Government finance to that of a household budget does not stand up for one very important reason: unlike you or me, the Government is the monopoly issuer of Australian dollars. This changes the game and breaks the analogy utterly.

Second, the opposition’s policy would still involve raising significant amounts of debt, just not issued by the Federal Government. If that debt is all incurred instead by State Governments, should that really be a cause for celebration? After all, unlike the Commonwealth, State Governments do not control issuance of currency, so they really could go bankrupt and indeed, recent history has shown that many of the State Governments are loath to increase their debt levels too significantly for fear of having their credit rating downgraded. What if the borrowers are in the private sector? Well, that would be worse still! Back in March I updated my chart showing private and government sector debt. The debt level we should all be worried about in Australia is private sector debt, which is far higher than government sector debt.

Third, infrastructure bonds have form. Back in the 90s, the then Labor government introduced an infrastructure bond scheme which also featured tax incentives. Of course, it did not take long for clever investment bankers to work out how to surgically isolate the tax benefit so that wealthy individuals could take advantage of the concession without actually taking on any investment risk. In the end, the whole scheme was shut down, although some of the transactions that were done still survive today. I would expect exactly the same thing to happen with this policy. Any special tax treatment is always a red rag to the tax expert bull.

So, it may sound clever, but to me it does not seem to be sound policy.

As a follow up to our guest post on the numbers behind Labor’s broadband policy, here is a quick poll to see whose policy you prefer. Let us know what you think!

Our regular guest contributor James Glover (aka @zebra) returns today with a look at the numbers behind the National Broadband Network. He asks: do you think it would be value for money?

The Labor Government’s proposed National Broadband Network (NBN) has many things to recommend it, not least speeds of up to 1GB/s (currently I am on 10Mb/s for ADSL; theoretical speeds of 24MB/s on ADSL2+ and 100MB/s on VDSL are also soon to be widely available, though the reality is dependent on many variables such as distance from an exchange). It would revolutionise the way we communicate as the higher bandwidth would allow not just interactive entertainment and fast downloads, but genuinely accessible cloud applications that really felt like they sat on your computer…and of course dishwashers waking up at 3.00am to negotiate the best electricity price. I doubt whether anybody on either side of politics would disagree that, in a perfect world, this is all desirable. But like all utopias, it comes at a cost and that is where the real divergence between the Labor Party and the Coalition’s broadband policies exists. I hope to cast some light on this cost argument using the power of the Time Value of Money, in particular calculating the real cost to you on a monthly basis so you can compare it with your existing broadband cost.

Labor wants an all-connecting fibre optic network (with subsidised satellite to cover really remote areas) that will cost an estimated $46bn. The Coalition wants a more modest effort: a fibre optic “backbone” network that uses existing copper wiring in urban areas and relies on market competition to pay for further improvements. It is estimated to cost about $8bn plus later commercial costs. Both of these figures seem extraordinarily high. How to decide if it is really worth it? Well if I told you that Labor’s NBN would cost you $10 per month would that sound too high? After all that only includes the infrastructure cost, not the access cost via an ISP. But most of us don’t pay upfront for our broadband or mobile (cell) phone bills, we pay monthly. The Coalition’s figure of $8bn works out at less than $2/month each (for those so inclined, you can read the details behind these figures). But it doesn’t include any additional costs charged by commercial companies building additional infrastructure. It also only claims to provide “peak speeds” of 10Mb/s which I already get on my ADSL+.

Is $10/month a lot of money? Or $2/month for that matter? It obviously depends on what your income is and how much you are currently prepared to pay for broadband. My broadband plan costs $50 for 120GB/month. I also live in a one-person household. It doesn’t sound much to me, but all those $10/month costs add up to the thousands we pay in tax each year. There’s no point paying more for little for no benefit. Of course it’s not going to be charged directly, but through increased taxes (or decreased services). I estimate $10/month to represent an average increase in the tax rate of about 0.5%. This seems reasonable to me. After all, if in 2020 a businessperson (or BusinessBot2020) came to Australia and found our broadband to be the equivalent of dial-up today, they’d hardly be impressed enough to invest in a technology business. Of course, by 2020 with super-fast broadband we should really be able to do most business remotely, right? But we’ve been saying that since the invention of the telephone.

A month ago I posted about “SURF”, the newly-established Sydney R user forum (R being an excellent open-source statistics tool). Shortly after publishing that post, I attended the inaugural forum meeting.

While we waited for attendees to arrive, a few people introduced themselves, explaining why they were interested in R and how much experience they had with the system. I was surprised at the diversity of backgrounds represented: there was someone from the department of immigration, a few from various areas within the health-care industry, a group from the Australian Copyright Council (I think I’ve got that right—it was certainly something to do with copyright), a few from finance, some academics and even someone from the office of state revenue.

Of the 30 or so people who came to the meeting, many classed themselves as beginners when it came to R (although most had experience with other systems, such as SAS). So if there’s anyone out there who was toying with the idea of signing up but hesitated out of concern that they know nothing about R, do not fear. You will not be alone.

The forum organizer, Eugene Dubossarsky, proceeded to give an overview of the recent growth in R’s popularity and also gave a live demo of how quickly and easily you can get R installed and running. Since there were so many beginners, Eugene suggested that a few of the more experienced users could act as mentors to those interested in learning more about R. As someone who has used R for over 10 years, I volunteered my services. So feel free to ask me any and all of your R questions!

As well as being a volunteer mentor, I will have the pleasure of being the presenter at the next forum meeting on the 18th of August. Regular readers of the Stubborn Mule will not be surprised to learn that the topic I have chosen is The Power of Graphics in R. Here’s the overview of what I will be talking about:

In addition to its statistical computing prowess, R is one of the most sophisticated and flexible tools around for visualizing quantitative data. It can produce a wide variety of chart types, including scatter plots, box plots, dot plots, mosaic plots, 3D charts and more. Tweaking chart settings and adding customized annotations is a breeze and the charts can readily be output to a range of formats including images (jpeg or png), PDF and metafile formats.

Topics covered in this talk include:

• Getting started with graphing in R
• The basic charting types available
• Customising charts (labels, axes, colour, annotations and more)
• Managing different output formats
• A look at the more advanced charting packages: lattice and ggplot2

Anyone who ever has a need to visualize their data, whether simply for exploration or for producing slick graphics for reports and presentations can benefit from learning to use R’s graphics features. The material presented here will get you well on your way. If you have ever been frustrated when trying to get charts in Excel to behave themselves, you will never look back once you switch to R.

For those of you in Sydney who are interested in a glimpse of how I use R to produce the charts you see here on the blog, feel free to come along. I hope to see you there!

Have you ever heard the question “Would you like a tea or a coffee” answered with a simple “Yes”? If so, the respondent almost certainly considers their response to be extremely witty. The questioner is unlikely to agree. There is also a high probability that the joker is someone’s Dad…or perhaps a mathematician.

I have to admit to having indulged in this “joke” in my time (more than once), but until recently it had not occurred to me that it in fact reflects a violation of a general principle of conversation. Enlightenment came when I read the seminal 1975 paper “Logic and Conversation” [1] by the philosopher H.P.Grice.

The humour (or lack thereof) of the coffee/tea gag lies in the conflict between the logical truth of the statement and its inappropriateness in conversation. While the statement “A or B” is logically true as long as at least one of A and B is true , in the context of conversation, logical truth is not enough. If you knew A was true and B was false, you would not bother saying “A or B”, you would just say “A”. Moreover, that is what others would expect of you. If I ask you to pass me a hammer, I don’t expect you to pass me a hammer and a spanner. In the same way, if you know you are going to Spain for your holidays, I don’t expect you to say “I’m either going to Spain or Canada”, despite the fact that, strictly speaking, it is a true statement. It is this distinction between simple logical truth and appropriateness in conversation that is the subject of Grice’s paper.

Grice bases his ideas on the notion of the “Cooperative Principle”, which he summarises as the requirement to

Make your conversation such as is required, at the stage at which it occurs, by the accepted purpose or direction of the talk exchange in which you are engaged.

People have conversations of many types for many reasons: to do business, to gossip, to seduce, to educate, to inform or simply for the pleasure of conversation itself. In every case, conversation involves (at least) two participants and the conversations that work best are the ones that take the needs of all of the participants into account. So it makes sense that a bit of cooperation is the foundation of a good conversation.

Based on the cooperative principle, Grice goes on to postulate a number of “maxims of conversation”. Here are the maxims as he describes them:

Quantity

1. Make your contribution as informative as is required (for the current purposes of the exchange).
2. Do not make your contribution more informative than is required.

Quality

1. Do not say what you believe to be false.
2. Do not say that for which you lack adequate evidence.

Relation

1. Be relevant.

Manner

1. Avoid obscurity of expression.
2. Avoid ambiguity.
3. Be brief (avoid unnecessary prolixity).
4. Be orderly.

The term “maxim” is carefully chosen as Grice notes that one need not follow all of the maxims at all times, while still being cooperative. The main reason that a maxim could be violated is if it is in conflict with another maxim. An example would be providing less information than required (violating Quantity 1) because you are not confident you have the facts right (and you don’t want to violate Quality 2).

Viewed in terms of Grice’s maxims, the coffee/tea joke is a clear violation of the first maxim of quantity.

As I have already admitted to this particular breach, the obvious question is: have I violated any other maxims? Some who know me well would take the view that, while I may take pains to avoid a violation of either of the maxims of quality, I regularly and flagrantly violate Quantity 2 and Manner 3 and probably Relation 1. I need to learn to stick to the point or risk being branded an uncooperative conversationalist! Or perhaps it’s too late.

[1] Available in the collection “Studies in the Way of Words” by H.P.Grice.