Category Archives: sections

Meta-category for grouping categories on the nav menu.

Protovis now working in Chrome and Safari

Thanks to everyone who responded to my experimental Protovis post*, whether in the survey, via twitter or in comments on the post. It quickly became clear that my trick for including the code to generate the chart completely failed to work in Chrome and Safari browsers. I still do not fully understand why that is, but I have now worked out a completely different approach to the problem which (fingers crossed) seems to work in more browsers, although I still cannot vouch for all versions of Internet Explorer.

So here is the chart one more time. I hope it now works for (almost) everyone!

[pvis src=”http://stubbornmule.net/scripts/pv/test.js” img=”/blog/wp-content/PV-CDO-circles.png” height=”125px”]CDO deals: total and recycled[/pvis]
I will also be updating the howto post very shortly to explain my new technique.

UPDATE: at the moment, this trick is not working on mobile devices. It should now be working on mobile devices except for Android. The only remaining problem is IE, but I think that will not be possible. I will instead try to make it fail more gracefully on IE.

* Protovis is a javascript data visualisation library being developed at Stanford, which allows the creation of interactive charts on web pages.

Getting Protovis working on WordPress

1 Reply

When I started experimenting with Protovis*, I quickly found that getting it to work in a WordPress blog was rather fiddly. With a lot of help from Google (and this page in particular), I managed to piece together what needed to be done, but since I did not find any explanations specifically focused on Protovis in WordPress, I thought it may be useful for others attempting the same thing if I summarised the steps involved. Most readers will not have the slightest interest in this, so I will not expect many of you to keep reading!

First of all, if you are hosting your blog on WordPress.com, give up now! These instructions will only work if you have a self-hosted installation of WordPress.

1. Download Protovis and unzip it on your server it a convenient location. I put it in a folder called “protovis”, accessible from the root of the webserver.

2. Ensure your headers include a pointer to the Protovis script. How exactly you do this, depends on your theme, but since I am using Thesis, it’s quite straightforward: under Site Options > Additional Scripts, I added the following code.

<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico">
 <script type="text/javascript" src="/protovis/protovis-r3.2.js"></script>

3. Wrap your Protovis code up in a function and save it in a .js file on the server. For example, here is the code I used to produce the chart in the last post. You will see that I wrapped everything up in a function called “cdodraw”. I saved the file in the folder /script/pv.

4. Edit your post in HTML mode and use the following code to load and call your function.

<script src="/scripts/pv/cdodraw.js" type="text/javascript"></script>
<script type="text/javascript+protovis"><!--
cdodraw();
--></script>

You should replace “/scripts/pv/cdodraw.js” with the location of your own Protovis chart code and replace “cdodraw()”. With the name of your own function. Note that the first script command has type “text/javascript” and the second “text/javascript+protovis”. This is important!

Part of the reason for the difficulty is that WordPress has a tendency to mash up the text you enter, which is fine most of the time, but not when you are trying to write Javascript. An alternative may be to try the Text Control plugin, which allows finer control over WordPress’s mashing of your text. I have not tried the plugin myself, so I cannot be sure how well it works.

Good luck, and let me know how you go! If you have any suggestions on how to do this a better way, please let me know. Better still, you could write a WordPress plugin to make it all much easier.

UPDATE: it appears that this function-wrapping trick does not work for Google Chrome or Safari. I’m looking into it!

FURTHER UPDATE: I could not work out why the approach described here does not work for Chrome or Safari, so instead I got it working by creating a custom shortcode that slurps in a javascript file and includes it in the post. I am in the process of wrapping this up in a plugin to save others the trouble of working out how to do it. For those who cannot wait, here is the code I used:

function sProtovis($atts, $content = null) {
 extract(shortcode_atts(array('src' => '#'), $atts));
 return '<script type="text/javascript+protovis">'."\n".file_get_contents($src).'</script><noscript>Scripts disabled -- cannot display chart!</noscript>'.'<p align="center"><strong>'.do_shortcode($content).'</strong></p>';
}
add_shortcode('protovis', 'sProtovis');

It still relies on the Protovis code already being added into the header (as described above). In the plugin, this will no longer be necessary.

PLUGIN: pv-loader is now available on github.

* Protovis is a javascript data visualisation library being developed at Stanford, which allows the creation of interactive charts on web pages.

Experimenting with Protovis

13 Replies

A couple of weeks ago I gave a talk on using graphics in R. During the question session, someone asked whether I had tried using Protovis, a javascript data visualisation library being developed at Stanford. It was an easy question to answer: no!

However, a bit of subsequent investigation revealed that Protovis has been developed very much in the spirit of Leland Wilkinson’s book The Grammar of Graphics, which I am currently reading, so I have decided to experiment with it here on the blog.

The charts I generate with R are all static images, while a tool like Protovis allows for user interaction which opens up some interesting possibilities. Compared to R, which I have been using for around 10 years, Protovis presents a double challenge: not only do I have to come to grips with Protovis itself, but I will also have to learn some basic Javascript programming. So, I expect it to be a slow journey.

As a tentative first step, I have reproduced the CDO chart from a recent post ranting about bubble charts. At first glance, it is essentially identical to the chart I produced using R. However, if you hover your mouse over the points on the chart, you should see the figures appear! It is by no means perfect (for example, it would probably look better if single points appeared, rather than every point on the chart and it could do with a legend), but it’s a start and I will persevere.

[pvis src=”http://stubbornmule.net/scripts/pv/test.js” img=”http://stubbornmule.net/blog/wp-content/CDO-circles.png” height=”125px”]CDO deals: total and recycled[/pvis]

Producing scripts using a Javascript library does have its drawbacks. For a start, it means the chart will only be visible when scripts can be run, so if you are reading this in an email or an RSS news reader, you will probably not see very much and will have to visit the page on the blog to see it. Even then, some of you may use script-blockers such as NoScript which will also break the chart (mind you, you can trust the Mule, so you could always whitelist this site!). Finally, I believe that some older browsers (such as IE6) will not support Protovis. It would be useful to see how many people can or cannot see the chart, so please let me know using this poll whether you can see the chart.

Getting Protovis to work on the blog was a little fiddly, so for anyone interested, I have also written up a quick guide to using Protovis on a WordPress blog.

UPDATE: Reports in so far indicate that the chart is not working in Google Chrome or on mobile devices. More work to do it would seem!

Keynes on Economics

8 Replies

I have always enjoyed the way John Maynard Keynes had with words. He was responsible for many a bon mot, such as “in the long run we are dead” (skewering the idea of long-run equilibrium in economics), “It is better to be roughly right than precisely wrong”, “Capitalism is the astounding belief that the most wickedest of men will do the most wickedest of things for the greatest good of everyone” and (my favourite) “Markets can remain irrational a lot longer than you and I can remain solvent”.

Today I read a rather scathing piece on the performance of economists in the lead up to and throughout the financial crisis, which included this rather longer but nevertheless brilliant quotation from Keynes.

The completeness of the [orthodox] victory is something of a curiosity and a mystery. It must have been due to a complex of suitabilities in the doctrine to the environment into which it was projected. That it reached conclusions quite different from what the ordinary uninstructed person would expect, added, I suppose, to its intellectual prestige. That its teaching, translated into practice, was austere and often unpalatable, lent it virtue. That it was adapted to carry a vast and consistent logical superstructure, gave it beauty. That it could explain much social injustice and apparent cruelty as an inevitable incident in the scheme of progress, [with] the attempt to change such things as likely on the whole to do more harm than good, commended it to authority. That it afforded a measure of justification to the free activities of the individual capitalist, attracted to it the support of the dominant social force behind authority. But although the doctrine itself has remained unquestioned by orthodox economists up to a late date, its signal failure for purposes of scientific prediction has greatly impaired, in the course of time, the prestige of its practitioners. For professional economists…were apparently unmoved by the lack of correspondence between the results of their theory and the facts of observation—a discrepancy which the ordinary man has not failed to observe, with the result of his growing unwillingness to accord to economists that measure of respect which he gives to other groups of scientists…

John Maynard Keynes, The General Theory of Employment, Interest and Money (1936; London: Macmillan, 1964) 32–3

I have a copy of the General Theory, which I have only skimmed. One day I really should read the whole thing. In fact, there’s even a Kindle edition for $3, so maybe that day is not too far away…

Photo credit: Wikimedia Commons.

More Informality

4 Replies

Yesterday’s post on informal votes generated a lot of questions, both on and off the blog. One commenter was interested in understanding why there was so much variability in informal votes in New South Wales. It is a good question, and one I do not have an answer to. Presumably demographic differences across electorates (such as varying facility with reading English among non-native speakers) would come into play. But this still leaves open the question as to why the swing in informal votes varies so much across New South Wales. I will have to leave it to you to explore: the table below has the informal vote in all 48 New South Wales seats for your perusal. Let me know if you have any theories!

Division ID	Division	Informal (%)	Informal Swing (%)
107	Blaxland	14.22	5.33
251	Watson	13.57	4.48
119	Fowler	13.2	4.72
111	Chifley	11.09	3.18
315	McMahon	11.01	3.41
153	Werriwa	10.59	4.01
104	Barton	10.28	3.71
122	Greenway	9.47	3.29
144	Reid	9.21	3.63
140	Parramatta	8.73	2.11
103	Banks	8.55	2.79
127	Kingsford Smith	8.23	2.92
128	Lindsay	8	2.48
131	Macarthur	7.93	2.36
121	Grayndler	7.13	1.15
110	Charlton	7.11	2.44
150	Throsby	7.01	2.14
118	Farrer	6.85	3.02
124	Hughes	6.78	2.34
148	Shortland	6.58	2.42
105	Bennelong	6.51	0.29
117	Eden-Monaro	6.44	2.71
126	Hunter	6.23	1.96
146	Robertson	6.2	2.76
250	Riverina	6.11	2.22
115	Dobell	6.07	1.75
136	Newcastle	5.82	1.39
112	Cook	5.73	1.92
114	Cunningham	5.73	1.82
134	Mitchell	5.62	1.55
133	Macquarie	5.4	1.75
249	Paterson	5.37	1.77
149	Sydney	5.35	1.17
132	Mackellar	5.27	0.6
139	Parkes	5.25	1.17
120	Gilmore	5.24	1
145	Richmond	5.15	0.87
125	Hume	5.14	1.57
109	Calare	4.69	1.07
151	Warringah	4.66	1.22
137	North Sydney	4.62	0.9
138	Page	4.48	0.18
106	Berowra	4.44	-0.24
152	Wentworth	4.43	-0.47
113	Cowper	4.31	0.34
108	Bradfield	4.23	0.26
130	Lyne	3.67	-1.36
135	New England	3.6	0.63

An email correspondent asked whether it was in fact the 2007 election that was anomalous rather than the 2010 election, so I have also compared the 2010 informal vote to the 2004 election. Interestingly, the uptick in informal votes from 2004 to 2010 is indeed smaller. In fact, Western Australia had a lower rate of informal votes in this election than in 2004. New South Wales still shows significant increases in informal votes in a number of electorates, which helps drive a national trend. Overall, compared to 2004 there does still seem to be something going on with informal votes, but the effect is certainly less marked.

I also received various questions about whether correlations could be seen between informal votes and Green votes, whether the increase in informal votes was greater in more marginal seats and so on. Unfortunately, as yet my data mining has not revealed anything of substance. Here, for example, is the increase in the rate of informal votes versus the absolute two-party preferred margin. The regression lines show no simple relationship.

Informal Vote versus Two-Party Preferred Margin

Comparing Green votes to informal votes is just as unenlightening. That, at least, seems to make sense. While it is reasonable to consider some of the Green vote as a protest vote and some of the informal votes likewise as a protest vote, it may be that in some electorates more voters were inclined to protest by voting Green than informal, or vice versa. This would mean that there would be negligible correlation between the Green and informal swings at the division level.

So, despite my efforts, I am yet to squeeze further insight from the data. Of course I remain open to further suggestions! If you would like to do your own analysis, the current 2010 data is available from the AEC as is past data.

UPDATE: If you sort the table at the top by informal vote, you’ll see that the two electorates with the lowest rates of informal voting were New England and Lyne, the seats of the independents Tony Windsor and Rob Oakeshott respectively!

Also, here is a national table of informal votes (just to avoid being to NSW-centric).

Dress: Informal

13 Replies

While Australia still waits to see which party will manage to scrape into power, the Australian Electoral Commission (AEC) has announced an investigation into the unusually high rate of informal votes. Veteran ABC analyst Antony Green observed that the rate of informal votes was the highest since 1984. Some are attributing the rise to the “Latham effect” following the exhortation by former Labor leader now professional provocateur, Mark Latham, that voters should spoil their ballots to thumb their noses at both major parties.

It will be interesting to see what conclusions the AEC draws, but there is no doubt that the informal votes in this election were significant.There are more votes to be counted and the trends in postal votes may differ somewhat from votes cast in person, but enough of the votes are in to get a reasonable picture of what has been going on. The figures here are based on the AEC data for the House of Representatives as at 23 August 2010. Informal votes rose in every state from the rate seen in the 2007 election, increasing by a margin of between 1.0% and 2.4%.

State	2007	2010	Change
NSW	4.95	6.87	1.91
NT	3.85	6.07	2.21
QLD	3.56	5.63	2.07
SA	3.78	5.54	1.76
WA	3.85	4.80	0.95
ACT	2.31	4.73	2.42
VIC	3.25	4.54	1.29
TAS	2.92	4.20	1.27

Informal Votes by State (%)

One way to visualize the changes is to plot the informal vote rate in 2010 against that of 2007. The chart below does this at a state level and also adds in a 45 degree line. Points falling above this line (as they all do) show an increase from 2007 to 2010, while points below the line would indicate a decrease.

Aggregating to a state level hides a lot of the interesting detail and can be misleading. For example, the ACT shows the biggest increase in informal votes, but with only two electorates, these figures have less statistical value. A more interesting picture emerges when the changes are shown by division. The chart below groups the changes by state, but plots points for each division*. Once again, 45 degree lines provide a guide as to whether informal voting rates increased or decreased.

Leaping out from this picture is the extraordinarily high rate of informal votes in some divisions in New South Wales. It is also striking that the rate of informal votes has increased in almost every division. At this point, there are only 4 divisions in the whole country (one in Victoria and three in New South Wales) to see the rate of informal votes drop.

It is hard to escape the conclusion that the increase in informal votes reflects a protest vote arising from deep voter dissatisfaction with both major parties. The Greens are pleased with the “Greenslide” they have experienced, but some of their success is likely to amount to the same voter protest, only expressed another way, rather than a permanent shift in commitment to the Greens.

* For the purists, there were changes to electorates between elections, and the chart only shows divisions which existed in both 2007 and 2010. Given changes to boundaries, some of these electorates are, strictly speaking, no longer perfectly comparable, but they have been plotted regardless.

Recognise this?

11 Replies

Last night I was watching the Chaser’s Yes We Canberra (only a day late), and jumped out of my chair when I saw Craig Reucassel corner Tony Abbott to challenge him about his obsession with reducing Government debt. Have a look at this to see why!

Here is the post referred to in the video.

UPDATE: here’s a tweet from Craig on the topic of attribution (or lack thereof): http://twitter.com/craigreucassel/status/21647798173

Infrastructure Bonds

15 Replies

With Australia’s Federal election looming, the opposition has today proudly announced a new policy to fund infrastructure without actually increasing Government debt! What are we to make of this?

It’s hard to determine the details from a media announcement, but based on the text posted by Peter Martin on his blog, it would seem that the idea is to provide tax incentives for entities other than the Federal Government to borrow to fund infrastructure:

Private infrastructure operators and State and Local Governments will be eligible to apply for the concessional treatment.

The way the scheme would seem to work is that eligible projects could issue bonds and investors would receive a tax rebate amounting to 10% of the interest on the bond. So, if you received a $100 interest payment and your earning put you in the top marginal tax bracket, you would pay $45 in tax. Under this scheme, you would only pay $35 in tax.

So, the cost to the Federal Government would simply be forgone tax revenue (and this would be capped at $150 million per annum) and the Opposition believes that the program could support up to $20 billion in infrastructure financing. Presumably, investors currently buying plasma TVs would rush to buy these bonds instead.

Seems like a neat trick, but I have a number of reservations about the scheme.

First, I have argued in the past that the near-hysterical concern about Government debt is overdone. For a start, Government debt in Australia is far lower than in other developed countries around the world. More importantly, the facile analogy that compares Government finance to that of a household budget does not stand up for one very important reason: unlike you or me, the Government is the monopoly issuer of Australian dollars. This changes the game and breaks the analogy utterly.

Second, the opposition’s policy would still involve raising significant amounts of debt, just not issued by the Federal Government. If that debt is all incurred instead by State Governments, should that really be a cause for celebration? After all, unlike the Commonwealth, State Governments do not control issuance of currency, so they really could go bankrupt and indeed, recent history has shown that many of the State Governments are loath to increase their debt levels too significantly for fear of having their credit rating downgraded. What if the borrowers are in the private sector? Well, that would be worse still! Back in March I updated my chart showing private and government sector debt. The debt level we should all be worried about in Australia is private sector debt, which is far higher than government sector debt.

Third, infrastructure bonds have form. Back in the 90s, the then Labor government introduced an infrastructure bond scheme which also featured tax incentives. Of course, it did not take long for clever investment bankers to work out how to surgically isolate the tax benefit so that wealthy individuals could take advantage of the concession without actually taking on any investment risk. In the end, the whole scheme was shut down, although some of the transactions that were done still survive today. I would expect exactly the same thing to happen with this policy. Any special tax treatment is always a red rag to the tax expert bull.

So, it may sound clever, but to me it does not seem to be sound policy.

Broadband Poll

16 Replies

As a follow up to our guest post on the numbers behind Labor’s broadband policy, here is a quick poll to see whose policy you prefer. Let us know what you think!

Labor’s National Broadband Network – Less than $10/month

43 Replies

Our regular guest contributor James Glover (aka @zebra) returns today with a look at the numbers behind the National Broadband Network. He asks: do you think it would be value for money?

The Labor Government’s proposed National Broadband Network (NBN) has many things to recommend it, not least speeds of up to 1GB/s (currently I am on 10Mb/s for ADSL; theoretical speeds of 24MB/s on ADSL2+ and 100MB/s on VDSL are also soon to be widely available, though the reality is dependent on many variables such as distance from an exchange). It would revolutionise the way we communicate as the higher bandwidth would allow not just interactive entertainment and fast downloads, but genuinely accessible cloud applications that really felt like they sat on your computer…and of course dishwashers waking up at 3.00am to negotiate the best electricity price. I doubt whether anybody on either side of politics would disagree that, in a perfect world, this is all desirable. But like all utopias, it comes at a cost and that is where the real divergence between the Labor Party and the Coalition’s broadband policies exists. I hope to cast some light on this cost argument using the power of the Time Value of Money, in particular calculating the real cost to you on a monthly basis so you can compare it with your existing broadband cost.

Labor wants an all-connecting fibre optic network (with subsidised satellite to cover really remote areas) that will cost an estimated $46bn. The Coalition wants a more modest effort: a fibre optic “backbone” network that uses existing copper wiring in urban areas and relies on market competition to pay for further improvements. It is estimated to cost about $8bn plus later commercial costs. Both of these figures seem extraordinarily high. How to decide if it is really worth it? Well if I told you that Labor’s NBN would cost you $10 per month would that sound too high? After all that only includes the infrastructure cost, not the access cost via an ISP. But most of us don’t pay upfront for our broadband or mobile (cell) phone bills, we pay monthly. The Coalition’s figure of $8bn works out at less than $2/month each (for those so inclined, you can read the details behind these figures). But it doesn’t include any additional costs charged by commercial companies building additional infrastructure. It also only claims to provide “peak speeds” of 10Mb/s which I already get on my ADSL+.

Is $10/month a lot of money? Or $2/month for that matter? It obviously depends on what your income is and how much you are currently prepared to pay for broadband. My broadband plan costs $50 for 120GB/month. I also live in a one-person household. It doesn’t sound much to me, but all those $10/month costs add up to the thousands we pay in tax each year. There’s no point paying more for little for no benefit. Of course it’s not going to be charged directly, but through increased taxes (or decreased services). I estimate $10/month to represent an average increase in the tax rate of about 0.5%. This seems reasonable to me. After all, if in 2020 a businessperson (or BusinessBot2020) came to Australia and found our broadband to be the equivalent of dial-up today, they’d hardly be impressed enough to invest in a technology business. Of course, by 2020 with super-fast broadband we should really be able to do most business remotely, right? But we’ve been saying that since the invention of the telephone.

So I’m for the Government’s NBN plan…but what do you think?

Update: I have since writing this post changed my mind based on readers’ comments and some research. It appears that many of the benefits of the NBN are available already on ADSL2+, VDSL and 4G and the Coalition’s more modest plan to build a fibre-optic network backbone might be sufficient. There is also the question of whether a Government entity is best placed to oversee such a large scale project – it’s not like Peter Garrett is going to personally project manage the NBN but Governments in general are not (IMO) best placed to predict and respond to consumer demand. But I accept there are strong feelings on both sides. Sometimes that bright shiny thing in your vision is a light on a hill and sometimes it’s a white elephant blocking your view.

UPDATE: Let us know what you think by voting in this broadband poll.

Stubborn Mule

Obstinately objective