Following on from the last post on the Hottest 100, I received a few tweets from @mjdart demanding a deeper dive into the data. One of his questions was

Of artists charting in at least 5 yrs, are Oz artists higher represented?

I decided to broaden the questions to look at artists with at least five tracks in Hottest 100s (so artists with two tracks in one year and one track in three other years would be in). On this criterion, Australia still comes out on top.

Number of artists with at least 5 “Hot” tracks

In the last post, I complained that 2010 data is currently missing from Wikipedia. It seems that this is because Wikipedia is yet to get permission from the ABC. I have decided to risk the wrath of Auntie and have posted the full chart in the table at the bottom of the post. Having pieced it together, I have updated my original chart to include 2010.

As you can see, 2010 was a good year for Australian artists. It also turns out that the artist with the most Hottest 100 tracks is also Australian: Powderfinger. Here are all the artists with at least 10 Hottest 100 tracks.

UPDATE:

@mjdart asked another question which I thought I should also answer:

@stubbornmule If you assign 100 pts for #1 thru 1 pt for #100 each year, is Oz proportion higher/lower? eg Oz filling out top or bottom 50?

To answer this, I assigned 100 points for the top spot, 99 points for the second and so on down to one point for last place. I summed the score for each country and then scaled it by dividing by 50.5. This odd choice arises because 100 + 99 + 98 + … + 2 + 1 = 5050 and so dividing by 50.5 would give a score of 100 for a country that managed to win every spot in the top 100. This makes the score directly comparable to a simple count of places in the top 100. So, how does this weighted score compare to a simple count? The answer, evident in the chart below is not much! So, each country’s artists must be fairly evenly spread through the top 100 over time.

Finally, here is the complete listing of the 2010 Hottest 100, including country of origin. If you are feeling brave, you may wish to update Wikipedia. Just keep in mind, the list may be deleted again if the ABC does not provide permission for the list to be published!

Another year, another Australia Day. Another Australia Day, another Triple J Hottest 100. And that, of course, means an excellent excuse to  set R to work on the chart data.

For those outside Australia, the Hottest 100 is a chart of the most popular songs of the previous year, as voted by the listeners of the radio station Triple J. The tradition began in 1991, but initially people voted for their favourite song of all time. From 1993 onwards, the poll took its current form* and was restricted to tracks released in the year in question.

Since the Hottest 100 Wikipedia pages include country of origin**, I thought I would see whether there is any pattern in whose music Australians like best. Since it is Australia Day, it is only appropriate that we are partial to Australian artists and they typically make up close to half of the 100 entries. Interestingly, in the early 90s, Australian artists did not do so well. The United Kingdom has put in a good showing over the last two years, pulling ahead of the United States. Beyond the big three, Australia, UK and US, the pickings get slim very quickly, so I have only included Canada and New Zealand in the chart below.

Number of Hottest 100 tracks by Country

If you have excellent eyesight, you may notice that 2010 is missing from the chart. For some reason, this is the only year which does not include the full chart listing on the Wikipedia page. There is a link to a list on the ABC website, but unfortunately it does not include the country of origin. Maybe a keen Wikipedian reading this post will help by updating the page.

I make no great claims for the sophistication or the insight of this analysis: it was really an excuse to learn about using the XML package for R to pull data from tables in web pages.

require(XML)
require(ggplot2)
require(reshape2)

results <- data.frame()
col.names <- c("year", "rank", "title", "artist", "country")

# Skip 2010: full list is missing from Wikipedia page
years <- c(1993:2009, 2011)

for (year in years) {
    base.url <- "http://en.wikipedia.org/wiki/Triple_J_Hottest_100,"
    year.url <- paste(base.url, year, sep="_")
    tables <- readHTMLTable(year.url, stringsAsFactor=FALSE)
    table.len <- sapply(tables, length)
    hot <- cbind(year=year, tables[[which(table.len==4)]])
    names(hot) <- col.names
    results <- rbind(results, hot)
}

# Remap a few countries
results$country[results$country=="Australia [1]"] <- "Australia"
results$country[results$country=="England"] <- "United Kingdom"
results$country[results$country=="Scotland"] <- "United Kingdom"
results$country[results$country=="Wales"] <- "United Kingdom"
results$country[results$country=="England, Wales"] <-"United Kingdom"

# Countries to plot
top5 <- c("Australia", "United States", "United Kingdom",
  "Canada", "New Zealand")

# Create a colourful ggplot chart
plt <- ggplot(subset(results, country %in% top5),
    aes(factor(year), fill=factor(country)))
plt <- plt + geom_bar() + facet_grid(country ~ .)
plt <- plt + labs(x="", y="") + opts(legend.position = "none")

Created by Pretty R at inside-R.org

UPDATE: there is a little bit more analysis in this follow-up post.

* Since the shift to single year charts, there have been two all-time Hottest 100s: 1998 and 2009.

** There are some country combinations, such as “Australia/England”, but the numbers are so small I have simply excluded them from the analysis.

For some time now, I have been meaning to have a look at Beautiful Soup, a python library designed to make it easy to scrape data from web-sites. Now that I have finally tried it out, I wish I had got to it sooner. It really is very handy and easy to use.

As my first Soup project, I turned to an old Mule favourite: petrol data. The Australian Institute of Petroleum (AIP) publishes retail petrol price data, which it sources in turn from MotorMouth. The price data is spread across individual pages for each state, like this one for Victoria and there are separate pages for unleaded and diesel. It would be nice to pull together all of the data and, since the pages are all laid out in exactly the same way and there is a straightforward naming convention for the urls of each page, this is very easy to do using Beautiful Soup. You can see the results showing average weekly petrol prices for the week ending 18 December in the table below.

With all the data in hand, the obvious questions to ask are: where is the cheapest petrol and where is the most expensive petrol? As you can see in the chart below, Adelaide came in as the cheapest place to fill your tank in late December at 134.9 cent/L, while Broome was the most expensive at 165.9, an impressive 23% mark up over Adelaide.

Top 10 and bottom 10 unleaded petrol prices
(average for the week ending 18 December 2011)

Of course this only gives a snapshot at a point in time: Adelaide may not always offer such good value for money and Broome residents may not always pay such a premium. Unfortunately, I have not been able to find any historical data by town on the AIP website. So I have set my data-scraping routine up to collect the data each week. Some time late this year I will revisit this data to see if any patterns emerge over time.

Returning for another guest post, James Glover is once again drawn to a beer coaster for some quick, if somewhat morbid, calculations. For those taking to the road over the Christmas period, this post should also serve as a reminder to drive carefully!

Why did the chicken cross the road? To get to The Other Side. This twist on the ending to the iconic joke was based on the observation of a single dead chicken on the road while I was returning by car from Sydney to Melbourne at the end of my recent touring holiday. The holiday in fact started two weeks earlier when I was driving around Tasmania. While there were no chickens, there was a dead sperm whale on the road.

In Tasmania I noted that in addition to its abundance of quality food, beer and wines, it has a remarkable supply of one other thing compared to the mainland: native fauna road kill. If you think that the occasional dead kangaroo (or more likely fox) you see driving in the country is plenty, then you haven’t been to Tassie. We are talking a dead possum or wallaby every kilometre or so, which means if you are driving at 100km/hr means you see one every 36 seconds. While it is unfortunate, particularly for the animals involved, to see so many native animals dead (but no foxes because they have been eradicated from the island), it is actually cause for joy because it indicates a very healthy population of native wildlife.

This got me thinking. Could you actually use the number of road kill to estimate the density of animals living in the bush? The answer is yes, and without any derivation or proof I present it here:

density = road kill ratio/kill zone area

where

road kill ratio = av. distance between cars/
                   av. distance between road kill
kill zone area = 2 x car velocity x time x car width

It’s a surprisingly simple formula, and can also be handy for keeping children occupied on long car trips (at least in Tasmania). The model it is derived from is admittedly fairly simplistic – let’s just call it the “Frogger model of vehicle/animal interaction”. Here the time is important because clearly if we had an infinite amount of time and no method of disposal of road kill then the number would build up without limit. In practice the attendant carrion birds on each road kill and its, shall we say, “freshness” (blood, guts, brains you get the picture) suggests that they were all products of the previous nocturnal period’s collisions. In fact there are road signs indicating to drivers to be particularly careful between dusk and dawn to avoid animals so I take “time” to be 10 hours and assumed all carrion are fresh. Taking the average distance between road kill to be 1km and cars to be 10km gives a road kill ratio of 10. The average car speed was 100 km/hr and my car is about 2.5m wide so putting this all together gives an estimated bush density of 20 animals per square km. That’s about one per 5 hectares. That seems a little on the low side for a densely populated area but as “beer coaster” estimates go, it’s probably not a bad start.

This reasoning got me thinking that there probably true that there are no Tasmanian tigers left, because one would have shown up as road kill by now. Tasmanian rangers patrol the roads every morning looking for Tassie devils to monitor the spread of that awful face tumour disease they get. In fact, I saw two Tassie devils on the road and they were both alive and moving. I also saw three live echidnas: three more than I have seen in my life up to now. As mentioned I also saw a dead sperm whale on the road. Not some replica or whale skeleton either: it had died the day before. It was on the beach near Strahan which I was driving on at the time. So, yes, it was on my road and hence I feel justified classifying it as road kill. And no, I don’t know why the whale crossed the road. Perhaps it mis-heard someone say that in Tasmania there was an abundance of “road krill”.

It has been quite a long time since art was the subject of a post here on the Mule, but today we took the kids to see Sculptures by the Sea. Held each year, this exhibition consists of a series of large sculptures arranged along the coast from Bondi beach to beach. As usual, parking was challenging, but as usual, the effort was worthwhile. We did not make it along the full length of the exhibition (small legs got a bit too tired), but there were some excellent pieces. The family favourite was, without a doubt, the magnificent stag. Here are a few photos.

I have been resting on my laurels for too long. Two years ago I had Shocking success tipping a winner for the Melbourne Cup. Needless to say the analysis was entirely bogus, but it was fun. Since then I have been reluctant to tarnish my spotless prediction record, but fortune favours the brave, so I really should try again.

Inspired by my last analysis, The Hoopla is tipping Lost in the Moment, which is a 5 year-old bay horse starting at barrier three and is carrying 53 kg. Unfortunately, my statistics tell me that over the last 150 races going back to 1861, there have been five Cups in which a horse with three words in its name has won, but never one with four words. That does not bode well for Lost in the Moment.

I will not give up entirely on a good formula, so based on the analysis from two years ago, I would still like to tip a bay horse with a handicap in the 50-55 kg range starting in the barrier range 1 to 5. This time, however, I will take into account the fact that over half of previous Cup winners only had a one-word name.

Three of the horses in barriers 1 through 5 have a one-word name, and they are all carrying around 53 kg:

• Illo
• Precedence
• Modun

According to this form guide (which is the only one not blocked at my place of work, which limits access to “gambling sites” and yet is doubtless hosting many Melbourne Cup festivities), Precedence and Modun are geldings, which rules them out (they come second to horses in the chart below).

So: the Stubborn Mule tip for the 2011 Melbourne Cup is Illo.

For anyone foolish enough to take heed of this tip, I appreciate it, but you are foolish.

UPDATE: after a good run for much of the race (always a bad sign) Illo failed to win or even place. So it seems that the Mule has lost his crown.

As Australia’s economic fortunes continue to surpass the likes of the US, UK and Europe, it is hard to escape a lingering nervousness about what could happen if the mining boom were to collapse. What if the Chinese juggernaut were to falter? Would we be doomed?

Having a conversation exactly like this earlier in the week, I was reminded of a post I wrote more than a year ago which showed surprisingly (to me at least) that exports to China were contributing only 3% to Australia’s gross domestic product (GDP). In yesterday’s Sydney Morning Herald, economist Ross Gittins tried to bring some perspective to the nervous by pointing out that 80% of Australia’s economic activity is domestic and concluded that:

Take away mining and we wouldn’t be quite as rich as we are, but most of the economy would look much the same as it does. Most of us would still have good, secure, well-paid jobs.

Of course, not everyone is taking such an encouraging line. Over on the Mule Stable, one econo-pessimist drew my attention to this interview with hedge fund manager John Chanos, who has been predicting a bursting of the Chinese economic bubble for some time now. As well as showing a very detailed knowledge of China’s construction industry, Chanos notes that were China’s economy to stall, the US would be much better positioned to cope with it than countries like, say, Australia. That was supposed to be good news for American viewers…not so cheering for those of us on this side of the globe!

All of this suggested that an update of the trade statistics was overdue. The results are as one might expect: the contribution that exports to China make to Australia’s GDP has risen from the 3% I noted in August 2009 to almost 4% as at September 2010.

GDP from Exports to China (Dec-1988 to Sep-2010)

So, while 4% may still be small compared to the 80% of activity that is generated internally in Australia, the real story here is growth, as the steepness of the chart dramatically illustrates. That increase in exports has contributed almost 1% to Australia’s GDP growth for the year! Here is the rolling annual change in the contribution that exports to China make to Australia’s GDP.

Annual Change in GDP from Exports to China (Dec-1988 to Sep-2010)

Not wishing to forget Gittins’ point that we should consider total contributions to the economy, not just exports, it is hard to resist wondering how many of our exports now go to China. The answer is: a lot and growing.

China’s Share of Exports (Dec-1988 to Sep-2010)

So, where does that leave us? Gittins is not wrong, and a collapse in the Chinese economy would not suddenly put everyone in Australia out of work. Nevertheless, it would certainly take a lot of the wind out of our economic sails. Furthermore, given the amount of attention China and the mining industry have in our national consciousness at the moment, it is worth recalling the words of that sage John Maynard Keynes:

Even apart from the instability due to speculation, there is the instability due to the characteristic of human nature that a large proportion of our positive activities depend on spontaneous optimism rather than mathematical expectations, whether moral or hedonistic or economic. Most, probably, of our decisions to do something positive, the full consequences of which will be drawn out over many days to come, can only be taken as the result of animal spirits – a spontaneous urge to action rather than inaction, and not as the outcome of a weighted average of quantitative benefits multiplied by quantitative probabilities.

There is little doubt that if Chanos is right about China, our animal spirits would not take it too well.

Data source: based on Australian Bureau of Statistics (copyright Creative Commons Attribution). Note that all export figures here represent exports of merchandise, so exclude services.

Some readers will recall last year’s astonishing correct tip for the Melbourne Cup here on the Mule. I was vaguely contemplating an equally spurious analysis this year, but inspiration has eluded me and so I will instead rest on my thoroughly undeserved laurels and preserve my perfect prediction record.

Ever since Julia Gillard managed to wangle the support of two of the three country independents and scrape through to a second term in government, speculation has focused on how long the arrangement can last…and not only in the media but also on the Mule Stable.

Challenging though the road ahead may be for the new government, with so many different interests to juggle, I am of the view that Labor will do whatever they can to hold on to power. Even if they are unable to pass “crucial” legislation, they would be very unlikely to go to the polls early lest they lose the election. After all, if they did not have the courage to trigger a double dissolution when they failed to pass emissions trading legislation to combat the “greatest moral challenge of our time”, it is hard to see what issue could be important enough to them to jeopardise their power.

As for the independents, another election would risk their own new-found power. Furthermore, in siding with Labor they have not really promised very much. All they are committing to is to pass supply and to support the government in the event that no confidence motions are brought against it. On each and every particular piece of legislation they are free to horse-trade once more and potentially vote against the government. Also, as Bob Brown recently pointed out, there is nothing to stop the independents and the Greens backing legislation initiatives brought forward by the Liberals. So there really is no good reason for the independents to withdraw their support from Labor.

Without the numbers, the Liberals and Nationals are powerless to bring on an early election. So, this unlikely new coalition government is likely to be here to stay. The only scenario I can see that could undo Labor is a by-election. If one of the MPs supporting Labor were to fall under a bus, retire, disgrace themselves and resign or in some other way leave the Parliament, the Liberals would have the chance to win the by-election and chance the numbers on the floor. Failing that, I would expect to see Labor ruling for a full term.

What do you think? While it may take some time to see the result, this seems like a good opportunity for another poll on the Mule, so have your say!

(polls)