Monthly Archives: January 2012

More on the Hottest 100

Following on from the last post on the Hottest 100, I received a few tweets from @mjdart demanding a deeper dive into the data. One of his questions was

Of artists charting in at least 5 yrs, are Oz artists higher represented?

I decided to broaden the questions to look at artists with at least five tracks in Hottest 100s (so artists with two tracks in one year and one track in three other years would be in). On this criterion, Australia still comes out on top.

Number of artists with at least 5 “Hot” tracks

In the last post, I complained that 2010 data is currently missing from Wikipedia. It seems that this is because Wikipedia is yet to get permission from the ABC. I have decided to risk the wrath of Auntie and have posted the full chart in the table at the bottom of the post. Having pieced it together, I have updated my original chart to include 2010.

As you can see, 2010 was a good year for Australian artists. It also turns out that the artist with the most Hottest 100 tracks is also Australian: Powderfinger. Here are all the artists with at least 10 Hottest 100 tracks.

Artist Count Country
Powderfinger 22 Australia
Foo Fighters 20 United States
Grinspoon 17 Australia
Silverchair 17 Australia
Muse 16 United Kingdom
The Living End 16 Australia
Regurgitator 15 Australia
Pearl Jam 14 United States
Placebo 14 United Kingdom
You Am I 14 Australia
Green Day 13 United States
Something for Kate 13 Australia
Eskimo Joe 12 Australia
Garbage 12 United States
Red Hot Chili Peppers 12 United States
Hilltop Hoods 11 Australia
Radiohead 11 United Kingdom
Spiderbait 11 Australia
The White Stripes 11 United States
The Whitlams 11 Australia
Wolfmother 11 Australia
Beck 10 United States
Ben Harper 10 United States
Jebediah 10 Australia
Metallica 10 United States
Rage Against the Machine 10 United States
The Strokes 10 United States

UPDATE:

@mjdart asked another question which I thought I should also answer:

@stubbornmule If you assign 100 pts for #1 thru 1 pt for #100 each year, is Oz proportion higher/lower? eg Oz filling out top or bottom 50?

To answer this, I assigned 100 points for the top spot, 99 points for the second and so on down to one point for last place. I summed the score for each country and then scaled it by dividing by 50.5. This odd choice arises because 100 + 99 + 98 + … + 2 + 1 = 5050 and so dividing by 50.5 would give a score of 100 for a country that managed to win every spot in the top 100. This makes the score directly comparable to a simple count of places in the top 100. So, how does this weighted score compare to a simple count? The answer, evident in the chart below is not much! So, each country’s artists must be fairly evenly spread through the top 100 over time.

Finally, here is the complete listing of the 2010 Hottest 100, including country of origin. If you are feeling brave, you may wish to update Wikipedia. Just keep in mind, the list may be deleted again if the ABC does not provide permission for the list to be published!

RankTitleArtistCountry
1Big Jet planeAngus & Julia StoneAustralia
2Rock ItLittle RedAustralia
3Dance The Way I FeelOu Est Le Swimming PoolEngland
4PlansBirds Of TokyoAustralia
5Fall At Your FeetBoy & BearAustralia
6Teenage CrimeAdrian LuxSweden
7Fuck You!Cee Lo GreenUnited States
8Tokyo (Vampires & Wolves)The WombatsEngland
9Magic FountainArt Vs. ScienceAustralia
10Somebody To Love Me {Ft. Boy George & Andrew Wyatt}Mark Ronson & The Business Intl.England
11ABC News Theme {Remix}PendulumAustralia
12RapunzelDraphtAustralia
13Clap Your HandsSiaAustralia
14Runaway {Ft. Pusha T}Kanye WestUnited States
15Barbara StreisandDuck SauceUnited States
16Mace SprayThe JezabelsAustralia
17Bang Bang Bang {Ft. MNDR & Q-Tip}Mark Ronson & The Business Intl.England
18There’s Nothing In The Water We Can’t FightCloud ControlAustralia
19Crave You {Ft. Giselle}Flight FacilitiesAustralia
20Sunday BestWashingtonAustralia
21Undercover MartynUTwo Door Cinema ClubNorthern Ireland
22JellylegsChildren CollideAustralia
23AddictedBliss N EsoAustralia
24Talking Like I’m Falling Down StairsSparkadiaAustralia
25Eyes Wide OpenGotyeAustralia
26Not In Love {Ft. Robert Smith}Crystal CastlesCanada
27You’ve Got The Dirtee Love {Live}Florence & The Machine/Dizzee RascalEngland
28Radar DetectorDarwin DeezUnited States
29It Can Wait {Ft. Owl Eyes}IllyAustralia
30O.N.EYeasayerUnited States
31Bloodbuzz OhioThe NationalUnited States
32Pumped Up KicksFoster The PeopleUnited States
33Solitude Is BlissTame ImpalaAustralia
34Punching In A DreamThe Naked And FamousNew Zealand
35The Bike Song {Ft. Kyle Falconer & Spank Rock}Mark Ronson & The Business Intl.England
36Opposite Of AdultsChiddy BangUnited States
37Doncamatic {Ft. Daley}GorillazEngland
38Young BloodThe Naked And FamousNew Zealand
39RevolutionJohn Butler TrioAustralia
40Baby, I’m Getting BetterGyroscopeAustralia
41Down By The RiverBliss N EsoAustralia
42On Melancholy HillGorillazEngland
43We No Speak AmericanoYolanda Be CoolAustralia
44BaptismCrystal CastlesCanada
45Rabbit SongBoy & BearAustralia
46Way Back HomeBag RaidersAustralia
47Wild At HeartBirds Of TokyoAustralia
48WitchcraftPendulumAustralia
49Easy To LoveThe JezabelsAustralia
50One Life StandHot ChipEngland
51AmblingYeasayerUnited States
52OverpassThe John Steel SingersAustralia
53ReflectionsBliss N EsoAustralia
54Holidays {Ft. Alan Palomo}Miami HorrorAustralia
55Giving Up The GunVampire WeekendUnited States
56Bring NightSiaAustralia
57KickstartsExampleEngland
58The SuburbsArcade FireCanada
59Rich KidsWashingtonAustralia
60My EagleChildren CollideAustralia
61Jackson’s Last StandOu Est Le Swimming PoolEngland
62Hold OnAngus and Julia StoneAustralia
63Ready To StartArcade FireCanada
64Jona VarkGypsy & The CatAustralia
65One StepDead Letter CircusAustralia
66Audience =Cold War KidsUnited States
67HolidayVampire WeekendUnited States
68Dog {Ft. Lisa Mitchell}Andy BullAustralia
69WatercolourPendulumAustralia
70Paper RomanceGroove ArmadaEngland
71Piper’s SongGypsy & The CatAustralia
72I Can TalkTwo Door Cinema ClubNorthern Ireland
73Time To WanderGypsy & The CatAustralia
74LucidityTame ImpalaAustralia
75Coming Around Hungry Kids Of HungaryAustralia
76RadioactivedKings Of LeonUnited States
77Shutterbugg {Ft. Cutty}Big BoiUnited States
78Stylo {Ft. Bobby Womack and Mos Def}GorillazEngland
79Slow Motion Slow MotionLittle RedAustralia
80Howlin’ For You Howlin’ For YouThe Black KeysUnited States
81Echoes EchoesKlaxonsEngland
82Tighten Up Tighten UpThe Black KeysUnited States
83Modern Man Modern ManArcade FireCanada
84The Hardest Part The Hardest PartWashingtonAustralia
85I Feel Better I Feel BetterHot ChipEngland
86Queensland QueenslandEvil EddieAustralia
87The Saddest Thing I Know The Saddest Thing I KnowBirds Of TokyoAustralia
88Monster {Ft. JAY-Z, Rick Ross, Nicki Minaj & Bon Iver}Kanye WestUnited States
89Barricade BarricadeInterpolUnited States
90Finally See Our Way Finally See Our WayArt Vs. ScienceAustralia
91Northcote (So Hungover)The Bedroom PhilosopherAustralia
92I Can ChangeI Can ChangeLCD SoundsystemUnited States
93Anyone’s Ghost Anyone’s GhostThe NationalUnited States
94Time To Smile Time To SmileXavier RuddAustralia
95The High Road The High RoadBroken BellsUnited States
96Go Do Go DoJonsiIceland
97SleepwalkerParkway DriveAustralia
98Spanish SaharaFoalsEngland
99BigBigDead Letter CircusAustralia
100Neutron Star Collision (Love Is Forever)MuseEngland

Hottest 100 for 2011

Another year, another Australia Day. Another Australia Day, another Triple J Hottest 100. And that, of course, means an excellent excuse to  set R to work on the chart data.

For those outside Australia, the Hottest 100 is a chart of the most popular songs of the previous year, as voted by the listeners of the radio station Triple J. The tradition began in 1991, but initially people voted for their favourite song of all time. From 1993 onwards, the poll took its current form* and was restricted to tracks released in the year in question.

Since the Hottest 100 Wikipedia pages include country of origin**, I thought I would see whether there is any pattern in whose music Australians like best. Since it is Australia Day, it is only appropriate that we are partial to Australian artists and they typically make up close to half of the 100 entries. Interestingly, in the early 90s, Australian artists did not do so well. The United Kingdom has put in a good showing over the last two years, pulling ahead of the United States. Beyond the big three, Australia, UK and US, the pickings get slim very quickly, so I have only included Canada and New Zealand in the chart below.

Number of Hottest 100 tracks by Country

If you have excellent eyesight, you may notice that 2010 is missing from the chart. For some reason, this is the only year which does not include the full chart listing on the Wikipedia page. There is a link to a list on the ABC website, but unfortunately it does not include the country of origin. Maybe a keen Wikipedian reading this post will help by updating the page.

I make no great claims for the sophistication or the insight of this analysis: it was really an excuse to learn about using the XML package for R to pull data from tables in web pages.

require(XML)
require(ggplot2)
require(reshape2)

results <- data.frame()
col.names <- c("year", "rank", "title", "artist", "country")

# Skip 2010: full list is missing from Wikipedia page
years <- c(1993:2009, 2011)

for (year in years) {
    base.url <- "http://en.wikipedia.org/wiki/Triple_J_Hottest_100,"
    year.url <- paste(base.url, year, sep="_")
    tables <- readHTMLTable(year.url, stringsAsFactor=FALSE)
    table.len <- sapply(tables, length)
    hot <- cbind(year=year, tables[[which(table.len==4)]])
    names(hot) <- col.names
    results <- rbind(results, hot)
}

# Remap a few countries
results$country[results$country=="Australia [1]"] <- "Australia"
results$country[results$country=="England"] <- "United Kingdom"
results$country[results$country=="Scotland"] <- "United Kingdom"
results$country[results$country=="Wales"] <- "United Kingdom"
results$country[results$country=="England, Wales"] <-"United Kingdom"

# Countries to plot
top5 <- c("Australia", "United States", "United Kingdom",
  "Canada", "New Zealand")

# Create a colourful ggplot chart
plt <- ggplot(subset(results, country %in% top5),
    aes(factor(year), fill=factor(country)))
plt <- plt + geom_bar() + facet_grid(country ~ .)
plt <- plt + labs(x="", y="") + opts(legend.position = "none")

Created by Pretty R at inside-R.org

UPDATE: there is a little bit more analysis in this follow-up post.

* Since the shift to single year charts, there have been two all-time Hottest 100s: 1998 and 2009.

** There are some country combinations, such as “Australia/England”, but the numbers are so small I have simply excluded them from the analysis.

Have wheelchair, will travel…probably

Spending couple of weeks down the south coast of New South Wales, spotting dolphins and echidnas, has slowed down my blogging. Fortunately, regular contributor James Glover has once more come to the rescue with a guest post. This time his topic is wheelchairs and air-travel.

Perhaps you’ve heard of a recent court case in which a wheelchair user, Sheila King, took Jetstar to court (and lost) on the basis of the Disabilities Discrimination Act? If you are a wheelchair user and you book a flight on one of our airline carriers then a fairly obvious thing won’t happen. Unlike say a bus you won’t be able to board the aircraft in your chair and be strapped in for the journey. What actually happens is that when making the booking you tick a box (or tell the booker on the phone) that you are in a wheelchair. If there are seats available for wheelies when you get to the airport you will give up your chair and be made to use a specially designed “wheelchair” (its a chair, it has wheels) that is designed to be fit the narrow corridor of most planes which I am sure you are aware of – their narrowness, for you, only apparent when the person ahead of you is blocking the aisle loading 3 pieces of carry on luggage into the overhead lockers while chatting to their new friends in the seat they are meant to occupy. We all suffer this situation. These “wheelchairs” are not designed to be used without help, they are more like children’s toy carts and cannot be operated by the user as the wheels are very small and low down. For a wheelchair user to be taken out of their wheelchair in a public place can be quite discombobulating. Many wheelchair users develop a personal relationship with their chair – it is after all a place you spend many of your waking hours.

Digression. The very first time I was in a wheelchair outside the confines of a hospital ward (it was a hospital wheelchair but is the exact same model I now own, like I said it is personal) I was being pushed by none other than the proprietor of this very website! Without going into the details let’s just say it was a pretty dramatic event and we both learned a valuable lesson in wheelchair use and the wheelchair repair workshop at the hospital was kept busy. But I digress.

So here is the thing. According to Google about 1% of the population uses wheelchairs. And a Jetstar plane has about 200 seats so they expect to get about 2 wheelchair users on average per flight. So what is the problem with only allowing this same number on each flight, as some airlines do? Well the problem is that statistically wheelchair users don’t travel in pairs and sometimes there will be less than 2 users and sometimes there will be more. Just as if you toss 10 coins sometimes there will be fewer than 5 heads (the average or expected number) and sometimes there will be more. Only on average will there be 5. In fact it is a simple problem to work out the probability of there being, say, n wheelchair users, given the average of 1% on a 200 seat plane. This is called the Binomial Distribution. If you have access to Excel then the function Binomdist(n,200,1%) will tell you this probability. Before I give you some numbers I admit that the overall population average may not be the same as the average flying on planes. It may be less than 1% due to wheelchair users being put off flying. But maybe on some routes it is higher: but I am guessing the annual “snowbird” migration of retired people from the northern United States to Florida at the start of Winter would track above the 1% rate.

So here are the Binomial probability figures.

Count Probability
0 13%
1 27%
2 27%
3 18%
4 9%
5 4%
6+ 2%

Binomial Probabilities (N=200, p=1%)

For example, assuming a 1% chance of any given passenger a 200 hundred seat plane being in a wheelchair, the probability that there will be exactly 4 wheelchair passengers wanting seats is 9%. To work out the probability of a passenger being denied a seat on their preferred flight, we will assume that we’re dealing with an airline where more than two wheelchair passenges book on a flight, then at least on passenger will have to change their travel plans. From the table above, the chance of the flight only having 0, 1 or 2 wheelchair passengers totals 68%, so there’s a 32% chance that there will be at least one wheelchair passenger who cannot fly. For any one wheelchair passenger, there is a (n-1)/(n+1) chance of being bumped if n other wheelchair passengers book on the flight. Weighting that by the probably that there are n passengers and adding it up for all n>1 gives a probability of 27%. As a frequent flyer in a wheelchair, you can expect to miss out on a seat quite regularly! [Note: these calculations have been updated: the editor’s “corrections” were undone. Ed.]

I am quite fortunate now that I no longer need to travel in my wheelchair. But as I still use a walking stick I wait for everyone else to get off the plane. You sit there, looking behind you to see if everyone else has left. But there are always these strange people who seem to sit there at the back of the plane and wait for 10 minutes or more, after everyone has disembarked, before even moving. You wonder why the airline staff don’t just hurry them off? I assume they aren’t disabled because they are sitting at the back of the plane. If airlines really had a problem with the extra time that getting wheelies off the plane then they could make this up by just moving these people along.

When I first read about this case my initial response was that being disabled and traveling is a bit of challenge anyway and you just get on with it. But the more I thought about it I wondered if the airlines just took it for granted that wheelchair users would change their plans to fit in with the rules. I am glad Sheila King took the issue up!

Where is the cheapest petrol?

For some time now, I have been meaning to have a look at Beautiful Soup, a python library designed to make it easy to scrape data from web-sites. Now that I have finally tried it out, I wish I had got to it sooner. It really is very handy and easy to use.

As my first Soup project, I turned to an old Mule favourite: petrol data. The Australian Institute of Petroleum (AIP) publishes retail petrol price data, which it sources in turn from MotorMouth. The price data is spread across individual pages for each state, like this one for Victoria and there are separate pages for unleaded and diesel. It would be nice to pull together all of the data and, since the pages are all laid out in exactly the same way and there is a straightforward naming convention for the urls of each page, this is very easy to do using Beautiful Soup. You can see the results showing average weekly petrol prices for the week ending 18 December in the table below.

With all the data in hand, the obvious questions to ask are: where is the cheapest petrol and where is the most expensive petrol? As you can see in the chart below, Adelaide came in as the cheapest place to fill your tank in late December at 134.9 cent/L, while Broome was the most expensive at 165.9, an impressive 23% mark up over Adelaide.

Australian Petrol PricesTop 10 and bottom 10 unleaded petrol prices
(average for the week ending 18 December 2011)

Of course this only gives a snapshot at a point in time: Adelaide may not always offer such good value for money and Broome residents may not always pay such a premium. Unfortunately, I have not been able to find any historical data by town on the AIP website. So I have set my data-scraping routine up to collect the data each week. Some time late this year I will revisit this data to see if any patterns emerge over time.

StateTownWeekly Average
NSWAlbury135.5
NSWArmidale143.6
NSWBallina146.9
NSWBatemans Bay144.3
NSWBathurst145.9
NSWBega150.1
NSWBroken Hill145.9
NSWBulahdelah138.1
NSWBuronga146
NSWCanberra146.4
NSWCasino144.8
NSWCentral Coast143.2
NSWCoffs Harbour147
NSWCooma150.3
NSWCootamundra149.2
NSWDeniliquin148.9
NSWDubbo143.1
NSWForbes147.5
NSWForster146.9
NSWGlen Innes141.9
NSWGoulburn141.9
NSWGrafton146.3
NSWGriffith144.4
NSWGundagai141.7
NSWGunnedah144
NSWHay145.8
NSWInverell146.9
NSWKempsey146.2
NSWLeeton145.5
NSWLismore144.5
NSWLithgow139.9
NSWMittagong149.9
NSWMoama145.9
NSWMoree148.8
NSWMoruya147.6
NSWMoss Vale142.1
NSWMudgee150.4
NSWMurwillumbah143.1
NSWMuswellbrook146.6
NSWNewcastle144.3
NSWNowra144.8
NSWOrange147.4
NSWParkes146.7
NSWPort Macquarie146.5
NSWQueanbeyan146.6
NSWSingleton143.8
NSWSydney138.2
NSWTamworth145.7
NSWTaree144.9
NSWTemora148
NSWTumut145.9
NSWTweed Heads South141
NSWUlladulla143.9
NSWWagga Wagga144.9
NSWWauchope143.6
NSWWest Wyalong149.9
NSWWollongong143
NSWWoolgoolga145.7
NSWYass148.4
NTAlice Springs163.4
NTDarwin151.8
NTKatherine146.3
NTTennant Creek164
QLDAtherton145.3
QLDAyr145.6
QLDBiloela148.6
QLDBlackall157.8
QLDBlackwater148.2
QLDBowen146.2
QLDBrisbane141.3
QLDBundaberg144
QLDCairns146
QLDCharters Towers149.4
QLDChilders142.9
QLDDalby142.4
QLDEmerald146.9
QLDGladstone143.5
QLDGold Coast141.3
QLDGoondiwindi148
QLDGympie143.3
QLDHervey Bay143.9
QLDIngham142.9
QLDInnisfail145.3
QLDKingaroy143.7
QLDLongreach153.9
QLDMackay141.2
QLDMareeba145.9
QLDMaryborough143.2
QLDMiles151.3
QLDMoranbah146.5
QLDMt Isa150.4
QLDRockhampton148.8
QLDRoma148.9
QLDSunshine Coast141
QLDToowoomba139.6
QLDTownsville141.9
QLDTully148.9
QLDWarwick143.7
QLDWhitsunday138.5
QLDYeppoon148.6
SAAdelaide134.9
SABordertown147.9
SACeduna150.6
SAClare138.2
SACoober Pedy160.4
SAKadina139.4
SAKeith145.9
SALoxton147.8
SAMt Gambier146.2
SAMurray Bridge140.6
SANaracoorte143.9
SAPort Augusta138.4
SAPort Lincoln145.2
SAPort Pirie139.1
SARenmark142
SATailem Bend146.4
SAVictor Harbour142.9
SAWhyalla143.5
TASBurnie150.3
TASDevonport149.8
TASHobart150.2
TASHuonville149.9
TASLaunceston149.9
TASNew Norfolk149.9
TASSmithton149.5
TASSorell144.9
TASUlverstone149.9
TASWynard152.5
VICArarat143.1
VICBairnsdale139.6
VICBallarat144.4
VICBenalla145.9
VICBendigo142.4
VICCobram142.5
VICColac145.9
VICCorryong148.2
VICEchuca145.9
VICEuroa140.2
VICGeelong136.3
VICHamilton146
VICHorsham145
VICKoo Wee Rup139.5
VICKyabram143.9
VICLeongatha142.9
VICMelbourne137.9
VICMildura147.5
VICMoe139.9
VICMorwell142.9
VICPortland146.9
VICSale140.9
VICSeymour137.9
VICShepparton143.6
VICSwan Hill146.7
VICTraralgon142.2
VICWallan138.1
VICWangaratta142.3
VICWarrnambool143.8
VICWodonga139.1
VICYarrawonga149.9
WAAlbany146.8
WABoulder151.7
WABridgetown144.9
WABroome165.9
WABunbury137.1
WABusselton141.3
WACarnarvon154.3
WADongara153.9
WAEsperance145.9
WAGeraldton149
WAKalgoorlie149.6
WAKarratha159.9
WAMajimup143.9
WAMount Barker149.2
WAPerth138.3
WAPort Hedland160.2
WAWaroona144.4