I just saw something thought provoking: “The rapid drop in intelligent Congressional speech”. It’s based on this analysis from the Sunlight Foundation, drawing out some comparisons between famous speeches & documents from years gone by and this chart of trends in the language complexity of congresspeoples’ speeches over the past 15 years or so:
Much as I want to read this chart as part “political discourse today has gone down the toilet” and part “those damn Republicans debasing political discourse”, I see several really important caveats:
- Is this statistically significant? My gut tells me it must be because it’s a consistent enough pattern, but it’s also a pretty small difference in scores overall and gut checks are no substitute for significance testing.
- It’s based on Flesch-Kincaid reading level scores – is this a useful measure? One gets a higher score by using longer words and longer sentences – not particularly good public speaking technique, nor a reliable benchmark of the complexity of ideas, and I would expect pre-20th Century prose to score higher just because that’s how the language has evolved (for an example, read Conrad, Greene & Grisham in sequence, and remember that each was writing popular thrillers in their time). For that reason I’m not at all surprised that MLK scored lower than Lincoln: each was using the language of their time, pitched right for a broad audience.
- I don’t think it makes sense to compare speeches with written documents – a successful speech is written using simpler language than a good constitutional document, precisely because of the ways in which it’s expected to be used. So the only comparison with pre-20th Century touchstones that makes sense is the Gettysburg Address, which at 11.2 is apparently on a par with the simpler speeches in 96-2005.
- Related to the previous two points: remember that before widespread radio, even addresses that were given as speeches would have reached most people as a transcript, whereas these days far more people will listen to a speech than read it. A good speech writer will be very conscious of this, and a good off-the-cuff speaker will have trained themselves differently in this light.
- Finally, I don’t remember the source, but there was a study in the 90s of Canada’s parliament that showed opposition parties consistently giving the simpler speeches. The authors’ contention was that in opposition it’s much easier to simply point out criticisms, while the governing party has to defend its actual record and put forward plans that won’t embarrass it in the coming months. The party trend in that chart above certainly fits this model, and I wonder if controlling for it would explain the whole difference between parties?
Actually this story is a bit of a rehash of something from a few months ago, with a different political slant on it (surprise surprise):
http://www.economist.com/blogs/johnson/2012/01/readability
See particularly this bit:
“However, there’s no normative weight to the Flesch-Kincaid grade level. The score is a function of how long the sentences are and how many syllables the words have. It’s a weak proxy for accessibility, not substance or value. I just tested a couple of recent articles in The Economist—which I hope we can all agree is a reasonably well-written publication—and found grade levels of 10.3, 10.6, and 10.8. George Orwell’s “Why I Write”: 9.5. “The Love Song of J. Alfred Prufrock”: 7.4, suggesting that the Fleisch-Kincaid formula isn’t that sensitive to context. In any case, such comparisons are a little silly; no one judges political speeches on their syntactic complexity. (Reagan’s address to the nation after the Challenger disaster: 5.7.)”
Right, hence points 2-4. Though I will say that including poetry is an even more invalid comparison – concepts like sentence length barely make sense in verse, and the Economist Style Guide explicitly encourages using the simplest vocabulary & grammar possible to make any given point.
I’m not sure if point 5 ties in with this or not. I seem to recall that that study actually looked at content rather than form, by counting things like the number of distinct ideas in each speech, but it’s a long time since I read it and I don’t exactly remember.
Ah, this meme is back, now with pretty graphics from the Grauniad showing a reduction in F-K reading levels over successive State Of The Union addresses: http://www.guardian.co.uk/world/interactive/2013/feb/12/state-of-the-union-reading-level
Helpfully, they also provide an excerpt from each one. Go and read some of the early ones, and tell me if you’re really convinced this says anything about intelligence or sophistication as opposed to dialect change.