Publication time – my first crack at a new (to me) field

Until 15 May 2022, you can get a copy of our new paper here. It’s published in Gynecologic Oncology (online from 26 March 2022). It will be freely available again later, once the journal has had a chance to make some money from holding it behind a paywall. In the meantime, do contact me (maybe on Twitter @confoundingfact) if you’d like a copy.

This study looks at the quality of life of women with gynaecological cancer when they receive their diagnosis, and one year on from that diagnosis. Quality of life is an umbrella term which covers a whole range of feelings and situations: energy levels, sociability, sex, and ability to concentrate, as well as physical symptoms such as pain, fatigue and weakness. There are lots of different accepted question batteries for getting at this, but we used two which are cancer-specific: they’re called QLACS and QLQ-C30. The idea of picking these as ‘outcomes’ for study is to look at the whole person. We didn’t concentrate on whether a woman’s cancer was well-treated, or whether she could be considered ‘cured’, or whether she had side-effects from treatment: we looked at a summary measure intended to represent many aspects of her quality of life.

Participants who were depressed or anxious before starting treatment had worse quality of life both at diagnosis and after 12 months, even after controlling for other factors which are known to be associated with both mental health and quality of life outcomes (such as socio-economic status, physical health status, social support, diet, and exercise). We know from previous research that there is a need for mental health screening for people with cancer so that appropriate support can be given. This paper shows that poorer mental health is associated with worse all-round outcomes.

Women who are more confident in living with or managing the effects of cancer and cancer treatment have better quality of life at diagnosis and one year on: this type of practical confidence is called “self-efficacy”. Our paper supports the idea that self-efficacy is an important protective factor for quality of life. We think that it’s something that clinicians should look for, and support women to gain confidence, as they treat patients with gynaecological cancer.

Posted in Academic research, Cancer | Tagged , , , , , , , , | Comments Off on Publication time – my first crack at a new (to me) field

The pain, the pain

I should have written “I’m done!”nearly three years ago. I couldn’t face it. Finishing my PhD was soul-destroying. I had several changes in supervision* and a sizeable disagreement with my incoming supervisors about the work I had already produced, which resulted in a lot of wasted time. Simultaneously, I was becoming increasingly unsure about whether I really wanted to work in academia. Although my department (social statistics and demography) was lively and apparently well-funded, there were a lot of rather depressed and aimless PhD students in the building, and a number of excellent post-doc researchers scrabbling around for short teaching contracts and otherwise working for free in an attempt to increase their publication count.

In academia you must publish or perish. It’s clear to me that academic publishing is broken in a pretty fundamental way. Academics research and write the content. Some of them are paid to do this and some aren’t. Then more academics run the journals, acting as editors and editorial assistants, and sitting on editorial boards. Most of them are not paid, though a few receive an honorarium. Articles are submitted and then go out to reviewers (this is the ‘peer review’ process that everyone gets so excited about). These reviewers are also academics. None of them are paid for reviewing articles. Finally, an article appears in print. None of the people who produced it made any money, and none of the universities or funding bodies made any money either, but you can be sure that the publishing company did. (You can tell that it did by checking online – most academic research is held behind a pay-wall.) If I wanted to publish papers from my PhD now, I wouldn’t be paid for that labour.

Additionally, a big change is underway in academic journals, from a subscription model to ‘pay to play’. I first wrote about this in 2013. It’s clear (to me, to a Martian, to anyone really) that publicly funded research should be publicly available. So public funders (in the UK, that’s the research councils) have begun to insist on it. Journals have obliged by offering a ‘gold’ payment option, which allows academics to effectively pay the subscription fee themselves to allow their article to appear in the public domain. (By the way, of course it’s usually the research councils paying the fee, not individuals.) Alternatively, there is a free ‘green’ embargoed option, which a research council may be minded to accept instead. This gives the journal publishers some months to make money from the publicly funded article before it appears in the public domain.

Of course, nothing in academic life could be as simple as just ‘making research freely available’. As John Holmwood argues, this only really stands up well to scrutiny when there’s no money to be made directly from the research concerned. That might well be true in the humanities and social science, but is less likely to be true in the more eminently patent-able STEM subjects. In any case, my intention here is not to solve the problem, but merely to demonstrate its scope and apparent intractability.

Thus, if I wanted to make a career in academia, I would first have to do considerable unpaid work to produce the publications which are a prerequisite. In addition, since I no longer have funding or an institution from which to obtain funding, I would be limited in the journals I could target, assuming I didn’t wish to pay the ‘gold’ open access publishing fee myself. The (post-award) requirements of my (long since spent) funding require that I pursue open access publishing for my work, so I am prevented from choosing a subscription only journal.

Even if I could overcome these objections, my work would still have to pass a peer review. Since I haven’t worked on it for three years, I suspect that the first reviewer request would be to update my references. But I have only very limited access to academic research (that access comes from my undergraduate institution which funds an alumni subscription to JSTOR – god bless the University of Oxford and all who sail in her) which makes an update a particularly frustrating task. And if you want to read a no doubt excellent BMJ article on the problems of peer review, you’ll have to stump up £23.

And this is just ONE of the ways that we are funding and then losing perfectly good research and researchers. I may be back to enumerate the others. Or I may cut the strings and never look back. It remains to be seen.

*Through it all, the wonderful John Mohan never gave up on me and my research, and he still hasn’t. These days you can find his research group, the Third Sector Research Centre, here at the University of Birmingham.

Posted in Uncategorized | Tagged , , , | Comments Off on The pain, the pain

I’m done!

I finished my PhD. It was quite a while ago now. You can download a pdf here, or via the University of Southampton. In short, I found that volunteering was associated with a small, positive effect on political activity. However, that relationship is heavily influenced by social class.

Early volunteering seems to be somewhat formative, in that it is associated with higher rates of political activity in middle age. However, that relationship is much stronger for upper and middle class people, and breaks down completely for people from the most working class backgrounds. There’s a circle, but it’s more privileged than it is virtuous.

Secondly, volunteering last year is associated with slightly raised political interest next year, even after heroic attempts are made to ‘net out’ vital explanatory factors such as upbringing and culture. The effect is small, though, and, as for the longer term relationship, analysis suggests that it is highly dependent on social class.

These small, positive effects should not be overstated. They do not offer sufficient evidence to encourage the use of volunteering as a policy tool to improve political engagement.

I also investigated generalised social trust as a potential mediator for the relationship between volunteering and political activity and found no evidence that trust plays an important role.

So there it is. Four years’ work and many, many tears, in one blog post. Read it and weep.

Posted in Uncategorized | Tagged , , | Comments Off on I’m done!

I trust we’re all in this together…?

Spot the odd one out.

  1. I trust you.
  2. She’s a trusting soul.
  3. Do you trust him?
  4. Generally speaking, would you say that most people can be trusted or that you can’t be too careful in dealing with people?

Of course it’s 4, a survey question designed to get at “generalised” or social trust.  It’s so ubiquitous in social surveys that it’s known by an acronym: GTQ – the Generalised Trust Question.  Generalised trust is the sense we have that the people around us – people we don’t know personally, but who occupy the same public spaces as us – are generally safe to be around.  A generalised truster sees friends he hasn’t met yet; a non-truster sees a paedophile behind every bush and a pickpocket in every crowd.

I’ll be honest and admit that after writing that sentence I checked the correlation between ‘trusting’ and ‘reading the Daily Mail’ using the 2009/10 Citizenship Survey.  I was slightly disappointed to find that there’s no difference between Mail readers and the rest of the population.  Guardian readers, on the other hand, are a trusting lot.  Generalised trust stands at around 40% in the population (and among Mail readers) but 65% for Guardian readers.  It’s a small sample (383 Guardian readers out of a core sample of 9305).  Anyway…

Generalised trust is an odd concept.  Trusters and non-trusters must be moving in different circles, as well as having different habits of mind.  An intelligent truster in a neighbourhood full of cheats, thieves and liars would soon change her attitude.  Patrick Sturgis has an interesting paper on the subject which links trust to intelligence.  His hypothesis runs roughly that bright people (well, children who scored well on intelligence tests) will go on to be more trusting adults (measured by the GTQ) because they make good judgements.  Bright people give their trust to unknown others under circumstances in which that trust is most likely to be rewarded, so they tend to have good experiences, so they continue to be trusting: essentially, they make good judgements.  People who maybe aren’t so bright invest their trust less wisely and so have bad experiences, and end up less trusting.  Of course, as Sturgis points out, intelligence test scores in childhood and generalised trust are both strongly social class-related: rich kids are brighter and rich adults are more trusting.  Using the 2009/10 Citizenship Survey data again, 53% of adults in management are trusting, compared to 32% of those in routine or semi-routine jobs.

Trust is an attitude, linked to personal circumstances but also to personal history and character.  Robert Putnam (he of Bowling Alone, social capital, the evils of TV and the demise of schmoozing) uses it as the key social attitude linking community cohesion and political and civic health.  People who schmooze, who meet with their neighbours for coffee, who volunteer in their communities, are more trusting than loners, he says, and their trust grows as a result of their positive contacts with the people around them.  He also says that trust is good for democracy: trusting people have more reasons to get involved with running their communities, they vote more often, and engage more fully with the democratic process, contacting politicians and local officials more often than non-trusters.

This tangled “bowl of spaghetti” (that’s Putnam again) is pretty much impossible to unpick, but it seems to me, that the fork around which the spaghetti twirls is social class.  That muffled twang, by the way, was the sound of the spaghetti metaphor unravelling.  Social class at birth predicts IQ, trust, future social class (forget about social mobility right now), future salary, civic and political engagement, newspaper-readership, and pretty much anything else you can think of.  Your father’s job title when you are born offers enormous insight into your future civic and political life (and everything else) – or certainly it has done up until now.  I just wish I could see that changing any time soon.

Posted in Academic research, PhD | Tagged , , , , , , | Comments Off on I trust we’re all in this together…?

A sense of dignity

I went to hear Jonathan Wolff (UCL Philosophy) give a lecture on relative poverty and social inquality tonight.  It’s part of a piece of work he’s doing for the Joseph Rowntree Foundation (JRF). The JRF is bringing together academic ideas on poverty from across disciplines – economics, political science, and also philosophy. Wolff said that poverty is much ignored by modern political philosophers, in favour of equality and justice: Rawls himself doesn’t mention it once.  He then emphasised his point by quoting extensively from economists and social statisticians, referencing in particular the works of Adam Smith and Benjamin Seebohm Rowntree.

Adam Smith in The Wealth of Nations addressed the relative, cultural aspects of poverty, writing: “A linen shirt, for example, is, strictly speaking, not a necessary of life…  But in the present times, through the greater part of Europe, a creditable day-labourer would be ashamed to appear in public without a linen shirt…”  There are some activities and goods which are so central to life in our society that we are prepared to forgo even food to buy into them. These activities and goods are strongly status-linked. For a school-child, it might be a set of clothes which are not school uniform. For a worker, it might be a drink in the bar on a Friday night. For a teenager, it might be a smartphone.

Poverty is essentially relative – and this relativity is local. Some groups are so far from the societal goodies that they turn inward. There is no point reaching for the next rung on the social ladder – it’s at ceiling height – so status is measured against a more local norm. Wolff talked about gangs in South Africa which prioritise an unusual dental arrangement: removing the front four incisors.  People with this modification are marked to the extent that they will be unable to compete for status in South African society’s mainstream, but they have high status in Cape Town’s gang culture.

In this, as in so much political science and philosophy, The West Wing has got there first.  In the episode Isaac and Ishmael, we hear the following words spoken by Charlie, the President’s bag-carrier:

Gangs give you a sense of belonging, and usually, an income. But mostly, they give you a sense of dignity. Men are men, and men’ll seek pride… You think bangers are walking around with their heads down, saying, “Oh man, I didn’t make anything out of my life. I’m in a gang.” No, man! They’re walking around saying, “Man, I’m in a gang. I’m with them.”

My colleague Rich Penny (@Rich_Penny) has written his PhD on the subject of Rawls, justice and self-respect: “the process of securing the social basis of self-respect for citizens is much less a matter of ‘traditional’ distribution, and much more a case of cultivating a particular set of social relations and a public political culture that serves to secure for all citizens the space, recognition and encouragement needed for them to develop a sense of self-respect on their own terms.”

Poverty is about so much more than money.

Posted in Uncategorized | Comments Off on A sense of dignity

Back in the computer lab – questions answered

I signed up to teach again this semester.  I am demonstrating on a rather nice course covering social science data and methods using Stata (STAT6077, for those in Southampton: it was the course that made me realise I was doing the right MSc).  Today was computer lab number one.  These are the questions I answered:

Q: What does the 25th income percentile mean? A: It’s the level of income below which one quarter of the incomes are found).

Q: Which of these equivalized incomes gives the right answer?  A: None of them. Annoying isn’t it? For the uninitiated, income data is often collected at a household level.  Analysis, meanwhile, frequently takes place at an individual level.  To bridge the gap, you have some choices.  You can pretend that one person in the household gets to control the whole income (i.e. pretend that individual income equals household income). Or you can pretend that household income is shared evenly among all household members.  Or you can assign 20% of the income to the kids and divide the rest evenly between the adults.  Or you can do as the OECD does and treat individual income as household income divided by the square root of household size.  I think this might be the time (Monday of Week One) to quote George Box: “Essentially, all models are wrong, but some are useful”.

Q: How do I access the University filestore? A: It’s on the H: drive. If you want to, for example, keep a log of your Stata workshop session, you can tell Stata:

log using h:\workshop.log, t

This will open a log file for your session and save it in your personal filestore (don’t forget to close your log at the end of the session).  You can access your filestore from any computer in the university and also via a VPN connection at home.

Q: Why are there any negative household incomes in the Family Resources Survey?  A: Negative incomes are usually discounted for analysis purposes (thank you Dr Google!).  The variable is constructed using all income, minus council tax, pension contributions, some particular insurance payments, student loan repayments and child maintenance.  There are 36 individuals (out of more than 52,000) in 17 households with a negative income.  And 19 of those were children.  Must be pretty grim.

Q: How does this .do file work?  A: It’s magic. And you can keep a log of everything you do too. In fact, you probably should.  All the details were in the introductory session on Stata in Week 0 and are available on Yves Berger’s website.

Q: Can I use Stata at home?  A: Sadly not. University of Southampton only has on-site licenses. You can, however, buy a student license from Timberlake on their GradPlan.  You’ll end up buying Stata 13 when the computer labs are running Stata 12, though…  And it will cost a minimum of £120.

Q: What is this coefficent of variation, then?  A: Instead of answering this question, allow me to introduce you to the statistical equivalent of Wikipedia.  It’s great, up-to-date, and (unlike Wikipedia) completely acceptable as a reference/citation.  It’s based at UCLA:


Posted in Pedagogy | Tagged | Comments Off on Back in the computer lab – questions answered


I spoke yesterday at the Voluntary Sector and Volunteering Research Conference about the first paper of my PhD. I finally answered the question I asked two years ago in this post: For those who are interested, the answer is: the Raising of the School Leaving Age. Those extra 16 year old volunteers were essentially timetabled into it. More soon.

Posted in Uncategorized | Comments Off on Mystery

Another advantage of longitudinal data

I have been having trouble formulating an important argument in one of my PhD papers, so I’m going to rehearse it here.  If anyone could help me to refine it, I’d be eternally grateful if you’d let me know…

Longitudinal data is good stuff if you want to try to understand a cause/effect relationship.  For example, causes generally precede effects, and having longitudinal data allows you to check whether your hypothesised relationship runs in the right direction through time.  I am interested in whether volunteering ’causes’ political activity.  I have a dataset (the NCDS) which allows me to check whether volunteering precedes political activity (like voting, signing petitions and attending public meetings) in time.  It does.  So far, so good.

There are still problems, though. In survey terms, volunteering is bound to precede political activity.  The cohort is asked about volunteering for the first time at age 16 and political activity for the first time at age 23.  Therefore, if volunteering and political activity are related at all, it will look as if volunteering is the cause and political activity the effect, even if that’s not really the case.  For example, volunteering and political activity could be jointly caused.  This is incredibly plausible.  Volunteers are generally well-educated and middle-class: political acts are also linked to education and class.  Furthermore, one could argue that there is a ‘civic type’: people with a particular personality or socialisation which predisposes them to volunteer and vote.  Causal inference, therefore, continues to elude me.

There is hope, however.  Longitudinal data like this allows the user to account, to a certain extent, for the effects of character, intelligence and socialisation.  If a person’s volunteering is driven by their character, their innate intelligence, or their familial socialisation, those effects are likely to be evident at any given age.  That is, if there is a ‘volunteering type’ we would expect to see those people over-represented among the active volunteers in the NCDS at age 16 and age 23 (and indeed at 50, when these questions are asked again).  Instead what we see is that volunteers at age 16 are less likely to go on to get an education, less likely to go on to vote and more likely to be from families which are supported by an unskilled worker.  The reasons for THAT are a whole other blog post, by the way.

This pattern in the data makes me all the more confident about my findings.  Volunteers at age 16 are 40 per cent more likely to volunteer at age 50 than n0n-volunteers (even controlling for an array of potential confounders).  This is true even though the 16 year old volunteers have quite different demographic and socio-economic characteristics to the 50 year old volunteers. When I look at the data and see that the relationship persists even though adolescent volunteering in the NCDS looks so unusual, I feel more confident that I may be looking at a causal relationship. Furthermore, when I perform logistic regression using volunteering at age 16 to predict volunteering at age 50, I am controlling not only for volunteering at age 16, but implicitly for all the things which ’caused’ that adolescent volunteering. Therefore, I have controlled for being a ‘volunteering type’, for socialisation by the family, for innate intelligence, and for a whole host of other things which the literature has linked to volunteering. I remain within the shadow of doubt for sure, but I feel a lot more confident than I would have done without the longitudinal data.

And that’s it. Criticism welcome. I am not attempting formal, statistical causal inference, by the way. This is more ‘weight of evidence’. I hope.

Posted in PhD | Tagged , , , | 5 Comments

Survey Data Not Be All And End All – Shock

Apparently there’s more to research than survey data – who knew?

Those who have been paying attention will remember that I have used the National Child Development Study (NCDS) to examine the relationship between volunteering and political activity for the first paper of my PhD.  It turns out that, as well as 50 years of fascinating survey data on a cohort of people born in a single week in 1958, the NCDS also includes the transcripts of 220 semi-structured interviews (each lasting around an hour and a half) given when the cohort members were aged 50.  This collection of 220 interviews is known as the Social Participation and Identity Study (SPIS).

I’ve just taken a short break from my PhD research to do a collaborative (ESRC-funded) research project using this lovely stuff.  I collaborated with Jane Parry (Southampton) and Katherine Brookfield (soon to arrive at Edinburgh) on a mixed-methods report looking at a sample of 50 of the semi-structured interviews (called “50 at 50” – geddit?).  Our full report will be available shortly at   It focuses on participation: in clubs, associations, voluntary organisations etc.  It covers the differences between participation data collected in surveys (quantitative) and in open-ended interviews (qualitative).  We also looked at people’s motivations for involvement and the barriers to participation that they face.  Finally, we included a chapter on participating in a cohort study (actually, more accurately, I should say that Katherine and Jane did that – I stuck mostly to the numbers and to formal social and civic participation).

For now, I’ll stick to the differences between quantitative and qualitative data when it comes to studying participation.

We used NCDS quantitative data to make a ‘sample’ of 50 out of the SPIS 220.  The quantitative data covers the whole of the cohort members’ 50 years, but we used only data from the five adult waves (cohort members were surveyed at age 23, 33, 42, 46 and 50).  We singled out:

  • non-participants,
  • perennial participants, and
  • frequent participants.

Non-participants answered ‘no’ to a slew of participation questions at every wave – they didn’t even go to discos when they were 23!  Perennial participants answered ‘yes’ to ONE question per wave about active involvement with a formal/membership organisation.  Frequent participants were active weekly in either (a) many spheres at once, or (b) voluntary work

Looking only at the quantitative survey data, perennial participants and non-participants are indistinguishable in terms of gender, social class, education, marital status, childrearing and care for elders.  Frequent participants (nb: only 8 of them in our sample) look noticeably better educated than their less-involved peers. They were also:

  • healthier
  • less engaged in caring for elderly parents
  • more likely to vote
  • more likely to be self-employed.

For those familiar with the literature, this should be raising some eyebrows.  We would generally expect volunteers, for example, to be better educated, and from the professional/managerial social classes.  We can certainly see some of those differences among the frequent participants, but not among the perennials.  The take-home, however, in spite of these small differences, is that it’s pretty hard to pick someone’s likely participation status from the rest of their NCDS data.

Looking at this from other side (the dark side?), the qualitative data shows that, as you might expect, frequent participants really are very busy people.  John, one of the frequent participants*, launches into a description of the sports coaching activities he and his wife engage in with the words: “My wife gets up at just after three.”  Frankly, just reading the interview is exhausting.  While both data sources generally agree for the frequent participants, the qualitative interviews show that the majority of non-participants are more engaged that their quantitative data suggests.  Out of 21 non-participants, just seven failed to come up with any participatory activity at all in their qualitative interviews.  Furthermore, some perennial participants do really very little to ‘earn’ that tag.  Some perennials had just attended church services, or played golf at a club – and this put them in the same category as others with much more active records.  Finally, some types of participation were all but absent from the qualitative interviews: political acts and trade union activity were far more prominent in the quantitative than the qualitative data.  (By the way, this last comes under the general heading of ‘if you don’t ask, you don’t get’ – there was little chance that cohort members would be prompted to describe their trade unionism or political acts, given the interview schedule.)

Survey timing clearly played a role here.  Some of the difference between quantitative and qualitative data is down to “events, dear boy, events”.  The gap between quantitative and qualitative data collection was anywhere between one month and two years!  Alison, for example, referred to a changing economic backdrop:  “I think probably we’ve felt the credit crunch crunching over the last few months, so we probably do stay at home a lot more”.  Iona had been involved in various activities and organisations at the time of the NCDS quantitative interview.  However, between then and the SPIS qualitative interview, she was diagnosed with cancer and had started a programme of treatment.  As a result, her participation and volunteering commitments had been suspended (“since September I’ve had somebody else’s life really”).  In that case, the time gap was only three months, but it was very significant for the findings.  The quantitative data identified Fiona as a frequent participant on the basis of her participation in multiple organisations at age 50. However, the qualitative interview data indicated that she rarely participated in any organisations. Highlighting the role of timing, in her SPIS interview Fiona mentioned that work commitments, and the demands of studying part time for a new qualification, had, in the past two years, forced a reduction in her participation activities.

There is also a strong question effect.  The NCDS participation questions are generally predicated on ‘joining’ or membership.  Before cohort members have a chance to tick a box describing their activities, they must first pass a ‘membership’ hurdle.  The survey asks first “Are you a member…?” and only then “How often do you join in the activity of…?”.  In the case of religious activity, this is visible in the quantitative data alone.  Three cohort members (Iain, Susan and Anwen) said that they attended weekly services but were not members of a religious organisation.  When quantitative and qualitative data are compared, it’s clear that ‘membership’ is perceived differently by different cohort members.  For example, Michael (a perennial participant, according to the quantitative data) appeared to identify online contact with several clubs and receiving email newsletters, both rather passive forms of engagement, as ‘joining in’ while Paul (a non-participant) did not identify his membership and subsequent attendance at National Trust properties as ‘joining in’.  The membership hurdle also rules out participatory activities like evening classes and episodic volunteering.  For example, Janet (a quantitative non-participant) was involved informally with a children’s charity through her husband.  She says:

“My husband is a taxi driver and there’s a–, the only volun–, the only charity work that I do is help–, he takes part in a–, it’s a one day kiddies’ outing to PLACE IN SCOTLAND… so we’ve got to blow up the balloons and get everything ready but it’s–, it’s a couple of weeks of preparation, getting everything in, then you’re up at four, five in the morning to put this stuff onto the taxi and things like–, so that’s–, that’s about it.”  The interviewer then asks: “Have you been doing that for quite a while?”  She replies: “Oh, 20-odd years now…  It’s–, it’s maybe not a lot but it’s a lot to me [coughs] ‘cause we pay for it all ourselves as well.”

This certainly sounds like volunteering – but Janet’s quantitative NCDS record suggests no such activity.  The survey questions did not capture her episodic, family-linked volunteering (perhaps partly because she seemed reluctant even to use that word).

Including religious activity as a form of participation has also affected the findings. Some religious participants are very active: running choirs, or Sunday schools, or involved in church administration.  Others just go to the service.  Iain, for example, said: It’s a sanctuary type thing and really, that’s all it is. If somebody were to say to me, “What did the priest say last week?” I couldn’t tell you what he says. I listen in wee bits, but the majority of the time it’s just me and that’s what I get out of chapel.“  This is about as non-participatory as participation gets!  Louise, on the other hand, revealed in her qualitative interview that she was spending 1.5 to 2 days a week on her church-related activity.  Depending on how the cohort member views their own activity, these people can look pretty much identical in the quantitative data.

So the message here…?

  • When you use a survey question, you rely heavily both on question-order and on the respondent’s interpretation.
  • Unless you are interested purely in church-going, it may be better to ask specific questions about church-related participation.
  • If you don’t ask, you don’t get.

And finally – don’t for a moment think that this has put me off using survey data; it’s just made me determined to use it better.


*All names are pseudonyms.

Posted in Methods, PhD | Tagged , , , , , , , | 2 Comments

A civic recovery? Or a graph of er… something-or-other.

Last year, I bought a copy of Political Participation in Britain (Paul Whiteley, 2012).  On page 86 there is a graph (bear with me…) of responses to one of the questions from the British Election Study Continuous Monitoring Survey.  This study asks 1000 people a month their views on a variety of political issues.  This particular question reads: “Over the past few years, have you volunteered to get involved in politics or community affairs?”  Whiteley describes “a slow but noticeable decline” from April 2004 to December 2010.  He writes: “It is no coincidence that this was the beginning of the worst financial crisis and recession in eighty years, suggesting that the state of the economy has an influence on social capital”.  At the anniversary of the London Olympics, I wondered whether the The Games could have made a difference to how involved people get with their communities, so I went back to the data to bring the graph up to date.

Before we go too much further into unpacking some of this, I’ll show you my version of Whiteley’s graph.  He uses monthly data, while I collapse mine to quarters; his stops at the end of 2010, whereas I now have the data up until the end of Q1 2013.  I also annotated mine with pretty red arrows.

04Q2_to_13Q1_civics_BES_CMS_annotatedThere are quite a few things here that need more thought.  First up, I don’t see an obvious Olympic dividend.  The pick-up in civic activity began well before the Olympics.

Secondly, Whiteley claimed that the data he used (up to Q4 2010) showed a “noticeable” decline.  Looking at it now, with the 2011 and 2012 data, I’m not at all sure that’s the case.  Hell, even if you put your hand up to the screen to cover the later data, I’m not sure you’d really describe the decline as particularly “noticeable”.

Thirdly, look at the question on which the graph is based: “Over the past few years, have you volunteered to get involved in politics or community affairs?”  What’s this really asking? “Over the past few years” is such a woolly timeframe that this is more of an attitudinal question than an activity question.  My best guess is that the question stands for ‘Would you get involved…?’ more than it stands for ‘Did you get involved…?’.

So what about the last segment of the graph?  It looks as though people got more enthusiastic about getting involved in politics and community affairs sometime shortly after the last General Election.  As far as I’m concerned the jury’s still out, but it certainly seems to suggest a political driver, rather than Whiteley’s economic driver, for the change in responses to this question.

I wonder (and I’m whispering very quietly here), could the Big Society have anything to do with this…?


EDITED 26/7/13: After a tweet from @karlwilding, I plotted the confidence intervals for the means in the graph above.  This one was done in Excel and has ended up looking less spiky, but it’s the same data.


The eye is now drawn much more to the dips than to the peaks: if there’s anything here to explain at all, it now looks as though it’s in the 2010 and 2004 troughs.

Posted in Academic research | Tagged , , , , | 2 Comments