Computers, creativity and learning

Athena Irony

Last week was steeped in irony. I attended an Athena Swan self-assessment team meeting my university. Athena Swan is a scheme by which universities get accredited for not being rubbish about gender (and other sorts) of equality. Currently my university doesn’t have accreditation; it lost it in mysterious circumstances some years ago. Or at least, the circumstances are mysterious to me as they were not discussed at this meeting. Athena provide a giant checklist of practices which universities should try to follow to get themselves out of their mire of favouring white middle aged blokes (WMABs). They all seem perfectly sensible to me but let’s just say they may seem exotic – luxurious – to university senior management who after all are predominantly WMABs (and not just at my university).

The first serving of irony was the meeting itself. It was chaired by a WMAB,  and to an extent dominated by another WMAB who gave the impression of being somewhat dismissive of the scale of the problem.  There were women at the meeting. A lot of the preparatory work had been done by a well-informed woman. There was a female former member of Athena Swan panels there. So why the bias towards WMAB even in the meetings about gender equality? I have a suspicion, which might be completely unfair of course: the committee had to be taken seriously by the heads of academics schools so it was decided to have it chaired by an academic head of school to give it credibility. And due to the very problem which Athena Swan exists to solve, the heads of school in science and technology are WMAB. This is problematic in my view. There is a lot to be said for understanding stemming from lived experience, and clearly being a bloke academic prevents you from having experience of being a woman academic.  (Mostly. Apart from that psychologist who had a sex change and found that people took her a lot less seriously than her “brother”.) At any rate, I have decided not to continue as a committee member mostly because I think it is too important to be done “off the side of my desk”. The whole point of the accreditation process is to give serious time and resource to gender equality issues and I couldn’t realistically see it being factored into my work load. Further, I am increasingly taking the view that just because I could do something doesn’t mean I should do it. There are, after all, other women in my department who are working mothers and who would contribute to such an activity.

The second portion of irony was served as I desperately tried to do my share of an EPSRC Culture and Communities network proposal on Communities of Motherhood. This was in a week of potty training angst (my angst, toddler’s potty). I was attempting to appear like a normal sane mother to grandparents while costing a proposal during nap times and after bed time. My collaborators on this (and other proposals) all had way too much to do as well. I wanted to help earlier career researchers to get this (tiny amount of) funding in these times of frugal funding. (Don’t get me started on the futility of funding research networks rather than research.) It seemed like a lot of effort for not very much reward. And yet, academia is structured in such a way that people routinely put in mega efforts to achieve yet more seemingly impossible things – meet this funding proposal deadline, write just another cheeky wee paper. Because there are core teaching activities which simply must get done within working hours, research sometimes becomes a strange hobby. I have a colleague who saves his paper writing for Friday evenings.  And the problem with this for gender equality is that working mothers don’t have extra hobby time to spend on the research which advances their careers. There just isn’t any slack time in the week for anything extra.

So, what could universities (or funding councils) do to improve gender equality? Here are my thoughts:

• Stop having stupid sand pits where people get walled up in a country house for 5 days at a time, in a terrible Lord of the Flies scenario. This makes it hard for people with family responsibilities to attend. It is costly in terms of childcare, puts strain on other family members to manage the child care in one parent’s absence and is particularly hard for breast feeding women.
• Don’t spring funding calls with short deadlines on people, particularly not during the school holidays.
• Make targets and expectations clearer and more feasible. Academics seem to carry a burden of guilt that they should have more grant funding or write more papers. It would help to know when one has done enough.
• Introduce careers paths where people can succeed through high quality work which can be done without extensive foreign travel. For example, this might mean weighting journal articles more highly than conference articles, and considering reviewing duties rather than key notes as indicators of esteem. Or not expecting EU grant funding (which often requires travel).
• Make an effort to put women on interview panels for promoted posts, even if this means asking women from outwith the university to serve on the panel.

Research excellence and the gender gap

Monday was the "the point in the year when women in effect start working for nothing because of the gender pay gap – for every £100 men take home, women are getting £85 – a 14.9 per cent difference."

Doesn't that make you happy to be alive?  That makes me feel great about all those essays I marked this week. To cheer me up, I can look forward to a lifetime pay gap of £423,000, if I make it to a managerial position. My university is feeling very pleased with itself because it now has gender equality in the strategic plan. Huzzah! That'll make all the difference then.

One of my reasons for starting this rant is that I just filled in the form to apply for reduction in my expected REF outputs. You may recall me blogging about this before when it appeared that those who took a period of maternity leave would be expected to produce the same number of publications over the 4 year period as those who did not. Luckily the REF people saw sense and decided not to pursue this policy and now people who have been off work get a reduction in the number of papers they were expected to write.

What made me snort (loudly) was there was also a box to fill in asking you to record any other factors which might affect your work (like breastfeeding or maternity). Sleep deprivation? General distractedness caused by raising a small person? Babies don't just go away once they are born. Turns out they grow into toddlers who also need looking after. Apparently this state of affairs continues for some time. Like 18 years or so.

What bothers me about this is that research careers (and in the UK, REF is one of the main ways of measuring research success) are built on hours of dedicated work above and beyond contracted hours. If you have small children (or indeed are caring for other family members) you don't have any extra hours. None. (If you want a further depressing read, here's an article on the impact of child rearing on female academics.)

What I want to know is this: why do academics feel compelled to work so hard? I can understand the lure of data analysis and the pursuit of new knowledge That's exciting work, worth doing and worth spending spare time on it. But applying for more and more funding for projects you don't really want and writing more and more obscure papers? That's make-work. I refuse to spend my very precious working hours on it.

 

Better

I have been rereading Atul Gawadne's book "Better", mostly because I have been thinking about how we can improve teaching in my department since I became a course director. Here's an inspiring quote. Betterment is: "to live a life of responsibility. The question... is not whether one accepts the responsibility. Just by doing this work, one has. The question is, having accepted the responsbility, how does one do such work well?"p9. Gawadne suggests there are three aspects to making things better: diligence ("giving sufficient attention to detail to avoid error and prevail against obstacles"), doing right, and ingenuity (thinking anew). He also has some suggestions for how you can become a positive deviant (i.e. striving to make things better even if the people around you don't). 

1. Ask an unscripted question (when talking to people)

2. Don't complain

3. Count something (i.e. monitor something which interests you numerically)

4. Write something (to reflect on what you have counted)

5. Change

This is me doing number 4 - writing!

I find the quote above striking because even although it was written by a doctor about medical practice, it applies nicely to teaching. I believe that as lecturers, we have signed up to educate and that we can always do better. Even if you get great course feedback and high averages and win teaching awards, there will always be something which you can improve on. Having spent 6 months being a course director and talking to both staff and students about teaching, I am beginning to see that not everyone sees things that way. I think that maybe some staff see the pursuit of betterment only as a criticism, not something which is just built into the job.Maybe this is a "if it ain't broke, don't fix it" attitude. But my point of view is that you have to check whether it is broken regularly, and try to keep it in good working order! It's part of the cycle of reflective practice, planning, teaching, reflecting, then replanning. Trying new teaching methods or content from time to time will make things more interesting for you, and might well make things even better for the students.

In being positively deviant, I am also doing numbers 3. and 5. from Gawadne's list. The counting is to do with the dreaded national student survey results - logging students' attitudes towards their teaching and seeing if in the long term we can increase scores. I am not convinced this is an end in itself, but it is a benchmark for us to use to see whether our changes are actually making things better. Number 5 - change- is hard, it turns out. Universities tend to be conservative beurocracies which work on long time scales. It is hard to achieve anything within one academic year simply because of when meetings at which approval decisions can be taken are scheduled. Staff and students are both busy and busy people tend not to want to make changes which will cost them time. It is hard to coax people out of the habit of complaining and into the more positive frame of mind where they will make constructive suggestions.

But I am trying, and I will keep trying.

We needn't be malevolent grumps in 2012

[cross posted from my CACM blog]

A few months back, Bertrand Meyer wrote about the nastiness problem in computer science, questioning whether we as reviewers are “malevolent grumps”. Judging by the user comments on the page, this hit a nerve with readers who were the victims of such grumpiness!  Jeanette Wing then followed up on this with some numbers from NSF grant rejections which did indeed indicate that computer scientists are hyper-critical. Much as I enjoy the colourful phrasing, I feel that a field full of malevolent grumps is not something we should simply accept. In fact, even if there are only a few grumps out there, it’s in all our interests to civilise them.

So what can computer scientists do to reduce the nastiness problem when reviewing? Reviewers, authors, programme committee members, conference chairs and journal editors can all do their bit by simply refusing to tolerate discourtesy. Let’s embrace the “no asshole” rule: we no longer ignore bad behaviour. As reviewers, we can aim to be polite (yet stringent) ourselves but also to point out to co-reviewers if we find their impoliteness unacceptable. As authors, we don’t have to accept a rude review and just lie down to lick our wounds. We can (politely!) raise the issue of rudeness with the programme chair or editor so it is less likely to occur in the future. As editors, chairs and programme committee members we can include the issue of courtesy in the reviewing guidelines and be firm about requesting reviewers to moderate their tone if we notice inappropriate remarks.

One of the first steps is to separate intellectual rigour from discourtesy. It is possible to be critical without being rude or dismissive.  We can maintain standards in the field without resorting to ill-natured comments. (Believe it or not, it is also possible to ask genuine questions at a conference without seeking to show off one’s own intellectual chops, but that is another matter). The purpose of reviewing, in my view, is to help an author improve their work, not to crush them under the weight of your own cleverness. It’s not the author’s fault that you had a bad day, or that some other reviewer just rejected your own paper. 

Of course, there are some pockets of good reviewing practice within the field which we can draw on. I am sure there are many, but I have chosen CHI because I have been writing for it recently. The CHI conference is one of the biggest well respected annual human computer interaction conferences. Last year there were 2000 attendees from 38 countries. This year there were 1577 paper submissions with a 23% acceptance rate.  This was the first year I submitted papers to it, and I have been impressed by the quality of the reviews in terms of their fairness, constructiveness and level of detail. They contained greater insight and intellectual oomph than the reviews I had from a high impact journal recently. For one of my CHI submissions, the reviewers did not agree with the paper on some points – it is on a controversial topic- but they still offered suggestions for how to resolve these issues rather than simply rejecting the paper. Was I just lucky in the reviewers I was allocated? Possibly, but the CHI reviewing process has some interesting features built in to maintain review quality*.

1. In the guidelines for reviewers, courtesy is explicitly mentioned: “please be polite to authors. Even if you rate a paper poorly, you can critique it in a positive voice. As part of polite reviewing practice, you should always state what is good about a paper first, followed by your criticisms. If possible, you should offer suggestions for improvement along with your criticism.” 

2. Authors can select both the sub-committee and the contribution type for a paper, which maximises the chance that the paper will end up with reviewers with appropriate expertise , and that the reviewers will use criteria appropriate to the paper when assessing its suitability (e.g. not insisting on empirical evidence for a theoretical contribution).
3.  The reviewing process is thorough and has several opportunities for unfairness or discourtesy to be weeded out. Each paper is blind reviewed by three or more experts, and then an associate chair writes a meta-review to summarise the assessment of the paper, and what action (if any) should be taken to improve it. In this way, individual grumpiness is moderated. A variant of this good practice from other conferences is when reviewers of the same paper can see each other’s reviews (once they have submitted their own), thus introducing peer pressure not to be awful.
4.  Authors have a right to reply by writing a rebuttal of the review. The rebuttal is taken into account along with a revised meta-review (and potentially revised individual reviews) at a two day committee meeting when final accept/reject decisions are made.
5. All submitting authors are surveyed about their opinions of the reviewing process – yet another chance to raise issues about unfairness or discourtesy which have not been addressed in a rebuttal.
6. This point is more about the nature of the conference itself, rather than the reviewing procedures. Because CHI is so interdisciplinary, participants have a wide range of backgrounds from art and design to hard core engineering. They are therefore exposed to – and may in fact seek out- different perspectives which may make them open to different paradigms as reviewers. Could colleagues from the arts and social sciences be having a civilising influence on the grumpy computer scientists?

Ts This is a fairly heavy weight process, but if conference organisers adopted even just one more of the practices from points 1-5, or if journal editors added a courtesy clause to their review instructions the world would be a slightly better place. Make it your New Year's resolution to stop tolerating malevolvent grumps in 2012!

 

*Thanks to Tom Erickson – the person who runs the CHI author survey – for kindly raising some of these points.

 

Celebrate Open Access Week with ACM’s Author-Izer tool

[cross posted from my ACM blog]

Happy Open Access Week! If you would like your work to reach the widest possible audience, Open Access week encourages you to take steps to make this happen. Think of all those unfortunate readers out there who would just love to read your latest article but can’t because it is hidden behind a pay wall. Luckily, the nice people at the ACM have developed a new tool called Author-Izer which can help you reach these readers by allowing them to download your ACM published articles for free via a link from your own web page.

This is a great step forward from the ACM in the support of the democratisation of academic knowledge. I’ve been following the arguments for open access publishing for a while. The arguments go like this: academics invest their time in research and in writing up their findings for conference and journal articles. Conferences and journals do not pay these authors for their contributions. Reviewers and editors volunteer their time to ensure the high quality of the publications. Publishers then charge readers a small fortune for the articles which they got for free, either through institutional library subscriptions or individual charges per article online. This means that potential readers of articles can be barred from accessing the articles they need, for example because they belong to a small institution which does not subscribe to a particular journal, or because they are not professional researchers and so don’t have access to institutional resources of this sort. Yet, for science to progress, researchers need access to the latest findings. And if science is to take the rest of the world with it, members of the public should also have access to articles which relate to the issues in their lives.

A lot of academics attempt to get round the problem by publishing early versions of their articles on their personal web pages for readers to download. The problem with that is that the final version which appears in the published journal may contain important corrections which your readers will miss. Further, if a reader downloads articles from your web page, this download is not included in bibliometric download data which the publisher collects so your true brilliance is not quantified for those who care about such things! Author-Izer takes care of both of these issues. Here’s how to set it up. First, get your free ACM web account, and then go the ACM digital library. From there you can edit your author profile page. Once you get an email notification that the author profile has been updated you will find that the ACM Digital Library shows an Author-Izer option under every article by you which was published by the ACM. When you click on it, you can authorise the article to appear the url of your personal publication web page. It evens generates the code for you to cut and paste into your webpage so it appears with the smart little ACM logo and – glory of glory- the bibliometric information about how many times it has been downloaded and cited. That last touch appealed to my academic vanity no end. There you are – next time a reader visits your web page, they will be redirected for free to an ACM archive of your article.  They will be humbly pleased to read your great work, and their click will be added to your download statistics. And the cause of science will have advanced, one article view at a time.

Gender discrimination in the UK Research Excellence Framework

[cross posted from my CACM blog]

All over the UK, academics are frantically writing papers. Yes, I know academics everywhere are frantically writing papers – there are after all over a million papers published per year. But UK academics are writing now to publish in high impact journals in time for the 2014 Research Excellence Framework (REF), the result of which determines how government block funding for research is allocated. We are meant to publish 4 world beating Turing-award level papers between 2008 and 2014 to bring in the maximum funding for our cash strapped departments. However, this is not a post about the problems of channelling researchers’ efforts into stalking their own bibliometrics on Publish or Perish instead of doing something productive. This is a post about how the REF discriminates against female researchers. It's an issue which could prevent the recruitment of more female computer science academics (or encourage existing ones to leave) at a time when as a discipline we should be trying to address the gender imbalance.

You might expect that female researchers who have been on maternity leave during the REF period would be expected to produce a number of papers proportional to their time spent at work in that period. No. Instead, the draft REF guidelines indicate that women will be expected to produce one paper fewer only if they have taken more than 14 months off for maternity during the REF period. Because of the way statutory maternity leave works in the UK, it is common to take between 6 months and a year off. So to qualify to produce one less journal paper output, you’d probably need to produce more than two human outputs (babies!) over 5 years  (See http://www.swipuk.org/notices/02-09-11/ for a good summary).  An alternative proposal is to reduce the REF output by one paper for each child. I do find it bizarre - almost comical- to equate the effort of carrying, delivering and raising a child to researching and writing a paper but at least this proposal acknowledges that children have an impact on their carers’ careers.

This proposal, if carried out, would further divide academia along gender lines. It’s already a fairly bleak situation. In Europe, only 18% of professors are female, only 9% of universities have a female head , and there is a pay gap of 20% in the US.  An interesting article in the Times Higher from last year cites studies which show that women’s publication output falls after childbirth, although it does bounce back later. Further, in countries which invest in high quality childcare, female researchers are more productive. So it is not inevitable that women who have had children must be less effective researchers in the long run. It is however probable that their productivity will drop in the period when they take maternity leave.  It would be strange indeed if being off work for 6 months to a year didn't affect one's output.

What about computer science in particular? A report from the main research council which funds computer science in the UK (EPSRC) shows that only 13% of grants are awarded to female researchers. Of course, the prospects in the computing industry outside academia are not good either. In the US, an NCWIT report states that there is a gender pay gap in CS jobs  starting at 7% and increasing to 11% for workers with more than 15 years experience. Also worrying is the trend reported in the same source that two years after graduation from CS degrees, only one third of women remain in science and technology jobs. 

The odds are stacked against women succeeding in academia, and in computer science  as it is. The discrimination built into the REF would penalise women who have had children, increasing their stress at work and reducing their chances of promotion. The good news is that the REF document is still under consultation. There is a chance to change the way the REF is conducted by writing comments here before October 5^th. Please do this for your sister computer scientists!

 

New students' hopes and fears

Today I inducted 66 new students into our department for Computer Science and Information Systems degrees, having the previous day been part of the exam board which graduated a seemingly endless stream of MSc students. Such is the cycle of university life. 

I like to get the students to write down their hopes and fears for starting uni so we can discuss a few of them with staff from the student advice centre. I actually find it quite touching. There is something so evocative about seeing a scribbled note on a bit of paper: "Will people like me?" Bless you, who ever wrote that. Of course they will.

Here is a selection of representative comments.

Looking forward to:

Learning from experts in their subject

More knowledge on computer science and using it in real life

I am looking forward to study great and interesting things

Learning new programming languages and meeting like minded people

Using all 6 cores!

Find out at what I’m really good

I look forward to graduating with high marks and living and working in Scotland

Meeting new people

Worried about:

I am worried about money (loads of people said this!)

Debt at the end of the course

Will people like me?

Being away from family and friends

Cooking/ looking after myself

Not making many new friends

Leaving home and having only myself to blame for what I do (See Nick, people were listening!)

Not being able to keep up with the class

Failing

High work load

Exams

Full time job + full time university

English language

 

 

 

Checklist before submitting a journal paper

This is a note to myself really, about all the things I should do before submitting a journal paper., based on what I have read this summer, and mistakes I have made recently. Happily (for my self esteem, not the quality of papers in the field) I know I am not alone, judging by  the papers I review.

So maybe you will find them useful. This of course is an exercise in procrastination as I have stack of dissertations to read which I can't quite face.

• If you're doing experimental work have you thought up a meaningful hypothesis which says more than just "there will be a difference between the groups?" (See Diennes 2008 for why) Have you thought about what sort of effect size you expect based on previously reported effect sizes?Necessary if you're doing Bayesian analysis but useful any to avoid lame hypotheses.You should be doing this well before the writing phase, Judy, you should have done it months ago before the experiment started!
• Again for experimental work,have you done power calculations? No, Judy, thought not. And again, you should before you plan the experiment. See Cohen (1992) for some shocking estimates of required sample sizes.
• Has anyone written about this recently? Go check. Yes, I know it's obvious but I also know I nearly forgot to do it just yesterday.
• Have you read recent papers in the target journal to make sure they are interested in this stuff?
• If you're doing any sort of analysis that requires judgement, have you done inter-rater reliability checking and stats? The last section of Lombard et al (2002) has a good checklist.
• If you're doing inferential stats have you reported your descriptive statistics to avoid irritating future researchers who might want to include your study in a meta-analysis?
• If you're doing t-test or ANOVA, have you checked the assumptions (normality (see http://www.statsguides.bham.ac.uk/HTG/HTGtextonly/ot/normality.htm) and homogenity of variance? (Levene's test))
• If you're doing inferential stats, have you calculated and reported effect size? See Sun et al (2010) for a practical checklist (last section)

• Have you proof read it? Have you got someone else to proof read it? 
• Have you downloaded the review forms used by the journal and assessed your own paper? (Arrggg.. no.. better do that now!)

 

If you can think of any to grow my collection,  please add a comment!

Cohen, J. (1992). A power primer. Psychological bulletin, 112(1), 155. American Psychological Association. Retrieved from http://psycnet.apa.org/journals/bul/112/1/155/

Dienes, Z. (2008). Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference (1st ed., p. 150). Palgrave Macmillan.

Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human communication research, 28(4), 587–604. Wiley Online Library. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1468-2958.2002.tb00826.x/abstract

 

Sun, S., Pan, W., & Wang, L. L. (2010). A comprehensive review of effect size reporting and interpreting practices in academic journals in education and psychology. Journal of Educational Psychology, 102(4), 989-1004. doi:10.1037/a0019507

Visiting the Queen

See me? See the Queen?* I went to her garden party on Monday. It was pure dead brilliant. This is me outside her house. Note my fascintator (the thing on my head is not actually a bizarre bee hive).

It was fun actually. I went with my lovely sister in law and we hung out in the Palace gardens drinking tea and discussing everyone's outfits. I also ate a considerable number of royal cakes, including one with a crown on it.

Thanks to EPSRC for nominating me. It was for helping the little children and teachers with computer games, and that.

* Forgive the Glasweigen syntax.

Lost calendar

I have lost my calendar in a freak iTunes accident. If you have an upcoming appointment with me, it would be a good idea to email me beforehand to remind me!