Computers, creativity and learning

How do you know your PhD results aren't just the Hawthorne effect?

An external examiner put the following question to my student  last week: "how do you know your results aren't just caused by the Hawthorne effect?" This is a perfectly reasonable question, but  I spent quite a lot of time trying to work out what a good answer would be. You could legitimately ask it of almost any study I have conducted when developing new technology for children.

The Hawthorne effect refers to the idea that observing people's behaviour changes that behaviour. It's a kind of observer effect, related to expectations, and a second cousin of the placebo effect. In the 1920s and 30s, a psychologist conducted a series of studies about the working conditions in a factory in a suburb of Chicago called Hawthorne. If he decreased the light levels, workers' performance increased. But if he decreased the light, their performance increased too. He concluded from this that the increases in performance were due to the workers being observed, rather than the physical changes to their work environment. There was a threshold, because performance would obviously drop if the light levels were so low that no-one could see to work. By analogy, the examiner wanted to know "how do you know that the young people in your study changed their career aspirations because of the educational robotics intervention they took part in? Could it be that they were simply inspired by the presence of an enthusiastic expert from a university who was interested in them?" 

The answer that it is entirely possible that the young people's interests changed as a result of interactions with an enthusiastic researcher. In fact, in this study of an existing educational robotics after school programme, a purpose of the programme was to connect young people with mentors from the technology industry to help them find jobs afterwards. Opportunities to work with enthusiastic technology experts was an active ingredient of the intervention. There might be a stronger effect for a mentor who is also a researcher from a university in another country but my guess is that wouldn't be very large. A mean examiner could pounce on such a wooly guesstimate by asking: "just how small would the effect be, and how do you know?". 

With the luxury of time and google scholar open in front of me, I can find an answer to that question. A 2014 meta-analysis of the Hawthorne effect reviewed 19 studies to find an overall odds ratio of 1.17 (95%CI: 1.06, 1.30) [1]. So one might expect an effect to the lower end of that confidence interval. Without getting too hung up on how to interpret an odds ratio (see [2] if you care), this is pretty small (similar to finding that Cohen's d < 0.2).  The meta-analysis concluded that there is evidence of some kind of observer effect, but the evidence is all over the place. The different studies included in the review had different ways of defining and measuring the effect, and we don't know much  about the exact mechanism by which is mean to work, and how it might vary in different circumstances. The authors of the meta-analysis recommend that some better theory is required to guide new experimental work in this area. Don't worry - it would be unreasonable of an examiner to expect an answer like that in the heat of the moment!

The examiner's question is about how a researcher would know whether the Hawthorne effect is present. This is rather hard to tell with this sort of qualitative study. The original Hawthorne studies were quite simple. The interventions which manipulate light were time bounded and reversible: higher light levels today impact your performance today but presumably not how you perform next week if you the light has gone back to normal. (Unless there is some kind of cumulative effect by which you perform best after working for 5 days in a row with optimum light levels).  So it is relatively easy to design a study to pick up quantitative evidence by looking at differences between treatment and control groups in a one off experimental setup which controls a single variable in this case by increasing and then decreasing it.  However, in education, it is harder to switch effects on and off in this way - it is more likely that an intervention would need to run for sometime before becoming effective, and then there would be a slow decay (or no change) once the intervention has finished. In a study about methods of teaching people to read, there is presumably not a slow decay once the intervention has finished because people don't typically forget how to read once they are secure with reading.In the technology career example, it would be extremely unlikely to change young peoples' career aspirations in a single afternoon, whereas classes over the course of a year in positive circumstances might gradually change persuade people to consider technology careers, until the classes stop and life circumstances gradually intervene. 

I suppose warning signs from qualitative which might indicate that the Hawthorne effect is at play could be the young people mentioning that the researcher personally inspired them, or that being in a study made them reconsider their lives. Or perhaps one could compare the outcomes for other young people who took part in the robot classes at another centre but who did not have exposure to the researcher. The problem there would be that variation would be introduced by other differences in this comparison.

It is hard to distinguish the Hawthorne effect from a set of related factors. It is quite common across many disciplines for the effects of positive pilot studies to diminish when they are rolled out at scale. Let's think about it within classroom interventions as an example. Suppose there was a classroom intervention which was originally delivered by a subject expert researcher, or by a teacher who was closely collaborating with such as researcher. When it comes to rolling it out across many classrooms, the researcher no longer delivers all the lessons. It's the class teachers who now do it, and they don't have the specialist knowledge of the researchers and in addition, each teacher has less support from the research team. There's more variability in delivering the intervention as intended because the teachers have different backgrounds and a million other things to do as well as this intervention. The expectancy effect is another factor here. There are studies which demonstrate that teachers' behaviour towards students varies according to their expectations about their students' abilities at the outset. It becomes a self fulfilling prophecy: those who are labelled as the brightest students will always seem bright to the teachers, and the lowest test performers will seem like lost causes.

In short, if an external examiner ever challenges you about the Hawthorne effect, here's what you could say. It is, of course, something I am aware of, but  a recent meta-analysis indicates  the concept is not well defined, and the effects are small. From my evidence, I saw no red flags which would suggest it was the case, but in future studies when the study is rolled out across more classrooms (or equivalent) I'll be alert for the effects becoming smaller or more variable as more practitioners are involved.

The story ended happily because my student now has a PhD.  And I have another tricky question up my sleeve for next time I have to examine a thesis.

 

[1] McCambridge, J., Witton, J., & Elbourne, D. R. (2014). Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. Journal of Clinical Epidemiology, 67(3), 267–277. https://doi.org/10.1016/j.jclinepi.2013.08.015

[2] Chen, H., Cohen, P., & Chen, S. (2010). How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics: Simulation and Computation, 39(4), 860–864. https://doi.org/10.1080/03610911003650383

Primary Teachers' Guide to Computational Thinking

Primary school teachers across Scotland will be getting a free printed copy of Teach Computing Science: A Guide for Early Years and Primary Practitioners this week. It's a guide to teaching the new Computing Education aspects of the Technologies curriculum in Curriculum for Excellence to help teachers who need an in depth introduction to the new expectations and outcomes.

The handbook was written by Kate Farrell, Judy Robertson, Quintin Cutts and Richard Connor. Education Scotland, Skills Development Scotland, SICSA Education and the Royal Society of Edinburgh all contributed to the cost of producing the handbook.

You can download a free digital copy here: http://teachcs.scot, as well as sign up for professional development courses.

Blogging again

I'm blogging for Communications of the ACM again after a break of four years! I got bored writing business cases and REF self assessments. Here is my first post about what children want to know about computers.

EqualBITE: recipe about Imposter Syndrome

For my academic colleagues: if you've ever felt a fraud in your job, or worried that people will find out that you're not really very clever, you're a) wrong and b) not alone. This recipe (from the EqualBITE book) is for you. 

Dealing with Imposter Syndrome

Judy Robertson

 

Background

Do you feel like a fraud? Do you secretly think that you know less than your colleagues and that it is only a matter of time before you get found out? If so, you may have fallen prey to Imposter Syndrome. And you’re not alone. Imposter syndrome, characterised by “persistent thoughts of intellectual phoniness” (Hutchins & Rainbolt, 2016), is common in high performers across a range of careers. Graduate students, early career researchers, women, and minority groups suffer from it. But you know what? So do professors and so do men (Hutchins & Rainbolt, 2016; Woolston, 2016). When you think about it, it’s not that surprising that so many academics are wracked with self-doubt. Academia is a high pressure environment with too many people competing for limited resources and where performance is measured against ridiculously high standards.  To be an academic – even a successful one- is to live with regular rejection from reviewers, journal editors, promotion committees and grant funding panels. The continual cycle of trying one’s best, being rejected in highly competitive circumstances and then mustering the strength to try once more is draining.

Imposter syndrome provokes anxiety and stress, and can lead people to give up their academic careers because they feel they are not good enough (Woolston, 2016). This is unfortunate because people in this position are often highly talented when considered objectively: they are their own worst critics. However, imposter syndrome can cause people to underperform despite their talent, for example by micromanaging their team, by delaying making decisions, by procrastinating, or by insisting that all tasks must be done perfectly rather than prioritising (Mount & Tardanico, 2016).

It is important for managers to recognise that different groups may experience imposter syndrome differently.  It is likely to be more common among staff vulnerable to stereotype threat (often women and minority groups) (Dasgupta, 2011) and women and men may employ different coping mechanisms. In particular, men may be more prone to maladaptive coping strategies which could lead to health problems (Hutchins & Rainbolt, 2016). Here are some ways we can overcome imposter syndrome by taking care of ourselves and our colleagues.

Ingredients

• Empathy (as a manager) and appreciation
• Self-awareness and introspection
• Accurate feedback on your performance
• A mentor or peer with whom you identify
• Understanding of imposter syndrome

Method

1. Look out for what triggers imposter syndrome for you so that you can avoid or prepare for such situations in the future. In a qualitative critical incident analysis of how academics are affected by imposter syndrome Hutchins and Rainbolt identify a series of events which trigger or revive persistent feelings of inadequacy:
   1. Questioning expertise. “Do I know enough? Do I belong here?” Questions triggered by conversations with colleagues who appear to be much more knowledgeable, or who are rude enough to tell you that you are wrong. In my experience there will also generally be a few top-of-the-class students who are perfectly happy to comment on how wrong you are, in public or in earnest after the class conversations. It’s not surprising that you can start questioning yourself, particularly in situations of stereotype threat.
   2. Comparisons among colleagues. “Why is everyone doing so much better than me? I’ll never measure up”. In my view, the Research Excellence Framework really stirs this up. During the week I wrote this recipe our College conducted a “REF readiness” exercise. This required staff to enter all their publications and evidence of other scholarly super-powers onto a database so the outputs could be counted. This database feeds public facing webpages so it is very easy to see your colleagues’ achievements and equally easy to worry that your performance is being evaluated negatively against others’. In fact, new proposals for restructuring the REF are intended to have a more collegiate setup where everyone contributes to the pool of papers for the entire department. Logically, the success of others should be no threat to you. But imposter syndrome is not logical.
   3. Scholarly productivity. “Will I ever get this paper finished?”; “What will the reviewers say?” ;“Aaargh, look what the reviewers said”. The pressure to produce high quality work can lead to performance anxiety and writers’ block. Expectations (in research, teaching and admin) have to be fulfilled, particularly for those on probationary period like the Chancellor’s Fellows at Edinburgh or new lecturers,and much effort can be expended on high stakes grant applications and papers. Many tears can be expended over the brutal rejections of those applications and papers. If you want more proof of this, have a look at the tumblr site “Shit my reviewers say” (http://shitmyreviewerssay.tumblr.com/) and marvel.
   4. Experiencing successes. As if all these triggers were not enough, by some perverse twist of psychology, some people find that experiencing success triggers imposter syndrome. Some people squirm when introduced as an expert, others worry that they will not be up to the challenge of managing a grant or assuming a more senior role.

Among this woeful catalogue of triggers, the women in Hutchin and Rainbolt’s study were more prone to worries resulting from peer comparison and harsh reviews. The men spoke about it in more general terms, although they did mention being labelled an expert in the media as a trigger. I am sure this problem would crop up for women too, if there were more opportunities to appear in the media as experts in their field rather than wearers of pretty dresses.

2. Own your successes. People with imposter syndrome are often reluctant to attribute success to their own abilities, preferring to say that they were just lucky, or they just worked hard. While modesty is a virtue, failure to acknowledge your own success is self-defeating. It can be very valuable to learn to take the time to acknowledge your success and consider what skills led to it so that you can build on this for the future. (Mount & Tardanico, 2016) suggest setting a timer for 5 minutes and writing down 10 things you do well. If you find this difficult, ask a trusted friend or colleague to help you. Reflect on these skills to see if you can identify a common factor and then consider which future roles would benefit from such a skill set.  
3. Remind yourself of the facts. Imposter syndrome thrives on half-conscious misperceptions about your lack of success and what it takes to succeed. It is worth taking some time to challenge these beliefs next time you notice them crop up. For example, if you find yourself thinking “I could never be a professor because professors all speak perfect BBC English” then challenge yourself to collect examples of professors with regional accents. If you think “I’ll never be able to do this”, then remind yourself of the last time you achieved something similar. It is possible to use the offensively superlative descriptions in the REF to your favour here. If you know what your papers were rated at through internal peer review then you can use this as a way to describe your work to yourself. According to the REF 2014 assessment criteria (http://www.ref.ac.uk/panels/assessmentcriteriaandleveldefinitions/), papers which are rated as one star are considered to be of “Quality that is recognised nationally in terms of originality, significance and rigour”. Two star is “recognised internationally”, three star is “internationally excellent” and four star is “world-leading”. It amuses me that internationally recognised papers only get two stars in this ranking scheme. Outside academia, international recognition would be something to aspire to, not to give half marks to! At any rate, try telling yourself “According to a panel of subject experts, my work is nationally/internationally recognised/excellent/universe beating”. Changing to this way of thinking gave me confidence to apply for my current post.
4. Talk to yourself kindly. Get in the habit of positive self-talk rather than mentally running yourself down. The women academics in the Hutchin and Rainbolt study found that they had remind themselves over and over again of achievements which made them proud. Think of the range of things you have achieved and why they matter. I like to save emails and cards from students to remember that when I am a teacher or a mentor I make a difference to people’s lives. Some of the women in the study liked to remind themselves of their successes in other areas of their lives when they feel their confidence drain.  Last week when I had a paper rejected, my family were surprised to witness me stop brooding in order to blurt out “Well, at least I can grow lemons!”.  I understand that not everyone takes comfort from citrus fruit farming, but there is robust empirical evidence that even short exercises in self-affirmation such as writing about the values which are most important to you can protect you from stereotype threat over a surprisingly long period (Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009).
5. Find social support. Talking to a trusted colleague, mentor or coach may help. You may be surprised to learn that some of your peers have similar difficulties and would appreciate reciprocal support. Ask the person you confide in to help you acknowledge your strengths and objectively evaluate whether your limitations are as crippling as you think they are (Mount & Tardanico, 2016). You could arrange to meet each other at potential trigger points, such as helping each other to read grant reviews or prepare key note talks.
6. What to avoid. Some academics develop maladaptive coping strategies which are counter-productive in the long run. A common and understandable response to imposter syndrome is to work harder and longer. But there is a risk here of in the longer term of workaholism which is associated with a range of health problems such as burn out, stress, insomnia, work -life conflict and poor job satisfaction (Hogan, Hogan, & Hodgins, 2016).
7. If you’re a manager, be aware of and address the potential impact of imposter syndrome among your staff. Imposter syndrome can lead to anxiety and stress, and in some cases reduced work performance and burn-out through workaholism (Mount & Tardanico, 2016). You could well have members of staff in your team who suffer from imposter syndrome, but who are unwilling to admit it to you. This underlies the importance of making sure that all your staff know that you value their work across the spectrum of different academic activities, nourishing them “when you reward or recognise their good work, encourage them or offer emotional support” (Amabile & Kramer 2012:33). I also believe that as the university places strain on staff to fulfil REF obligations, managers have a responsibility to help their staff through it by reducing anxiety and offering support. Minimising circumstances where staff feel they are under obvious pressure from social comparison could help. Another useful avenue for this is social support with colleagues. Mentors and peers who belong to the same in-group can “socially vaccinate” against negative self-perceptions if the staff member identifies with them (Dasgupta, 2011). For example, a new female lecturer would benefit from mentoring from a senior female academic particularly in disciplines where women are in the minority. This can be hard to achieve without over-burdening the few existing female staff with additional mentoring roles, but it may be possible to trade across discipline boundaries where the gender mix is the opposite (e.g. engineering with education).  Mentoring and positive role models are particularly beneficial for those entering academia, and those at transition points in their career (Dasgupta, 2011). 

As Amabile & Kramer observe: “Although nourishers may matter more to some people than others, none of us can truly thrive without them” (2012:33)

 

Note: for fellow members of staff at the University of Edinburgh:

If you are finding it hard to cope, The University of Edinburgh provides a free and confidential counselling service to staff members, on issues including debilitating stress, anxiety, depression, and alcohol and drug abuse. You can get help at http://www.ed.ac.uk/counselling-services/staff.

References

Amabile, T. M. & Kramer, S. (2012). The Progress Principle: Optimizing Inner Work Life to Create Value. Rotman Magazine Winter 2012.

Cohen, G. L., Garcia, J., Purdie-Vaughns, V., Apfel, N., & Brzustoski, P. (2009). Recursive Processes in Self-Affirmation: Intervening to Close the Minority Achievement Gap. Science, 324(5925), 400–403. http://doi.org/10.1126/science.1170769

Dasgupta, N. (2011). Ingroup Experts and Peers as Social Vaccines Who Inoculate the Self-Concept: The Stereotype Inoculation Model. Psychological Inquiry, 22(September 2012), 231–246. http://doi.org/10.1080/1047840X.2011.607313

Hogan, V., Hogan, M., & Hodgins, M. (2016). A study of workaholism in Irish academics. Occupational Medicine, 66(6), 460–465. http://doi.org/10.1093/occmed/kqw032

Hutchins, H. M., & Rainbolt, H. (2016). What triggers imposter phenomenon among academic faculty? A critical incident study exploring antecedents, coping, and development opportunities. Human Resource Development International, 0(0), 1–21. http://doi.org/10.1080/13678868.2016.1248205

Mount, P., & Tardanico, S. (2016). Beating the Imposter Syndrome. Center for Creative Leadership.

Woolston, C. (2016). Faking it. Nature, 529, 555–557. http://doi.org/10.1038/nj7587-555a

EqualBITE recipe: Salary negotiation

To my women colleagues: have you ever looked at your pay cheque and wished it was bigger? Do you get to November and growl when you realise you'll effectively be working for free for the rest of the year?

This recipe (from the EqualBITE book) is for you.

Salary negotiation

Judy Robertson & Alison Williams

Background

We should make sure that, when they leave university, all of our women graduates know and understand salary negotiation skills. And because we firmly believe that institutions shouldn’t exploit people, let’s make sure that anyone in charge of a hiring process understands how gender relates to salary negotiation.

According to the University and College Union (UCU), there is a 16.4% gender pay gap in the UK despite equal pay legislation being in force for forty years (UCU, 2016). The gap is slightly less in higher education at 12.3%, and it tends to increase at higher grades. At Edinburgh, the gender pay gap across all staff grades is 4%: the average woman member of staff earns £1,814 a year less than the average male member of staff (UCU, 2016). Over a career of forty years, this is worth over £70,000. What would you spend the back pay on?

Of course, it’s not just an issue in academia – there are eye-watering salary differentials in many industries. For example, in technology jobs, women earn around £17K a year less than men (Mediaworks). And in case you’re not already completely in despair, try this: there is a motherhood pay penalty (Budig, 2015). Yes, that is as depressing as it sounds – women who are mothers earn less than women who are not mothers. Sadly, there is also a fatherhood pay premium. A report by AAUW (American Association of University Women) found that it takes mothers an extra five months to earn what fathers earn in a year (AAUW, 2016).

There are many complex societal reasons for this inequity which we will not go into here. But you’re not powerless to change your own circumstances, and the first step is to ask for more. It is vitally important for a woman to negotiate her salary in her first job because if she doesn’t, she will start her career earning less than her male counterparts, and the gap will be compounded over the years. Half of the readers may be thinking: “Well of course you negotiate a salary – duh!” Other readers might be thinking: “Oh, but I couldn’t possibly do that. I should just be grateful to get a job”. When Judy started working, she was in the latter group. It simply didn’t occur to her to do anything except to accept the employer’s first offer.  In general, women tend to negotiate salary less frequently, and ask for less money when they do (Bohnet, 2016). This is because there are societal expectations about how women are “meant” to behave.  Studies show that managers are less likely to want to work with female employees who had asked for a pay increase, but this was not the case for such a request from a male employee (Bohnet, 2016).  It is the price for being seen to violate gender norms. This recipe is about how to negotiate a salary, and also how to create an environment where women don’t pay a social price for doing so.

 

Ingredients

• A spark of indignation or burning sense of injustice (adjust to taste).
• Facts about salaries in your sector.
• Steely resolve to ignore any childhood socialisation about how “ladies don’t mention money”.

Method

1. Find out what the average salary is in your sector. This external legitimacy can give you confidence in asking for your starting salary. You can use the average to anchor your request in a reasonable range. In academia, you could use the UCU Rate for the Job website (https://www.ucu.org.uk/rateforthejob) for information on expected salaries at various stages.
2. If you’re recruiting staff, advertise the expected salary range in the job particulars. Add the phrase “salary negotiable” to the advert. Transparency is required because gender bias thrives with ambiguity. If women are given information about the range of wages for the position, they are as good at negotiation as men (Bohnet, 2016).
3. Ask for a salary increase when you switch jobs. When Judy last moved job, she spent a sleepless night trying to work out what salary increase she should ask for after receiving the initial offer. To save you the effort, 15% seems to be about right (Miller & Miller, 2011). In fact, this is what the University did offer her, so she could have enjoyed her sleep instead.
4. These three qualities for successful negotiation cropped up in Judy’s desperate nocturnal reading: be confident, be prepared and be willing to walk away (Miller & Miller, 2011, p. 16). You need confidence to reach an agreement, and to convince the other party that your request is based on an accurate understanding of the facts, it is fair and it will benefit you both. Part of being confident is not being apologetic about your position, or weakening your argument by saying: “I may be wrong but...”. You can prepare by thinking through your priorities and goals – perhaps what really matters to you is flexible working or a good holiday allowance – but know your bottom line and be prepared to reject an offer which does not meet it. “You do need to understand the value of what you bring to the table and refuse to accept less” (Miller & Miller, 2011, p. 16).
5. Be persistent – don’t give up at the first attempt. When Alison was invited to take over running an undergraduate seminar programme at a university which shall remain nameless, the assumption was that she would work on the basis of pre-existing expenses and a small honorarium. Running the entire programme, however, involved a lot of work: planning, facilitating, formative and summative marking, and student guidance.  The university didn’t seem minded to be generous – or even fair – and was trading on her love of the work, and her desire to keep the programme running.  Complaining to a woman friend that “they want to pay me peanuts” there was a pause, and then the reply: “I’ve always thought of you as more of a pistachio person, myself”. Alison went back to the university with a fair, not extravagant, pistachio proposal.  The thought “I’m a pistachio person” (confident of the value she was bringing – see point 4 above) carried her through two attempts to knock her back: and the pistachio proposal was accepted. 
6. Review your salary against the sector average and compare your work to promotion criteria regularly to avoid stagnation. You can frame it in terms of someone else’s suggestion if you prefer: “My team leader suggested I talk to you about a pay rise”  (Bohnet, 2016). You could request someone to negotiate on your behalf if the whole thing gives you the horrors.
7. If you’re a senior manager, make sure there are routine procedures through which all staff can apply for a pay increase with transparent guidance about what is required. At the University of Edinburgh, there is an annual timetable for regrading, promotion and one-off contribution payments (Reward Processes Timetable, 2016). Consider all applications systematically, and if you’re really serious about closing the gender pay gap, consider all staff for grade increases whether they have applied for them or not. Evidence across the University of Edinburgh Athena SWAN applications and the senior mangers’ focus groups shows that women are less likely to apply for promotion or compensation than men: they may need encouragement to apply. If you really want to put your money where your mouth is, do what the University of Essex did and increase women professors’ grades by three points to raise their average salaries to those of male professors (Times HE, 2016).

References

1. Budig M, England P. THE WAGE PENALTY FOR MOTHERHOOD. Am Sociol Rev. 2015;66: 204–225. doi:10.1017/CBO9781107415324.004
2. Bohnet I. What Works: Gender Equality By Design. Cambridge, MA: The Belknap Press of Harvard University Press; 2016.
3. Miller L, Miller J. A Woman’s Guide to Successful Negotiating: How to Convince, Collaborate and Create Your Way to An Agreement. 2nd ed. New York: McGraw Hill; 2011. (Page 16).

    https://www.ucu.org.uk/media/7959/Holding-down-womens-pay-Feb-16/pdf/ucu_IWDpayreport_mar16.pdf

   https://www.ucu.org.uk/media/7959/Holding-down-womens-pay-Feb-16/pdf/ucu_IWDpayreport_mar16.pdfhttps://www.mediaworks.co.uk/insight/visualising-the-gender-pay-gap-infographic-infographic/

   http://www.aauw.org/2016/05/06/dads-get-a-bonus-but-moms-get-a-penalty

    https://www.ucu.org.uk/rateforthejob http://www.ed.ac.uk/files/atoms/files/reward_processes_timetable_16_17.pdf

    https://www.timeshighereducation.com/news/university-of-essex-hikes-salaries-for-female-professors-to-eliminate-pay-gap

EqualBITE- our book about Gender Equality in Higher Education

One of the reasons why I have not blogged for an embarrassingly long time is that I have been busy working on a new book about gender equality in higher education. It's an academic recipe book, containing recipes, articles, photos an illustrations by student and staff from the University of Edinburgh. My co-editors were Alison Williams and Derek Jones (we worked together on the original BITE book about remarkable research), as well as Lara Isbel and Daphne Loads. Alan Tait and Elspeth Maxwell did a cracking job of the design and illustration, and I still live in trepidation and awe of Vince Robbins, the proof reader.

We thought writing about gender equality would be hard, but it turned out to be much harder, and much more satisfying than we ever imagined. There are 71 pieces of written content and 18 student drawings and photos, all from 67 contributors. Working with an editorial team of five was necessary to manage the work load (although it did bring a communication overhead which we largely managed to deal with by using Basecamp). Somehow my co-editors and I made it through 2 years without any serious fallings out and with a lot of (slightly hysterical?) laughter along the way. The book is so much better for having all of our points of view. My advice to you is never attempt a project on this scale on such a sensitive topic without a strong team of people who will challenge you and support you and disagree with you and cheer you up when it all goes to hell.

EqualBITE is now published! You can get a free PDF or buy a paper copy.  (Even my dad bought a copy, which is cool but unprecedented) The launch party will be in March 2018. I'll be blogging a few of the recipes over the next few months to give you a flavour of what it is like.

 

In memory of my PhD supervisor, Jon Oberlander

This week Jon Oberlander died. Jon was professor at University of Edinburgh School of Informatics.  He was my PhD supervisor between 1998 and 2001, and recently our paths had met once more on the City Deal project on data driven innovation. 

I find it hard to believe that Jon is gone, because he was one of the most vividly alive people I ever met. He had liquid silver intelligence and extraordinary energy. I have had the good fortune to meet a lot of highly intelligent people over the years (a perk of being an academic) but he was exceptional.  I remember him at my viva, nodding vigorously, encouragingly, almost bursting with the effort of remaining silent while there was an interesting discussion to be had. He said a couple of times after I graduated that he didn't supervise my PhD, so much as witness it. I took this as a Jon way of saying that I was quite an independent student. But now I wish I had told him how glad I am to have had him as a supervisor and what it means to me.

One of my other colleagues was joking yesterday about how universities are getting a lot of bad press in the UK at the moment - "everyone hates universities", he said. But I don't. I feel very lucky to work at a university - at Edinburgh in particular - and that is because Jon taught me by example to love the best of academia. He loved ideas. They would come tumbling out of him, playfully, seemingly effortlessly. He would listen intently, pounce on new ideas and generously share them with colleagues. He connected people. He revelled in knowledge. I get to do all of these things in my job every day. All I need to do is remember once more why I work as an academic, and avoid being swept away by the flotsam and jetsam that goes along with it. This lifelong deep appreciation of creating and sharing ideas is the most wonderful gift and legacy from my PhD supervisor. Thank you, Jon.

Help - could a computer think?

I'm working on some materials for primary school teachers to help them teach children about computers. Can you help? I am now focusing on artificial intelligence after have some fascinating philosophical discussions with kids about whether machines can think. I wanted to summarise what is known about AI in a non-confusing way without going too deep into the raging debate about our Robot Overlords or indeed giving children nightmares. Below is my attempt at an explanation of the state of the art of AI. 

• If you're a teacher, is there anything else you need to know about this topic, or anything which is not clear?
• If you're an AI researcher (or an AI or that matter), is there a key concept I have missed, or anything which is deeply misleading?

Could a computer think?

Most people would agree that the computers we use every day don’t think for themselves in the way that people do– they follow instructions which people have given them. Sometimes it seems like the computer or phone is more clever than it really is, especially if you are using a program which seems to understand what you say. Computers are good at calculating, searching huge amounts of information very quickly and accurately and sometimes making decisions based on the information.  Humans are not so good at this sort of task, but we’re usually much better than computers at working out answers to new problems, being creative and understanding feelings.

Scientists and engineers who research artificial intelligence have been trying to make intelligent machines since the 1950s. Because of their work, your phone can recognise what you say to it, satellite navigation programs (“Sat Nav”) can speak directions out loud, robots (such as the Mars Rover) can navigate for themselves, and even the best human chess players can be beaten by a computer. Although these programs are impressive, they can’t do all of the many amazing things which even young children can do. Activities which we take for granted, such as walking, understanding how your friend feels or learning new languages turn out to be surprisingly hard to program into computers.

Could computers in the future think for themselves? And if they did, would they be friendly to humans, or would they want to wipe us out? Nobody really knows yet, and experts disagree about it. But it is worth thinking hard about it today because it may be possible to design intelligent computers for the future which can help us to make Planet Earth a fairer, safer and more interesting place to be.

What's inside a computer? - help!

I'm working on some materials for primary school teachers to help them teach children about computers. Can you help? For the moment, I am concentrating on how a computer works, based on some interview work I did with primary kids which showed they were confused about this and would like to know more. Below are my attempts at clear and concise explanations. Turns out it's hard to write these well!

• If you're a teacher, is there anything else you need to know about this topic, or anything which is not clear?
• If you're a computer scientist, is there a key concept I have missed, or anything which is deeply misleading?

What’s inside a computer?

There’s an electronic chip inside every computer, laptop, tablet or phone. The chip needs electrical power to work, which is why you need to plug in your computer, or charge the batteries of your tablet. The chip contains a processor which is like the computer’s brain. It also has memory for storing information it is working on now, and a hard drive for information it will need to store for a long time. The processor has got billions of little electronic switches inside it. The switches can each be set to “off” or “on”. The computer has instructions in its memory which tell it what to do. The messages are written in binary code which means they only contain 1s and 0s. A “1” means “turn a switch on”, and a “0” means “turn a switch off”. When you type, the keyboard sends a message to the processor about which key you have pressed and the processor uses the instructions in its program to decide what to do.

For older children, the following additional information might be appropriate:

Over the years computer chips have got faster and smaller and they’re capable of following instructions to do very complicated tasks. But the basic design of the chip hasn’t changed very much since it was invented in 1945 by the American mathematician John von Neumann.

 

What does a computer do when it follows instructions?

Imagine you have written a program in Scratch which adds together two numbers. When you press the green flag to make your program run, how does the computer follow the instructions you have written? Scratch is designed to be easy for people to learn using words and pictures we can understand. The computer processor works with binary code but it would be very confusing for people to have to write instructions all in 1s and 0s. So programs written in Scratch or other high level languages get translated before they run on the processor. When you press the green flag to tell Scratch to run your program, the computer follows rules which eventually turn your jigsaw blocks into binary code instructions which the processor can follow by turning its switches on and off. Once it has worked out the answer to the sum you gave it, the answer is translated from binary code and passed back to Scratch to show on the screen.

For older children, the following additional information might be appropriate:

Another name for a computer programs is software. The programs which translate the instructions are called compilers or interpreters. In fact, a program might get translated several times between different computer languages before it becomes binary code. Computers nowadays usually have lots of different layers of software stacked on top of each other to make them easier to program. You can think of it like a tower of stacking cups with the computer processor at the bottom. Your Scratch program is on the top cup, and it gets translated into a language called Action Script in the next level down in the stack. Action Script runs within a web browser program like Chrome. In turn, instructions from Chrome get translated to C++, which is then turned into assembly code and then finally binary code to run on the processor.

[INSERT IMAGE OF ANNOTATED STACKING CUPS HERE]

Why trading cards are great for computational thinking

My six year old son has got the bug for trading cards.  I began taking an unnaturally close interest in this craze as soon as I realised the cards were a launching point for all sorts of computational thinking. Ian loves Star Wars cards, but it doesn't matter in the least what the topic of the cards is. If you ever wanted to try out these activities with a class, you could get the kids to make their own card sets which appeal to them. That would avoid problems with inclusion, either to do with the money for buying card packets or the marketing of most trading cards towards stereo-typically boys' interests. I refuse to believe that collecting or sorting cards is inherently either "for" boys or girls. 

So what do trading cards have to do with computational thinking? They are a collection of objects, each of which has a type (such as rebel, empire, mercenary), and numerical attributes (such as star rating, attack or defense). Each card also has a unique identification number. Computer scientists love collections. Their collections are usually digital objects (data) but the same computational principles apply. Your set of trading cards is information, or data, and what you do with the cards are the processes.

 

What do kids like to do with trading cards?

1. Collect a complete set. This involves knowing how many items there are in a complete set, and what those items are and knowing how many items you have in your own collection and what those items are. This enables you to work out how many cards you are missing, and which you need to acquire.
2. Trade with friends. Some kids trade to acquire the cards they are missing for their collections. Others seems to just love trading for the sake of it. 

I'll focus on number 1 for today, but there is a lot of interesting learning about game theory to be done with 2.

Collecting a complete set: data structures and sorting

How do you know what's in the complete set?

The company which makes the star wars cards supplies a poster which shows a photo of each card. The photos are grouped by type and the unique identification number is printed underneath. The numbers start at the top left, and the highest card number is at the bottom right. There is a checkbox under each so you can tick when you collect it. My son and I spent a happy half hour executing this algorithm:

For each card in your own set{

    Look at the picture on your card

    Find the picture on the poster which matches your card picture

    Tick the box on the poster once you have found a match}

But wait! There is more than one Yoda card (you get two sorts of subtly different shiny ones and a plain one). How do we decide whether we have the right one? We changed our algorithm:

For each card in your own set{

    Look at the unique identification number on your card

    Find the card on the poster with the same identification number

    Tick the box on the poster once you have found a match}

And that, my friends, exactly captures why unique identifiers are necessary as any database designer will tell you.

Then you can count how many ticks you have, and subtract from the total number in the complete collection. You can spot doubles easily too, which means you can identify what to trade. Note that every time you make a trade you'd need to keep the poster up to date. That's because it's a copy of the data in your actual card set, so when the cards get updated, you need to update the copy too.

 

How do you know what's in your own set?

If you have a set of cards, you need to store them somehow. You could keep them in a stack with an elastic band round it. My son loves stationary, so he was very keen to get a folder to keep his cards in. So was I when I realised that folder consisted of transparent pages with 9 slots on each page -it suggested to me an array data structure. At first Ian loved putting the cards into the folder and then taking them out again just for the sheer physical fun of it. But as his card collection grew, it became harder to know where to find his cards when he wanted them and he grew more ambitious.

I asked him (and Daddy who is a software developer) what is the best way to get the cards from the pile into the right place in the book? Ian wasn’t sure at first but then he suggested searching the card pile for the lowest card (which would be 1) and inserting it into first free slot in the folder. Then look for the highest number in the pile (243) and put it in the last free slot in the folder and then keep doing that. As I understand it, this is Ian's algorithm:

set lowest to 1

set highest to 243

While there are still cards left in your pile{

   for each card in your own set{

    if (card id equals lowest){

        insert card into folder in slot labelled with lowest

    }

}

lowest = lowest +1

 for each card in your own set{

    if (card id equals highest){

        insert card into folder in slot labelled with highest

    }

}

lowest = lowest +1

highest = highest -1

}

Now, this isn't the most efficient way of doing it but it would work (and is pretty good for a six year, she notes bursting with motherly pride). 

Then I suggested other ways of doing it. 1) Label all the slots in the book and then iterate over the pile, inserting each card into the slot with the corresponding identication number. 2) Sorting all the cards in the pile, then inserting each card from the sorted pile into the next free slot in the folder, starting from a blank folder.

Now we really get to the heart of computer science. We have a couple of possible ways of solving the problem correctly but we need to decide which to use. Which is optimal for our purpose? That really depends how you define optimal - what your priorities are. When sorting a collection of objects in computer science, you usually worry about efficiency: what is the quickest way to solve this given the size of the collection? If this is our priority, then 1) is better. * YOU MAY NOT CARE ABOUT THIS BIT ***As Daddy pointed out, 2) is less efficient because the sort is nlogn or worse, and for 1) labelling all the book slots is linear plus constant time insertions. What this means is that theoretical computer scientists do mathematical proofs about the efficiency of sorting algorithms based on the size of collection (referred to as "n"). The fastest known way of sorting will take the time it takes to process every item in the collection (n) multiplied by the logarithm of n. Because method 2) relies on sorting, we know it can't be faster than nlogn. For method 1, we need to write a number on every slot to store the collections - this takes n time. In both cases, inserting an item into an array is considered to be constant time - the time for it doesn't vary according to the size of the collection. START READING AGAIN* For computer science education, we might choose to define optimal as something different.To teach computational thinking we could think about optimising algorithms for cognitive load.We should think about the working memory load on the kid who is doing the sorting, and how hard they find every individual operation (such as comparing numbers or writing labels). In this case, 1 is also easier because it is easier on the memory and builds on writing numbers in sequence, which is common at school.

Anyway, back to Ian's Star Wars collection. We decided to label every slot with a number, starting at 1 and going to 243 (the size of the complete collection). More messy black ink and smudges than the nice clean digital equivalent:

int[] myIntArray = new int[243]*

 * Yes alright, I know arrays in programming start at 0, but evidently the manufacturers of trading cards don't

 

 Then we applied this algorithm

While there are still cards left in your pile{

    Pick up the card on the top of the pile

    Read the card id number

    Insert the card into the slot labelled with the card id number

}

Simple! So now we can see where there are gaps in the collection, and it's easy to insert into the collection in the right place when you get a new card. The see through slots make it easy to gleefully browse your collection. Isn't computer science wonderful?