Computers, creativity and learning

Ranty paper about why HCI with kids researchers need to do better evaluations

In my continued campaign to alienate colleagues and generally stir up trouble, here is our journal paper which was just accepted in Journal of Child Computer Interaction. The main point is that IDC is old enough now to be doing respectable, replicable longitudinal research in ecologically valid settings.We need to get away from the emphasis on cute technology innovation. If you want to try embedding your research in a real world classroom setting, in a sustainable way, give our Train the Teacher Method a go!

 

Robertson, J., Macvean, A. and Howland, K. (2013) Robust evaluation for a maturing field: the train the teacher method. International Journal of Child-Computer Interaction. http://dx.doi.org/10.1016/j.ijcci.2013.05.001 Download CCIPaperForWeb (close to final version)

Abstract:

This paper highlights the importance of evaluating educational technology for children in naturalistic classroom contexts. We present the Train the Teacher Method (TTM) which formalizes a method for IDC researchers to use when deploying and validating an educational system. Our key findings indicate the need to work in partnership with classroom teachers, providing both initial training and continued support. This will both result in more valuable research data, and address a gap in teachers’ continued professional development. The TTM aims to ensure that teachers, students and researchers can benefit from innovative educational systems deployed in real classroom contexts. We believe that the TTM could be adapted to other domains of study within IDC.

The paper goes on to argue the more general point that the interaction design and children community should shift emphasis away from technology innovation and towards reporting replicable longitudinal research findings from real world settings. We discuss possible approaches to achieving this, for example through changes to reviewing criteria and by actively supporting data and source code sharing.

Advert for game creation job for students

Here's an advert which I have been asked to pass on by a charity which needs some CS students to do a game design project. Questions to the email below.

*****************************************************************************************************************

We’re developing  Rock Operator a location based game, using iPhones  & maps to guide groups round the environment of Holyrood Park by GPS.

There's an established trail taking in 4 locations on Arthurs Seat beginning and ending at Dynamic Earth, which is aimed at helping 10-12 year old autistic kids learn more about their environment.

There is a short animated film called Rock Opera, from which you can see a trailer here. Also some more  characters viewable on the blog.
We are a small charity running on a limited budget, but you will get paid expenses and renumeration for your time. You will also have access to relevant footage for portfolio purposes and references on submission of a satisfactory app.

Please send an e mail me to [email protected] and  I'd be happy to meet up and discuss any queries you have.

Visiting the Queen

See me? See the Queen?* I went to her garden party on Monday. It was pure dead brilliant. This is me outside her house. Note my fascintator (the thing on my head is not actually a bizarre bee hive).

It was fun actually. I went with my lovely sister in law and we hung out in the Palace gardens drinking tea and discussing everyone's outfits. I also ate a considerable number of royal cakes, including one with a crown on it.

Thanks to EPSRC for nominating me. It was for helping the little children and teachers with computer games, and that.

* Forgive the Glasweigen syntax.

Looking back over 8 years of research

Let me tell you some stories about why I love research.  Anecdotes, of course, are not data, but they are a good way to capture powerful moments which make a difference to people’s lives. 

First here’s the back story. In 2003 a student of mine showed me the Neverwinter Nights toolset. He thought I might be interested in a tool for people to make their own games when I told him about a prototype I was writing to do something similar. I decided I would try it out at a summer workshop run by the local council. So I did. And there started a research programme of eight years. The kids loved that first workshop. I was fascinated by their learning. So I ran more workshops, conducted more interviews, analysed more games, published more papers on it.  I got research funding for Adventure Author to make the Neverwinter Nights interface more usable. We got follow on funding for a public engagement project called Making Games in Schools to train teachers so that it would be more widely available to school pupils. My long time collaborator Judith Good got funding for Flip, to look at game making from a programming perspective, based on our previous work. Flip is a sister project to Making Games in Schools and Adventure Author, and is also about helping kids to make their own games. It focuses on making it easier for kids to write programs which respond to events in their games, thus enabling more interesting stories to evolve. 

 

At this point over 1000 kids have taken part in game making workshops, all stemming from that initial casual decision of mine to try out something new because I thought it would be fun. Yesterday I had an opportunity to think back on it all because some of the teachers we trained on Making Games in Schools were discussing their experiences of game making with their pupils. Here are some of the stories which stuck in my mind.

• One of the teachers came to a workshop we ran at university for her 3rd year pupils and was “blown away” by the pupils’ learning. She piloted it as soon as she could at her own school and now four years later has involved 8 other school departments in an inter-disciplinary scheme which includes all their pupils from years 1-3. She explained that she had been offered early retirement but she refused because she couldn’t bear to give up the project.
• Many of the teachers told us how their pupils were clamouring to work on their games at lunchtime, how they didn’t want to leave when the bell went, and how they even turned up for class early to spend as long as possible on their games.
• Several teachers spoke of kids who normally hated school and who gradually became hooked by this project and even started singing the praises of the class to their friends in the years below. We also heard about autistic children who took to game making exceptionally well and how it opens up a whole new way of working with them.
• But my favourite story was about a boy who took part in a transition project at primary school to prepare him to start high school. At primary school he was an elective mute. He learned, with the others in his class, how to use the game making software at the high school. When this class arrived at secondary school for real, they were of course spread throughout different classes in the year group. In their computing classes these children are the experts to help their classmates to learn how to make games.  Now, half way through his first year at secondary, he has begun to talk, particularly to share ideas and give tips to his fellow pupils about their games. I’m not saying that the project was the reason for this. But the teacher thought it was part of the picture: a real reason for communication.

 

Two things are important about this, I think. One is that it’s worth taking a chance and trying out an idea, just to see what happens. It could have a big impact. The second is that for me, research should always have a positive influence on people’s lives. These game making projects have given a lot of pleasure to a lot of children. They have changed people’s attitudes to learning, to their careers and transformed how they relate to other people. And that seems well worth it to me.

 

Game Design through Mentoring and Collaboration

[cross posted from my CACM blog]

I was interested to read about a really fantastic National Science Foundation funded project called Game Design Through Mentoring and Collaboration. Taking place at McKinley Tech and George Mason University, the project encourages young people into STEM careers through weekend and summer courses in computer game design. I particularly like two aspects of Game Design Through Mentoring and Collaboration (GDMC). Firstly, the students learn with slightly more experienced peer mentors as well as an instructor. This can be a very effective model because both the mentor and the mentee can learn a lot, and it gives the teacher much needed assistance in a busy class full of temperamental computers and children. (If you want to know more about different models for effective mentoring Kafai et al (2009) is a good place to start). Secondly, the students also learn about science subjects and integrate their new knowledge into their games e.g. after input from a Federation of American Scientists biologist, the students’ games included accurate information about antibiotics, glial cells and neurotransmitters. If we can’t convince every child to become a computer scientist, any kind of scientist will have to do.

Game design projects are increasingly popular in education, and the evidence is starting to accumulate about the effectiveness of such schemes. In an article published earlier this year in Computers and Education, Vos and colleagues (Vos, van der Meijden and Denessen, 2010) compared students’ motivation and use of strategies for deep learning when they either played a simple memory drag and drop game, or constructed their own such game. The children enjoyed making games more than playing them, and were more likely to use deep learning strategies while doing so. A notable finding from this study was that the children were less motivated in the play condition than by their normal classroom lessons. This just goes to show that if you’re going to spend classroom time on a game, it had better be good, or you might as well not bother. Or perhaps a more positive way of looking at that would be to say it takes a high quality game to beat an enthusiastic teacher.

Game genre and graphical quality are likely to be factors here. The simple 2D board game style application in this study looks rather dull in comparison to the sort of action game you might find gracing the screen of a Wii. It may well be that making an online board game is more fun than playing it only because playing it isn’t that exciting to start with. In contrast, the Game Design Through Mentoring and Collaboration students learn a wider range of technical skills which enables them to make 3D games with proper physics. I think this is pretty important because in my experience kids want to make games which look and feel as good as the games they play at home. After all, they want their friends to be impressed when they play them. So hats off to the students on GDMC: your counterparts across the pond in Scotland salute you!

 

Kafai, Desai, Peppler, Chiu and Moya (2009). “The Multiple Roles of Mentoring”  From Kafai, Peppler, Chapman (Eds). The Computer Clubhouse: Constructionism and Creativity in Youth Communities”. Teachers College Press, New York. “

 

Nienke Vos, Henny van der Meijden, Eddie Denessen, Effects of constructing versus playing an educational game on student motivation and deep learning strategy use, Computers & Education, Volume 56, Issue 1, Serious Games, January 2011, Pages 127-137, ISSN 0360-1315, DOI: 10.1016/j.compedu.2010.08.013.

(http://www.sciencedirect.com/science/article/B6VCJ-50XS6C5-1/2/06e4ca67e68d2f5495080e31814a2062)

Quest to Learn

[cross posted from CACM blog]

There was a thought provoking article in the New York Times last week on games in education. Author of the indigestible yet indispensable academic game design bible Rules of Play, Katie Salen has project in a New York city school called Quest to Learn in which children's learning is based around game playing and design. It sounds like a big budget, integrated version of what teachers in the UK have been doing in their own practice (look out for details of this in an upcoming Futurelab report): the teachers playing games on a big screen in front of the class while soliciting advice from a super-engaged drooling set of kids; learners using game authoring software to explore concepts with game design. What's different is that this project has been systematically devised and rolled out across a school, rather than relying on the individual innovative practice of some teachers. Particularly telling in the fact that there are three game designers working with eleven teachers to devise games around interdisciplinary curricular content. That's a fantastic resource, and unfortunately unlikely to happen in the UK anytime soon given the current monumental cuts in public spending.

Furthermore, there are many hearts and minds to be won over here about the efficacy of games in education. The article acknowledges that 6th graders who took part in Quest to Learn did no better on standardized tests than learners who had not had the privilege. This makes me uneasy. Sure, you can argue that the Quest to Learn project values different educational goals which are more relevant to 21st century society than ordinary curricula (such as team work or problem solving) and that therefore different assessment tools are required. But is this just special pleading? It may be that we need to win the wider battle of making school assessment more appropriate to the skills required in current society before we worry about installing Wiis in the class.

One of the most interesting aspects of the article was related to the metaphor used to organise school life. The author writes:

"What if we blurred the line between academic subjects and remained the typical American classroom so that, at least in theory, it came to resemble a typical American living room, or child's bedroom or even a child's pocket circa 2010...what if, instead of seeing school the way we have alway seen it, we saw school for what our children dreamed it might be: a big, delicious video game?". School as a video game is certainly likely to be a more appealing metaphor to children than the current ones. Guy Claxton argues that schools are currently organized around a factory metaphor, full of grim tasks which must be completed and overseen by a supervisor figure, and time strictly marshalled by bells. He proposes a metaphor which he considers to be more fruitful: learning as gymnasium, where learners strive to improve their own performance by exercising their brains in an enjoyable fashion. This is consistent with the learning as game metaphor, but perhaps more general purpose. In any case, it is about time we reconsidered how schools work as learning spaces. The industrial revolution has had its day, and we are beginning to realise it is counter productive to coop children up in learning factories. That's why we need experiments like Quest to Learn.

I am now a volunteer blogger for Communications of the ACM blog.(ACM is the Association of Computing Machinery. If you are a card carrying geek, your card says "ACM") You can see my first post at:
http://cacm.acm.org/blogs/blog-cacm/34871-no-were-not-losing-our-ability-to-think-critically/fulltext

I'm going to try to blog there a couple of times a week, so I'll cross post the articles here. I think I will need to keep my madder witterings for here only, though.

Outraged researcher required to justify her "fragile" research

I'm applying for promotion at the moment, and I was gobsmacked to learn that someone in the senior management team thinks my research impact is fragile. Fragile? WTF?

I have to write a paragraph justifying the impact my research has on the UK economy and society. Help, blog readers! Answers on the comments page, pls: how does educational technology research impact society?

I don't think I can get away with my standard shorthand this time: "I help the little children".

Hooray!

I just got an EPSRC dissemination grant! Hooray!  The word "hooray" does not sum up the immense relief that I feel at this point. :-)  This is a resubmission of the grant which was rejected last time. Previously it was ranked 12th, and they funded the first 11. This time it was ranked 3rd out of 22. The moral of the story? If at first you don't succeed, try again. (Or "use the knee and then the heid", as my dad would say). Now I feel like a respectable citizen of my department and can (possibly) stop writing so many grant proposals.

The funding is to carry on my previous EPSRC funded Adventure Author project but this time to do a lot of teacher training. The idea is to train teachers how to help their pupils make their own computer games. My good friend Cathrin will be working on the project with me because she knows all abou training teachers. Briefly, our argument was as follows:  we think little children should learn how to make computer games because it is fun and it will make them want to learn computer science.( And everyone knows that the world needs more computer scientists. Because if it doesn't get them the ICT infrastructure of the world will stagnate and collapse. Then you'll be sorry you called us nerds and stole our dinner money.) The way to get kids making games is to get teachers involved because teachers can keep teaching classes long after individual bits of funding run out, thus ensuring a steady stream of happy baby computer scientists. A cunning argument, yes? Well it worked for the EPSRC reviewers...

We will train 32 teachers over 18 months (starting May 1st), and also train a bunch of computer science students to give them hands on help in the classroom. Thus ensuring that the almost grown up computer scientists inspire the baby ones.If you're a teacher within a 100 mile radius of Heriot-Watt, or a student who might want to be paid to work on such a project, let me know.

Wheeeeee!

Scientific reasoning when playing games

[Cross posted from the Adventure Author blog]

Thanks to Hywel, who sent me a link to a Wired article about Constance Steinkuehler's work on scientific reasoning in game players. http://www.wired.com/gaming/gamingreviews/commentary/games/2008/09/gamesfrontiers_0908

Constance does really interesting ethnographic work in massively multiplayer games, by which I mean that she spent hours playing Lineage to immerse herself in the culture to really understand it. It was a tough sacrifice, but it now puts her in the position where she has a healthy respect for, and an in depth knowledge of, how MMO gamers think. In this particular study, she was analysing discussion forum posts from players who were trying to systematically work out the rules governing the behaviour of boss monsters so they could defeat them. She identified that these groups of informal learners were in fact engaged in scientific reasoning and hypothesis testing. This is interesting for a number of reasons: 1. it could be a really good approach to teaching scientific inquiry skills 2. It just goes to show that teenagers on discussion boards are doing more than typing random strings of txt gibberish: they are self directed learners communicating in a specialised language and 3. Those who comment on the culture of computer games based on surface observations may miss a lot. Like say... Susan Greenfield, whose talk at the Edinburgh Book Festival revealed a lot of assumptions about what is actually going on when people play games.

In Adventure Author, we have also noticed kids hypothesis testing when they are making games. It's the same sort of process. Because Neverwinter Nights has underlying game rules (the dungeons and dragons rule set), your game creatures will obey these rules. Are you going to sit an read the D&D manual to understand it? No! You're going to guess what might be happening and then create little tests to try it out. For example, when we worked in Ancrum Rd School in 2006, we had a class of ten year olds who became fascinated by why their game characters kept fighting each other. After testing and class discussions they worked out it was to do with the different factions assigned to the characters they had chosen, and that they could control the behaviour by altering the faction tag.

There are many ways to teach scientific inquiry, but the Wired article makes a good point in that it is the process of science which learners need to know, not just scientific facts. Simulation and gaming environments are good predictable mini systems to practice in.