Computers, creativity and learning

Classroom computers kipper-slapped in OECD report

 This week the OECD published a report on Students , Computers and Learning: Making the Connection which will be a slap in the face with a wet kipper* to many a technology evangelists. Based on a large survey and standardised test dataset of 15 year olds across many countries, it concluded that "the reality in our schools lags considerably behind the promise of technology", "even where computers are used in the classroom, their impact on student performance is mixed at best" and "technology is of little help in bridging the skills divide between advantaged and disadvantaged students." What exactly did the report find, and what are the implications of this? (And no, I don't really think you should feed your school iPad collection to the fishes).

Figure 1. A slightly damp kipper

What sort of research is this?

 This is an analysis of survey data and standardised maths and reading tests collected in the OECD Programme for International Student Assessment (PISA) in 2012. Data is available from 510,000 fifteen year olds (representative of 28 million students across OECD countries) from 64 countries. Each country submitted data from a minimum of 150 schools. Studies of this sort are great for describing the generalities and providing basic facts about performance in standardised tests, technology access and ownership. However, as there are many factors which are not controlled for (it is not designed as an experiment in this sense), it is not possible to draw conclusions about cause and effect. The study might suggest patterns of behaviour or relationships between variables but it will not be able to explain why these patterns exist.

What are the main findings?

Access at home and at school

• On average, 96% of the school students surveyed in the OECD countries had access to a computer at home. The UK is slightly higher than average, with 98.8%. Children across OECD countries spend on average 104 minutes using the Internet at home on weekdays, and 138 minutes on weekend days. In contrast at school, they spend 25 minutes using the internet. (No figures are reported for the UK). 
• In OECD countries on average there are 4.7 students to one computer in school. There are more computers per head in UK schools (1.4 students to computers). In OECD countries, 72% of the students have access to computers at school. (UK information is not available on this measure).

Lack of technology effectiveness in the classroom

• Investment in educational technology infrastructure does not appear to be associated with better performance in reading, maths or science. Countries that have invested less in introducing computers in school have improved faster, on average, than countries that have invested more.There is a hill shaped curve in the relationship between technology and learning as measured by the PISA tests: limited use of technology in school is better than none but also computer usage at school above the OECD average appears to be associated with significantly poorer results. These calculations controlled for GDP and previous performance in PISA. The same pattern is evident for computer based homework. It is worth noting that there are countries where this pattern does not hold e.g. Denmark and Norway.  As the report does a good job of acknowledging, these are associations rather than statements about cause and effect. We cannot conclude from this that having too many  extra computers around is bad for children's learning.

 Figure 2. The relationship between reading performance and ICT provision at school

 

• The sort of activities performed on school computers make a difference: drill/practice software and chatting online are associated with lower performance, while browsing the Internet or using email once or twice a week is more beneficial.

Digital Divide

• There is a 13% difference in access to the Internet at home for students from socio-economically disadvantaged backgrounds across OECD countries, whereas it is only 3% in the UK. On average across OECD countries, disadvantaged students spend 7 minutes less using the Internet at weekends (presumably they are able to access the Internet outwith the home at weekends, or else the aggregate data is masking different patterns of behaviour within countries). 
• The purpose of using the Internet at home is different for those in social-economically disadvantaged students: 18% fewer disadvantaged studens use the Internet for finding practical information, while there is only a 0.5% difference for game playing. (No figures are available for the UK).
• The report draws the conclusion that in order the reduce digital inequalities, schools should focus on reducing inequalities in literacy and numeracy. Access to technology will not by itself ensure that the children have the skills to use the technology effectively for learning. 

Association between excessive internet use and negative behaviour/experiences

• Students who reported spending more than 6 hours per weekday using Internet (outside school time) are more likely to report feeling lonely at school and are more likely to be late for school or absent. It is worth pointing out here that these are associations, not causations. E.g. we don't know whether they are lonely because they use the Internet or that they use the Internet so much because they feel lonely, or whether there is some common underlying factor.

So what?

It is not entirely surprising that the overall picture for technology in schools is a damp squid. Previous meta-analyses( statistical syntheses of findings across many quantitative studies) indicate that computers in the classroom have only a small-moderate effect on learning. Two recent meta-analyses show that educational technology makes a modest improvement in learning of mathematics (Cheung & Slavin, 2013) and a small positive improvement in reading in comparison to traditional instruction (Cheung & Slavin, 2012).  Hattie's meta-meta-analysis (yes, there is such a thing!), across about 5000 studies, found a only moderate effect for computer assisted learning, and it is ranked pretty low down on his list of strategies which make a difference in the classroom.

What does surprise me is the hill-shaped curve finding, where more use of technology was associated with poorer performance. I have not come across a finding like this for technology before (which may be a reflection of my patchy knowledge rather than anything else). It may well be because class time spent using computers to do pointless stuff means less time available to do valuable stuff (face to face or on computer).  And money spent on vast banks of iPads (not looking at any council in particular here...) means less money available to be spent on teacher professional learning. 

I think us educational technologists need to pause a moment to feel the wet kipper. It's a good moment to reflect on the fact that technology in the classroom just isn't as effective as we claim it ought to be. This doesn't mean we should give up on it, but that we need to be more savvy about how it it used.  I also don't think we should let ourselves off the hook by saying "technology really is effective, but we're not measuring it the effects properly". Sure, technology may have lots of benefits which were not measured in the PISA test, ( such as enabling collaboration) but if we're deciding how to spend public money, it seems wise to invest in things which will enable gains in basic underlying skills such as literacy,  numeracy and science. I would argue it could potentially increase higher order cognitive skills too, but we need to measure that.

This study and Hattie's meta-analysis tells us that computers in the class can be more or less effective depending on the features of the software and how they are used. There are some circumstances in which they can be highly effective, and clearly some countries which do this better than others. This can give us hope, and ideas for fruitful directions for the future. Hattie found that benefits are more likely to occur when technology is used: as supplement to teacher (not replacement); collaboratively; when the learner is in control; when teachers have been educated about the use of computers as a pedagogical tool; and when feedback via the technology is optimised. So as well as investing in educational technology hardware (instead?), it seems that our education authorities need to be investing in teacher professional development (and quite possibly higher quality online learning materials). Teachers need support in learning to use technology but also how to use it effectively in the service of learning. As the OECD report put it: "In the end, technology can amplify amplify great teaching, but great technology cannot replace poor teaching".

 

 

References

Main report on which this post is based: OECD. (2015). Students , Computers and Learning: Making the Connection. http://www.oecd.org/education/students-computers-and-learning-9789264239555-en.htm

Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113. doi:10.1016/j.edurev.2013.01.001

Cheung, A. C. K., & Slavin, R. E. (2012). How features of educational technology applications affect student reading outcomes: A meta-analysis. Educational Research Review, 7(3), 198–215. doi:10.1016/j.edurev.2012.05.002

 Hattie, J. (2009). Visible Learning. New York: Routledge.

Technical report on PISA survey: http://www.oecd.org/pisa/pisaproducts/PISA%202012%20Technical%20Report_Chapter%204.pdf

 

* An expression meaning "receive an unpleasant surprise" for those of you unfamiliar with the phrase. Why kippers? No idea.

Reflections on using AppInventor to teach first year programming

[cross posted from my CACM blog]

Every so often I take the plunge and completely revamp the courses I teach. This year, after some nagging from students that they really really wanted to learn more about mobile app development, I decided to introduce Android development to my first year programming class. In first semester our first years learn logic, an introduction to Java, a study skills course and my course which is called Interactive Systems. I designed Interactive Systems to be a fun introduction to programming where the students come up with creative design ideas, attempt to implement these ideas, tie themselves in knots adjusting their plans and generally get to grips with independent learning. Previously the students made an interactive 3D pet in Second Life , but as much as I love seeing their pets, I couldn’t escape the feeling that Second Life was getting old and creaky.  I wanted some other environment where the students could build something visual and immediate, something motivating, something of which they could be proud. Developing Android applications seemed to fit the bill, but only if I could find a way to pitch it at the correct level.

As in many universities, our first years have a range of programming experience when they come to us: people who have never programmed before, visual basic hackers, increasingly Scratch adepts and the inevitable few whose first words were in syntactically correct C.  It can be difficult to find the correct pace.  It seemed unfair on the novices to expect them to learn Android development at the same time as they learned the basics of Java in my colleagues’ classes, yet I wanted them to have some early successes in programming. AppInventor seemed like a good approach, so my co-instructor Gudmund Grov and I decided to adopt it. It’s a Scratch-like visual programming environment which targets Android. Better still, it has a wealth of ready-developed pedagogically sound teaching materials such as Course in a Box. I like the flipped classroom model where the students read  textbook chapters, tried lab exercises and then came to lectures to consolidate underlying concepts.

The practical examples which come in the text-book are well structured and easy to follow. The students liked them but were not particularly challenged by them. (In fact I have seen a ten year old whizz through the same exercises). Sometimes the students told me that they worked through the exercises mechanically without really absorbing the material.  

The real challenge came when we asked the students to come up with their own ideas for an app and implement it over the next ten weeks. It can be difficult for students to estimate how difficult a task will be.  The representation and features of the AppInventor interface make certain tasks (which would be hard for beginners in an ordinary development environment) trivial. For example, working with data bases and using QR codes are both easy and quick to accomplish. Other tasks are almost impossible in AppInventor, or are tedious to achieve.  It was therefore hard to guide students on how ambitious they should be in their designs. Next year it will be much easier to judge this as we have examples to show them from this year’s cohort.

 

Example code from a student’s game

Two-thirds of the way through the course, we introduced more conventional Android development using Eclipse and helped the students to map between the blocks representation in AppInventor and the Java code they were learning in their other courses. The more experienced programmers lapped up Eclipse, while the novices despaired.

At the end of semester, after a showcase of student nominated contestants for best app, we were faced with 94 portfolios to mark including a learning log entries. These logs can be quite entertaining, and offered us useful insight into the students’ experiences with AppInventor.

Many of the students had become quite frustrated by bugs with the AppInventor environment e.g. "It [AppInventor] was still in the beta version and lots of bugs and errors kept occurring which caused my app to fail or delete parts of code randomly." Or "App Inventor is very much a beta stage product, missing some vital stuff and while using it complete my app had it corrupt a few of my screen resulting in me having to redo work that had taken me a while to create." There is a new version of AppInventor which we will use next academic year, so I am hopeful these issues will be resolved.

Students who were new to programming certainly appreciated AppInventor, commenting: "As App Inventor works in modules, I found that I now understand how the programming process is structured better than I would have if we had dived straight in with Eclipse.",

"I remember at the start of the course remarking that App Inventor would be beneficial to me as a beginner programmer as it breaks down the structure of conventional programming and would let me understand the big picture from the beginning. Now, when I am at the end of this course, I know that that is exactly what it has allowed me to do.",

"I really loved making my app, from the design stage through to sorting out that last piece of code. It has been a nice learning curve and I can confidently say that I am proud of what came out of it."

However, the more experienced students were exasperated by it. Some of the more polite comments from this group included "At the beginning of this course, I was very excited to create my own app as I loved programming at school so could not wait to get started. In the early stages, I found app inventor very easy to use and useful; however as I began to create more complex apps, I found that app inventor was very restrictive…I personally found Eclipse better to develop Android Apps in due to more flexibility and freedom with the software, even though this is challenging, the reward is much greater when completing each task". It is gratifying that this student was intrinsically motivated by problem solving in a more challenging environment. Another student reflected on the transition between environments as follows: "although Eclipse seems to be very difficult to get to grips with at first, after a little time working in App Inventor it begins to become more of a hindrance than a helping tool, and the user begins to find the limits of the program." One of his classmates had an alternative view: "While I thought app inventor was too simple, and as a result hindered the possibilities, I would now go so far as to say that the ease of use of app inventor makes it a fantastic entry point for future programmers. Whereas the eclipse IDE has so many points that can go wrong and as a result derail the stability of the program."

Summary

Would I recommend using AppInventor to teach first year computer science students to program? The answer is a qualified "yes". If you have a mixed ability class, I suggest spending perhaps six weeks on it before moving the students on to a more fully featured development environment. If you’re going to do this, the students will need a lot of support in managing the transition (taking their training wheels off!).   If you’re teaching complete novices, non-computer science students, or younger students, or a course in which the emphasis is on the product ideas rather than implementation, App Inventor will be ideal as it was developed with these scenarios in mind. One last thought: in my experience, computer scientists love to complain about the technology they are required to use. It’s part of a geek-macho culture to blame development tools in order to show one’s programming prowess. So don’t take student griping to heart too much as there may be no goldilocks zone for instructors where everything is "just right".

Ranty paper about why HCI with kids researchers need to do better evaluations

In my continued campaign to alienate colleagues and generally stir up trouble, here is our journal paper which was just accepted in Journal of Child Computer Interaction. The main point is that IDC is old enough now to be doing respectable, replicable longitudinal research in ecologically valid settings.We need to get away from the emphasis on cute technology innovation. If you want to try embedding your research in a real world classroom setting, in a sustainable way, give our Train the Teacher Method a go!

 

Robertson, J., Macvean, A. and Howland, K. (2013) Robust evaluation for a maturing field: the train the teacher method. International Journal of Child-Computer Interaction. http://dx.doi.org/10.1016/j.ijcci.2013.05.001 Download CCIPaperForWeb (close to final version)

Abstract:

This paper highlights the importance of evaluating educational technology for children in naturalistic classroom contexts. We present the Train the Teacher Method (TTM) which formalizes a method for IDC researchers to use when deploying and validating an educational system. Our key findings indicate the need to work in partnership with classroom teachers, providing both initial training and continued support. This will both result in more valuable research data, and address a gap in teachers’ continued professional development. The TTM aims to ensure that teachers, students and researchers can benefit from innovative educational systems deployed in real classroom contexts. We believe that the TTM could be adapted to other domains of study within IDC.

The paper goes on to argue the more general point that the interaction design and children community should shift emphasis away from technology innovation and towards reporting replicable longitudinal research findings from real world settings. We discuss possible approaches to achieving this, for example through changes to reviewing criteria and by actively supporting data and source code sharing.

UK Students turned into goldfish by social media

[cross posted from my CACM blog]

Apparently UK students get distracted by social networking at least once an hour, goldfish that they are. This rare talent for multi-tasking (or astonishing rate of dim-wittedness depending on your point of view) is unparalleled worldwide. They're also the second most connected student body in the world after Chinese students, as uncovered in a cutting edge survey commissioned by Cisco (who couldn't possibly have brought an agenda to the research). The CEO of CISCO in the UK, Phil Smith, claims that this means that employers need to consider young people's working habits, with more options for working at home and so on. Presumably working at home is advantageous for these hyper-connected young things because their bosses can't see they are using Facebook all the time. The problem with being distracted does not lie with the young people, in Mr Smith's view. Rather “if people are getting distracted from their jobs, it's often down to bad management.” By extension, if students are being distracted by social networking in my classes, this is clearly my fault for not being entertaining enough. However, I believe I can accommodate my learners' needs by having a 2 minute social networking break every 15 minutes, as suggested for films in the recent Orange cinema advert. That should enable them to catch up on any of the really important issues they're missing by being cooped up in a lecture theatre for a whole 50 minutes at a time.

But us old folk shouldn't mock. We shouldn't laugh at the younger generation who evidently find social networking crucial to their existence. One in three students and young professional surveyed by Cisco considers the internet to be as important as air, food and shelter. If you look at it that way it seems harsh of me to give lectures which last so long. After all, I wouldn't expect anyone to go without air for 50 minutes. It wouldn't be humane.

 

The impact of console games in the classroom

[crossposted from my CACM blog]

Futurelab recently published a report on the impact of console games in Scottish classrooms ( http://www2.futurelab.org.uk/projects/console-games). It contains case study evidence from 19 schools in which pupils are involved in game based learning projects, and considers the impact from the point of view of school leaders, teachers and pupils themselves. I find these sorts of studies very interesting because they’re about the way technology gets incorporated into educational practice in the every day messy world of schools.  Most of the console games used in the classes – Guitar Hero, Nintendogs, Endless Ocean- were never intended to be educational, and certainly not in school settings.  They have been enthusiastically adopted by teachers who happen notice their educational  potential while playing them at home. Contrast this to the academic development of interactive learning environments which may be carefully designed to solve a specific educational problem but are often sufficiently obscure that they never see the light of day in a classroom. The adoption of common of the shelf games (COTS) in classrooms is characterised by evangelism. Practitioners – once converted- tend to get quite over excited by it all and joyously sing the praises of such projects. This might go some way to explaining Futurelab’s  bias in choosing to identify the educational benefits of console game based learning rather than the drawbacks or challenges.

That being said, there are some interesting findings in the report which go beyond the excitement about motivating learners.  One is that teachers became motivated by the projects because they could see what an impact it had on their classes. Initially some teachers were worried about adopting games in the classrooms  because they did not see how it would fit with the curriculum, or they were panicked about not being able to use the technology themselves, or because they were worried about parents’ perceptions of it. However, the experience of seeing their learners become enthused about and engaged with their learning convinced many of the teachers that the risks were worth it.  From the perspective of keeping teachers invested in their work and extending their professional repertoire, managing a challenging games based learning project is likely to be beneficial.

A striking aspect of the classroom projects is that the pupils typically spend very little time actually playing the game at school – perhaps five minutes, three times a week. A lot of the classroom activities are based on the game but do not directly involve playing it e.g. balancing a budget for your rock band’s forthcoming tour or writing about how best to look after about your Nintendog pet. It seems like a small amount of game playing translates to a lot of motivational “buzz”. 

Critics of game based learning in schools sometimes argue that playing games in schools tarnishes children’s enjoyment of games at home, that it takes the fun out of it. The children interviewed in these case studies commented that they liked to receive support and advice from their classmates about solving puzzles in the games, and that they liked the increased competition. They also liked being challenged in school. Some children noticed that the titles they played at home were not as good for learning as those at school – perhaps the project taught them to expect more from games?

I’ll end on one of my favourite quotes from an interview with an eight year old who took part in a project based around Cooking Mama. “I burnt a cake at home and I learned that it’s just life – you have to cope with it not going right… and the topic helped us with that and keeps us from getting bad tempers”.  Coping gracefully with failure seems like a great skill to learn at school whether learned through real or virtual baked goods. 

 

High Priestess and learning zone updates

It's been a while since my last (somewhat psychotic) update on the student social area (aka crush area aka Learning Zone) and there have been some interesting developments. You might recall that the furniture was set up and that we were waiting for the screens to be installed and Goat Lady. Both are now here.

Goat Lady

Goat Lady is a reclining size 16 mannequin covered in goat hair and fibre optics. She lights up where you stroke her. She is an exhibit created by the artist Helen Storey and the real name of the exhibit is First, Last and Everything . She is currently on loan to us from our School of Textiles and Design. Originally she was designed to produce pheremones but we had to remove those as I was convinced the equipment for releasing them would be a fire hazard and we also though the pheremones had gone off after months in storage. Here is Goat Lady, with one the lecturers from Textiles and Design.

 
 When we started arranging the loan of Goat Lady, there was some disquiet about how she might offend female students. Or enflame the passions of the male students. In fact, the School of Textiles and Design refuse to display her for reasons of this kind. But you know what? It hasn't been a problem at all. In fact, people ignored her as best they could until recently. You'll see in the photo that I made a sign which says "stroke me" because people would never ever interact with her. At least not in public during the day. A few weeks back someone made another sign to put underneath mine which read "No, it's creepy". Which just about sums up how most people seem to feel about it. I personally am rather fond of Goat Lady. I like to stroke her because her fur reminds me of my big white cat. (Mr Bitey also likes to stroke her presumably because unlike the big white cat she doesn't run away.)

 In the last week events have taken an unexpected turn: people have started leaving offerings for Goat Lady! First there were coins left in front of her at night. Then origami birds. No-one knows why these offerings appeared but it is the subject of much spectulation! It has fascinated the staff in my department and is the main topic of conversation on the staff mailing list.  We hope it removes the need for tuition fees in this difficult economic climate. I have claimed the title of Goat Lady High Priestess, and it is now officially one of my admin duties in my boss's work load database.

Screens

Some of the wonderful PhD students and postdocs took on the task of finding things to display on the screens. Stefano wrote some code to visualise incoming tweets from a twitter aggregator in a pretty way.

 
Pawel is working on a slideshow app which enables users to identify inappropriate content using QR codes. The thing we are now realising about a public display is that it needs a lot of content because it's switched on for such a long time and the material cycles very quickly. We found that the screens were mysteriously being switched off - we think because people were fed up of looking at the same content. Which (predictably) irritates me. There's a message on the screens which clearly explains how to add more content. So in my view if the lack of content annoys you, you should either do everyone a favour and add more, or put up with it, but not passively agressively switch the whole display off. I spoke to some students about it and they seem annoyed by the screens and their lack of content as if their inalienable right to be entertained had been violated. So now they find themselves working on a project to produce content for the crush area. Hehe. I have another group of students working on producing an interactive application using Kinects for user input so we will see what that produces. Maybe some kind of app for GL to keep track of her devotees?

Robot army mobilised against absent students

You know what I'm gonna do? I'm gonna build me a robot to seek out students and bring them to classes.
Here's how. It will sit in the corner of the class and do face recognition on the students present, matching their faces to the photos in the module list. It will get the addresses of those absent from the student record system and use Google maps to go round to their homes and tell their mothers. It might need some help with this bit, but I figure a parrot AR drone navigated from my IPhone would solve this. It will also alert its creepy robot pals attached as web cams to the LCD screens throughout the department so it can pursue those actually on campus.
Having found the perps, it will employ a firm yet gentle regime of mild electric shocks to guide them to campus and into the class for the right time. In fact, maybe I need an army of them, one for each absentee. It would be their personal nemesis, assigned to torment them into turning up with their homework complete. My current favourite form of torment would be to generate infrasound (sound too low to hear) which makes people feel profoundly uneasy. Haunted, in fact.
I estimate that all this effort on developing my robot army would be worth it in the long run. Save me time on sending nagging emails after every frickin' class.
We are the student seeking robot army. Expect us.

Should students buy an iPad or a year's supply of Post Noodles?

[cross posted from my ACM blog]

As the leaves turn brown and the weather becomes crisper, the students are returning to their studies full of resolve and new purpose. When I was a lass, we used to celebrate the new academic year with a new pencil case or perhaps a new set of ring binder folders. Nowadays, the young 'uns like to invest in some shiny new tech. The question is: what to buy? A little netbook (now you can tinker with the linux kernel from where ever you happen to be)? Or an iPad (bound to impress the ladies)? I am a big fan of the iPad, I am the first to admit. But I am not convinced it actually makes sense as a student purchase unless you have a kindly uncle who is bankrolling your university career. Let's assume that you are an average student at my university in Scotland. According to the helpful budgeting guide published by our student support office, you need to budget £6978 for living expenses for an academic year. The current UK price for the iPad is £500, so you would be spending about 7% of your budget on it. A few comparisons help to bring it into perspective. Suppose you gave up washing clothes for a year, or relied on your mum to do it for you. According to the budgeting guide this would save £222, just about half of the iPad. You could go with dirty clothes for your first two years and buy an iPad for third year, I guess. How about food? The budget reckons £1295 for a year. Sheer extravagance! Two Pot Noodles a day would cost under half that at £547.50. Throw in some vitamin supplements at £1.69 and what more could you need? That saving alone would get you an iPad.

If you resisted the iPad in favour of eating or washing, what would you be losing? There has been much dewey-eyed hype about the use of iPads in education. Some universities have given free iPads to their students presumably in the belief that it will either help them to learn or at least convince them to study at their institution in the first place. iPads will transform education, apparently. “It occupies a new space in technology, one which is still undefined to some extent, yet which is likely to be significant over time”, according to an iPad education blog. The new space referred to is the technology niche between the smart phone and the laptop: extreme light weight portability combined with readable screen size and the potential for educational applications. No longer do you need to be stuck in computer labs to work on your essay. You can even stray away from power points because of the excellent battery life of the iPad. No more waiting for the computer to boot up – press one switch and the iPad springs to life, eager to serve your every learning need. Want to share an article you are reading in a tutorial with the others in your discussion group? The screen will automatically switch orientation as it is moved. The App Store can provide free or low cost applications to help you study: anatomy flash cards for medical students or a beautifully animated version of the periodic table for chemists. You can seamlessly listen to or watch podcasts of your lectures or view lecture notes in Keynote. All this on the alluring multi-touch screen, with a sleek sexy interface in the style only Apple seem to be able to pull off.

But hang on. If you didn't have an iPad, you wouldn't actually lose many of these capabilities. Working on a netbook or other laptop might be a less joyful user experience, I grant you. But it is still entirely possible to work on it with others, to listen to podcasts, read lectures notes or take advantage of the millions of educational applications which are available online. Furthermore, if you happened to need to use a page with Flash content, you would be able to view it. In fact, in my five months of using my iPad, I have come to realise its limitations for productivity tasks. I can easily imagine it becoming an excuse for my students to not do tasks in class. “I can't log into the VLE on my iPad”. “I can't edit that file in my iPad with this version of Pages”. “Joe can't transfer his notes to my iPad because we're not on the network and there is no slot to plug in my memory stick”. “Wait a bit.. it's taking me ages to cut and paste between applications on my iPad so I can't keep up”. Kind of like primary school kids make excuses to go to the toilet when they are bored, except the limitations of the iPad could give the students genuine reasons to become distracted from the key concepts of the class. Would there be any particular benefit of using an iPad to Computer Science students? Certainly developing for iPads (or iPhones) is popular with students, and can make for interesting projects for advanced students who want to learn a new programming language and environment. It is also an excellent case study for interaction design modules. For these purposes I would be inclined to say that the university should provide the hardware, rather than requiring students to get their own.

A netbook might be old fashioned and clunky in design terms but at least it will run the standard productivity software which lecturers tend to assume students have access to. They're also roughly half the price of iPads, thus requiring fewer sacrifices of nutrition or hygiene. So on the grounds of avoiding rumbling stomachs and malodorous students in my classes, I wouldn't recommend an iPad purchase on purely educational grounds.

Laptops in the classroom

(Cross posted from CACM)

Do you ever find yourself checking your email during a boring meeting? Do you drift off on a wave of rss feeds when you should be listening to your colleagues? Do you pretend to be taking studious notes during seminars while actually reading slashdot? In fact, should' t your full attention be somewhere else right now?

I find it increasingly tempting to do lots of things at once, or at least take micro breaks from activities to check mail or news. I do think it's rude during meetings so I try to stop myself. My students don't tend to have such scruples. They use their laptops openly in class, and they're not all conscientiously following along with my slides, I suspect. In fact, in a recent study published in AJET (http://www.ascilite.org.au/ajet/ajet26/lauricella.html), 70% of students spent half their time sending email during class (instant messaging, or playing games or other non academic activities were also popular). They did also take notes and other learning tasks, but they weren't exactly dedicated to staying on task. If you're interested in surveying your own class to find out what they really do behind their screens, the authors provide a reliable validated questionnaire tool about laptop usage in education.

Of course, there have always been distractions during class- as the margins of my Maths 101 notes demonstrate with their elaborate doodles. It's just that laptops make it so easy and seductive to drop your attention out of the lecture while still feeling that you are achieving something. ("I simply must update Facebook now. Otherwise people will not know I am in a boring lecture".)

Unsurprisingly, laptop usage in class has been associated with poorer learning outcomes, poorer self perception of learning and students report feeling distracted by their own screen as well as their neighbours' (Fried 200810.1016/j.compedu.2006.09.006 ). Many educators get frustrated by this ( see h http://doi.acm.org/10.1145/1151030.1151049 in CACM for example) and there is debate about whether laptops should be banned, or whether the lecturer should have a big red button to switch off wireless (or electrocute all students) when they can't stand it anymore.

Bear in mind,though,the studies I mention above were conducted in lecture style classes and the students were not given guidance on how to effectively use their laptops to help them learn rather than arrange their social lives. It is possible to design active classes around laptop use ( if you can make sure that students who don't own a laptop can borrow one) thereby making the technology work in your favour. For example, my students learn to do literature searches in class, or try out code snippets or critique the design of web pages. And yes, some of them still get distracted from these activities and wander off to Farmville. But at least I have given them the opportunity to integrate their technology with their learning in a meaningful way. They are adult learners after all. It's their decision how best to spend their brain cells in my class and my job to give them a compelling reason to spend them on computer science rather than solitaire.

   







   







Encouraging girls to study geeky subjects (part 1); emotionally intelligent tutoring systems

[Cross posted from CACM blog]

How can we encourage girls to enjoy maths and science subjects? Is there some way that we can tempt girls across the great geek divide to the point where they consider STEM careers? There are of course lots of possible answers to this. One which I came across recently in an interesting press release (http://www.umass.edu/newsoffice/newsreleases/articles/101058.php) is to develop intelligent tutoring systems tailored to improve maths scores among girls. Woolf and her colleagues at UMass Amherst were interested in providing extra support for girls, and decided to include an emotionally sensitive component to the tutoring system which would give encouragement and help based on the students’ detected affective state. As far as I can tell from the press release, the research results from their field studies are not yet available.

 

The tutoring system is personified with an optional animated agent who offers emotional support and other advice tuned around the users’ emotional state. It turns out that the boys more often turn off the agent than the girls, which is consistent with some of my own previous work where older boys (12 years) were more likely than girls or younger boys (10-11years) to prefer a tutoring system without an animated agent (see Robertson et al. 2004 if you care). Maybe boys don’t want emotional support. Or maybe they do but they don’t want it conveyed through an animated character. Users of the wretched Microsoft Clippy might sympathise with that view point. (“I see you are feeling upset. Would you like me to a) contort myself into a disturbing hug shape b) administer an electrical shock to see if it helps you focus or c) give you an easier maths problem?)

 

But how on earth do you reliably infer a user’s mood from their behaviour when using the system? This is a fascinating research topic in its own right, quite apart from the issue of how the tutoring system then responds sensibly to this information. In the UMass research, they use a combination of sensors and cameras with which “the computer can recognize when students are happy or stressed, fidgeting, frustrated or feeling confident”. They also use cues such as “the time taken to answer questions; number of hints requested and grip on the mouse.”  As you might imagine, there are huge problems in working out which items in this potentially massive volume of data are most indicative of the users’ emotional state. There is also a reliability problem; Woolf and colleagues report 70 – 80% accuracy when using the camera. Is this reliable enough to base a tutoring system action on? What are the repercussions when it gets it wrong? These are interesting open questions.

 

I recently heard a talk by Madeline Balaam (http://www.sussex.ac.uk/informatics/profile124388.html) which had some intriguing thoughts about how teachers might respond to learners’ emotional states (Balaam et al in press). She made the point that researchers have hereto assumed that human teachers know how to respond to their pupils’ feelings assuming they have the information to diagnose them. This turns out to be not necessarily true. In Madeline’s research, she designed Subtle Stones which are tangible devices which high school students can use to convey their emotional state by squeezing the stone to change its colour. The colour/emotion mappings are private to the student (blue might mean happy for me but bored for you) but the teacher has a special display which translates the individual emotion mappings to a standard display, meaning that she can get an overview of the emotional state of every class member in real time. In contrast to Woolf’s system, the users self-report their emotional states rather than having the system infer them from behavioural cues.

I have to admit that I would find it challenging to know what to do if I had access to such a rich source of information about the feelings of my students (beyond the obvious few who fall asleep on the desks). Are they unhappy because they split up with their girlfriend or because I bore them to tears? Is it OK for a learner to feel frustrated for a time if they eventually indicate that they are feeling proud because their work has paid off? I can imagine a first year programming lab class where everyone spends most of the class frustrated and then there are little bursts of pride as people finally get their compiler errors sorted out. What do I do if half the class is happy but the other half is bored? (This is presumably not a problem for an ITS which is dealing with one student at a time.)

 

To conclude: Woolf’s project is a great example of research which tries to solve a real world problem by tackling extremely challenging research issues. As this particular real world problem has an impact on girls’ attainment and interest in maths and sciences, various people in the ACM community will be watching with interest.

 

 

Balaam, M., Luckin, R., Good, J. (In press). Alsmeyer, M., Luckin, R., Good, J. Jump Starting Affective Communication in the Classroom with the Subtle Stone. International Journal of Learning Technologies.

 

Robertson, J., Cross, B., MacLeod, H., and Wiemer-Hastings, P. ( 2004) Children's interactions with animated agents in an Intelligent Tutoring System. International Journal of Artificial Intelligence in Education. 14.335-357.