Online assessment

Sorry... I forgot I was supposed to write things here :)

And thanks to David Cox's recent post on ExamView to prompt me to return to writing ... not to mention that I'm at Educon this weekend and everyone there is a prodigious blogger.

But back to ExamView. ExamView seems to be another online assessment tool and David writes how he is using it in his classroom and how he plans to use it. I've been using an alternate tool, MapleTA and had considerable success with it.

I think the strength of MapleTA over the other assessment tools is that it is based on the computer algebra system Maple -- and therefore, when you ask a question or the student enters a solution, it can involve any manner of mathematical content. Not just numbers but also algebra and graphs. And we managed to jury-rig Geometer's Sketchpad to provide dynamic diagrams (yes, we'll be switching over to GeoGebra when I find some time).

I think the most important aspect of online assessment is the use of a mastery protocol. A student can continue to practice their skills until they achieve some level of success. Obviously these are skill-based questions; yes, it's a technological drill-and-kill. But I'm okay with that. It's the efficiency that attracts me - a student who know how to factor will breeze through the questions and can stop. A student that is challenged can attempt the problems without penalty until they become proficient. The assessments are set up to show the worked solution (not just the answer) so they can see how others would have done the question. The frustration is something you have to work through with the student -- I keep track of the gradebook and can see which questions the individual students are challenged by, what their attempted solution and provide some prompting by email (thank you Jing for screencasting quickly) or in class.

There are issues of course. No one knows if your dog is doing your homework on the internet. There is a level of frustration when the student keeps getting the wrong answer -- but this is something that happens with regular homework and they can give up too easily on paper. And, when it comes to inputting algebra or matrices, say, online, that can be a challenge.

And the biggest impediment -- students don't read the question. While MapleTA understands that 2(x+3) and 2x+6 are the same answers and will mark both correct, it will not accept x2 for x^2.

Getting the word out on GeoGebra

Maria Droujkova has done some great work putting together some Elluminate sessions on Math 2.0... and she has more to come. On Saturday the 26th she had Markus Hohenwarter, the father of GeoGebra and the chief developer Michael Borcherds on for an hour discussing the past, present and future of GeoGebra. She recorded the session and it's available online.
What surprises me is that I still run in to teachers that have never heard of GeoGebra -- here you have free, open-source math software that almost any computer can run, it's multi-lingual, it's being used worldwide at all levels and has thousands of lesson plans and activities available on its wiki. And yet today I spoke to two Masters students who had never heard of it.
In Ontario, it's problematic since we (well, public and Catholic schools) have software purchased for them by the province and that set includes Geometer's Sketchpad. Now, GSP is an extraordinary program and we owe a great deal to Key Curriculum Press and Nick Jackiw but the development and growth of GeoGebra is a reflection of our brave new world -- collaboration on a global scale, the harnessing of our energies to support people we will never meet. What I do in my classroom can be given (instantaneously) to a classroom in Thailand, Kenya or Uruguay... and vice versa.
So how do we spread the word more effectively? How do we ensure that every preservice and practising teacher knows not only of its existence but also the community already formed?
And, most importantly, how can we port it on to an iPhone? :)

Coaching

As I mentioned in an earlier blog I was at the September meeting of the Math Forum; the theme for the meeting was coaching.
There was considerable disapproval of the term coaching; that it set up a hierarchy of ability or skill, that it brought up visions of movie-football coaches berating their athletes. The word facilitator was proposed as something more appropriate. But what a banal, uninspiring word.
I however suggested that coach was the right word -- so long as we envisioned it as an Olympic-level coach. An Olympic coach works with athletes that already have considerable ability; there's not a hierarchy, in fact, the athlete has the spotlight, the fame, the medals. The coach of an Olympian is a specialist; he doesn't focus on every football position but emphasizes one activity at considerable depth. It's not that the coach is the better athlete, it's that the coach has the knowledge and skill to help the athlete reach great competency and the background to be credible. The coach knows how to communicate, to decide the right next step, to plan the process to get the athlete to the next level. He sees the big picture; it's not just the athleticism but the diet, the lifestyle, the mental attitude. He knows when to use the soft touch and when to put his foot down.
It's certainly what I hope I achieve when working at PCMI - these are already good teachers who are looking to improve. It's a challenging role, and as much sleep as I miss or stress I endure I do enjoy it. There's not so much an opportunity at my school, where there's neither time nor appreciation for such a process.

Respect. It's not what you think...

I'm an occasional participant at the Math Forum at the Fields Institute in Toronto. It's a meeting of folks interested in math education research held monthly; I'd get there more but academic and other responsibilities often overlap. Even today I was supposed to be at school for Homecoming but it's been a year since I made it and the topic, on teacher-coaching, was well worth it.
At lunch, I sat myself amongst some folks I didn't know and the conversations ranged wildly. At one point, the conversation turned to how teachers had lost the respect of the public, that it was different in the past, and so on. Blame was placed on the former provincial government for taken an aggressive and demeaning approach to teachers. And I'm certainly not denying there is some truth in that effect that government had on the perception of our professionalism. But there's more to it than that.
The woman who initiated the conversation gave the example of a parent who had called her with a question. The teacher was quite offended that the parent said that his son "Chris doesn't believe you're helping him enough." Now, she even corrected herself when she changed the word "believe" from "think" and how she then explained what extra help options were available to Chris. I didn't get a chance to add to the conversation because another tablemate (thankfully) quickly changed the topic to the pronunciation of certain Swahili words.
This teacher seems to be mistaking respect with obeisance-- she seemed indignant; the parent had no right to ask her a question about the instruction in or out of her classroom. I even think the parent phrased the question respectfully; the teacher could have quoted the parent with "I don't think you're helping Chris enough" but the teacher was specific in how she remembered the conversation, the parent was already placing the responsibility for the misinformation on the student.
Our classrooms, our instruction, our approach, our philosophy should not only be clear and open with our parents but also open to being questioned -- the wonderful thing about the age of communication is the opening of discussion. And not just discussion -- the simple distribution of information on homework, assignments, testsextra help times. I still remember a time when you would go to the doctors and take their direction without questioning. Not nowadays -- there are other perspectives, updates in the field that an interested participant may bring to the table.
I know some of my parents aren't happy with my approach to mathematics teaching. They want pat formulas & algorithms that will help them help their kids at home; they don't want to see their children struggle with hard problems or not know all the answers when they used to in previous classes. They want to see worksheets and pages of questions like they remember. They want marks to be added up and averaged. And I understand their concerns and I'm always happy to take time out to explain the hows and whys of my choices in our classroom. Their questioning is not dis-respectful; in fact, I think it's part of their parental responsibility to question if they have concerns.
What is disrespectful is not supporting the teacher outside the school. Like a couple with shared custody, we have to work as a team and can't be disparaging of the other, even if we don't necessarily agree with them. It's not always easy to share custody but it is possible.

Math Video Markup

As I mentioned in an earlier post, my students will often be required to submit Jing videos of their worked solutions to a variety of problems. Basically it's the modern alternative to handing in paper copies of their homework but I get their voice, literally & figuratively, describing the solution with all the steps in-between. I think it helps to reinforce the importance of process over final answer since they have to go to all the work of explaining what they're doing and why, and also allows me to reinforce correct mathematical language.
When it came to providing feedback to the students, I've had to rely on just an email response, describing in text or providing a full worked solution in Jing on my own. What I'd really like is what we have for paper -- returning it with the markup on the product. Jing of course lets you mark up the image capture but what I need is video mark up, like they do on ESPN to describe football plays. There's this neat little website www.markupvideo.com that does this for YouTube videos but of course, I'd like it for Jing videos (it's all Flash anyways, eh?) Our PhysEd department has Dartfish but this seems more like LoggerPro on steroids and a bit more than what I need to mark up homework. But, we'll give it a try ...

I think I want to SMAK my kids

I've been thinking about how to assess my Grade 9 and 10 students this year ... I did a lot of experimentation last year with my accelerated Grade 8 students;they were open to trying things out since acquisition of skills and open-ended problem solving was right up their alley. Here's one change I can make:

So, there are four units in both MPM1D and MPM2D ("Ontario" for Grade 9 and 10 math respectively). For those paying close attention, there are only three in 2D but I break the Quadratics unit into two pieces. Beyond the typical written assessments (test and exam) that we're required to do by the school -- we have both Christmas and June sit-down exams -- and the evaluations & projects that the other teachers determined while I was in Utah, I'd like to introduce a SMAK at least once for each unit: Show Me Application and Knowledge. Yeah, it's a lame acronym... I'll try to think of something better.

My idea is that each student will choose a 10 minute period outside of classtime in which to show me their understanding by explaining pre-assigned or randomly chosen questions and by just explaining the important topics of the unit in their own words. I'd also like them to reflect on their learning process, homework, participation and all those other bits & pieces of our classroom. Pretty open ended on both sides of the conversation but I really want to evaluate their understanding of each unit based on a chat, 1 on 1. I tend to collect a lot of anecdotal observations (thank you iPod Touch!) in class during kikan-shido but this will provide me and them with a personal video asset for each student (oh, did I mention I was taping them?) I also haven't found blogging to be particularly effective in my classes and want to have a good record of their "voice".

I'm going to have to structure this: first, start by taping myself several times discussing learning & mathematics (first one, "Welcome to class") and having them use these as a model for their first SMAK. Using Jing to explain their problems from homework will also make them more comfortable in verbalizing their explanations (and hearing their own voices).

And this will be graded using a rubric. Because each SMAK will be unique in content there's no realistic marking scheme and the marks given will be for completion, thoroughness and quality. If one student spends most of the time given a great solution to an application while another devotes most of it to explaing how they finally learned how to factor, clearly explaining how they grew as a student then both would be graded highly.

By the end of the year, I'd like to have them create their own video ra

Image by mstephens7 via Flickr

ther than have me sit there for 10 minutes with them, guiding them through. It wouldn't surprise me if, as they realize I'm taping them, they'll suggest it themselves. Hopefully they'll have seen enough examples to gauge the depth I'm looking for and the breadth of options when it comes to their explanations. We'll see.
Is this feasible? My class size is usally about 16 kids: that's 160 minutes x 4 = 640 minutes = 11 hours per year. That's about the time it takes to mark a set of exams but spread across the whole year. Times three classes, of course.

Social Technology & Education @ Harvard

Sandwiched between two great motorcycle rides through upstate New York & Massachusetts, I attended the Social Technology & Education conference put on by the folks at Elgg. They held it in the Radcliffe Gymnasium, a former gym converted into a very elegant discussion space.
The conference evolved organically: people volunteered to present and participants came from a variety of academic, medical, non-profit and commercial situations. There was little advertisement and people heard of it through word-of-mouth (okay, well, Twitter). Now, unfortunately, almost 280 people signed up but not everyone showed; I think by making it free, people felt they could sign up, take a space and not show. Always have a nominal fee, just to show some level of commitment!
The presentations were varied so I'll pick out the high points for me; given my background, a lot of it covered issues we've already had under consideration for a while.

• It was a real pleasure to meet Dave Tosh, who despite his Scottish accent hails from Oshawa of all places! His most important reminder for me was that "Just because they use Facebook doesn't mean they are tech savvy... their mates are on Facebook so they are motivated; they're not motivated to do your site" So not only do we need to ensure they have a reason to use our online tools we also have to provide some level of training and support; it won't be automatic because the students (faculty, staff & parents) don't want it or need it to be.

• Real innovation comes when we take something for granted ... Christopher Sessum's presentation mentioned this, and apparently it comes from Clay Shirky's book Here Comes Everybody. I haven't read it but picked it up from Chapters when I got home from Boston. Sessum's notes and presentation are here; he has a similar presentation style to mine, so you'll need to read the notes. I can remember when we first started out at RCS and discussed this issue with Paul Kitchen. We wanted the laptop to be as fluid to the student and teacher as the pencil or chalkboard was. We were only a decade & a half ahead of our time.
  

• Christopher's (and later) presentations mentioned Etienne Wenger and Keith Sawyer a lot: I haven't done a lot of reading that discuss the development of communities of practice so they're now on my reading list. Developing communities is a lot of what we are trying to do with PCMI and so reading about the progression of professional learning networks has become important.
  

• Shelley Blake-Pollock, from TeachPaperless ran through his work with Twitter. Shelley takes a more blunt approach than I'm comfortable with although I think we perceive the end result similarly.
  

• Liz Davis did an excellent rundown of Ning; she's convinced me to use it for my courses (if I can't get Elgg up and running in time). We're using it right now for the PCMI group but Liz has given some great examples in her classrooms. There are some limitations, in particular using mathematics, but it's really the conversation and discussion, not notation.

• Jim Klein showed how his district in Canyon Country, CA used Elgg as a structure to build a community of faculty, staff, students and parents. I'm not sure whether or not his theoretical understanding of the process parallels Wenger & Sawyer (I've got read them, first) but the practical outcomes that he showed, linking students from across grade levels and subjects, speaks volumes. I can only make linkages between my own classes and classes outside my school but I think beginning a conversation with a larger academic community is important. I'd love to be able to use a tool like Elgg in this fashion but it would require considerably more time to develop & program than I have. Hence I'll likely be using Ning.

For a one day conference, there was lot of excellent discussion. That it was put together so quickly and with little budget gives me hope for things we have planned in the future!

(Most of this post was lost thanks to my crappy Dell tablet... I'll come back and relink things tomorrow.)

Working Groups

Before I start dealing with reflecting on the content of the classes, I've got two more aspects of PCMI to mention.
The first is the most productive: The working group. Each of the teachers is assigned a working group in a topic of secondary mathematics for the afternoon (Wednesdays off) in which they, typically in groups, will produce a product useful to classroom teachers.
As the person in charge of a group this can be very challenging: these are all energetic, enthusiastic and talented teachers -- who all teach in very different classrooms. So what may be appropriate for one school system could fail utterly in another, not just in terms of content but departmental expectations, school standards, etc. As the working group leader I have to steer these folks towards a consensus: a project that is meaningful to them, useful to others, and able to be accomplished in three weeks. Most of the time this takes the form of a lesson plan or activity that is refined throughout the three weeks -- I find that too limiting and I'll discuss what we did in a later post. I will say my group this year (go Discrete Math!) took on a huge challenge and did an amazing job; I was overwhelmed with how they took on their responsibilities and always questioned "how can we do this better, or different?"
We also spend some time looking at different problems in the mathematical area and we're always fortunate to have 200 world-class mathematicians running around in the corridor (well, they don't run so much as shuffle) to snag for a few hours. It's funny when you speak to them at lunch and then after lunch realize WHO they really are. They typically stride the mathematical world like colossus and you've asked them if they liked the carrot cake! :) We were lucky enough to have Joe Malkevitch (yes, THAT Joe Malkevitch) spend almost two hours discussing problems with us -- starting with the Art Gallery problem and then seeing where that took us. That is how lucky we are at PCMI!
The other group of activities I have to mention are the cross-program ones. This is a huge umbrella and can cover things like I mentioned below, James Heibert discussing the TIMSS Video Study results, at least two Clay Scholars every year discussing their work (with us! High school teachers!), Gov. Huntsman (at the time) speaking of math at the state/national level, and even Tom Garrity explaing how "Functions describe the world". The level and content varies so greatly, an exhaustive list would be its own (rather dull) blog post. Suffice it to say, it's the kind of opportunity you would have to hang around Harvard for, for at least a few years.

Reflecting on Practice

Once we're done the morning of math (with a brief coffee break) the teachers all get back together for an hour of math education pedagogy. Like the mathematics we cover, each year is something a little different. For example, in previous years we've focused on Lesson Design (with Drs. Nicole Bannister & Gail Burrill), Teaching through Problem Solving or Learning the Open-Ended Approach (with Dr. Akihiko Takahashi).
This year the organizers tried something a little different; they tapped six of the returning participants to look at Questioning in the Classroom from the practicing teachers' perspective. As one of those teachers leading the professional development it was a considerable challenge to not only meet the expectations of the participants and the organizers but also our own expectations -- my colleagues are amongst the premier educators in the States (National Board certified, AP consulants, you name it). We began with a working weekend in Denver in the spring, pulling together resources and a timeline -- our biggest fight was avoiding putting too much in. And then, when actually talking about pedagogy with professional teachers there is a huge struggle against anecdotes; everyone wants to share their stories. In discussing Questioning we want to move beyond what we do now and move towards something better. And so we start with what the research said.
This blog post is only to set the scene for a series of posts; I will go into this at greater depths in the future but our motivation was the results of the 1999 TIMSS video study -- James Hiebert presented the results to us in 2003 at PCMI and it was the most astonishing moment I've had in a lecture in a long time and it has been the prime motivator in my teaching ever since:

Almost all (ed: statistically 100%) of the problems in the U.S. that start out as making connections tasks are transformed, in a variety of ways. Often a teacher steps in and does the work for the students-sees students struggling, gives a hint that takes away the problematic nature of the lesson, and tells students how to solve it. These are not incompetent or poorly intentioned teachers but simply teachers who have picked up very well an American way of teaching mathematics. One of the cultural agreements we have made in this country, with ourselves as teachers and with students, is that it is the teacher's job to tell students how to do the problem and how to get the right answer-that it is not fair to allow students to struggle or be confused.

In other words: we are far too nice. So, for the past six years I have worked hard not to be nice and tried to persuade colleagues near and far to cowboy up¹. I've presented on this at OAME directly and in any other presentation that I've done I've pressed the point. It was encouraging to see Dan Meyer come to a similar conclusion in his presentation to open source programmers (yes, the context is a bit bizarre but makes sense if you follow his blog). Be sure you should watch the video.
_________
¹I include "cowboy up" only because I had to explain the phrase to Gail this year :)

Graph is created from data produced in the TIMSS video study and is from here: http://www.mathforum.com/pcmi/hstp/sum2009/reading/Hiebert_Improving_Math_Teaching_2004b.pdf

The 830 at PCMI

PCMI is a 3 week program; each day from about 830 to 1040 we have what can best be described as a math class. But it's unlike any math class most people have ever had.
Each day starts with its own problem set designed by the class' organizers, folks from the Education Development Center and Harvey Mudd College. The problem set is well structured, beginning with a simple idea or concept and then continually developing in both depth and breadth, although this may be obvious only several days later. The questions are also in categories: Important (things you'll need to know for upcoming days), Neat and Tough (can be really tough! Clay Prize tough!) -- we aim to get through at least the important stuff in our morning together.
The classroom is composed of 12 tables of 5-6 people each (we do have guests from the other programs) and as a table we tend to worth through things together; there's a table sandbox monitor who is there to ensure that the teachers exercise all those collaborative skills they try to encourage with their students. Not only that, but we never tell people ideas, we create a situation in which they can they discover it themselves. This is not easy and like any skill takes practice and continual reinforcement. It is at the heart of the whole morning class (indeed, of PCMI) and the mathematics could almost be the motivation for appreciating this whole process. It's why I call them "organizers" above and not teachers -- it's not instruction as you know it.
The math is very accessible and very deep - low threshold, high ceiling - and it is too easy to look at it only superficially. Teachers will occasionally race through the questions to get them done (remind you of any students?) and will miss out on the complexity of the mathematics. I remember my first year doing the same thing.
As one of the participants said "I've taken courses in number theory but never understood prime numbers until now." This has been true for every topic I've encountered at PCMI -- teachers seldom get the chance to think deeply of simple things that Al Cuoco of EDC, and one of the course's authors, encourages.
If you visit PCMI @ the Math Forum you can click on Class Notes to read over the problem sets from previous years. Or, to get a very insufficient glimpse of the questions, the MAA has a book of Al's work Mathematical Connections that includes material we've looked at during PCMI. It's condensed (remember, we get three weeks) and doesn't have the same level of personalization that our questions set have -- the authors adapt the problem sets from day-to-day to build off of our ideas, suggestions, questions & comments.

Precursor to PCMI

I've had the opportunity to come early to Park City and help set things up: there's actually a lot of infrastructure to put in to place. With (at least) 7 different groups running simultaneously around the conference centre, there's the usual classroom/lecture facilities to complete but nowadays we add on a considerable amount of technology: LCD projectors, wireless & wired networks, speaker systems, the typical. And, because we're mathematicians... a lot of chalk boards and coloured chalk. Lots. And old school overhead projectors.
But in the teacher room, because we've got at least 60 participants spread across 12 tables, we have a desk-based microphone/speaker system so that they can hear each other across the room, two Mimio electronic whiteboards (an excellent alternative to Smartboards!) tied into an ELMO document camera and three LCD projectors and, because we break this large room up into three smaller rooms, the need to have it all work as a common space and as separate rooms. Lots of cabling criss-crossing the room that has to be taped down.
So that's my first week - the participants all start to arrive on Sunday. I'm lucky that there are a number of returning folk along with the rest of the staff; it's good to see old friends! Most folks have the chance to come back for a second year -- and if you come back for a third you're conscripted to help out working with the new teachers.

Going back six weeks...

So the end of school was a bit of a flurry and I left meetings early to head out to Park City, Utah (home of the Sundance Film Festival) to participate in the Park City Mathematics Institute for the seventh time. If you're a math teacher and never been... you're missing out!
I first attended PCMI in 2002 -- by pure luck. I was teaching Ontario's Linear Algebra course and stumbled across their webpage which discussed that summer's topic, Gaussian Integers. I cross my fingers & applied. After attending as a participant for two years I got invited back to help out as staff. It's a lot of work and I don't get all the fun that participants have but I learn about math and teaching and learning in a different way. And I get to work some amazing people, both staff and participants, and great friends.
PCMI is hard to describe. I call it "math camp" when asked just to make things easier. Let me try to be more descriptive since I have the time: PCMI is a three week residential program that has about 60 teachers participate in daily 2.5 hour problem solving sessions that build around a topic, an hour of pedagogy, a 2 hour small working group session in the afternoon on a topic specific to the teacher's classes and a variety of afternoon and evening sessions, lectures and activities on recreational or research mathematics.
While the teachers are doing their thing, there are also about 250 undergraduates, graduate students, university faculty and research mathematicians doing their own courses & lectures on a specific theme, usually tangentially related to the teacher's morning problem solving topic. For example, this year's topic was L-functions -- this is a cutting edge area in number theory (and is the hot new thing in cryptography). Next year, it's image processing. The addition of all these "real" mathematicians running around (and these are sharp folk... Clay Scholars, Fields Medal winners, Nobel laureates -there's no math Nobel but sometimes the topics cross science/economics boundaries) lifts the matheamatical conversation and is an important reminder that math is continually developing... and is crucial to both our day-to-day life and to our future. Plus all these smart folks reminds me what it's like to be a student in my class...
The applications come out in the fall... if you're a math teacher, you should apply. Three weeks is a long time but the Park City area is beautiful, the PCMI teacher community is amazingly supportive and the math is a lot of fun.
Over the next couple of weeks I'll describe what went on this summer at PCMI. I did twitter throughout so feel free to Twitter Search but I didn't have time to blog.