Over the past four years, discussions about grading and assessment have turned towards questions of equity and whether certain traditional practices inherently advantage students from particular circumstances. Though valuable in disentangling behavioral from educational dimensions of performance, these efforts tend to ignore the potential of the time dimension to convey meaningful signals when assessing students, whether for summative or formative purposes. While allowing students to revise and resubmit assignments and exams until they attain the desired result has merit, it runs the risk of losing the information inherent in the time dimension.
Both learning and the demonstration of learning are time-extended processes. The former involves practice and repetition, resulting in a change over time. The latter typically consists of taking an exam or demonstrating a skill over a specified period. These activities will generally occur within the context of a course or a lesson, themselves things that span time. While timed examinations are ubiquitous in education, time is seldom considered in the context of student work process. Educators limit themselves by looking at work product detached from work process when assessing students. Time is the missing dimension.
Time and Traditional Examinations
There are reasons to impose time constraints on exams. Just as constraining the use of outside sources or the number of allowable words, constraining time provides a different type of insight into a student than the non-constrained question would. Time can be an essential element or irrelevant depending on the goal.
And yet this is only scratching the surface regarding the potential use of time in evaluating student knowledge. While a completed exam may capture a student’s knowledge state at a particular moment, the exam alone, divorced from the time dimension of its production, lacks essential information about the student’s work process.
The goal of assessment is to provide a clear signal about student understanding and capability. Whether this signal is used to optimize learning or convey information about a student to future educators or employers, the importance of time goes far beyond its role as a constraint.
Four brief examples illustrate the importance of time in this context.
The Sequence Of Response Matters
When determining whether a student practicing a given topic has mastered it, it is common to look at the percentage of questions answered correctly, equating mastery with a score of 80% or better. Consider two students answering questions, and let R and W stand for right and wrong. After eight questions, we might see an initial sequence of results as follows.
S1: R W R W W R R W
S2: W W W W R R R R
Both students have gotten 50% of their first eight questions correct. While the first student still has some confusion, for the second student, after the fourth question, the lightbulb seems to have switched on, and now she is off to the races. To satisfy an 80% threshold for mastery, she would have to answer the 12 more questions on that topic correctly, wasting time and turning learning into drudgery.
The Speed Of Response Matters
Consider three students answering 20 questions on an exam, all of whom scored 50%. Imagine that the average time each student spent on each question was as follows:
S1: questions answered on average two standard deviations faster than the mean
S2: questions answered on average two standard deviations slower than the mean
S3: questions answered on average at the mean
In the first case, the student needs someone to tell her to slow down. In the second case, the student may be struggling with language or missing important prerequisite information, so a score of 50% might be a heroic effort. The third student’s challenges are likely conceptual. Even though the students have the same score, the meaning of the result is very different.
Learning Is Best Measured By Where You Finish, Not How You Start
Consider a course with three comprehensive exams given at regular intervals. Suppose we have three students in this course who produce the following sequence of results on their exams.
S1: A B C
S2: B B B
S3: C B A
A traditional class might award a final grade of B. Someone looking at the final results would assume that all three students had the same level of understanding. Yet this is likely not the case. The first student started strong, maybe because he was familiar with the material from an earlier course but failed to keep up with the development of the material. The second student understands some elements, but not all, though he is moving along at a constant learning rate. In the third case, the student likely began the course confused, understood the material sometime in the middle, and was doing quite well by the end. Without including the time dimension, these students look the same; but when time is considered, it is clear that the third student has a much deeper understanding of the material. Moreover, from a character perspective, the third student has demonstrated resilience and an ability to come from behind to master something hard, essential traits for whatever their future may hold.
This type of situation often occurs in courses that span an academic year. In the case of a student who has a B at the end of the first semester but who finishes the course with an A, it may be that the A alone provides a better record of what he knows than the B snapshot taken in the middle of the year. Yet too often, high schools report these snapshots without such distinguishing information.
Work Process Conveys Important Signal
Consider an essay exam where students are given 90 minutes to analyze a set of documents and write an essay defending a thesis they formulate. In a mathematics exam, students are encouraged to make every step explicit. In this type of writing assignment, a student might be required to document their intermediate steps as well as produce the ultimate essay. More is needed, however, to really understand their work process.
Knowing how much time was spent on the initial reading, how much time was spent producing an outline, and then how much time was spent drafting the essay provides insight into the student’s thought process. Moreover, knowing whether enough time was spent reading and reflecting, whether the outline was produced before the writing began, whether the writing was a continuous fluid process, whether it was marked by fits and starts, or whether it was rushed out all at the end of the exam period, provide insights into the student’s conviction of the correctness of their work.
When it comes to capturing process, showing your work is only scratching the surface. The real work is only shown when the process is captured and this requires the dimension of time.
So Why Is Time Is Ignored?
Why don’t teachers pay more attention to time and work processes? Two reasons: class size and technology. Teachers can pay attention to the process when the number of students is small enough that they can watch students while they are working. In private tutoring, or during faculty office hours, it is common to ask a student to solve a problem, and then by watching the stops and starts, gain insight into a confusion that the student may not be able to articulate. But once a classroom has more than a handful of students, assessment inevitably moves from observed live demonstration to statistical sampling via work product. As for technology, while pencil and paper preserve the residue of work, they lose the process of its production, making meaningful use of time impossible.
Better tools for capturing and analyzing work processes are needed. Creating such tools will be challenging since unless they are natural, their use will not provide insight into student thinking as much as into the tool’s design. One example of such a tool would be “smart paper,” which tracks what a student writes and the time dimension around the writing. Another example would be an AI agent that observes the process of student work and extracts process information, producing a time-coded narrative summary of relevant information or taking steps to intervene, depending on context. Beyond capturing this information, tools will also be needed to analyze the findings and the feed analysis back to teachers in a manner that allows for ready action without requiring extension interpretation.
The signal is out there. What is missing are the tools to capture and process the signal so that it can be more than an academic curiosity. The time to create such tools is now.
Credit: Source link