online high schools: a disappointment?

Initial results from Washington's online high schools, at first glance, seem disappointing:

According to a state report released last month, nearly half of the students taking online classes in 2008 failed with an D or F grade.

Also, online school is not a get-out-of-WASL-free card. Full-time online students must take the test in person at a testing site set up by the school. Part-time students who take one or two classes online still test at their home districts.

Statewide, several online schools have a hard time getting their students to show up for the test, which results in mixed performance reviews.

In the six online schools the state studied, fewer than half of sophomores passed the reading WASL last spring, compared to 81 percent statewide. Less than 20 percent of those sophomores passed the math WASL.

Online school officials say the report is flawed. Students who skipped the WASL counted as a zero, which dragged the school average down.

Take out the “zeros” at Washington Virtual Academy for example, and 77 percent of their sophomores passed WASL reading. In math, 31 percent passed compared to 45 percent statewide....

Online school students at those six schools took the WASL 64 percent of the time, compared to nearly 98 percent statewide, the report said.

I'm pretty sure that the results can mostly be explained by the way online school is currently employed: as a second chance for students who've struggled in a traditional environment. A 50% passing rate, then, might actually represent a genuine success. We'll have to see longer-term results from districts that have a mixed approach. From a cost-benefit perspective, the lower infrastructure and instructional costs, even if the passing rate stays flat at 50%, might still make the project worthwhile.

standards-based grading in Spokane

Spokane's elementary schools will now employ a 4-point grading scale based on state standards, the local newspaper reports.

Most elementary students in Spokane Public Schools are seeing their new report cards for the first time this week; instead of A, B, C, D or F, it’s 4, 3, 2 or 1...

Instead of one letter grade for a whole topic, the numbers correlate to specific elements of learning within that topic. Instead of a grade for “writing,” for example, a student might receive separate grades for “writes in complete sentences” and “understands punctuation and capitalization.”

Spokane tested the grading scheme for three years before implementing it district-wide, a smart move.

I hope someday high schools adopt a similar scheme--one that translates directly into a grade point average, rather than the percentage-based adjustments we use now.

visualize data, visualize success

Blog-neighbor The Science Goddess, who is leading the charge in Washington state toward standards-based grading, shares some of her research-based data visualization practices. The upshot:

When I look at this with my teacher eyes, I see so much more of a story appearing about each student. It is no longer a sea of numbers. Now, these fancy-dancy charts won't help me know what to do next (e.g. If students are still below standard, what should the intervention be?), but it may be a better start for identifying issues.

Absolutely. I'll go one step further:

Have students visualize their own data.

Google Docs offers a basic spreadsheet program with enough chart-generating bells and whistles to make it effective for student use, provided enough teacher input. Here's how I set it up: first, I create a spreadsheet with a title row, formulas, and a blank chart inserted. Then I make copies, renaming each after its intended student, and share that copy with that student.

Then, with a little guidance, I have them input data that they've recorded on paper--gotta have a backup!--and the chart appears as if by magic.

It ends up looking like this:



I'll report back at the end of the semester as to whether it's an effective strategy for tracking progress in reading fluency. My gut says it's working, but then, my gut also thinks bacon is a food group.

Update: The Science Goddess adds Part II, with a sample report card.

grades are arbitrary?!!

Yep.

A "D" average is good enough to earn a diploma, according to the Seattle Public Schools.

The district is considering dropping graduation standards from a "C" average to a "D" average in order to boost the number of college-ready students.

The school board is doing its own math on the district's logic.

According to an administrator, the district wants to offer more advanced classes and raise the bar. In order to do so, it needs money. To get that money, it has to find a way to keep more kids in school. So a "D" might be good enough to graduate.

What does a "D" mean?

Some places, you pass with a C. Others, you pass with a D. Some places, a C starts at 70. Other places, a C starts at 62.

Yep. Grades are arbitrary.

Which is what makes arguments over changing grading schemes so much fun.

giddy with anticipation for the last time

For most, Christmas comes twice a year: once in December, and once in July, when Santa descends from the North Pole to deliver mattress sales.

For Washington educator-types, Christmas comes a third time, in August, when the state releases WASL results.

For one last year. After this, we'll have to get giddy about the High School Proficiency Exam, or HSPE, which, quite frankly, doesn't roll off the tongue.* What else does the future hold?

Students in third-through-eighth-grades will take the new Measurements of Student Progress. It will be shorter than the WASL.

About a quarter of the state’s sixth-through-eighth-graders are expected to take their exams online next spring, according to OSPI.

Dorn plans to have the majority of state testing online by the spring of 2012.

And who knows what the test will be like / called by then.

*Any experts at OSPI care to tell us how to pronounce the shorthand version? Is it H-S-P-E? HissPee? HizzPeh? H-Spee?

a little love for the WASL

This morning's Seattle Times offers an article on a recent survey of Washington teachers regarding the soon-to-be-obsolete WASL. An unfortunately small, probably unrepresentative sample reports that, yes, there are some good things about having a statewide standardized test that values critical thinking.

Teachers echoed many of the same old criticisms of the WASL — it's too long, the results are confusing and don't come back in time — but they also credited the WASL with improving students' writing and reasoning skills.

They pointed favorably to its "extended response" questions, which are to be eliminated from new exams favored by Randy Dorn, the new state superintendent of public instruction who campaigned to replace the WASL.

The new tests, to be introduced next spring, will continue to have some short-answer questions but will be largely multiple-choice.

That will be true for the state exams given to 10th-graders and those for students in third though eighth grades. The only exception will be the writing section, where students will still be judged on the quality of short essays.

Researchers found that most of the teachers they surveyed wanted to improve, rather than replace, the WASL. The difference is tough to discern. At some point, we enter into Ship of Theseus territory.

And what about the upcoming non-WASL?

[Superintendent Randy] Dorn is working to offer the test online; math sections will be updated; and the superintendent's office is working on classroom tests that would allow teachers to diagnose what help students need.

Missing: by 2014, math tests will be end-of-course exams directly linked to instruction in various subjects, since not all students take the same math sequence. (The way things have gone, who knows what'll be required by then, anyway.)

The key change is the new, technologically-mediated approach to the upcoming non-WASL's diagnostic capabilities. If the test, as its proponents claim, validly points out deficiencies in instruction, then that information needs to be in schools' and teachers' hands within days, not months. (And, as I argue elsewhere, a better diagnostic test might not even need to be linked to graduation to be effective.)

Last, if we can save at least ten of the sixteen hours currently spent administering the 10th-grade WASL, I'll see that as a win.

deconstructing merit pay

Ryan at I Thought a Think explains why merit pay is never a simple matter.

So, who is your Most Valuable Teacher?

Is it Teacher A, who added the most value to her class over the course of the year?
Is it Teacher B, who had more of her kids meet the year-end goal?
Is it Teacher C, whose class scored the highest in the spring?
Is it Teacher D, who turned around more failing kids than any of the others?

"Value" is a homophone; there's the value signified by the numbers, but there's also the values of the school, the district, and the state which have to be superimposed atop any effort to link the data to the teacher. If the incentive pay/merit pay/whatever pay in this case goes to only one of the four teachers, you're making a statement about the value of the work the other three did, and it's a pretty lousy thing to say to the other three who also made progress that their success didn't matter as much.

And, of course, there are even more fundamental assumptions at work:
1. That the differences across teachers are statistically significant. (In small sample sizes, chance is magnified.)
2. That there are no mitigating factors that better explain students' growth within and across classes. (How much is due to good ol' maturation? Are all relevant factors controlled for?)
3. That the test measures something important.

We run into more trouble when we deal with mobile populations, or when we consider high school teachers who see their students for only fifty-five minutes a day in a single subject.

This is not to say only nay to the prospect of performance pay for teachers. I'm sure with today's data collecting and crunching powers, some magic formula can be worked out--something akin to the Netflix prize for education--but as Ryan shows, first we have to agree on what we actually value.

a call for grading reform

In the pages of the vaunted Onion A.V. Club blog, noted educational theorist Josh Modell provides one of the best possible arguments for grading reform.

Letter grades are relative, and nowhere on the site (or in our reviewers' minds, I don't think) does an A or A- mean that a movie (or CD or book or videogame) is one of the best ever made. I certainly don't believe that Crank 2 is one of the best movies ever made. But what that A- did signify, and what I think high grades on The A.V. Club generally signify, is success. In my eyes (and in the eyes of Keith Phipps and Scott Tobias, who both enjoyed Crank 2), the movie was successful at what it was trying to be: weird, almost parodic, and over-the-top. Those seemed to be its goals, and it delivered. It was certainly never boring or cliched, and much of the time it was actually pretty incredible. For people who like these types of movies (and I can think of very few analogs for this particular movie), Crank 2 was pretty great. So what grade do you give to a movie that you think is highly entertaining and successful but exists in a genre that's not allowed a high grade?

That gets into what I like to think of as "genre profiling." You haven't seen the movie, and yet you know it's not possible that it deserves an A-. To some degree I agree with you: Action movies (even outliers like this one) are, by definition, incapable of being among the greatest films of all time. And that's fine, but it's a bit unfair to ghettoize them so much that an A (or even a B) grade is off the table. It's like deciding that the fat kid in gym class couldn't possibly do better than a B-, and then grading him accordingly. (What if he's a phenomenal goalie?) All of our writers understand--and I think our readers are sharp enough to understand--that Crank 2 isn't even in the same universe of greatness as The Godfather or (insert your all-time fave here).

It's true that A or A- grades in film are pretty rare at The A.V. Club, and I appreciate that that makes them more trustworthy. What I tell the music writers is that if something is getting an A or A-, it should be one of their personal favorites--top 10, if not higher--of the year. Crank 2 could very well end up one of my favorite movies of 2009. (And no, I'm not some sort of action junkie--my faves of last year were Synecdoche NY, Ballast, Snow Angels, Dear Zachary, and Wendy & Lucy, and I think this year's action hit, Taken, is one of the worst movies I've ever seen. It got a C-.)

By now you've guessed that Modell isn't actually an educational theorist. Which is too bad.

your cheatin' WASL

Surprise to no one: a high-stakes test induces "irregularities," including cheating by both students and teachers. KIRO's breathless story:

A four-month KIRO Team 7 Investigation uncovers hundreds of cases where students and teachers failed to play fair on mandatory assessment tests.

Investigative Reporter Chris Halsne proves how a little cheating can go a long way in improving scores and reputations.

Under the Washington Open Records Act, KIRO Team 7 Investigators acquired reports of around 650 WASL "irregularities" filed in the past three years.

We dug deep into that data and found accusations of cheating, which could falsely improve scores, spread out in 317 different schools in more than 100 districts.

Texting answers, using dictionaries and calculators, and teachers helping kids score math points are just a few examples of cheating reported to the Office of Superintendent of Public Instruction.

Our research also found that plenty of classroom cheating goes unreported – something students we interviewed confirmed.

Out of curiosity, I looked at the 2008 data for some local districts. The results are a little less than overwhelming. Some calculator mistakes (that didn't affect the outcome, apparently), an overzealous scribe, students passing notes, some "non-allowed accommodations," and, my personal favorite, "staff members [sic] cell phone went off during testing."

the report card, 167 years later

Last week, while visiting the Cabildo, a New Orleans museum chronicling the history of Louisiana, I chanced across a report card from 1842:

PUBLIC SCHOOLS -- SECOND MUNICIPALITY
Monthly Account of George Leahy from Monday, March 7, to Saturday, April 8, 1842

Credit Marks for Correct Lessons, 67 --duty requires 96
Merit Marks for Correct Conduct, 17 --duty requires 21
Checks for Misconduct,
Absent, 6 days Tardy, times
Remarks:--

The Parent or Guardian is requested to signify that he has examined this account, by signing it below the Teacher's name and returning it.

A few random thoughts.

1. I would love to see a report card coming from a contemporary public school that uses the word "duty" in any context.

2. Do public schools do "merit marks" these days? Mine sure doesn't.

3. As the Washington legislature plans to return education to 1842 funding levels, perhaps we oughtta return to 1842 pedagogy, too. Slates! Primers! Corporal punishment! (Double bonus: no more Dornian pseudo-WASL.)

standards-based grading: more popular by the minute

Blog-neighbor TRP's school is starting to have the same conversation we are: how to get smarter about grading. The TRP's initial take on one principle--no zeroes--isn't too favorable.

My colleague also said that if a student understands the material, than he or she is "an A student" and should get an A. She cited a positively brilliant kid--one of the best writers I've ever had in 12 years on the job--who got a C for me last year and is getting a C for her this year in AP. Why? She skips assignments. "But you've seen her writing," my colleague says. "She's an A student."

My response: "No. She has an A brain...but A students do the work. Therefore, she's not an A student."

We kicked around some possible alternatives: A 4-point system, where an A is worth four, a B three, a C two, a D 1, and an F zero. It penalizes kids less for screw-ups. (The aforementioned kid with 3 75s and a skip would get a C- under this system.)

I wouldn't buy into that either...it means that the kid who tried and struggled gets the same grade on a paper as the kid who played video games instead. That's unjust.

But then, that's because SBG requires something that most schools aren't doing: meaningful grades. As I explain in a comment over there,

The problem is that one grade (A, B, C) means too much and too little at the same time. I like some of the theory behind standards-based grading--but it requires a smarter report card. If "A" represents only learning, then we need separate categories (on the transcript, natch) for things like behavior, effort, and timeliness.

And it can be done.

We just need to be more--how do I say this gently?--Canadian.

See here for an example [pdf].

grading panacea / grading Pangea

Blog-neighbor The Science Goddess recently gave a talk on standards-based grading at the National Science Teachers of America conference. It was well-attended, which surprised her.

Instead, I had well over 100 people crammed into the room---sitting in the aisles, up at the presentation table and standing in the doorway straining to listen. I'm not sure how many others turned away when they saw the throng...and I know the fire marshal wasn't poking around because the number of people was well over the posted room occupancy. Wowser.

The experience was very validating---not so much for me personally as for the topic itself. Grading has arrived. When I talked to a few of the attendees about their "hardcore" attitude of staying to the end, they said that this was an area of need for them and I was the only one on the schedule talking about it. Others who chose to stay after the presentation to talk to me mentioned that they were trying to do some of these things at their schools---but it was a lonesome experience. It is indeed hard to implement something like this on your own. I got asked about presenting at other schools. Would I come? Would I talk to more than just science teachers? Would I answer the phone/e-mail if there were questions? Of course. But how sad is that people are all out there struggling on their own little islands of grading.

At my school, a miniature book club has met a couple times to discuss progressive grading practices. I get the same sense that TSG does: right now, standards-based grading is an archipelago in a vast, old-school ocean. But as teachers and principals get excited about it, and as word keeps spreading, within a decade I think we'll see a continent born.

the post-Bergeson future of the WASL

Dick Lilly, on the future of the WASL:

The door is now open to change the type of test from one that is based on a particular educational philosophy (the ultimate problem with the WASL and most tests of its ilk) to one that is content-based. At root, what’s wrong with the WASL is that it is designed in response to the dominant educational philosophy of the past 40 or so years — the “student-centered,” “discovery” learning that de-emphasizes content knowledge in favor of critical thinking. That's a short-hand description and a lot can be said about this approach, but what it amounts to is the contention that you can make profound decisions while looking at a spreadsheet in which every cell is blank.

To get away from this style of testing, Dorn should switch immediately at the high school level to the SAT subject tests (formerly the SAT IIs). These are good if not excellent content tests, as are the advanced placement (AP) tests. Such tests would provide a stunningly clear picture of what our high school graduates know.

a brief intro to standards-based grading

Want to get the jump on the trendiest trend in educational trends? Visit Dr. Pezz's classroom, where he introduces some of the basic principles of standards-based grading.

Oh, and here's a followup.

If your building isn't talking about standards-based grading, they will be soon. Trust me.

standards-based grading: coming to a school near me

I read about North Thurston's switch to standards-based grading in elementary schools...

Teachers no longer will include homework or students' behavior as part of a letter grade; the mark only will represent the student's learning achievement, measured by test scores, key assignments and observations. Work habits, homework and behavior will be evaluated separately.

Other changes include:

• Late assignments will not reduce a grade

• Extra credit or bonus points also will not typically factor into the grade

• Grades will not be based on attendance

• Homework will not be factored into grades, but rather used as practice and to guide teachers on what to focus on.

• Grades will be organized by standards or learning goals rather than on one specific subject. For example, the third grade report card will measure students' reading skills based on fluency, vocabulary and comprehension.

... and the next day was asked to read a slightly overgrown pamphlet about standards-based grading "fixes," including all the changes listed above. The goal: prepare me to participate in a book group examining Capital's assessment practices.

I detect a trend.

standards-based grading backlash in North Thurston

Change a letter grade to a standards-based number, and what do you get? Confusion and backlash.

Rich Semler sounds good on the radio box

I caught Rich Semler's appearance [mp3, 20:00 in] on the Dori Monson Show (Frank Shiers filling in) on KIRO this afternoon, and was impressed by his good humor, quick wit, and overall amiableness. I liked how Semler stressed the difference between being anti-WASL and anti-assessment. We can use less expensive, just as reliable instruments to get the results we need.

In an important interchange, Shiers asked Semler how he'd reform the WASL. A few steps:

1. Smarter use of multiple choice.
2. Get it back to teachers in 2-3 days.
3. 2-3 hour test, instead of days.
4. Testing complex thinking without taking 12-14 questions per student.
5. "Mime" systems like Delaware's.

It didn't help so much that Shiers, who's otherwise well-versed in educational issues, wasn't aware that Terry Bergeson is actually in the running, which perfectly illustrates the primary problem Semler faces. The Superintendent of Public Instruction just doesn't register on the average voter's radar.

If Semler can overcome that hurdle--and here, the ongoing publicity efforts by the WEA should help--he has a decent shot at an incumbent whose popular support, such as it is, weakens daily.

pissed pedagogue won't WASL

Seattle teacher Carl Chew refused to administer the sixth-grade WASL to his students, taking a two week unpaid vacation as a consequence.

Carl Chew, 60, who teaches science, wanted to take a stand against a test he considers harmful to students, teachers, schools and families.

"I did it because I think it's bad for kids," he said.

He said he knew he would face consequences, and might even be fired.

"When you do an act of civil disobedience, you gracefully accept what happens to you," he said.

Before the WASL started at Eckstein last week, Chew said he told Eckstein administrators that he would not give the exam. He said they tried to talk him out of it.

Or into it, really.

I'm surprised he wasn't fired straight up for insubordination. He must have a pretty good record--or a pretty forgiving administration. At the very least, he's sincere, knowing full well what his mini-rebellion would entail.

Click through and read the whole thing.

math WASL dumped for end-of-course exams

Hasta la geometría, baby:

Legislators and Gov. Chris Gregoire have decided to phase out the math portion of the Washington Assessment of Student Learning and replace it with end-of-course tests. The final budget announced on Wednesday included $3.2 million toward developing exit tests for each math course. The senior class of 2013 will still be the first group of students who must pass a math test to graduate, but they will be able to pass either the WASL math test or end-of-course exams. And the math WASL likely will be eliminated by 2014....

State Superintendent of Public Instruction Terry Bergeson supports the change, and Steve Mullin, president of the Washington Roundtable business group, said, “We’ve basically received a lot of assurances that while this was a different method, the rigor would be the same or perhaps higher.”

I can see a certain amount of pedagogical sense to the plan, detailed in HB 3166 [pdf]. Not sure it's going to be any less expensive, though.

Over at the Partnership for Learning blog, alisonm goes over the pros and cons. Her final assessment of the assessment:

Offering both tests gives students more options, which can be a good thing. But the reality is, if standards aren't aligned with curriculum taught by high quality teachers to motivated students, the kind of test given to students won't really make much of a difference.

WASL expenses set to rise

A week before March madness approaches again, a fun story: some of those classic WASL questions have to go.

The state superintendent's office estimates that the cost of administering the Washington Assessment of Student Learning could jump by $15 million to $25 million in 2009.

That's on top of the $22 million originally budgeted.

The increase is being driven mainly by a higher demand for testing because of the federal No Child Left Behind education law, said Joe Willhoft, an assistant state superintendent of public instruction in Olympia.

Sen. Rosemary McAuliffe, D-Bothell, and chairwoman of the Senate Education Committee, said paying $47 million for one year of testing is not an option.

Instead, state leaders are pressing for changes they say would shave millions off the cost and cut the amount of time spent testing without diminishing the accuracy of the WASL.

The savings would come primarily from chopping the number of open-ended, thought-provoking questions and delaying some extra features. And even then, the tests would still cost an extra $15 million or so.

The WASL takes eight days, and uses four booklets (reading, writing, math, and science). Even at today's lower rates, it costs the state roughly $70 per student to assess the WASL, assuming that student takes all four exams. Meanwhile, the SAT costs a student $43, while the PSAT costs $12. Is either any less valid than a proposed "mini-WASL?" Maybe it really is time to exorcise our Concorde effect demons and scrap the WASL.

Update 3/5: KOMO has more, noting that now, a bill to slice the WASL has passed the House. Meanwhile, the Senate wants to bring back end-of-course exams... Not while Gregoire's in office, methinks.