Donate now

Privacy Policy

Protection of privacy is our first concern, and SQE does not sell or trade information provided by its subscribers or supporters. Your information is used to process donations and newsletter subscriptions, and to contact you about upcoming publications and events.

feed iconSubscribe to our Blog

Follow Us
Follow SQESocQualEd
on Twitter

Please note Downloads require you to have the Adobe Reader installed, you can get it here for free Adobe.com

 

 
 
Society for Quality Education

Intelligent Testing

May 23, 2010 by .(JavaScript must be enabled to view this email address) at 06:30 AM

Since it’s the May 24 in Canada and our readers are probably at loose ends, here is a link to two one-hour videos of a symposium on high-stakes testing. It turns out that is not a simple, straight-forward issue at all. Much of the symposium is quite interesting and thought-provoking but, if you can’t watch all of it, we suggest you select the segment involving Dr. Daniel Koretz in Part I (Minute 16 - Minute 39). 

Comments

High-stakes testing of today, I disagree on what is being measure, and how the results are used. I believe in testing, but on the other hand the high-stakes testing have many problems that do not accurately show students achievement. Below are links that will show high-stakes testing, is not simple, but many in and out of the education field, portray it as being simplistic and a good measurement to decide on education policies.

1.  http://www.fairtest.org/arn/caseagainst.html

2.  http://www.youtube.com/watch?v=8YmO29-xRpY

3.  http://www.youtube.com/watch?v=zS2nNrtXrmQ&feature=related

4.  http://www.youtube.com/watch?v=vQEnV5tM_2Y&feature=related

5.  http://www.apa.org/pubs/info/brochures/testing.aspx#

On the video portion of Dr. Daniel Koretz, I found it very interesting, on the term coaching.  I have found many types of questions in math, where my child only had to learn two things, from the number of key facts in a whole concept. And we wonder why, children are not doing well in math. It probably explain why, my child fails portions of the math CRT, because the question is only asking for a partial answer, without requiring the need to know the full concept. My child will get the correct answer, but her method is not the correct one that is the approved one. The math CRT problems, are really measuring the ability to memorized certain methods of math, and not measuring the understanding of key concepts. If it was, then my child would be receiving a 4 or a 5 in the 5 areas of math measurements.

The most recent example, is where the school is predicting she will have trouble in the grade 9 CRT in exponents, and in particular the squared numbers. The example given by Dr.Koretz of the Pythagorean Theorem, is a question that would puzzled my child, who knows her squared roots, and the patterns. It is the question that is puzzling to her. The way it is worded, that will tripped her up. At the moment, the math teacher is working with her, not to improved her knowledge, but to give her proven strategies to break down the word problems, that are often found on CRTs, that really only measures how well one reads, and not their true math ability. It is odd to see the low CRT math scores, to any teacher that has taught her, knowing she is very skillful and adept in math.

Posted by Nancy on 05/23 at 04:46 PM

We do not have high-stakes testing in Ontario. Not even close. The EQAO tests sure aren’t.

Posted by Chuck on 05/24 at 06:42 AM

Chuck is correct that EQAO tests are not “high stakes” tests as generally understood; there are no consequences to students for poor performance. At one time successful performance on the OSSLT (aka the Grade 10 Literacy Test) was required for a secondary school diploma, but in recent years that has been changed to permit those who fail the test twice to take a specific course and compile a literacy portfolio to meet the graduation requirement.

EQAO tests are in the same genre as many of the state tests in the USA (“performance assessments” or “holistic assessments”). They are not norm-referenced (standardized on a bell curve); items are written anew very year, but with little opportunity to field-test or revise them; the level of difficulty and the rubrics for grading, as well as the content, changes every year, making year-to-year comparisons statistically invalid.

According to the Literacy and Numeracy Secretariat, the lack of improvement in EQAO performance is not due to poor basic skills (decoding, spelling, grammar, arithmetic and math algorithms) but to overemphasis on these in the schools, to the exclusion of “higher level thinking.”  Thus schools are now to concentrate on teaching inferencing, text-to-world connections,main idea, summarizing, writing from varying points of view, “explaining your thinking”  in mathematics (getting the right answer will only net you a Level 2), supporting arguments with details from the text and elaborations from personal experience, and so on.

There is some built-in class bias in this type of test, because the requirement that students relate texts to their own experience favours students who have relevant experiences to relate to.  One year a passage in the EQAO tests recounted camping in a provincial park and students were to relate the story to a family experience of their own in camping. Children in the high-rise urban jungle who have never set foot in a park of any kind simply have no relevant experience to “connect” to, however well they may read and comprehend the passage. This requirement for students to “connect to your own experience” is needed for a Level 3 or 4 on almost every item, so children without the rich experiences to relate to must either suffer a lower mark for it or fabricate an “experience” that never happened.

Posted by TDSBNW on 05/24 at 08:20 AM

It`s funny,this am I was reading the Globe and Mail and wondered how children taught -or rather adolescents taught will look say method would ever be able to read the name of the winner of the Palme D’Or in Cannes-

According to the Literacy and Numeracy Secretariat, the lack of improvement in EQAO performance is not due to poor basic skills (decoding, spelling, grammar, arithmetic and math algorithms) but to overemphasis on these in the schools, to the exclusion of “higher level thinking.“  Thus schools are now to concentrate on teaching inferencing, text-to-world connections,main idea, summarizing, writing from varying points of view, “explaining your thinking”  in mathematics (getting the right answer will only net you a Level 2), supporting arguments with details from the text and elaborations from personal experience, and so on.

I am really happy to see the Literacy and Numeracy`s take on this-Thank You

A-they live on an island and indulge fervently their opinions versus the results of collective research.
B-The fact that this goes on in the Google era is preposterous!
C-This is shameful!They are practicing education malpractice.

Thank You TDSBNW

Posted by Jo-Anne Gross on 05/24 at 08:36 AM

High-stakes testing: defined:

“Tests become high-stakes when they are utilized to assess students in a manner that has broadly-based and far-reaching implications on students’ future achievement potential. Such tests create additional pressure on schools and teachers to ensure that students perform well on single tests, while teacher and school performance is increasingly being assessed by the results of students’ achievement on such tests. Clearly, there is nothing progressive about the current inability to reconcile testing approaches with political aims. The teacher, the best fit to teach, is forced to fail a student who cannot pass one literacy test, heedless of why, or to teach only to the test to give the struggling student a chance of passing.”

http://findarticles.com/p/articles/mi_qa3965/is_200301/ai_n9228190/pg_5/?tag=content;col1

From Wikepedia: “In common usage, a high-stakes test is any test that has major consequences or is the basis of a major decision.[1][2][3]

Under a more precise definition, a high-stakes test is any test that:

is a single, defined assessment,
has clear line drawn between those who pass and those who fail, and
has direct consequences for passing or failing (something “at stake”).[4]”

http://en.wikipedia.org/wiki/High-stakes_testing

“HIGH-STAKES testing is the practice of attaching important consequences to standardized test scores, and it is the engine that drives the No Child Left Behind (NCLB) Act. The rationale for high-stakes testing is that the promise of rewards and the threat of punishments will cause teachers to work more effectively, students to be more motivated, and schools to run more smoothly — all of which will result in greater academic achievement for all students, but especially those from poverty and minority backgrounds. Although it is certainly arguable, we believe that, to date, there is no convincing evidence that high-stakes testing has the intended effect of increasing learning. By contrast, there is a growing literature suggesting that the unintended consequences are damaging to the education of students.”

http://www.britannica.com/bps/additionalcontent/18/35617813/Why-Has-HighStakes-Testing-So-Easily-Slipped-into-Contemporary-American-Life#

EQAO, and other CRTs are for all purposes are high-stakes tests, where it will impact a student’s future in receiving an education. In particular, the results of standard testing such as the EQAO, the schools have a tendency to include the results, when the students reached high school. Parents who have LD children, that are academically inclined, with 80 % averages or above, are the same ones who fail parts or all of the portions of the testing conducted throughout the years. The children are often pressured into going into applied courses, rather than the academic courses. This is a common issue among LD parents, where the standard testing is used against these students in the near future.

EQAO literacy test, has a big condition that unless the student passes it in whatever forms that are allowed, the student who fails the tests, and other alternative formats, will not received a diploma. I would think that is an important consequence for students.

Now what are the important consequences of a school failing to provide for students who have failed the tests?  None that I know of, but than it would mean, accountability in the public school system. The dreaded ‘A’ word that puts fear in any public education system.

Posted by Nancy on 05/24 at 10:06 AM

results of standard testing such as the EQAO, the schools have a tendency to include the results, when the students reached high school.

Not in Ontario.  The student’s EQAO results should not be in the student’s Ontario School Record (OSR) forwarded to the high school. However, sometimes documents that should not be in the OSR *do* get filed there, through accident or oversight.  Ontario parents, you have the right—and if you have any concerns, should act on this right—to request in writing an appointment with school administration to go through your child’s OSR. 

Some items *must* be left in the OSR: report cards, decision statements of an IPRC (Identification, Placement and Review Committee), legal notices such as custody orders, suspension notices, and the *most recent* IEP if the student has an official Special Education designation. Students who are not classified but have an IEP do not have to have the IEP included in the OSR unless the parent so wishes. In any case, only the most recent IEP should be there—not a collection over many years.  Work samples, test results, reports from social work, psychology, speech and language, and other services should only be included if they are recent and still applicable. Assessments from years earlier should be removed. EQAO results, since they are not current information, should also not be included.

In general, schools interpret a student’s EQAO performance with some healthy scepticism. While there are many “Level 1” students in the schools, there are very few “Level 1” EQAO results.  We see instead a phenomenon called “compression towards the middle” where top performers score lower than you would expect, and lower performers score better; this is part of the nature of holistic assessments. EQAO emphasizes that the student’s score is only a snapshot of his or her performance at a single point in time.

It used to be a big deal for a parent to go through the OSR: procedures have been greatly simplified and most schools are quite accommodating in going through the OSR with the parent and removing non-essential and outdated documents. While psychological assessments, speech-language assessments and the like are kept on file at the central school board offices, the parent has the right to determine whether these documents go in the student’s OSR. Generally, if the report contains information that will help staff support the student, parents want the teachers and support staff to have access to it.

As for the EQAO Grade 10 Literacy Test, it is no longer a “high stakes” test because the only way a student can fail to meet the graduation requirement is to not take the literacy course and submit a portfolio. Anyone who takes the course meets the grad requirement. A bigger obstacle for students getting a diploma is accumulating enough credits in a timely manner. Mathematics is a bigger issue here than literacy skills.

Posted by TDSBNW on 05/24 at 11:34 AM

The OSR file, has the same rules more or less, that is in other provinces concerning student files. What information is in it. for the most part, that students files include a lot of information, including the EQAO results. It depends on the board and for the most part it is at the discretion of the principal to include a hard copy within the file. Otherwise, the boards holds a electronic version of the individual’s student EQAO results, for a year. During that time, educators that have been granted access, have access to the electronic file, and where they can download and print off a hard copy. It is done only under the guidelines of supporting the student’s education and their learning needs.

That said, the school also receives the same letter as the parents do, on the individual student’s EQAO results.

One board:

Additional information: Part H
•
student registration form;
•
suspension letters - It is recommended that the principal review annually whether or not suspension letters should remain in the student’s OSR. It should be noted that suspension letters accompanying a Violent Incident Form shall not be removed from the OSR unless three consecutive years have passed during which no further suspensions for serious violent incidents have taken place;
•
work-education agreements for cooperative education students;
•
correspondence and/or forms relating to attendance and fees—Note: Individual schools are responsible for developing the means by which such information is recorded;
•
EQAO results.

http://www.tldsb.on.ca/documents/OSRProcedure.pdf

The above link, is the guidelines from the Ontario government provided for the boards.

The next link is the Ontario link, Ontario Student Record (OSR) Guideline, 2000. As far as I know, the OSR guidelines have not been updated.

http://www.edu.gov.on.ca/eng/document/curricul/osr/osr.html

As for parents seeking a look at the OSR file, that in itself is a bureaucratic nightmare. Removing an item, will be a testament to a parent’s persistence. Likewise for parents to use the EQAO results, as supporting evidence to get an assessment for their child, or enhanced targeted reading help for their child.

It really comes down to the principal, and how accommodating he is. That depends heavily on what the board’s goals and aims are. Often the goals of the board are much different than the individual student needs. As one poster has always said, and as much as I like to disagree with almost everything he says, I have to agree with him on this one. The group of students needs are far more important than the individual student.

The only item that I have found is that the EQAO results are not given to any post-secondary institute under any circumstances.

High schools, who do not have access to the previous EQAO tests results of individual students, must have a harder time to determine what grade 9 students are in need of help, to pass the grade 10 literacy tests. It actually adds another responsibility to the parents who are concern their child may not pass the grade 10 literacy, based on previous EQAO test results.

Posted by Nancy on 05/24 at 02:52 PM

As for this comment, “In general, schools interpret a student’s EQAO performance with some healthy scepticism. While there are many “Level 1” students in the schools, there are very few “Level 1” EQAO results.  We see instead a phenomenon called “compression towards the middle” where top performers score lower than you would expect, and lower performers score better; this is part of the nature of holistic assessments. EQAO emphasizes that the student’s score is only a snapshot of his or her performance at a single point in time.”

As stated on the Ontario site, describing EQAO assessment, “Teachers and administrators should be familiar with a variety of valid and reliable reading assessment techniques, including holistic, analytical, and performance-based tools. Provincial assessment provides a snapshot of student performance at the end of the primary division; ongoing assessment provides continuous feedback on student achievement in reading during the early school years.”

http://www.edu.gov.on.ca/eng/document/reports/reading/target.html#province

Now what does holistic mean in education -

It depends:

http://www.holistic-education.net/educate2.htm

“In holistic education there is no curriculum set by “experts” but rather it is developed by the immediate stakeholders—teachers, students, and parents. This ensures that what is studied is relevant and meaningful. However, this means that teachers must be creative and responsive to the individuality of their students. Teachers in holistic schools cannot simply “deliver” a pre-packaged curriculum, which is a challenge to some teachers but a great joy and inspiration to others.”

One can only wonder how a student who is struggling in reading and writing, are to received the supports that are needed at the school level.

On another site, ” In the end, standardized tests provide different information. A standardized test is essentially a snapshot in time using one method of assessment. Teachers assess their students using a broad range of tools. A more holistic and complete picture of a student’s performance can be seen from various types of assessment that the teacher will do throughout the year. Many educators believe that if educational decision-making is based on a teacher’s year long assessments of students, rather than one standardized test, improvements in student learning will be achieved.”

http://www.tvo.org/cfmx/tvoorg/tvoparents/index.cfm?page_id=145&action=article&article_id=361

Nice way, of avoiding the tangle web of under achievement or over achievement concerning standard testing. No longer do I need to wonder, on students who are maintaining 80 or above averages, but do poorly on the EQAO testing, or CRT testing conducted in other provinces. Apparently, it is a natural phenomenon, and should be of no concern at all to parents concerning reading and writing issues of their children.

Posted by Nancy on 05/24 at 02:54 PM

Depends what you mean by high in high stakes I guess. Grade 10 literacy either must be passed or a lit course substituted. Call it a moderately high stake.

Posted by Doug on 05/26 at 05:06 AM

There is no high-stakes testing in Ontario.

Posted by Chuck on 05/26 at 07:09 AM

Testing should reflect the syllabi—that simple.  Nowadays, we employ so many ‘experts’ to do the obvious…
Reading the blogs brings to mind some of my former co-workers, and unfortunately it reminds me of the ones who hated working—instead of just doing their job, they used to spend hours of time and energy covering up what they hadn’t done.  This is what the testing as described in the above blogs reminds me of—they won’t teach phonics, nor drill in math, so they have to engineer the tests to hide the fact that some of the children are totally illiterate and have zero math skills.  Educrats don’t want us to know how far advanced some students are also (and the advanced students would not be ahead because of what these educrats are doing).
By the time some of these poor children are in the tenth grade, in order to graduate, they have to do an alternative test because they can’t pass a simple literacy test.  If they would just do the right thing in the first place—teach properly, test the syllabi, and have the children who haven’t mastered the same repeat the year—all of this time and money would not have to be wasted.  Of course, then we would be spending far fewer dollars on education, as we did years ago, so obviously this is not in the educrats best interest. 
It’s becoming more and more apparent that almost no decision is made by our politicians nor the educrats that is in the best interests of the children.

Posted by Bev on 05/27 at 02:50 PM

One should only fail the grade, if they failed in one of the core subjects, in the elementary schooling. In high school, failing any course, should only have to repeat the course again. I hated that, fail one subject, one had to repeat the whole grade in high school. They did not change the rule until I was in grade 11. It did not matter what stream that a student was in. At the same time, they made it easier for students to switch from a 4 year to a 5 year. Before that, a 4 year student was forbidden to take a 5 year course. Likewise, for a 5 year student, to take a 4 year course. But again, they had the teachers, the resources, and it was the school that adapted to the various student’s time tables.  These days, students pick their courses, and keep their fingers cross, that they will get to take the course.

Posted by Nancy on 05/27 at 03:56 PM
Page 1 of 1 pages

Leave A Comment

Name:

Email (required but not displayed):

Emotions

Notify me of follow-up comments?

Submit the word you see below:


Next entry: Worth the Wait

Previous entry: It's Not Just About the Kids