No project is without it’s challenges, it is the challenges that allow for growth and encourage new thinking and deeper research. The Number Agility assessment was initially envisioned in response to a challenge. Teachers lack the tools to assess number sense as a stand alone concept the way addition, subtraction, multiplication or division are assessed. In creating the assessment there have been multiple challenges from determining what number sense is and how the different components work together, as well as with the assessment itself. One of the biggest challenge was the amount of information available on the topic, finding information specifically on number sense and not as a chapter or a few paragraphs in a math textbook.

Student experience with the assessment format

While some aspects of the assessment pages will be familiar to students such as completing the counting patterns, others may not be.  In Klinger et al (Klinger, et al., 2015) it is suggested that students have opportunity to practice with the assessment format prior to them being completed. The number representation pages which were inspired by the ANIE and SNAP assessments may not be familiar to students in their format and lay out, however what students are being asked to do on those pages should be familiar.

Student Math Anxiety

Math anxiety can be debilitating for many students. While there is not yet a single known cause of math anxiety personal experiences, culture and family perceptions can impact a student’s view of math and themselves as mathematicians. Math anxiety can be debilitating and severely impact a student’s ability to be successful on a math task. While some anxiety on a test or assessment is normal and can indicate that a student wants to do well excessive anxiety can be detrimental to student learning. Math anxiety is it’s own research area and merits a much deeper exploration then is possible for the purpose of the Number Agility assessment. Students who experience math anxiety should be encouraged to try their best and given supports to address the anxiety.

Challenges with engaging in Authentic Number Sense Assessment

The breadth of number sense skills and the limitations of many forms of assessment has created challenges to authentically assess number sense with a single test. Number sense is responsive to situations and ever evolving as people gain more experience with numbers and use them in a variety of ways. Even as adults’ number sense changes as we specialize into specific fields or choose a broader path, hobbies will also influence the development of our number sense.  There is the additional challenge present in every task that we do in classrooms in finding something that fits all our learners.  “Authentic assessment includes authentic tasks that are rich and contextualized within real-world problems and that replicate genuine intellectual challenges,” (Koh, Chapman, & Liu, 2021) this number sense assessment removes much of the context from the assessment particularly in the written form. While there is opportunity for working in context in the interview portion particularly with the object counting and building numbers with manipulatives to create authentic contextualized assessments for students would require a unique assessment for each student and take away the standardized function of this assessment. We need to ask our students all the same questions to be able to determine what skills they have and where there is room for growth. We also need to present the assessment to each student in the same way to ensure that it is fair and valid assessment. By using the same assessment for each student and presenting it in the same way we are creating an artificial environment. To truly understand a student’s number sense we must also give them the opportunity to demonstrate it in performance tasks and real world situations.

Challenges in creating the assessment

The breadth of skills covered by the five strands of number sense created challenges for writing both the written and oral components of the assessment. Too long and students lose stamina and interest which could lead to unreliable data, not long enough there is the risk of not collecting enough data. There was also the challenge of creating an assessment that was practical in a classroom context. Part of that was not relying on digital forms of collecting student data while keeping the forms approachable. Using traditional paper formats was chosen as digital formats presents an additional layer of challenge in programming the assessment as well as keeping them secure and compliant with the relative privacy legislation.

Predicted Implementation Challenges

This assessment has not yet been used in classrooms and introducing it to students and staff is the true test of its reliability and validity as well as if it is as easy to implement as intended. Staff will need training on how to implement the assessment and new assessments and programs are not always well received often being seen as just one more thing being added to teachers’ plates. Convincing teachers that this assessment is needed and will give them valuable information about student learning will be key. Teachers will need the opportunity to explore the new assessment and will need to be consulted and given the chance to suggest changes to it. An open period of consultation with staff, students and families may help mitigate some of the challenges with acceptance of the new number sense assessment. Running a pilot project with a few classes will also allow for changes to be made and the assessment better adapted to the classroom context. With anything new that is introduced there should be a healthy level of skepticism and there will be an adoption period as teachers learn more about number sense and the assessment.

Questions of Validity and Reliability

Content Validity

“Content validity refers to the extent to which a measurement instrument (e.g., a test, questionnaire, or survey) accurately and adequately measures the specific content or construct it is designed to assess. In simpler terms, it assesses whether the questions or items included in an assessment are relevant and representative of the subject matter or concept under investigation,” (Thompson, 2023) as this assessment was created by a single author ensuring content validity as been a challenge. Having an additional math expert look at the assessment and confirm content validity would be a good follow up step. Content validity can also be ensured when running the pilot project with the assessment and comparing student results to other number sense performance tasks. As this assessment was created using the Number Agility level documents and the BC curriculum as guides content validity can be assumed.

Construct Validity

Construct validity ensures that an assessment measures what it was intended to. As this assessment was created using the curriculum and level documents as a guide and in the simplest form possible construct validity can be assumed.  Like with content validity construct validity can only be confirmed by piloting the assessment and comparing the results to other number sense assessments to ensure that they are showing similar data. Construct validity is also supported in this assessment by providing students with multiple opportunities to demonstrate the same skills and in a variety of ways.

Predictive Validity

Predictive validity is the ability of an assessment to predict future outcomes. The Number Agility assessment while still untested will hopefully be able to predict which students will need additional supports in developing their number sense as well as general math skills. The predictive validity of the assessment will take a few rounds of piloting to determine as well as comparison to other measures of student achievement currently being used including the in depth assessments being used by the resource team. In terms of current predictive ability students who are not meeting grade level or close to grade level expectations on their Number Agility assessments could be grouped into Tier 2 number sense groups for focused practice and skill building as well as referred to school based team for further discussion.

Reliability

Reliability refers to how consistent the results of an assessment are. In the case of the Number Agility assessment reliability has not yet been established as a pilot project will be needed to determine how consistent the results are. In this assessment there is also room for human interpretation of what students write while many sections of the assessment are objective and have a correct answer there are portions of the assessment that are open ended. The open ended questions have many possible answers and can be interpreted by the person scoring the student’s work in many ways. In order to ensure reliability teachers can score their assessments together and discuss the student’s responses coming to an agreement of what each level looks like. Reliability will also be influenced by how teachers present the assessment to their students and the consistency in which the implementation directions are followed. This will be especially important in the interview portion where teachers need to phrase the questions consistently and use the same materials for counting to ensure each student as a consistent experience.

Choosing an Assessment

There are four versions of the assessment to choose from in this package. Please carefully consider your students when choosing an assessment, the assessment are appropriate for students between the ages of 7 and 11. Use the level correlation chart to determine which version of the assessment you would like to use. If unsure where to begin it is recommended that the long form assessment is utilized to give the full picture of each student’s strengths and areas for growth at the start of the school year. The single level assessments are designed to monitor student progress within a level over the course of a school year.

It should be noted that the materials in this assessment assume mastery of the work in the early and primary years. Students who have large gaps in these skills will benefit from being assessed with the primary assessment or early years assessment to determine the appropriate starting place for instruction. Similarly if students demonstrate mastery of the skills in the intermediate assessment the Middle Years assessment may be given to determine how to best guide the student and enhance their number sense.