Articles

We report on an innovative design of algorithmic analysis that supports automatic online assessment of students’ exploration of geometry propositions in a dynamic geometry environment. We hypothesized that difficulties with and misuse of terms or logic in conjectures are rooted in the early exploration stages of inquiry. We developed a generic activity format for if–then propositions and implemented the activity on a platform that collects and analyzes students’ work. Finally, we searched for ways to use variation theory to analyze ninth-grade students’ recorded work. We scored and classified data and found correlation between patterns in exploration stages and the conjectures students generated. We demonstrate how automatic identification of mistakes in the early stages is later reflected in the quality of conjectures.

Abstract:

We used an assessment platform to study the potential of rich student data obtained online to influence classroom instruction and help teachers respond to students’ needs. The article focuses on technology-based formative assessment in the course of teaching a unit on quadrilaterals. We studied example-eliciting tasks, requiring students to determine the truth of a given claim and to use interactive diagrams to construct examples to support their answer. Our aim was to teach the concept of parallelogram by exploring students’ concept images and responding adequately to their submissions. A group of 11-year-old students studied the unit as part of their geometry curriculum. We used an assessment platform, which provides tools designed to monitor examples produced by students interacting with a dynamic geometry environment and to characterize critical and non-critical attributes of the submitted constructions. We present the teaching decisions made by the teacher (the first author), who used automatic feedback on the characteristics of three assessment tasks.

Abstract:

While curricular change often begins with a new textbook, its eventual enactment in the hands of individual teachers depends on their orientation towards the curriculum and its representation in the textbook. Studying this teacher-textbook relationship is crucial for understanding how curricular reform intended by textbook designers is likely to be implemented in individual classrooms. We have been studying this by means of didactic tagging—a methodology where teachers assign metadata to textbook tasks. Categories of metadata include both descriptive aspects of tasks and aspects of their envisioned enactment. Prior research has focused on characteristics of the textbook that emerge as patterns in metadata that persist across multiple taggers. In the current study we turn our attention to individual differences across taggers. Four teachers tagged tasks that comprise an entire pre-calculus high school textbook. Analysis generated non-trivial findings regarding each tagger’s interaction with the textbook, in the form of individual patterns in the tagged metadata. These patterns allowed us to put forth tentative “profiles” of each of the four teachers’ interactions with the textbook. We propose this as a method for investigating and assessing teachers’ didactic orientations, and for predicting aspects of their enactment of textbook curricula, while avoiding time-consuming and intrusive observation of classroom teaching. 

Abstract:

This study addresses high school students’ conceptions of mathematical definitions of congruent and similar triangles. The findings indicate that many of the participants differentiated between definitions and theorems and did not always accept the congruent and similar triangles theorems as formal definitions of congruency and similarity. Based on the participants’ explanations of their responses and from the interviews performed, it appears that two issues prevented some participants from accepting or preferring these theorems as definitions. The first was a concern for uniformity: there is only one known and accepted definition of each concept. The second was a focus on the essence of the concepts: the essence of the concepts of similarity and congruency lies primarily in the lengths of the sides of a triangle. The students who accepted these theorems as formal definitions explained their reaction as arising from the equivalence and from the theorems including necessary and sufficient attributes. Students’ difficulties in understanding the characteristics and roles of mathematical definitions of geometric concepts affected their understandings of mathematical and geometric definitions. This behaviour indicates a tendency to interpret the content of theorems incorrectly and an inability to unpack the logical structure of the theorem.

Abstract:

With the emergence of e-textbooks, along with expectations to integrate technology in instruction, teachers are becoming significant co-designers of the curriculum. Informed selection and sequencing of learning resources requires sensitivity to didactic nuance, and tools to support development and application of such sensitivity. Teachers’ practices are constrained by the aspects of resources that are “searchable,” which are limited using standard search engines, and the selection of tasks is influenced by the engines’ ranking of search results, reflecting their popularity. We are developing and researching a coupled pair of tools to support mathematics teachers in making informed curricular decisions—a tool for tagging learning resources with prescribed categories of didactic metadata and a dashboard for browsing collections of resources according to this tagged metadata. In this article, we investigate affordances of these tools for the professional development of mathematics teachers—both practicing and pre-service teacher candidates. Viewing the dashboard, along with the metadata that it encodes, as a boundary object between the teachers’ and the researchers’ perspectives on curricular design, we show how teachers learned through acts of boundary crossing, conceived as transitions and interactions between the two communities’ curricular discourses. We show how using the dashboard in a task selection assignment encouraged teachers to reflect on their practice—making explicit the tacit considerations that they apply to curricular decisions and articulating them from the researchers’ perspective. We also describe the emergence of “hybrid” search strategies, integrating multiple perspectives to create practices that are both didactically informed and practically relevant for instruction. 

Abstract:

In the world of print, textbooks were the most important tools for dictating what and how student learn in schools. The introduction of Information and Communication Technology (ICT), however, gave rise to eTextbooks – a multi-modal, hardware mediated, and connectable, curriculum material. Indeed, the emergence of eTextbook creates fascinating opportunities for teaching and learning, but at the same time, it poses new challenges for both educational practices and policy making by revolutionizing the traditional pedagogical practices, classroom culture and the textbook publishing industry. These new challenges require rethinking and reexamining the appropriateness of the institutional and legal norms which govern the use and authorship of textbooks. This paper identifies the new challenges introduced by eTextbooks, and offers some insights on the policy and legal implications. 

Abstract:

We argue that examples can do more than serve the purpose of illustrating the truth of an existential statement or disconfirming the truth of a universal statement. Our argument is relevant to the use of technology in classroom assessment. A central challenge of computer-assisted assessment is to develop ways of collecting rich and complex data that can nevertheless be analyzed automatically. We report here on a study concerning a dedicated design pattern of a special kind of task for assessing student’s reasoning when establishing the validity of geometry statements that go beyond a single case, concerning the similarity of triangles. In each task students are given three relations that exist either in every triangle or in special types. Their task is to verify, by creating an example, claims that argue for logically compounded claims built out of the given relations. 50 students, aged 15–16, were asked to verify or disprove claims. Their submissions were automatically characterized along categories based on the correctness of the claims they chose and the examples they used to support the claims. We focus on characterizing the properties of the conjunction/disjunction design for automatically assessing conceptions related to examples generated by the learner with interactive diagrams. Our analysis shows that our automated scoring environment, which supports interactive example-eliciting-tasks, and the design principles of conjunction and disjunction of geometric relations, enable one to assess students’ exploration of the logic of universal claims, characterize successful and partial answers, and differentiate between students according to their work.

Abstract:

In their enactment of the curriculum, teachers have a substantial role as instructional designers. Accordingly, any evaluation of the progression of students’ learning should first be concerned with the pedagogical intentions of the teacher. In this article we present a method for reconstructing teachers’ implicit and tacit considerations in their selection, sequencing and enactment of tasks. Two 11th grade teachers tagged all of the tasks that comprised a 5-week learning progression. Tagging consisted of assigning values to prescribed categories of metadata. Visual representations of the metadata revealed patterns in the tagged progressions, and allowed the teachers to reflect upon these patterns. Both teachers, though guided by very different didactical considerations, validated that many of their explicit and implicit intentions were revealed in the representations of the progressions. Furthermore, both teachers had the opportunity to reflect on tacit aspects of their instructional design that they were not previously aware of.

Abstract:

With the growing availability of technology, teachers are becoming significant codesigners of the curriculum—resequencing, editing or supplementing a primary textbook with publicly available digital learning objects. This places a grave responsibility on teachers, who must have a clear view of the rationale and didactic intentions of the textbook in order to “re-source” it while maintaining its coherence. Yet, teachers are seldom provided with means to understand textbooks’ structure and underlying design principles, particularly at the grain size of individual learning objects. We are developing tools to express the “voice” of a textbook—one tool for assigning specific metadata to learning objects (i.e. “tagging”, sometimes referred to as “coding”) and another for analysing the coherence of the textbook based on the tagged metadata. In the current study, the 3 pre-calculus chapters of a high school textbook were tagged. Based on the tagged metadata, we used the second tool to reveal the relative prominence of different “types” of tasks, analysed by the categories of metadata. We focused on commonalities across taggers to explore the nature of the textbook. Triangulating our findings in an interview with the textbook author, we show that some patterns in the metadata reflect explicit design principles, such as avoiding symbolic representations in tasks that open a topic, while others reflect tacit principles—making sense to the author though not explicitly intended—such as eliciting non-technological student justifications in the topic of derivative. We conclude that this methodology offers novel opportunities for the study of textbook design.

Abstract:

This study investigated the experience of preservice mathematics teachers working with example-eliciting tasks (EET) in an online formative assessment platform (Seeing the Entire Picture, STEP). The study focused on the effect of working with EETs on the preservice teachers’ knowledge of pedagogical content and pedagogical design capacity (PDC). Participants consisted of three preservice teachers, studying for their Master’s degree at a teacher education college in Israel. The findings show that the preservice teachers’ experiences of engagement with EETs as students were reflected in their knowledge of content and of students (KCS): for example, which examples are easy, difficult, confusing, and which examples motivate for learning. The experiences also affected their knowledge of content and teaching (KCT), namely the proper sequence of instruction and the choice of examples for use in the classroom to facilitate deeper understanding of concepts. Findings also show that participants evaluated the collective example space submitted by their colleagues without focusing only on correctness, but also attending to variance between different correct examples. Finally, the findings indicate that experiencing EETs using STEP enhanced the PDC of preservice teachers.

Abstract:

What are mathematics teachers’ considerations in grouping students, and how could automated formative assessment systems help them in doing it? In this study, nine teachers were asked to use data on students’ performance in a mathematics task, derived from an automated formative assessment system, to create pairs in which students could contribute to their peers. We called this grouping strategy “mutual.” The teachers were also asked to explain their considerations for each grouping. We found two main grouping strategies in addition to the mutual one: based on similar answers (“similarity”), and based on dissimilar answers, in which one student performed better than another and could teach the other (“hierarchy”). Findings show that despite the experimenter’s request to group students based on mutuality, teachers mostly grouped based on other considerations, at times even grouping students whose answers were mutual using hierarchical considerations. In some cases, different teachers formed the same groups of students based on different grouping strategies. The findings confirm the hypothesis that informed grouping may be challenging for teachers, and may benefit greatly from an automated pairing system.

 Abstract:

Studying interactions across communities of mathematics teachers and educational researchers in professional development poses theoretical challenges. The authors suggest that it may be productive to view such encounters as the meetings of two professional communities that have different, possibly conflicting perspectives on the theory and practice of mathematics education. Drawing on the notions of boundary objects and boundary-crossing, the authors propose a framework for how teachers and researchers may learn from and with each other through joint work on common objects, through which they can explicate, reflect upon and modify their perspectives. Through the analysis of three educational projects from different countries – Italy, France and Israel – the authors describe ways in which the structure of boundary objects supports various aspects of dialogical learning, thus providing an initial frame for the proposed theoretical constructs. 

Abstract:

This study examined the changes that a group of nine Israeli 7th grade mathematics teachers, who collaborated on editing the textbook they used in class, chose to make in the textbook. Data sources included the project website (using a modified wiki-book platform), the group meetings, interviews with the teachers, individual papers written by the teachers, and a research journal. Data analysis revealed 4 types of change that the teachers wittingly and purposely suggested making. Three of them comprised changes related to ways suggested in the textbook to teaching and learning mathematical content: integrating technological tools, re-structuring textbook content presentation, and adding materials for students with low achievements. The 4th type of change was of a different nature, associated with creating organizing tools embedded in the textbook in order to make the textbook user-friendlier. The work on the 4 types of change involved 3 kinds of challenge: professional, conceptual, and technical. 

Abstract:

We studied a unique design principle for assessing online work and student solutions to rich interactive tasks. Students controlled the choice of givens in a task, creating a rich personal space of examples based on their personal preferences. We conjectured that using such a personal space for formative assessment provides the teachers with online information beyond correctness, provides a way of learning more about the student’s mathematics, and enables teachers to provide students with feedback on their idiosyncratic work. We studied a group of students in the 9th grade working on one e-task involving linear functions. We describe the principle of designing tasks that give students the choice of the givens, and investigate how the automatic platform for formative assessment makes it possible to identify and characterize students’ choices of givens to work with.

Abstract:

In this study we aim to characterize the challenge related to sketching functions by constrained sketching on a grid. Our research question is as follows: What are the characteristics of sketching graphs by dragging points vertically in assessment e-tasks? Specifically, which functionality does this design support? Towards this goal we explore how students construct a sketch of f'(x) based on a given graphic representation of f(x), and vice versa, how they construct a sketch of a function based on the graphic representation of its derivative. The analysis of 114 submissions of high-school students, support the formation of assessment tasks’ design principles for drawing a sketch. The sketch that students construct makes it possible to characterize each submission with respect to the actions the students took at critical or non-critical points; to whether they attend to certain points separately or to domain and to conjecture about various concept images.

Abstract:

Tasks that require students to construct examples that meet certain constraints are known to be used in mathematics education. It is also well established that while examples are not proofs (for general statements), they have a supporting role in the preliminary stages of making sense of a certain mathematical phenomenon. In this study we examine a task design in which students are required to submit a supporting example to their explanation negating an existential statement. We introduce the idea of limit confirming examples and present their use in an analytical geometry task in a 10th grade class in Italy, along with the explanations they support in negating an existential statement. The results show that the design was effective in fostering the construction of limit confirming examples that could be considered as means of argumentation in the initial parts of proof construction.

Abstract:

Our research focuses on the e-assessment of challenging ‘construction’ e-tasks designed to function as a dynamic interactive environment of multiple linked representations (MLR); we explore the effect of constraints on the variation in the students’ response space. Students are asked to determine whether an existential statement is correct. If they answer "yes," they construct an example in a MLR environment to support their answer; otherwise, they provide an explanation. The submitted example may be a sketch or an algebraic expression that can be checked automatically. Using a design-based research methodology, we describe a two-cycle study, focusing on one e-task on the topic of tangency to a function. Findings suggest that adding constraints to a logical mathematical statement enriches the variation of the response space and helps reveal different characteristics of students’ thinking.

Abstract:

Automated online formative assessment of students' work in a rich digital environment has the potential to support and develop students' reasoning process. Previous studies have presented the challenge of designing e-tasks. Here we focus on the challenge of designing a personal online formative assessment that supports the students' reasoning process. A common type of online formative assessment is elaborated feedback. We provide a design principle for elaborated online feedback of students’ work on an online example-eliciting task (EET) using the Seeing the Entire Picture (STEP) platform. We demonstrate two cases of attribute isolation elaborated feedback (AIEF) design, and the case of a pair of students who used the AIEF to support their reasoning process.

Abstract:

This study compares the different characteristics of student answers addressed during classroom discussion by mathematics school teachers in various settings. The study examines 3 9th grade mathematics teachers from the same school in Israel, all of whom taught the topic of quadratic functions. We examined and analyzed the teachers' choices and sequencing of the different characteristics of parabola sketches in a classroom discussion about student answers. The teachers analyzed student answers either manually or using the Seeing the Entire Picture (STEP) automatic formative assessment platform. Findings suggest a difference between the characteristics that teachers addressed in the classroom discussions in the two conditions. In the manual setting, teachers focused on incorrect features of the example, whereas in the automatic setting they focused on characteristics that emphasize different mathematical dimensions.

Abstract:

Rich technological environments present many opportunities for guided inquiry in the mathematics classroom. This chapter focuses on the teacher’s role in supporting the formation and justification of conjectures by students during whole-class discussion. I examine the practice of an expert teacher who conducts a classroom discussion based on students’ conjectures formed while working in pairs in a dynamic geometry environment (DGE). The way the teacher categorizes the different conjectures, then addresses them during the whole-class discussion is analyzed. Using this example and extending it to other guided-inquiry examples, I show how this type of categorization may be offloaded onto a technological platform that would do it automatically, making the categorization of student answers an option not only for teachers who could perform this categorization on the spot, but also for those who cannot

Abstract:

Using examples supports the construction of concept images and concept definitions of mathematical concepts. Examples can also serve to examine relationships and connections between different mathematical concepts that learners sometimes make during the learning process of different mathematical concepts. This paper focuses on the interactions between teachers and definitions of mathematical concepts, as they manifested when working with online mathematical tasks that require learners to create examples that meet a specified set of conditions. These interactions required the teachers to re-examine the definitions of mathematical concepts and their boundaries. We will describe how the automatic platform of formative assessment enables and sometimes forces reiteration and fine tuning of definitions of the mathematical concepts, and examine possible impact on the learning process.

Abstract:

Multiple choice (MC) items are the natural choice for automated online assessment. Ideally, making a choice should be based on knowledge and reasoning. Nevertheless, studies demonstrate that often various techniques (e.g. guessing) are the common practices. In the last decade technology has been employed to support real-time feedback as formative assessment for teaching and learning. This study examines whether and how learner generated examples, when required as support to the choice made in MC task, could be automatically identified to give insights into learners' understandings. Results show discrepancies between chosen correct statements and their supporting examples. Other automatically assessed characteristics are related to learner's approaches and strategies.

Abstract:

We explored the challenge of analyzing "symbolic sketching," using the Seeing the Entire Picture (STEP) online assessment platform. STEP enables students to submit examples of mathematical objects constructed as interactive diagrams. Students use various tools to draw functions and analyze specified properties. We describe submitted symbolic expressions that are (a) incorrect, although the graph expresses the main properties of the solution, which when analyzed as a sketch may meet the requirements of the task, and (b) correct sketches that are too complex to analyze symbolically, which must therefore be treated as a sketch.

Abstract:

This paper describes an analysis of a professional development program for teachers in automatic formative assessment that took place during the past two years. We focused on the use of a formative assessment platform (called STEP) to support mathematical classroom discussion. We will describe the process of using this platform, and the change in using it by novice and advanced teachers during the program.

Abstract:

In the present study, we ask whether and how online assessment can inform teaching about students’ understanding of advanced concepts. Our main goal is to illustrate how we study design of tasks that support reliable online formative assessment by automatically analyzing the objects and relations that characterize the students’ submissions. We aim to develop design principles for e-tasks that have the potential to support the creation of rich and varied response spaces that reflect convincingly the students’ perceptions. We focus on studying the design principles of such an interactive rich task, and the representations and tools that support automatic real-time analysis of submitted answers, primarily in the form of free-hand sketches. Using a design research methodology, we describe a two cycle study focusing on one e-task concerning tangency to the graph of a function. We characterize the mathematical attributes of examples constructed by students in an interactive environment. Checking the correctness of answers is only one of the functions of the STEP platform that we use. That platform can identify submissions that contain partial information (e.g., missing or misplaced tangency points), can categorize these answers, and allow the teacher to analyze them further. The reported analysis demonstrates how the design of the tasks and the characterization of mathematical attributes can help to expand our understanding of different ways of conceptualizing.

Abstract:

Paper tasks are often redesigned to function as digital tasks. The research and design literature (Pead, 2010; Burkhardt & Pead, 2003) has reported on the challenges of such a transformation. We report on a study exploring the design principles of an e-task, originally designed as a paper- and-pencil task and converted into an interactive diagram. We describe a paper task in the content area of parametric functions, and report on results from an experiment conducted with 39 high school students, who dealt with an e-task based on a paper task. Analyzing the results, we demonstrate that in a redesigned e-task based on a paper-and-pencil task, technology should allow self-reflection, promote learning, and guide the students to focus on the important details without unnecessary distractions.

Abstract:

Teachers face challenges when conducting lessons built on student- designed solution strategies. The work we describe here is part of an effort to explore how technological support for classroom formative assessment may play a role in attempts to reform mathematics instruction. We focus on the challenges involved in teachers' attempts to design and experience a new genre of formative assessment with an automated formative assessment platform in mathematics (STEP).

Abstract:

Reform efforts in mathematics education since the late 1980s have sought to change how mathematics is done in classrooms—to reduce the focus on technique and to encourage creativity. For example, in the US, the NCTM Standards movement sought to encourage classrooms where students present multiple solutions to open-ended tasks and then engage in discussion to justify and critique these solutions (NCTM, 1991). Similarly, the US Common Core State Standards Initiative (National Governors Association Center for Best Practices/Council of Chief State School Officers,2010) includes the Standards for Mathematical Practice that describe classrooms in which, for example, students use appropriate tools strategically, persevere to complete demanding tasks, construct viable arguments and critique the reasoning of others, and attend to precision. Still, teachers face challenges when trying to conduct lessons that productively build on student-designed solution strategies (e.g., Ball, 2001). In this article, we focus on formative assessment, conceptualized as: “all those activities undertaken by teachers, and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged” (Black & Wiliam, 1998, pp. 7-8). Specifically, we explore how technological supports for classroom formative assessment (Stacey & Wiliam, 2013) might play a role in attempts to reform mathematics instruction, particularly if they provide real time feedback on complex student performances that can be immediately used in the service of instruction. To do so, we use the construct of instructional exchanges (Chazan, Herbst & Clark, 2016) that: are marketplaces managed by teachers; teachers recognize in the midst of students’ mathematical activity those actions taken by students that “trade” as indicators of the acquisition of the knowledge that teachers are supposed to teach. In that sense, these student actions have become academic “gold”. (Buchbinder, Chazan & Fleming, 2015, p. 2). A challenge for teachers teaching in the institutional context of contemporary compulsory public schooling is to manage such exchanges given the quantity of student work produced by students during a school day. Pedagogical inventions like two column proof (Herbst, 2002) or the canonical method for solving equations (Buch-binder, Chazan & Fleming, 2015) support, among other functions, teachers’ management of instructional exchanges by formatting student work in ways that make it more easily assessed with a quick survey by the teacher. With these sorts of pedagogical inventions, each piece of student work does not need its own careful hermeneutic interpretation. But, while such inventions may help teachers manage instructional exchanges, at the same time, these sorts of inventions are often criticized for misrepresenting the discipline (Schoenfeld, 1988) or stunting student learning and removing opportunities for student creativity (Rittle-John-son & Star, 2007). Perhaps instruction could be changed if a large amount of the responsibility for managing the instructional exchange were off-loaded onto technology, freeing up teachers’ attention for student solutions that are out of the ordinary in different ways. We explore this possibility by first sharing vignettes of current classroom practice and an imagined alternative. We then articulate important characteristics of systems using computer checking of student responses that are intended to support teachers’ formative assessment in classrooms.

Abstract:

The M-TET (Mathematics Teachers Edit Textbooks) project invites mathematics teachers to collaborate in editing the textbooks they use in their classes as a means of transforming conventional connections among teachers, curriculum developers, and mathematicians into more productive connections. The unique aspects that characterize the work environment offered to teachers include the following: designing a textbook for a broad student population, producing a textbook by making changes to a textbook designed by expert curriculum developers, and consulting with professionals that are not part of the teachers’ usual milieu (textbook authors and mathematicians). This chapter explores the nature of the connections between teachers and textbook authors, and between teachers and mathematicians that participation in the M-TET project made possible, and it discusses what might be gained by offering such a work environment. 

Abstract:

Our research focuses on the e-assessment of challenging calculus construction e-tasks designed to function as a dynamic interactive environment of multiple linked representations (MLR) that provide feedback to the learner. A construction e-task requires students to use technological affordances to construct examples that satisfy specific conditions. The e-task is checked automatically and intermediate actions and submitted answers are reported. We present here an example of a construction e-task and report on a pilot experiment designed to elucidate the role of the dynamic MLR environment in solving and assessing construction e-tasks. Specifically, we examine the student’s submitted solutions and analyse whether it helps reflect the reasoning behind the answer.

Abstract:

This chapter examines how the conventional relationships between teachers and textbooks may be expanded so that teachers become more genuine participants in the process of textbook development. The Integrated Mathematics Wiki-book Project is used as a vehicle for investigating this matter. First, the work environment provided for teachers is described. Then, the chapter focuses on the ways in which teachers participated in the joint editing of a textbook they were using in class, during the first year of the project. The analysis focuses on three aspects that characterize the unique work environment provided for the teachers: (1) designing a textbook for a broad student population, (2) preparing a new textbook by making changes to a textbook designed by expert curriculum developers, and (3) consulting with professionals that are not part of the teachers’ usual milieu.